58 lines
3.0 KiB
JSON
58 lines
3.0 KiB
JSON
{
|
|
"generated_at": "2026-04-06T10:42:47.999309",
|
|
"embedding_model": "bge-m3:latest",
|
|
"index_name": "avap-docs-test-v4-bge",
|
|
"result": {
|
|
"status": "ok",
|
|
"questions_evaluated": 5,
|
|
"elapsed_seconds": 48.4,
|
|
"judge_model": "claude-sonnet-4-20250514",
|
|
"index": "avap-docs-test-v4-bge",
|
|
"category_filter": "all",
|
|
"scores": {
|
|
"faithfulness": 0.5378,
|
|
"answer_relevancy": 0.6515,
|
|
"context_recall": 0.3143,
|
|
"context_precision": 0.2425
|
|
},
|
|
"global_score": 0.4365,
|
|
"verdict": "INSUFFICIENT",
|
|
"details": [
|
|
{
|
|
"id": "GD-001",
|
|
"category": "RETRIEVAL",
|
|
"question": "What is AVAP and what is it designed for?",
|
|
"answer_preview": "AVAP, or Advanced Virtual Application Protocol, is a new proprietary language. It is designed to facilitate communication and interaction between different systems and applications in an efficient manner. The provided context does not contain specific details about the purpose of AVAP beyond its nam",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-002",
|
|
"category": "RETRIEVAL",
|
|
"question": "How does AVAP handle conditional logic? What commands are used and how are blocks closed?",
|
|
"answer_preview": "AVAP uses the `if` statement to handle conditional logic. The condition is checked using a comparison operator, such as `==`, which compares two values for equality.\n\nBlocks in AVAP are typically closed with curly braces `{}`. For example:\n\n```avap\nif(condition, true_value, false_value)\n // code ",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-003",
|
|
"category": "CODE_GENERATION",
|
|
"question": "Write an AVAP script that reads a 'password' parameter, generates a SHA-256 hash of it, and returns the hash.",
|
|
"answer_preview": "```avap\n[1] addParam(\"password\", pass_nueva)\npass_antigua = \"password\"\nif(pass_nueva, pass_antigua, \"!=\")\n addVar(cambio, \"Contraseña actualizada\")\nend()\naddResult(checksum)\n\nchecksum = encodeSHA256(\"payload_data\", checksum)\n```",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-004",
|
|
"category": "CODE_GENERATION",
|
|
"question": "Show an AVAP script that loops from 1 to 5, builds a JSON object with each iteration index as a key, and returns it.",
|
|
"answer_preview": "```avap\nstartLoop(i, 1, 5)\n item = \"item_%s\" % i\n AddvariableToJSON(item,'valor_generado',mi_json)\nendLoop()\naddResult(mi_json)\n```",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-005",
|
|
"category": "RETRIEVAL",
|
|
"question": "How does AVAP support external HTTP calls? What commands are available and how is timeout handled?",
|
|
"answer_preview": "AVAP supports external HTTP calls through the `RequestGet` function. This function allows you to make GET requests to a specified URL, with optional parameters such as headers or data.\n\nTimeout handling in AVAP is managed by the `addVar` command. When a variable named `timeout` is added using `addVa",
|
|
"n_chunks": 5
|
|
}
|
|
]
|
|
}
|
|
} |