58 lines
2.9 KiB
JSON
58 lines
2.9 KiB
JSON
{
|
|
"generated_at": "2026-04-06T10:57:01.947412",
|
|
"embedding_model": "bge-m3:latest",
|
|
"index_name": "avap-docs-test-v4-bge",
|
|
"result": {
|
|
"status": "ok",
|
|
"questions_evaluated": 5,
|
|
"elapsed_seconds": 48.5,
|
|
"judge_model": "claude-sonnet-4-20250514",
|
|
"index": "avap-docs-test-v4-bge",
|
|
"category_filter": "all",
|
|
"scores": {
|
|
"faithfulness": 0.4956,
|
|
"answer_relevancy": 0.508,
|
|
"context_recall": 0.3143,
|
|
"context_precision": 0.2425
|
|
},
|
|
"global_score": 0.3901,
|
|
"verdict": "INSUFFICIENT",
|
|
"details": [
|
|
{
|
|
"id": "GD-001",
|
|
"category": "RETRIEVAL",
|
|
"question": "What is AVAP and what is it designed for?",
|
|
"answer_preview": "AVAP, or Advanced Virtual Application Protocol, is a new proprietary language. The provided context does not contain specific details about the purpose of AVAP but indicates that it involves adding variables to a system, setting parameters, calculating discounts based on user type and purchase histo",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-002",
|
|
"category": "RETRIEVAL",
|
|
"question": "How does AVAP handle conditional logic? What commands are used and how are blocks closed?",
|
|
"answer_preview": "AVAP uses the `if` statement to handle conditional logic. The condition is checked using a comparison operator, such as `==`, which compares two values for equality.\n\nBlocks in AVAP are typically closed with curly braces `{}`. For example:\n\n```avap\nif(condition, true_value, false_value)\n // code ",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-003",
|
|
"category": "CODE_GENERATION",
|
|
"question": "Write an AVAP script that reads a 'password' parameter, generates a SHA-256 hash of it, and returns the hash.",
|
|
"answer_preview": "```avap\n[1] readParam(password)\nhash = encodeSHA256(password)\naddResult(hash)\n```",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-004",
|
|
"category": "CODE_GENERATION",
|
|
"question": "Show an AVAP script that loops from 1 to 5, builds a JSON object with each iteration index as a key, and returns it.",
|
|
"answer_preview": "```avap\nstartLoop(i, 1, 5)\n item = \"item_%s\" % i\n AddvariableToJSON(item,'valor_generado',mi_json)\nendLoop()\naddResult(mi_json)\n```",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-005",
|
|
"category": "RETRIEVAL",
|
|
"question": "How does AVAP support external HTTP calls? What commands are available and how is timeout handled?",
|
|
"answer_preview": "AVAP supports external HTTP calls through the `RequestGet` function. This function allows you to make GET requests to a specified URL, with optional parameters such as headers or data.\n\nTimeout handling in AVAP is managed by the `addVar` command. When a variable named `error_trace` is added using th",
|
|
"n_chunks": 5
|
|
}
|
|
]
|
|
}
|
|
} |