58 lines
3.1 KiB
JSON
58 lines
3.1 KiB
JSON
{
|
|
"generated_at": "2026-04-06T10:41:02.528045",
|
|
"embedding_model": "qwen3-0.6B-emb:latest",
|
|
"index_name": "avap-docs-test-v4",
|
|
"result": {
|
|
"status": "ok",
|
|
"questions_evaluated": 5,
|
|
"elapsed_seconds": 69.0,
|
|
"judge_model": "claude-sonnet-4-20250514",
|
|
"index": "avap-docs-test-v4",
|
|
"category_filter": "all",
|
|
"scores": {
|
|
"faithfulness": 0.7141,
|
|
"answer_relevancy": 0.8949,
|
|
"context_recall": 0.755,
|
|
"context_precision": 0.6578
|
|
},
|
|
"global_score": 0.7554,
|
|
"verdict": "ACCEPTABLE",
|
|
"details": [
|
|
{
|
|
"id": "GD-001",
|
|
"category": "RETRIEVAL",
|
|
"question": "What is AVAP and what is it designed for?",
|
|
"answer_preview": "AVAP (Advanced Virtual API Programming) is a proprietary language that is specifically designed to facilitate the orchestration of secure, concurrent, and deterministic microservices and I/O operations. It is not intended as a general-purpose programming language but rather as a domain-specific lang",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-002",
|
|
"category": "RETRIEVAL",
|
|
"question": "How does AVAP handle conditional logic? What commands are used and how are blocks closed?",
|
|
"answer_preview": "AVAP handles conditional logic using two modes of invocation:\n\n1. **Modo 1 (Comparación Structurada):** `if(variable, value, comparison)`. This mode evaluates a comparison between the variable and value using an operator enclosed in double quotes (`\"==\"`, `\"!=\"`, etc.). The first two arguments must ",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-003",
|
|
"category": "CODE_GENERATION",
|
|
"question": "Write an AVAP script that reads a 'password' parameter, generates a SHA-256 hash of it, and returns the hash.",
|
|
"answer_preview": "```avap\naddParam(password, pass_nueva)\npass_antigua = \"password\"\nif(pass_nueva, pass_antigua, \"!=\")\n addVar(cambio, \"Contraseña actualizada\")\nend()\naddResult(cambio)\n\n// Generate SHA-256 hash of the password parameter\naddResult(encodeSHA256(password, \"hash\"))\n```",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-004",
|
|
"category": "CODE_GENERATION",
|
|
"question": "Show an AVAP script that loops from 1 to 5, builds a JSON object with each iteration index as a key, and returns it.",
|
|
"answer_preview": "```avap\nstartLoop(i, 1, 5)\n item = \"item_%s\" % i\n AddVariableToJSON(item, 'valor_generado', mi_json)\nendLoop()\nreturn(mi_json)\n```",
|
|
"n_chunks": 5
|
|
},
|
|
{
|
|
"id": "GD-005",
|
|
"category": "RETRIEVAL",
|
|
"question": "How does AVAP support external HTTP calls? What commands are available and how is timeout handled?",
|
|
"answer_preview": "AVAP supports external HTTP calls through the `RequestPost` and `RequestGet` commands. These commands allow for making POST and GET requests, respectively.\n\nTimeout handling in AVAP involves a parameter called `timeout`. If this parameter exceeds its set value, the destination variable receives `Non",
|
|
"n_chunks": 5
|
|
}
|
|
]
|
|
}
|
|
} |