- Created `n00 Beir Analysis.ipynb` for analyzing BEIR dataset with Ollama embeddings.
- Added `n00 Beir Analysis_cosqa.ipynb` for evaluating the CosQA dataset using similar embedding techniques.
- Introduced `n00 first Analysis.ipynb` for initial analysis with Ragas embeddings and semantic similarity evaluation.
- Implemented data loading and processing for each notebook, including downloading datasets and saving results.
- Included evaluation metrics such as NDCG, MAP, Recall, and Precision for model performance assessment.
- Changed Elasticsearch index from "avap-docs-test-v3" to "avap-docs-test-v4" in elasticsearch_ingestion.py.
- Removed unused import SystemMessage from langchain_core.messages in translate_mbpp.py.
- Added import for Lark in chunk.py to support new functionality.
- Changed Elasticsearch index from "avap-docs-test-v3" to "avap-docs-test-v4" in elasticsearch_ingestion.py.
- Added Lark parser for AVAP code processing in chunk.py.
- Enhanced metadata extraction for processed documents, including AST for AVAP files.
- Improved error handling for AVAP code parsing.
- Implemented parser for executing AVAP files within a Docker container (parser v1.py).
- Created a script to send AVAP code to a local server and handle responses (parser v2.py).
- Introduced a mock MBAP test harness to validate AVAP code against expected outputs (mbap_tester.py).
- Added transformation logic to convert AVAP code into Python-like syntax for testing purposes.
- Enhanced error handling and output formatting in the testing harness.
- Implemented `parser v1.py` to run AVAP files in a Docker container using subprocess.
- Created `parser v2.py` to send AVAP code to a local server and handle JSON responses.
- Introduced `mbap_tester.py` as a heuristic mock executor for testing AVAP code against predefined test cases.
- Added functions for transforming AVAP code to Python and executing it in a controlled environment.
- Included error handling and summary reporting for test results in `mbap_tester.py`.
- Added `export_documents` function to save processed documents to JSON.
- Extended `ElasticHandshake` to include chunk metadata during ingestion.
- Updated `process_documents` to include extra metadata for each chunk.
- Modified `ingest_documents` to return Elasticsearch responses for further processing.
- Adjusted `elasticsearch_ingestion` command to accept output path for exported JSON.
- Added output path parameter to elasticsearch_ingestion command for exporting processed documents.
- Implemented ElasticHandshakeWithMetadata class to preserve chunk metadata during ingestion.
- Updated process_documents function to include extra metadata for each chunk.
- Modified ingest_documents function to return Elasticsearch response for each chunk.
- Introduced export_documents function to save processed documents as JSON files.
acano
pseco
rafa-ruiz