- Created a new JSON file containing evaluation results for the AVAP knowledge models, including scores for faithfulness, answer relevancy, context recall, and context precision.
- Updated the evaluation notebook to use a new embedding model and fixed execution counts for code cells.
- Created `n00 Beir Analysis_cosqa.ipynb` for analyzing CoSQA dataset with BEIR.
- Created `n00 first Analysis.ipynb` for initial analysis with embeddings.
- Implemented `evaluate_embeddings_pipeline.py` to evaluate embedding models across CodexGlue, CoSQA, and SciFact benchmarks.
- Added adapters for Ollama and HuggingFace embeddings to ensure compatibility with BEIR.
- Enhanced error handling and data normalization in embedding processes.
- Included functionality to load datasets from local cache or download if not present.
- Introduced a new notebook for generating synthetic datasets for AVAP, including loading AVAP and MBPP data, and creating prompts for LLM interactions.
- Implemented parser for executing AVAP files within a Docker container (parser v1.py).
- Created a script to send AVAP code to a local server and handle responses (parser v2.py).
- Introduced a mock MBAP test harness to validate AVAP code against expected outputs (mbap_tester.py).
- Added transformation logic to convert AVAP code into Python-like syntax for testing purposes.
- Enhanced error handling and output formatting in the testing harness.
- Implemented code to utilize OllamaEmbeddings for embedding documents.
- Included example usage with sample text inputs.
- Demonstrated response handling from the Ollama LLM.
- Noted deprecation warning for the Ollama class in LangChain.
- Updated `read_files` function to return a list of dictionaries containing 'content' and 'title' keys.
- Added logic to handle concatenation of file contents and improved handling of file prefixes.
- Introduced `get_chunk_docs` function to chunk document contents using `SemanticChunker`.
- Added `convert_chunks_to_document` function to convert chunked content into `Document` objects.
- Integrated logging for chunking process.
- Updated dependencies in `uv.lock` to include `chonkie` and other related packages.
- Created a new Jupyter notebook for analyzing BEIR dataset with CosQA using Ollama embeddings.
- Implemented a custom embedding class to integrate LangChain's OllamaEmbeddings with BEIR.
- Added data loading and evaluation logic for the CosQA dataset.
- Updated `uv.lock` to remove unnecessary dependencies (`mteb` and `polars`) and incremented revision number.
pseco
acano