mirror of
https://github.com/Canner/WrenAI.git
synced 2026-10-29 17:51:48 +08:00
Pao ShengandChih-Yu Yeh
946f727a4f
* add missing prompt builder * remove the traced component * todo comment to mention the eval process * run the example for the ragas evaluator * update the ignoring pattern for the spider dataset and output folder * produce the eval context * refactor the function to produce the eval context * check whether the evaluation context exists * test ragas evaluator * implement a collector to execute the evaluation process * cost, latency analysis * modify the structure and todo comment * modify the result structure * implement LLM as a Judge * implement a class to collect all result and summarize the evaluation * adjust minor details * support eval different pipeline via make * remove duplicated pattern in gitignore file * restructure the eval package * extract the functions and classes to the utils module * add streamlit app to present the result * adjust the structure * remove the testing suffix for evaluation context * update the makefile and fix the wrong parameter name in ask details pipeline * rename the init parameter to make them simplify * add the original response in post_processor * modify the report file name format * use post_process_result to evaluate the sql execution correctness * use entire CTE query to engine for validating the SQL * modify the output name --------- Co-authored-by: Chih-Yu Yeh <chihyu.jimmy.yeh@gmail.com>