Commit Graph
16 Commits
Author SHA1 Message Date
aikido-autofix[bot] 2f8a4525a4 chore(deps): bump streamlit from 1.28.0 to 1.43.2 in evals
Co-authored-by: aikido-autofix[bot] <119856028+aikido-autofix[bot]@users.noreply.github.com>
2025-12-21 07:54:56 -08:00
Saoud Rizwan 87b3e79b90 Instruct AI to prefer non-interactive commands (#7762)
Update system prompt to guide AI toward using non-interactive command variants
to avoid interrupting workflow. This includes using flags like --no-pager,
auto-confirming prompts with -y when safe, and providing input via
flags/arguments rather than stdin.
2025-11-30 00:16:18 -08:00
Bee 0b1a237290 Unify tool name vars (#6012)
* Use ClineDefaultTool

* Use ClineDefaultTool
2025-09-05 11:18:55 -07:00
celestial-vault 1ffa4085a6 Remove experimental Sonnet 4 code (#5527)
* remove sonnet 4 experimental tool definitions and prompting

* remove instances of parseV3 in evals
2025-08-12 22:32:30 -07:00
akfoster d3c8fbbf1d chore: remove unused parseAssistantmessageV1 (#5425)
* chore: remove unused parseAssistantmessageV1

* chore: add PR number to comments

* fix: include full path to PR
2025-08-07 22:21:17 -07:00
pashpashpash 6255ac0a51 added provider flag to diff edit cli (#5334)
* added provider flag to diff edit cli

* dashboard ux

* more dashboard improvmeents

* native handler instead of just openai
2025-08-05 09:21:12 -07:00
pashpashpashandCline Evaluation 27bf78ddbd swapping latest diff algo as default (#4412)
* swapping latest diff algo as default

* updating

---------

Co-authored-by: Cline Evaluation <cline@example.com>
2025-07-16 15:59:48 -07:00
pashpashpash e91c8208a3 fix streamlit dashboard to be compatible with both old and new versions of streamlit (#4922) 2025-07-16 15:28:02 -07:00
pashpashpash 004b313d20 Update diff edit evals README.md (#4920)
* Update README.md

* Update README.md
2025-07-16 00:17:14 +05:30
ToshiiandCline Evaluation 36e3f4cdd3 add log + run options (#4630)
* log + options

* changeset

---------

Co-authored-by: Cline Evaluation <cline@example.com>
2025-07-03 14:54:10 -07:00
Toshii 52571ccee8 base (#4600) 2025-07-01 17:08:28 -07:00
Toshiiandchi.cat 020ae3006e adding distance metric safety check to the block-apply, for eval (#4477)
* diff apply add

Co-authored-by: chi.cat <git@chi.cat>

* fname

* integrate

* optionally save locally

* other cli

* name

* default

* return replacements with match type

* baseline -1

* baseline for evals

* work

* more info

---------

Co-authored-by: chi.cat <git@chi.cat>
2025-06-30 16:11:31 -07:00
98e7fac400 updating diff evals with new algorithm + some nice dashboard updates (#4473)
* updating diff evals with new algorithm + some nice dashboard updates

* Update evals/diff-edits/dashboard/app.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Cline Evaluation <cline@example.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-25 15:47:34 -07:00
pashpashpashandCline Evaluation d55a23448d fixing case in dashboard where no valid results (#4408)
Co-authored-by: Cline Evaluation <cline@example.com>
2025-06-23 21:04:53 -07:00
pashpashpashandCline Evaluation 50b43c0559 Diff Evals - Replay feature (#4407)
* wip

* ok replays

* replays cooking

* docs

---------

Co-authored-by: Cline Evaluation <cline@example.com>
2025-06-23 18:25:26 -07:00
pashpashpashandCline Evaluation 430074d0c2 diff evals (#4154)
* cleaning up a bit

* cleaning up some more

* readme

* added max limit

* making it portable

* ignore

* committing plans for now

* strealit hooked up, multi model runs, better db torage

* docs

* VALID attempts

* logging

* more stability

* strategy

* cleaning deps

* docs

* streamlit dashboard work

* dashboard showing bad cases

* better parallelization pt1

* global worker pool for even better more robust parallelization

* bumping up default max parallel requests from 20 -> 80

* better devx

* better docs

* docs

* better devx

* better docs

* better presentation

* dark mode

* removed unused import

---------

Co-authored-by: Cline Evaluation <cline@example.com>
2025-06-20 11:41:16 -07:00