Revalvo
Run prompts on every model at once. Score. Version. Ship.
78 followers
Run prompts on every model at once. Score. Version. Ship.
78 followers
Revalvo is a local-first workbench for prompt engineering and LLM evaluation. Run the same prompt against every model in parallel, score responses with 40 built-in evaluators, version prompts like code, and batch-test on datasets — before anything hits production. No account, no hosted database: your API keys stay in your browser.
Revalvo
hey, classic question: how is it different than langsmith evals or weights and biases evals?
Revalvo
@paul_from_dentro Hi Paul, LangSmith & W&B are great for hosted tracing + team evals in production. Revalvo is a local-first prompt workbench: multi-model playground, prompt versioning/diffs, batch eval (40 evaluators, mostly rules + some judges), BYOK, no account, Ollama offline. We’re optimized for fast iteration before you wire up a full observability stack — not replacing enterprise eval infra.
40 evaluators is the number I'd push on. Most eval suites I've used come down to another model grading the output, so the eval inherits the same failure mode as the thing it's grading. For each of those 40 I'd want to know upfront whether it's deterministic or a judge model, because I trust those two very differently. BYOK with no markup on API spend is the right call though.
Revalvo
@asadmalik901 Really fair point — and it's why we don't treat "40 evaluators" as "40 LLM judges."
In Revalvo today it's roughly ~25 rule-based checks (exact match, regex, JSON schema, length, PII patterns, etc. — deterministic, no extra API spend) and ~14 that need a model (LLM judge, rubric, faithfulness/hallucination-style scorers, plus embedding similarity). In the Evaluators workspace each one is labeled Rule-based, LLM judge, or Code before you attach it to a dataset.
Our bias for batch eval is the same as yours: stack deterministic gates first (cheap, stable, CI-friendly), then use judges only where rules can't express the rubric. Judges are powerful but you're right that they inherit the grader's failure modes — we don't pretend otherwise.
BYOK / no markup is non-negotiable for us too. If you try it, I'd love to know whether the kind labels are visible enough on first run or if we should surface "deterministic vs judge" even earlier in the dataset flow.
Thanks for the thoughtful comment — this is exactly the feedback we were hoping for on PH.