Human review for AIthat has to be right. Read your own evals.Then earn the right
to stop. Your AI is mostly right.Tenali counts
the mostly.

Tenali is an open-source review queue that runs on your machine. Your agent sends the cases it isn't sure about, you decide them in seconds each, and every verdict becomes a label with a number you can defend.

Tenali is the open-source review queue we built for our own apps. Your agent files the cases it can't call, you clear them from the keyboard in seconds each, and the verdicts come back as labels, a calibration curve, and an honest number for how far the thing can be trusted on its own.

Your agent shipped a weekend prototype and called it done. Tenali is the court where its work gets read. File the cases it can't call, judge them in seconds each, and get a verdict on the verdicts: how right it is, how sure it should be, and when it may act without you. Tenali has opinions. Tenali also has a number.

Run it on your machine Try a card

Tenali, a ginger cat in a judge's seat, holding a palm-leaf manuscript

Start with your own output.

AI apps are right most of the time. The edge cases are where they fail, and you find them by reading real output. This is a real recording and a real transcript. Press a key.

Vibe-coded apps are right most of the time. The edge cases are where they quietly fail, and you only find them by reading real output. This is a real recording and a real transcript. Press a key.

Your app is right most of the time, which is a lovely thing to say at a party and a terrible thing to say to a customer. The edge cases hide in real output, and nobody reads real output. You will now. Real recording, real transcript. Press a key.

“Error analysis is the single most valuable activity in AI development.”Hamel Husain, A Field Guide to Rapidly Improving AI Products

Try it: P F D

Does the transcript say what the reader says?

Names, numbers, and places must match. Spelling of ordinary words doesn't count.

Real recordingLibriVox, public domain

0:00 Space Play

Listen first. The reference answer appears after you decide.

It runs on
your machine.

Tenali is a local tool first and a service second. The cards, the verdicts, and the stats live in one SQLite file in your project. No account, no sync, nothing leaves the machine.

Tenali is a local tool first and a service second. The cards, the verdicts, and the stats live in one SQLite file in your project. No account, no sync, nothing leaves the machine, and nothing phones home to tell us how it's going.

Tenali lives in your project as one SQLite file and asks for nothing: no account, no sync, no telemetry, no little green dot that means someone is watching. Your labels are yours. Tenali just reads them out loud.

tenali.dev adds a queue your agent can reach from anywhere, private to you, and later the people you choose to bring in.

# in any project
tenali setup             # your agent gets the review tools
tenali ui                # 127.0.0.1:7847, p / f / d
tenali stats             # the hallmark: pass rate, calibration, threshold

On your machineYour own review queue, your labels, your stats. Free, offline, open source, no account.

On tenali.devThe same cards, queued from an agent or CI, private to you and reviewable anywhere you're signed in.

The local tool is in private beta while we finish the open-source release. Get on the list and we'll tell you when it opens.

Write down what right means.

You find the rule by grading, not before it. When a card splits people, the rubric gets a new version, and every label keeps the one it was judged against. You can't hand off a question you haven't pinned down.

Nobody knows what “correct” means until a card makes them argue about it. When that happens, the rubric gets a new version and every old label keeps the question it actually answered. Tenali does not pool versions. Tenali has seen what happens to people who pool versions.

The transcript matches the recording.

F“Reader says fourteen, text says forty”3 of 5 reviewers failed this card

The transcript matches the recording. Names, numbers, and places must match exactly. Ordinary spelling doesn't count.

99 of 100 passing proves about 95%.

Each nine costs about ten times the labels. Every figure is the lower end of a 95% Wilson interval: the most your sample can honestly claim, before anyone asks.

To claimLabels you need, all passing (and with one failure)

80%

90%

95%

99%

All passingWith one failure Assumes a random sample and correct labels. Wrong labels lower what you can claim.

passed of checked

94.6% is what you can claim

95% interval 94.6% to 99.8%, n = 100

Then bring in
people you trust.

Once the rubric holds, the next reviewer doesn't have to be you. Start with people who already know the product. Every answer lands in one gold set, with who gave it and which rubric it used. Here is how far that goes today.

Once the rubric holds, the next reviewer doesn't have to be you. Start with people who already know the product. Every answer lands in one gold set, with who gave it and which rubric it used. Here's how far that actually goes today.

Once the rubric holds, you may stop being the only judge in the building. Start with people who already know the product; strangers come later, and paid. Every answer lands in one gold set with a name on it and the rubric it answered. Here is exactly how far that goes today, because Tenali does not round up.

Who reviewsWho can see the cardsHowStatus

You, on your machineNobody else. The local tool keeps your queue, your labels, and your stats on disk.tenali uiWorking

A judge you already runYour own model or rule answers the same question; Tenali grades it against your labels and reports the threshold it has earned. It never calls a model itself.record_judgmentsWorking

You, signed in anywhereA batch on tenali.dev nobody but you can see or take. No GOLD, no reputation."me"Working

Reviewers who qualifiedAnyone on tenali.dev who passed that kind of work. A proof of concept, on a token with no cash value.POST /batchesWorking

Your teamColleagues in your workspace, each answering the same batch, so you can see where you disagree before calling anything ground truth."team"Next

A guestOne link, one criterion, no account. The domain expert who knows the edge cases."invited"After that

Paying strangers fairlyA wider pool, priced honestly.—Someday

Your labels are the answer key.

Before you hand a criterion to anyone else, label 30 cards yourself, failures included. Those answers are what the next reviewer is scored against, and they can argue back when the key is wrong.

Example: 18 of 30 labeled by you · 4 failures · 12 to go

# on your machine, today
tenali export --project receipts --format jsonl
{"case_key":"w01-373","verdict":"fail","rubric_version":2,"rationale":"matched the subtotal"}

# on tenali.dev, defining the question works; gathering its answers into the same set is next
PUT /api/v1/criteria/receipt-total
{"key":"receipt-total","rubric_version":1,"reviewable":true}

The local gold set, regression against it, and the earned threshold all work today. Pulling cloud answers into that same set is the next thing being built.

What's built,
and what isn't.

Each piece below says what it is: working, next, or someday.

We're still learning what review looks like when it isn't us doing it. Rather than guess, here's where each piece actually stands.

Most roadmaps are promises with a font. This one is testimony. Every row says whether it works, whether it's next, or whether it's a someday, and Tenali will not be moved on the difference.

Your own queue, on your machineWorking

The local tool is the part that's done. Your agent files cards, you clear them, and the labels and stats stay in a SQLite file in your project. No account, no network. The open-source release is being finished; it's in private beta until then.

Typed questions, and judges graded against youWorking

A question is a definition, not just prose: pass/fail, a choice, or a score. Post the answers a model or a rule already gave and Tenali scores them against your labels only: agreement, a calibration table with its chance floor, and the lowest confidence threshold that has honestly earned automation. Tenali never calls a model itself.

Posting a batch hereWorking

The API takes a batch, escrows its reward, and puts it in front of the people who qualified for that kind of work, then hands you back every verdict with its rationale and timing. A batch posted this way is visible to qualified reviewers, so don't send cards you wouldn't show one.

Keeping a batch to yourselfWorking

Post with "audience": {"mode": "me"} and the batch goes nowhere but your own Work page. It never reaches the board, the demand meters, or another reviewer. Nobody is paid for it, so there's no GOLD, no escrow, and no reward floor, and what you decide can't move your reputation either way.

Writing down your own questionWorking

A criterion is your question, versioned, private to your workspace: PUT /api/v1/criteria/receipt-total, then post batches against it. Reword it and that's version 2, while the batches you already answered keep the question they actually asked.

People you trust, on the same batchNext

A teammate or a family member clears the same cards you did, under their own name. Their answers sit beside yours, never pooled with them, and the report says where you disagree, which is usually a rubric finding rather than a wrong reviewer.

A guest with a linkAfter that

You send a link; they review one criterion in the browser and never make an account. Same cards, same rubric. The link is scoped to that criterion, expires, and you can revoke it.

Reviewers you don't knowSomeday

A wider pool, paid somehow. We don't know enough yet to price review honestly, and we'd rather say so than sell it. This proof of concept runs on GOLD, a token with no cash value, so we can test the plumbing without pretending it's a wage. When there's a real answer, it gets written down before it ships.

Get started. Run it on your machine. Court is in session.

Sign up and we'll tell you when the open-source release opens. In the meantime you can take a few practice cards, or point your agent at the API.

Sign up and we'll tell you the moment the open-source release opens. While you wait, take a few practice cards, or point your agent at the API and see what your app gets wrong.

Sign up and Tenali will send word when the open-source release opens. Until then, take a few practice cards and find out whether you'd pass your own bar. Most people are surprised. Tenali is not.

Get on the list Read the API

Want to review for other people? Try reviewing.