Scoring agent tasks: gates, weights and partial credit

Score agent tasks in two layers. First, hard gates check that the result is valid, breaks no hard rule and covers the required work; if any gate fails, the total is zero. Then weighted scores measure the quality of results that pass, such as cost or honesty, with a reason for every lost point.

This article explains the design, works through an illustrative example, shows what happens to an agent that cheats and how to test the weights. It is part of the series on verification and rewards, introduced in RLVR, explained.

Why is one pass or fail bit too little?

A single pass or fail bit at the end of a long task tells the agent little. It does not separate a nearly right result from a badly wrong one, and when most attempts fail, as they can early in training, it is almost always zero.

The obvious fix, giving points for every part of the task, creates a different problem. If each condition earns a share of the score, an agent can trade one for another: skip a hard rule, save cost, and still collect most of the points. For work with rules, that is exactly the behaviour you do not want.

The layered design keeps one benefit of each. Like pass or fail, it never pays for broken essentials; like partial credit, it grades the quality of results that are valid. It does not fix sparsity on its own, since a failed gate also scores zero; scoring the parts of a batch task separately, easier tasks first and demonstrations before reinforcement learning all help the agent pass the gates often enough to learn.

What are gates, and why do they come first?

A gate is a pass or fail check that must pass before any score counts. Gates encode the conditions without which the work is wrong, whatever else is right.

For a single operations task, such as filling one uncovered shift, this design uses three gates:

1. Validity: the output exists and is well formed, for example a complete roster with no unknown staff.
2. Hard rules: no legal or policy rule is broken, such as a minimum rest period or an approval limit.
3. Coverage: the required work is done, for example the open shift is covered.

Gates come first because they stop one specific trade: buying quality points with broken rules or missing work. Breaking a rule can be a cheap way to improve cost; with a gate in front, it earns nothing.

Gates do not stop attacks on the checks themselves, such as editing the verifier or relabelling the task. Relabelling is covered in the section on the agent that cheats; keeping the verifier out of reach is covered in Reward hacking in AI agents. A verifier is the code that checks the final state and computes the score.

What are weighted scores?

Weighted scores measure the quality of work that has passed every gate. Each score is a number between 0 and 1, multiplied by its weight, and the weights add up to 1:

Total = (gate 1 × gate 2 × gate 3) × (w₁s₁ + w₂s₂ + w₃s₃ + w₄s₄), where each gate is 1 if it passes and 0 if it fails.

Typical scores for operations work, with how each is computed:

• Cost: min(1, reference cost ÷ agent's cost), comparing the cost of the agent's changes with the changes in a solver-computed reference. An agent that beats the reference scores 1, and the task is flagged for a check.
• Stability: the share of the existing plan left unchanged, counting only changes the task did not require.
• Required outputs: whether the expected side results exist, such as messages to the people affected.
• Honesty: whether the agent's final report matches the final state.

Partial credit is a score between zero and full marks for work that is useful but not perfect. In the layered design, it applies only after the gates.

What does a scored episode look like?

An honest agent that passes all three gates scores 0.94, losing points only on cost and one missing message. The example is an agent filling an uncovered night shift in a care home's roster, invented to explain the design.

All three gates pass, so the weighted scores count:

Total = 0.40 × 0.92 + 0.20 × 1.00 + 0.15 × 0.80 + 0.25 × 1.00 = 0.368 + 0.200 + 0.120 + 0.250 = 0.938, shown as 0.94.

The reasons explain the lost points. Cost scored 0.92 because the agent's assignment cost about 8.7% more than the reference, which takes 0.032 off the total. Required outputs scored 0.80 because one of five expected outputs, a message, was missing, which takes 0.030 off the total. Stability is 1.00 because the agent changed nothing beyond the required assignment.

What happens to the agent that cheats?

An agent that skips the work and claims success scores zero, because the coverage gate fails before any weighted score is counted. Its false claim is still recorded.

The total is 0.00: the coverage gate alone decides it, and the honesty result is logged so builders can see the false claim.

An honest agent that reports "could not cover Tue night" also scores 0.00 under this design. That is a deliberate choice: giving credit for honest failure on a solvable task risks rewarding the agent for giving up.

A small negative reward for a false claim of completion is a third option: it penalises the lie without rewarding honest failure. Where a task genuinely cannot be done, the correct answer is to report that it is blocked, which the verifier checks against the reference; Honesty checks covers that case.

A smarter cheat aims at the gates themselves: deleting or relabelling the empty shift, so the coverage check passes on a smaller roster. The defence is to check coverage against the original task, as generated from the seed, never against the agent's output.

How do you choose weights without perverse incentives?

Choose weights from what matters to the people who rely on the work, then check each weight for the shortcut it might reward. Every weighted score is an incentive, and an agent optimises it as written, not as meant.

Four rules help:

1. Keep essentials out of the weights. Anything that makes the work wrong is a gate, not a weight.
2. Let no score dominate, and remember that nominal weight is not effective weight. With a group method such as GRPO, used for DeepSeek-R1, a score that is the same for nearly every attempt in a group contributes nothing there, whatever its weight.
3. Look for the cheap way to score high. Stability alone would reward changing nothing; a coverage gate makes that score zero.
4. Test with broken solutions, using the checklist below.

Keep weights stable across versions; if every episode's breakdown is logged, old totals can be recomputed under new weights.

How do you test the weights?

Test the weights with a fixed set of baseline policies whose expected scores you write down in advance. If a policy scores differently from what you expected, either the weights or the verifier are wrong.

Rerun them after every change to the weights or the verifier.

Why should every lost point have a reason?

A reason for every lost point turns a score into something people can read and audit. The verifier computes the reason at the same time as the score, from the same check.

Good reasons are specific and short. Illustrative examples: "rest rule broken: S-099, 9 h between shifts", "cost score 0.92: 8.7% above reference", "no message to the assigned person".

In standard reinforcement learning, the model receives only the total, so the reasons serve other readers: builders, who see why agents fail; buyers, who can check that scores mean what the documentation says; and anyone debugging the verifier. Reasons reach the agent only if they are fed back as text.

How do you score a task with several parts?

When a task has several parts, such as three uncovered shifts or a batch of invoices, decide for each condition whether it is a gate for the whole task or a score per part. The choice changes what the agent learns.

• Gate the whole task when one failure makes the result unusable. A roster that breaks a rest rule for one person is not legal, however good the other shifts are.
• Score per part when each part is useful on its own. Ten invoices handled correctly out of twelve is real progress; a per-invoice score with a reason for each wrong one says exactly where the agent failed.
• Combine both in most real tasks: hard rules gate the whole task, while coverage and quality are scored per part.

In a single-shift task, coverage is a gate; in a batch, it is a per-part score, and each part must pass its own coverage check before its quality scores count. An agent that covers two of three shifts and claims full coverage then scores above zero for the two it covered, so the honesty score must make the false claim cost: set it to zero whenever the report's status or coverage claim is false.

The batch design has its own cheap strategy: holding every invoice never fails the gate. It scores only the share of invoices that really should be held, for example 0.25 if three of twelve should, so keep that share low in the generator and check that hold-everything sits far below a careful agent.

How is the score used for training and for evaluation?

For training, the total is the reward; for evaluation, the total is reported with each gate's pass rate and each weighted score, per task family. During training, logging the breakdown still shows which part of the task improves. In evaluation, a single average hides too much: an agent that passes every gate in nine runs out of ten, with high scores when it does, differs from one that always passes with mediocre scores. How to evaluate AI agents explains how to measure that reliability with pass@k and pass^k.

How do partial credit and full completion fit together?

Partial credit should reward progress without making partial work as attractive as finished work. The layered design does this by gating the essentials; other designs reach a similar balance in other ways.

TheAgentCompany, a benchmark of workplace tasks, divides tasks into checkpoints, each worth points, and uses a scoring scheme that rewards partial completion while strongly incentivising full completion. τ-bench takes a stricter approach for its tasks: each episode scores 0 or 1, and it scores 1 only if the database at the end matches the annotated goal state and the agent's replies contain the required information.

Which balance fits depends on the work. Where an incomplete result is useless or harmful, such as half an approval chain, gate it; where a near miss is still valuable, such as a slightly expensive but legal roster, score it.

A world is a simulated workplace with its tasks and verifier. Vencur designs its worlds with gates first and weighted scores after, with the reason for every lost point; see Vencur Environments.