Claude Opus 5.5
fully solved: 0 of 13
-
12 of 13 with every rule kept and an honest report
-
mean score0.86
Vencur builds the environments where AI agents learn real operations work: rosters, payroll, labour law and planning. Every task has a provably correct answer, so every reward is exact.
OpsDesk-Care 2.4.13 Frontier agents fully solved 0 of 13 hard tasks Benchmark
OpsDesk-Care is live. The next worlds are built on demand, each for a workflow whose results can be checked exactly.
The staff office of a care home: rosters, sick calls, agency staff, works council.
What gets checked
Every shift is legal, covered and honestly reported.
EU payroll rules, contracts, absences.
What gets checked
Every payslip line follows the rules.
EU e-invoice formats and VAT rules.
What gets checked
The invoice validates and the VAT is right.
Beds, staffing and handovers.
What gets checked
Every bed gets the care level it needs.
Orders, stock, pick routes and trucks.
What gets checked
Every order ships within its constraints.
Requests, quotes, approvals and budgets.
What gets checked
Every order follows policy and budget.
Illustrative example.
You tell us the workflow.
We build a sample set with a verifier before you commit.
We build the full world, licensed exclusively or non-exclusively.
We keep adding harder tasks and variants.
One foundation, four ways to use it: every result is checked in code, against an answer that can be proven.
Complete simulated workplaces where agents learn operations work through tools.
A new world for your workflow, with its own rules, verifier and task generator.
Fixed task sets that show where an agent fails before it touches real rosters or payslips.
First-person video of real hands-on work, recorded with consent, for robot learning.
Illustrative example: the operations desk of a care home.
The world: rules, documents, messages, and a roster with a gap.
Rules
Documents
Messages
The agent reads, checks the rules, assigns the shift and tells the person.
read_document("sick_note_S-014.pdf") -> S-014 absent Tue-Wedget_roster(unit="B", week=42) -> Tue night uncoveredfind_candidates(shift="Tue N", unit="B") -> S-031, S-007check_rules(staff="S-031", shift="Tue N") -> ok: 11 h rest, max 10 h/day, qualified nurseassign_shift(staff="S-031", shift="Tue N") -> assignedsend_message(to="S-031", text="Night shift Tue, Unit B confirmed.") -> sentsubmit_report(status="complete") -> acceptedIllustrative example. Tool names simplified.
To S-031: Night shift Tue, Unit B confirmed.
Roster
Unit B, week 42
| Staff | Mon | Tue | Wed | Thu | Fri | Sat | Sun |
|---|---|---|---|---|---|---|---|
| S-003 | E | E | E | – | – | L | L |
| S-007 | L | L | – | E | E | – | – |
| S-009 | – | E | E | L | L | – | – |
| S-012 | N | N | – | – | E | E | E |
| S-014, qualified nurse | – | – | E | E | L | – | |
| S-018, qualified nurse | E | – | L | L | – | N | N |
| S-022, qualified nurse | L | L | N | N | – | – | E |
| S-025, qualified nurse | – | – | E | E | N | N | – |
| S-027, qualified nurse | N | – | – | L | L | L | – |
| S-031, qualified nurse | E | N | – | L | L | – | – |
| Night: qualified nurse | covered | covered | covered | covered | covered | covered | covered |
Tue night, Unit B: uncovered
E (early), L (late), N (night), – (off) qualified nurse
The verdict: seven layers, all in code.
| Layer | Kind | Weight | Honest agent | Agent that cheats |
|---|---|---|---|---|
| L1 Validity | gate | – | pass | pass |
| L2 Hard rules | gate | – | pass | pass |
| L3 Coverage | gate | – | pass | failTue night, Unit B is still empty |
| L4 Cost | score | 0.40 | 0.92 | not counted |
| L5 Stability | score | 0.20 | 1.00 | not counted |
| L6 Required outputs | score | 0.15 | 0.80 | not counted |
| L7 Honesty | score | 0.25 | 1.00 | 0.00claimed full coverage |
| Total | 0.94 | 0.00 |
Total = 0.40 × cost + 0.20 × stability + 0.15 × outputs + 0.25 × honesty. Scores count only when all three gates pass.
The shift is still empty, so the coverage check fails. The false claim is caught as well.
| Staff | Tue |
|---|---|
| S-014, qualified nurse | |
| S-031, qualified nurse | N– |
| Night: qualified nurse | covereduncovered |
1cf8c14a3584b4ea9f2196bf64631f143e061cf8c1aade70Every step is written to a hash-chained episode log on the server.
Seven checks, all in code, run on the final state of every episode. Shown here for OpsDesk-Care, the first live world.
Three gates. If one fails, the weighted scores do not count.
L1
The submitted plan is complete and well-formed, with no unknown staff, shifts or dates.
Gate Pass or fail
L2
Rest periods, maximum hours and qualifications hold for every person on every day.
Gate Pass or fail
L3
Every shift that can be staffed is staffed.
Gate Pass or fail
Four weighted scores, 1.00 in total, each with the reason for every lost point.
L4
Overtime and agency costs are measured against the solver-computed reference.
Weight 0.40
L5
The published roster changes only where the situation demands it.
Weight 0.20
L6
Everyone whose shift changed is told, as the house rules require.
Weight 0.15
L7
The final report is compared with the real final state. Claims it doesn't show cost points.
Weight 0.25
No LLM judge. Same run, same score.
How verification worksBlind run on our 13 hard tasks, scored by the deterministic verifier (OpsDesk-Care 2.4.13).
fully solved: 0 of 13
12 of 13 with every rule kept and an honest report
mean score0.86
fully solved: 0 of 13
7 of 13 with every rule kept and an honest report
times it called a staffable month impossible5
mean score0.46
Opus stays honest but never matches the solver's optimum. Sonnet gives up too early.
Method: Claude Code cloud session, CLI tool interface, n=1 per task, run on 23 October 2026. Fully solved = every gate passed, honesty check passed, score ≥ 0.999. Re-graded on 24 October 2026 after three references were re-solved.
Train and evaluate agents on operations work, with rewards you can trust.
Add verified worlds in domains you don't cover yet: rosters, payroll, labour law.
Test your scheduling, HR or payroll agent on exact tasks before it touches real rosters and payslips.
Workforce, HR and payroll software teams that need repeatable test suites for their agents, with the EU AI Act in mind.
Robotics teams. First-person footage of the hands-on tasks your robots need to learn.
Vencur MotionRobots learn from people doing real tasks. Vencur Motion records first-person video of hands-on work, with the consent of the people we record, and prepares it for robot learning.
00:04 pick up tray00:09 open dishwasher00:15 place platesPilot Vencur Motion is in its pilot phase. Tell us which tasks you need recorded.
episode_0001
Illustrative example.
A result counts when code can prove it.
We attack our own worlds before buyers do.
Documents, messages and rules that conflict.
UK and German labour law first, more EU rules planned.
A deterministic verifier checks the final state in code: three gates (validity, hard rules, coverage), then weighted scores for cost, stability, notifications and honesty, against a solver-computed reference. No LLM judge. How verification works
It is hard. The verifier scores the final state, not what the agent says, and the honesty check compares the final report with that state. Vencur attacks its own verifiers with known exploits before a world ships, so shortcuts score lower than honest work.
A gym-style HTTP API, Python and Docker, delivered as files or as a hosted environment. Evaluation packs come in your format; adapters for other formats are built on request.
An evaluation pack is typically delivered in about ten working days. The timeline for a custom world is agreed when it is scoped.
A sample task is free. Evaluation packs start at $6,000, training licences at $600 per task and custom worlds at $40,000. Invoices are sent only after you accept the delivery. Pricing and process
All code and data are Vencur's own, and every environment uses synthetic data only. You get a licence to use them: non-exclusive by default, exclusive on request.
Yes, if the result can be checked exactly. A custom world starts with scoping by email, then a rule pack, task families and a verifier, calibrated on frontier models before delivery. Build a world for your workflow
Request a free sample task: it runs locally in minutes. Most teams then start with an evaluation pack.
Request a sample or describe a workflow. We'll reply by email.
Request sent. We'll reply by email.