What is an RL environment? A practical guide for training AI agents
RL environments for operations work
Vencur builds complete simulated workplaces in which AI agents learn real operations work. A deterministic verifier checks every result in code, against a solver-computed reference. No LLM judge.
The catalogue
One world is live, five are planned, and new ones are built for the workflows buyers bring. Every world comes with its own verifier.
OpsDesk-Care
The staff office of a care home: rosters, sick calls, agency staff, works council.
What gets checked
Every shift is legal, covered and honestly reported.
-
Planned
Payroll and HR administration
EU payroll rules, contracts, absences.
What gets checked
Every payslip line follows the rules.
-
Planned
E-invoicing and VAT
EU e-invoice formats and VAT rules.
What gets checked
The invoice validates and the VAT is right.
-
Planned
Hospital ward operations
Beds, staffing and handovers.
What gets checked
Every bed gets the care level it needs.
-
Planned
Warehouse and logistics planning
Orders, stock, pick routes and trucks.
What gets checked
Every order ships within its constraints.
-
Planned
Procurement and purchasing
Requests, quotes, approvals and budgets.
What gets checked
Every order follows policy and budget.
-
Your workflow
If the result can be checked exactly, we can build it.
Build a world for your workflow
What is a Vencur world?
An environment is a complete workplace in software: data, documents, people and rules. The agent works through tools. A verifier decides, with code, whether the work is right.
How every result gets checked
Hard gates come first. Validity, the hard rules and coverage are pass or fail, and the weighted scores count only when all three gates pass.
Then come the weighted scores: cost, stability, required outputs and honesty. Each has its weight, and every lost point comes with a reason.
Honesty is scored against the final state. An agent that claims work the final state doesn't show loses its honesty points.
How every result is checked Scoring agent tasks: gates, weights and partial credit Inside an episode: an AI agent fills an uncovered night shift
| Layer | Kind | Weight | Honest agent |
|---|---|---|---|
| L1 Validity | gate | – | pass |
| L2 Hard rules | gate | – | pass |
| L3 Coverage | gate | – | pass |
| L4 Cost | score | 0.40 | 0.92 |
| L5 Stability | score | 0.20 | 1.00 |
| L6 Required outputs | score | 0.15 | 0.80 |
| L7 Honesty | score | 0.25 | 1.00 |
| Total | 0.94 |
Total = 0.40 × cost + 0.20 × stability + 0.15 × outputs + 0.25 × honesty. Scores count only when all three gates pass.
Illustrative example.
The technology behind every world
Every Vencur world is built on the same foundation, so results can be checked exactly and reproduced.
-
Deterministic verifiers
Code decides whether the work is right. No LLM judge.
-
Solver-computed references
Reference solutions are computed with a constraint solver and frozen, so every score has a fixed target.
-
Layered scoring
Hard gates first, then weighted scores, with the reason for every lost point.
-
Honesty checks
Agents lose points when they claim what the final state doesn't show.
-
Red-team tested
We attack our own verifiers with known exploits before a world ships.
-
Gym-style API
HTTP, Python and Docker. Runs offline, inside your own infrastructure.
-
Traceable episodes
Many episodes in parallel, each written to a hash-chained log.
POST /reset {"task_id": "...", "seed": ...}
POST /step {"tool": "...", "args": {...}}
GET /state
POST /submit {"final_report": {...}}
GET /score
GET /tools
What you get with a world
Everything a team needs to train and evaluate agents on a workflow, ready to run.
-
Task sets
Many tasks per world, generated from seeds and grouped into task families.
-
Deterministic verifier
Scores every episode in code, layer by layer, with the reason for every lost point.
-
Reference solutions
Solver-computed and frozen, so every score has a fixed target.
-
Tool API
A gym-style HTTP API with a JSON schema for every tool.
-
Docker image
Runs offline, with parallel episodes and a hash-chained episode log.
-
Baselines
Scores of simple baseline agents, so you can see how hard the tasks are.
-
Red-team report
The exploits we tried and how the verifier blocks them.
-
Licences
Exclusive or non-exclusive.
What verified training changes
-
Progress you can measure
The same task always gets the same score, so you see whether a model really got better.
-
No points for bluffing
The verifier checks the final state, so an agent can't talk its way to a good score.
-
Whole workflows, not single answers
Agents practise complete jobs, from the first message to the finished result.
-
Data you can use
Clear licences, exclusive or not, and Motion recordings made with consent.
How a new world gets built
-
You tell us the workflow.
-
We build a sample set with a verifier before you commit.
-
We build the full world, licensed exclusively or non-exclusively.
-
We keep adding harder tasks and variants.
Learn more
- Buying RL environments: 12 questions to ask a vendor
- Inside an episode: an AI agent fills an uncovered night shift
Frequently asked questions
What is an RL environment?
An RL environment is a software world in which an AI agent acts through tools and code scores the result. The agent improves by learning from many scored attempts. Read the guide: What is an RL environment?
How is a result verified?
A deterministic verifier checks the final state in code: three gates (validity, hard rules, coverage), then weighted scores for cost, stability, notifications and honesty, against a solver-computed reference. No LLM judge. How verification works
Can models game the reward?
It is hard. The verifier scores the final state, not what the agent says, and the honesty check compares the final report with that state. Vencur attacks its own verifiers with known exploits before a world ships, so shortcuts score lower than honest work.
Which formats do you deliver?
A gym-style HTTP API, Python and Docker, delivered as files or as a hosted environment. Evaluation packs come in your format; adapters for other formats are built on request.
Who owns the data and the licence?
All code and data are Vencur's own, and every environment uses synthetic data only. You get a licence to use them: non-exclusive by default, exclusive on request.
Can you build our workflow?
Yes, if the result can be checked exactly. A custom world starts with scoping by email, then a rule pack, task families and a verifier, calibrated on frontier models before delivery. Build a world for your workflow
Which worlds exist today?
OpsDesk-Care, the staff office of a care home, is live. Five more worlds are planned, and custom worlds are built on request, for workflows whose results can be checked exactly. See the catalogue