Original tasks
We start with a decision a model struggles to make. A task specifies the goal, the constraints, and what counts as success. Varying those constraints lets us ask whether the model has learned a strategy it can use again.
AI TRAINING DATA RESEARCH
Beyond the
training horizon.
We design original reinforcement learning tasks and environments to study what models can discover when familiar strategies stop working.
Explore our researchWhy Fontal
Where does a new
capability begin?
Imagine a model halfway through a plan. A tool it was counting on goes offline. The model finds another way. What in its training taught it to do that?
Our name draws on fontal, an old word that evokes a spring or source. That question of origins shapes our research. We design tasks where a model can try a strategy, see what happens, and revise its next move. Then we change the conditions to discover which lessons travel with it.
Reinforcement learning
Tasks and environments.
How do you give a model room to explore, and enough feedback to learn from its choices?
Interactive environments
The environment determines what a model can observe and which tools it can use. Each action changes what happens next. Following those changes lets us study how a model plans ahead, interprets feedback, and recovers from a mistake.
Alongside our reinforcement learning work, we develop supervised fine-tuning and pretraining data for reasoning, planning, and tool use.
Environment explorer
Follow a trajectory.
Take the assistant’s next turn in three fictional workspaces. Inspect evidence, call a tool, or test a patch, then see what your choice changes.
User request
Did the new planning tasks help the model generalize?
A researcher has sent you a training log and an evaluation note. Decide how far the evidence supports their claim.
Success condition
Cite both records and limit your conclusion to the settings actually tested.
Experiment report
Report ATraining log
Training rollouts: 62 of 100 completed before training, 84 of 100 after training. Both runs used the same map pool.
Evaluation note
The evaluation rewrote the instructions but reused the training maps and movement constraints. On this set, completions went from 60 to 80 out of 100.
| Parcel | Status | Destination |
|---|---|---|
| Desk lamp | Packed | Home |
| Monitor arm | In transit | Home |
Customer: “The Annex office, please.”
Shipping policy: a packed parcel can be rerouted until carrier handoff. An in-transit parcel keeps its current destination.
| Customer name | Result |
|---|
- Action
- Read the researcher’s request.
- Observation
- Two records are available in this report.
- Final check
Sources cited: 0 of 2
Research method
When the
conditions change.
Solving a training task leaves a question open. Which part of the solution will still work when the tools, constraints, or starting state change?
Define the capability
Specify the decision the model should learn to make. For planning, that might mean recognizing when a promising first move leaves no way to reach the goal.
Inspect the reward
Compare the reward with the actual outcome. Could a model earn a high score while leaving the goal unmet? A shortcut like that points to a flaw in the task or its scoring rule.
Test what transfers
Hold the goal fixed and vary a condition the model did not encounter during training. Compare the decisions it makes in both settings to see where its strategy holds and where it breaks.
Research collaborations
What should a model
learn next?
We welcome joint work on original tasks, training environments, and data. Tell us about a capability you are investigating and a setting where your model gets stuck.