What Survives the Next Task? A Python Walkthrough of Agent Self-Improvement

A tool router sends a question about yesterday’s release to a calculator. It sees the failed tool call, tries search, and answers correctly. Now start a fresh task about another recent release. Does the router still reach for the calculator?

That fresh task is the first experiment in TensorDrill’s new Self-Improvement & RSI course. I maintain TensorDrill. We use the same toy router throughout the course so you can see what changes between tasks, from a single answer to the rules used to evaluate future updates. Open the L0–L4 course

Follow the state into the next task

The first three chapters track three different places where feedback can take effect.

Level	What changes in the router	What the next task uses
L0 · Output refinement	The router revises the current decision after a failed tool call.	The original router. The repair ends with the current task.
L1 · Parameter learning	Verified traces become training examples for a small router with trainable biases.	The updated parameters, if the training step is applied and retained.
L2 · Scaffold evolution	A fixed update procedure selects a routing prompt or skill to keep.	The new prompt or skill around the same model weights.

For L2, imagine a candidate prompt that says to use search for any question involving a date. It may fix release lookups and send a date arithmetic question to the wrong tool. Once the prompt persists, the change reaches more tasks than the one that inspired it.

A version can improve on average and still be a bad update

Suppose both prompts are evaluated on the same two development tasks. Higher scores are better. These scores are supplied toy data.

Development task	Current prompt	Candidate prompt	Change
Look up a release	0.500	0.875	+0.375
Calculate a date difference	0.750	0.500	−0.250
Mean	0.625	0.6875	+0.0625

The mean gain clears a minimum of 0.0625. The date arithmetic task drops by 0.25, twice the permitted drop of 0.125. The course’s promotion exercise checks both conditions before accepting a candidate. Promote a Version Without Forgetting Old Tasks

import math

def promote_scaffold(before, after, min_gain, max_drop):
    differences = [a - b for b, a in zip(before, after)]
    return (
        math.fsum(differences) / len(differences) >= min_gain
        and all(d >= -max_drop for d in differences)
    )

The function assumes nonempty, equally long score lists, aligned by task, and nonnegative thresholds. It compares the floats directly and includes equality at both boundaries. It takes O(n) time and O(n) auxiliary space.

before = [0.5, 0.75]
after = [0.875, 0.5]

assert not promote_scaffold(before, after, min_gain=0.0625, max_drop=0.125)
assert promote_scaffold(before, after, min_gain=0.0625, max_drop=0.25)
assert not promote_scaffold([0.5], [0.5], min_gain=0.125, max_drop=0)
assert promote_scaffold([0.5], [0.5], min_gain=0, max_drop=0)
assert before == [0.5, 0.75] and after == [0.875, 0.5]

The first assertion catches a rule that looks only at the mean. The second catches a strict comparison that rejects a candidate exactly at the limit. The third keeps the gain threshold meaningful even when every old task holds steady.

Keep the final audit separate

This gate makes a decision on the development tasks. If you adjust the prompt or its thresholds after seeing an audit result, that result has now informed development. Reserve fresh tasks for the next independent audit and include both the new behavior and the old capabilities you intend to retain.

The course adapts the L0–L4 taxonomy from Wang et al.’s survey preprint. GEPA supplies a research example of reflective prompt evolution. The two-threshold gate here is TensorDrill’s teaching component, with supplied scores and a fixed policy. Wang et al., Diving into Reliable Self-Evolving Agents · Agrawal et al., GEPA

The visual walkthrough lets you vary the allowed drop and see exactly which task blocks promotion. Open the visual editorial