Those choices optimize different objectives. The difference becomes obvious when a model assigns A a very large score advantage: a hard winner encourages certainty, while a soft target can penalize it.

While adding Constitutional AI exercises to TensorDrill, the AI coding practice site I maintain, I separated this into two small components: transform supplied judge scores, then compute a preference loss. Neither component needs a model API or a GPU.

First, identify the stage you are implementing

The Constitutional AI paper separates critique-and-revision training from learning with AI-generated preferences. This article focuses on transforming supplied preference judgments, rather than generating critiques or training a complete aligned assistant.

For this exercise, the judge provides natural log probabilities for two answer options from the same prompt. These are not the preference model's response scores. We normalize the two option probabilities to obtain a target for that later model.

Normalize probabilities, not negative log values

For two log probabilities a and b, the normalized probability of option A is:

target_A = exp(a) / (exp(a) + exp(b))

Dividing a by a+b is a different operation. Exponentiating both values directly also fails for inputs such as -1000 and -1001: both exponentials underflow to zero.

Subtracting the maximum solves the numerical problem:

import math

def preference_label(logp_a, logp_b, clamp=False):
    peak = max(logp_a, logp_b)
    a = math.exp(logp_a - peak)
    b = math.exp(logp_b - peak)
    p_a = a / (a + b)
    if clamp:
        p_a = min(0.6, max(0.4, p_a))
    return p_a, 1.0 - p_a

assert preference_label(-2, -2) == (0.5, 0.5)
p, q = preference_label(-1000, -1001)
assert abs(p - 0.7310585786300049) < 1e-12
assert abs(p + q - 1.0) < 1e-12
assert preference_label(0, -1000, True) == (0.6, 0.4)
assert preference_label(-1000, 0, True) == (0.4, 0.6)

The inputs here are finite natural log probabilities at most zero. The optional 0.4–0.6 clamp illustrates the paper's main chain-of-thought feedback setting. It is not a general instruction to clamp every preference dataset, and it does not establish that a judge is calibrated.

Always construct the second probability after clamping the first. Otherwise the pair can stop summing to one.

A target is not a predicted probability

Now suppose a preference model assigns response scores s_a and s_b. Their difference determines the predicted preference:

d = s_a - s_b
prediction_A = sigmoid(d)

The judge's normalized probability is the target, not that prediction. Pairwise binary cross-entropy compares the two:

loss = -target * log(prediction_A)
       -(1-target) * log(1-prediction_A)

For a target of 0.6, the loss is minimized when the prediction is 0.6. If we round the target to 1, increasingly confident predictions keep reducing the loss instead.

This is a standard soft-label pairwise logistic objective. It exposes one preference-model component; it is not a claim to reproduce every training choice in the Constitutional AI paper.

Avoid computing log of a rounded zero

Large score differences can make a floating-point sigmoid return exactly zero or one. Computing the logarithms directly can then fail, even though the mathematical loss is finite.

A stable expression works in score-difference space:

def preference_loss(scores_a, scores_b, targets):
    losses = []
    for a, b, target in zip(scores_a, scores_b, targets):
        d = a - b
        losses.append(
            max(d, 0.0) - target*d
            + math.log1p(math.exp(-abs(d)))
        )
    return math.fsum(losses) / len(losses)

assert abs(preference_loss([0], [0], [.6]) - math.log(2)) < 1e-12
assert preference_loss([1000], [0], [1]) == 0
assert preference_loss([1000], [0], [0]) == 1000
assert preference_loss([1000], [0], [.6]) == 400
assert preference_loss([-1000], [0], [1]) == 1000

The lists are equal-length and nonempty, targets lie in [0,1], and scores are finite with absolute value at most one million. The function computes a mean loss in nats; it does not update parameters.

The 400 result is the useful counterexample. An extremely confident preference is cheap under a compatible hard label, but expensive when the target still assigns meaningful probability to the other answer.

Test the objective, not just whether it runs

Useful tests distinguish these common mistakes:

base = preference_loss([2], [0], [.7])
assert abs(preference_loss([2, 2], [0, 0], [.7, .7]) - base) < 1e-12
assert abs(preference_loss([0], [2], [.3]) - base) < 1e-12
assert abs(preference_loss([7], [5], [.7]) - base) < 1e-12

The last case checks that a common score offset changes neither preference nor loss.

Agreement is not a harmlessness guarantee

A low loss shows agreement with supplied targets. It does not independently establish that responses are harmless, correct, or helpful. Judge errors can become training targets too. The paper's broader pipeline also includes helpfulness data and reinforcement learning.

To practice the components in sequence, I put revision data preparation, soft-label normalization, and preference loss in the TensorDrill Constitutional AI learning unit. It includes inspectable browser tests and complete Python solutions. The wider library covers agents, world models, Attention, Flow Matching and other AI components.

References

• Constitutional AI: Harmlessness from AI Feedback, especially the supervised and AI-feedback stages.
• Anthropic's research overview.
• Python math.log1p.

AI assisted the wording of this article. Every Python block and assertion was executed and checked.

You finish reading a tutorial on Transformers, understand the basic idea of Attention, and feel reasonably confident about the equations. Then you open a blank Python file and try to implement it yourself.

Suddenly, things get complicated.

Which dimension should Softmax operate on? How exactly should the mask be applied? What happens when the input values become very large?

Understanding a concept and implementing it correctly are two different skills.

This gap becomes even more noticeable when moving beyond classical machine learning into areas like Diffusion Models, Flow Matching, AI Agents, and World Models.

That got me thinking: we have LeetCode for algorithms, but where do we go to practice the actual building blocks of modern AI?

There are already a few interesting platforms trying to answer that question, although they approach it in different ways.

1. Four Platforms, Four Different Approaches

Before getting into specific AI topics, it's worth looking at what different coding platforms are designed to teach.

Platform

Main Focus

What You Can Practice

LeetCode

Data structures and algorithms

General programming, algorithmic thinking, coding interviews

Deep-ML

Machine learning fundamentals

ML algorithms, mathematical operations, numerical implementations

TensorTonic

AI and ML engineering

Machine learning implementations, PyTorch, CUDA, Triton, and related technologies

TensorDrill

AI concepts from foundations to newer research topics

Transformer components, generative models, Agents, World Models, and more

I don't think there's a single winner here. The platforms serve somewhat different purposes.

LeetCode is still a great place to develop general programming skills. If you're preparing for software engineering interviews or want to get better at algorithms, it's hard to ignore.

Deep-ML brings the coding-challenge format into machine learning. Instead of solving another graph traversal problem, you might implement a numerical operation or an ML algorithm from scratch. It's a useful bridge between mathematical knowledge and working code.

TensorTonic is interesting for people who want to explore practical AI implementation across different technologies. In particular, PyTorch, CUDA, and Triton introduce a different set of engineering challenges from pure Python algorithm exercises.

TensorDrill takes a more topic-oriented approach, organizing small Python challenges around AI concepts, including some areas that appear frequently in recent research.

The important distinction isn't simply how many questions each platform offers. It's what kind of understanding you're trying to develop.

1. Can Research Concepts Become Coding Challenges?

At first, the idea of turning research topics into LeetCode-style problems might sound a little strange.

After all, a modern generative model can involve billions of parameters, distributed training, complicated data pipelines, and substantial GPU resources.

You obviously can't reproduce all of that in a ten-minute coding challenge.

But many of the underlying mechanisms are surprisingly small.

Consider a few examples.

Attention

You don't need to train a language model to implement Scaled Dot-Product Attention.

The core computation is:

[ \operatorname{Attention}(Q,K,V)=\operatorname{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V ]

Implementing this correctly forces you to think about tensor shapes, scaling, masks, and numerical stability.

Diffusion Models

You can explore diffusion without training an image generator.

For example, implementing the forward noising process, a noise prediction loss, or a DDIM sampling step helps connect the equations in a paper to executable operations.

AI Agents

Even though agents often involve large language models, many of their supporting mechanisms can be tested independently.

A ReAct trace parser, a tool-call validator, a retry policy, or a dependency-aware task scheduler can all become small programming exercises.

World Models

The same idea applies to world models.

State transitions, belief updates, trajectory rollouts, and planning objectives can be studied with compact numerical examples.

A particularly interesting exercise is comparing two predictive models: one might achieve a lower prediction error but still lead to worse planning decisions.

That is a concept worth understanding through code rather than simply memorizing as a statement.

The common theme is that you don't always need to build the entire system to understand one of its essential mechanisms.

1. A Concrete Example: Learning Flow Matching Through Code

Flow Matching is a good example because its core mathematics is elegant, yet implementation details can still be confusing.

Suppose we have two points, (x_0) and (x_1), and define a simple linear interpolation:

[ x_t=(1-t)x_0+tx_1 ]

The corresponding target velocity is:

[ u_t=x_1-x_0 ]

These two equations already suggest several useful exercises.

Exercise 1: Implement the interpolation.

Write a function that takes (x_0), (x_1), and (t), and returns (x_t).

Test the boundary cases. At (t=0), the result should be (x_0). At (t=1), it should be (x_1).

Exercise 2: Implement the target velocity.

For this particular linear path, the target velocity is constant.

A simple implementation can help you see why it does not depend on (t), even though the interpolated state does.

Exercise 3: Implement a velocity regression loss.

Given predicted velocities and target velocities, compute a mean squared error objective.

Pay attention to reduction conventions and tensor shapes.

Exercise 4: Implement a numerical ODE solver.

Start with the Euler method, then try Heun's method.

Compare the approximation errors on a velocity field with a known solution.

None of these exercises requires training a full generative model.

Yet together, they provide a much clearer understanding of what happens inside a Flow Matching implementation.

And once you've worked through them, reading a more complete paper or codebase becomes easier because the individual operations are already familiar.

This is the kind of progression I find useful: start with a small mathematical mechanism, implement it, verify it, and only then move to the larger architecture.

1. The Most Useful Part of a Coding Problem Might Be the Explanation

Online judges are good at answering one question:

Does this implementation produce the expected output?

But when learning AI, that's only part of what matters.

Suppose you're implementing Softmax.

You know the formula:

[ \operatorname{softmax}(x_i)=\frac{e^{x_i}}{\sum_j e^{x_j}} ]

A straightforward implementation might appear correct for ordinary inputs.

But what happens when an input contains a value like 1000?

Directly computing its exponential can overflow.

A more numerically stable implementation subtracts the maximum value before exponentiation.

This is a small implementation detail, but it's also an important general lesson about numerical computing.

A good coding exercise should help you understand both the mathematical operation and the engineering decisions needed to implement it reliably.

That is why I prefer exercises that provide more than a reference solution.

Ideally, they should explain the mathematical reasoning, important edge cases, complexity considerations, and common implementation mistakes.

For example, if an Attention implementation fails a test, it's much more helpful to understand whether the problem comes from an incorrect mask, a shape mismatch, or numerical instability than to receive a generic "Wrong Answer."

The goal shouldn't be to memorize solutions. It should be to understand why one implementation works and another doesn't.

1. Why Actually Running the Code Matters

There's also a meaningful difference between answering a conceptual question and implementing the concept correctly.

Someone might easily explain why we divide Attention scores by (\sqrt{d_k}).

But can they implement Attention correctly with a causal mask?

Can they handle batch dimensions?

Can they write a Softmax function that remains stable with large inputs?

Those details are difficult to evaluate through multiple-choice questions alone.

With executable coding exercises, the feedback loop is more concrete:

Write code → Run tests → Inspect failures → Revise → Understand

The tests themselves matter, too.

An implementation that passes one example isn't necessarily correct. Good test cases should include boundary conditions and common mistakes, rather than checking only the most straightforward input.

One approach I've been exploring with TensorDrill is keeping the exercises small enough to run directly in the browser while providing reference implementations and detailed explanations.

As of October 2026, the project includes 100 Python challenges, 26 concept pages, and 19 practice sets. Its reference implementations have been checked against 553 assertions, along with tests designed to reject 91 deliberately incorrect implementations.

Those numbers don't guarantee that every edge case has been covered, but they reflect an attempt to treat test quality as part of the learning experience.

There are trade-offs, of course.

The platform currently runs Python locally in the browser, and its tests are public. That's convenient for practicing and debugging, but it isn't the same as a secure competitive programming judge with private tests.

It also isn't a replacement for a proper PyTorch or GPU environment when you need to train models or optimize large computational kernels.

For learning individual concepts, though, small executable exercises can be a useful intermediate step.

1. How I Would Structure an AI Coding Practice Roadmap

If you're trying to learn AI through programming, I wouldn't recommend jumping randomly between unrelated topics.

There's a natural dependency structure worth following.

Start With Numerical Foundations

Before worrying about the latest model architecture, make sure you're comfortable implementing basic operations.

Matrix multiplication, normalization, Softmax, gradients, and optimization algorithms appear across a huge range of AI systems.

Being able to implement and debug them is useful almost everywhere.

Move to Transformer Components

Once the foundations are in place, start exploring Attention, causal masking, multi-head operations, positional embeddings such as RoPE, and feed-forward components such as SwiGLU.

These exercises help turn a Transformer from a complicated diagram into a collection of understandable computations.

Explore Generative Modeling

From there, move into VAEs, GANs, Diffusion Models, DDPM/DDIM sampling, and Flow Matching.

Try to connect the underlying mathematics rather than treating every new model family as an entirely separate subject.

Numerical integration is especially useful to understand because it appears in several continuous-time generative modeling approaches.

Then Explore Agents and World Models

Finally, consider mechanisms such as tool calling, ReAct execution, retrieval evaluation, state transitions, trajectory rollouts, and planning.

These topics are particularly interesting because they connect mathematical predictions and model outputs to actual decisions and actions.

Of course, this is just one possible path.

If you're already comfortable with the fundamentals, you can jump directly into a topic that matches your interests.

The important thing is to build connections between concepts instead of collecting isolated solutions.

1. So Which Platform Should You Use?

It really depends on what you're trying to get better at.

If you're preparing for a conventional coding interview, LeetCode is probably the most obvious starting point.

If you want to strengthen your understanding of classical machine learning through implementation, Deep-ML is worth checking out.

If you're more interested in AI systems engineering or lower-level implementation technologies, TensorTonic may be a better fit.

And if you're looking for topic-based exercises that extend from AI fundamentals into Transformers, generative models, agents, and world models, TensorDrill is another option to explore.

These platforms can also complement one another.

You might practice general algorithms on LeetCode, implement numerical ML operations on Deep-ML, explore different engineering stacks through TensorTonic, and work through modern AI components on TensorDrill.

What matters most is choosing exercises that expose the gaps in your understanding.

Final Thoughts

AI moves quickly. New model architectures, training methods, and agent frameworks appear all the time.

Trying to keep up by reading every new paper is difficult enough.

But reading alone doesn't always translate into practical understanding.

For me, a more useful question is:

Can I take a mathematical equation or model component, implement it without copying the original code, and verify that it behaves as expected?

If the answer is yes, I've probably learned something meaningful.

If not, I've found a gap worth working on.

That's what makes small, well-designed coding challenges valuable, even in research areas that normally involve enormous models and complicated infrastructure.

Sometimes the best way to understand a complicated AI system is to start by implementing one small piece of it.