TensorDrill update: visual AI coding practice and interview labs
TensorDrill is a Python practice site for implementing AI components: Softmax, Attention, Transformers, diffusion, Flow Matching, agents, world models and more. It also has a separate set of exercises for working with existing code.
I'm building the project, and the latest update adds three interactive walkthroughs: Top-K token probabilities, classifier-free guidance (CFG), and global gradient clipping. Each connects a small numerical example to the function you implement, its tests, and a complete Python solution.
Here is what changed, with runnable examples of the ideas behind the visuals.
Top-K probabilities: filtering is only half the job
Suppose the logits are [2, 4, 1, 3] and k = 2. The retained token indices are 1 and 3. Their final probabilities are approximately:
[0.0, 0.731059, 0.0, 0.268941]
Applying Softmax to the whole vocabulary and then zeroing the tail gives a different result: the surviving probability mass is about 0.880797, rather than 1. You still need to renormalize.
The new walkthrough places three bar charts side by side: full Softmax, the zeroed tail without renormalization, and the required Top-K output. Changing k makes the redistribution visible. All three charts use the same probability scale.
A complete implementation is small:
import math
def top_k_probabilities(logits, k):
selected = sorted(range(len(logits)), key=lambda i: (-logits[i], i))[:k]
peak = max(logits[i] for i in selected)
weights = [math.exp(logits[i] - peak) for i in selected]
total = sum(weights)
result = [0.0] * len(logits)
for index, weight in zip(selected, weights):
result[index] = weight / total
return result
assert top_k_probabilities([0, 2, 2, 2], 2) == [0.0, 0.5, 0.5, 0.0]
The tie rule matters: this exercise retains exactly k indices, preferring lower indices for equal logits. It returns probabilities in the original vocabulary order and does not sample a token. Library tie handling may differ.
CFG: see when interpolation becomes extrapolation
The CFG exercise combines unconditional and conditional epsilon predictions at the same noisy input and timestep:
def cfg_epsilon(eps_uncond, eps_cond, guidance_scale):
return [u + guidance_scale * (c - u) for u, c in zip(eps_uncond, eps_cond)]
assert cfg_epsilon([-1, 0], [1, 1], 0) == [-1, 0]
assert cfg_epsilon([-1, 0], [1, 1], 1) == [1, 1]
assert cfg_epsilon([-1, 0], [1, 1], 1.5) == [2, 1.5]
The two-dimensional view shows the two predictions and the combined result. Between 0 and 1, the point interpolates. Above 1, it continues beyond the conditional prediction. The table exposes the difference vector and its scaled contribution coordinate by coordinate.
This also makes a notation trap easier to explain. The expression c + w * (c - u) matches u + s * (c - u) when s = 1 + w. The same numeric scale is not interchangeable between the formulas.
The diagram is prediction space, not an image or a denoising trajectory. This helper does not run a model or a sampler, and increasing the scale does not guarantee better images.
Global clipping: a vector norm is not a coordinate limit
For a gradient [3, 4], the L2 norm is 5. With max_norm = 2.5, global clipping multiplies both coordinates by 0.5 and returns [1.5, 2.0].
Clamping each coordinate separately instead gives [2.5, 2.5]. Its norm is about 3.535534, still above the requested limit, and its direction has changed.
import math
def clip_global_norm(grad, max_norm):
norm = math.sqrt(sum(value * value for value in grad))
factor = min(1.0, max_norm / norm) if norm > 0 else 1.0
return [value * factor for value in grad]
assert clip_global_norm([3, 4], 2.5) == [1.5, 2.0]
assert clip_global_norm([0, 0], 2.5) == [0.0, 0.0]
assert clip_global_norm([], 2.5) == []
The visualization compares these operations against a circle whose radius is max_norm. Both axes use equal units, so the circle and vector directions remain meaningful. The displayed coordinate range adapts to the input and threshold; the axis ticks show that change.
Try the gradient-clipping walkthrough.
A second practice format: take over code, review a suggestion, test a change
Implementing an AI component and safely changing an existing program exercise different skills. TensorDrill now has seven existing-code labs alongside the implementation problem bank.
Their workflow is:
Read the task and existing implementation.
Review a preset AI suggestion and decide what it misses.
Debug, repair, or extend the program.
Run tests and check the behavior that must remain intact.
Examples include maze traversal, load balancing, template rendering, and invoice pagination. The suggestions are preset teaching material; the site does not connect to a live AI assistant. These are original practice exercises, not a claim to offer official company interview questions.
Explore the AI-assisted coding interview labs.
What stays practical
The current implementation bank contains 100 Python challenges. Practice and reference solutions are free, the interface is available in English and Chinese, and Python runs in the browser. No sign-up or model API key is required. Progress is stored on the device, with export/import for backups.
Visual walkthroughs are available on selected problems; the rest of the bank has written explanations and reference code. The new three walkthroughs were checked with 1,085 numerical traces against Python references and independent formulas, including every reachable numerical control setting.
The goal is to make it easier to connect a formula, an implementation, and a test that would catch a plausible mistake. I plan to keep expanding the exercises and the visual explanations. Feedback on unclear examples or useful next topics is welcome.
Further reading
Writing assistance: this post was drafted with AI assistance. The numerical examples were checked against Python reference implementations.
