William Chang Projects How it works

Theory Demos

Five small experiments from machine-learning theory that you can play with. Move a slider or click a plot and watch the effect happen, live in your browser.

1

More parameters, less error

Fit 12 noisy points with p random wave-shaped features. Test error rises as p approaches 12, peaks exactly at p = 12, then falls again as p keeps growing. Add a little ridge penalty and the peak disappears.

Feature draws per p
median test error, your λ (band: middle half of the draws) no penalty, for reference · click to pick p
One fit with p = 12

Same setup as double_descent.py: the same 12 points, cosine features with frequency scale 15, 500 test points. Your browser draws its own random features, so the numbers wobble a little from run to run.

2

High dimensions are strange

Pick a dimension. In high dimensions two random directions are almost always nearly at right angles, almost all of a ball's volume sits just under its surface, and far more than d directions fit that are all nearly at right angles to each other.

Angle between 5,000 random pairs of directions, as a cosine (0 = right angle). the bell curve with spread 1/√d
Share of a ball's volume in its outer ε of radius, 1 − (1 − ε)d.
How many directions fit with every pair within |cos| < 0.1. random set d greedy search
20,000 random tries; keep each one that is nearly at right angles to all kept so far (d ≤ 1,000).
3

A perfect diffusion model only copies

The red crosses are training points on a circle. Start 2,000 points from pure noise and run a diffusion model backwards. With the mathematically ideal model every sample lands on a training point. Blur the model or use a small trained network, and samples spread along the circle instead.

generated training points
Model
4

Private polls

Each person flips a hidden, biased coin and answers either the real question or its opposite, so no single "yes" gives anyone away. The pollster can still estimate the true rate. Lower ε means more privacy, and wider error bars.

 
100 polls, each with its estimate and 95% interval.
typical error (formula) spread of the 100 polls honest answers

Analyse a real poll

A CSV with one yes/no answer per row (a header row is fine; the column named "answer" is used if there is one). Nothing leaves your browser.

 
The example poll has 800 answers, collected with p = 0.75.
5

A guarantee against small nudges

A small network separates two interleaved half-moons. Click anywhere: the circle is a proven safe zone, where no change to the input can flip the answer. Randomized smoothing gets that proof by asking the network about many noisy copies of the point.

Shaded regions show the classifier's answer. Dots are the 200 test points.
Classifier
Click the plot to certify a point.

Uses the exact networks trained by certified_robustness.py, and the same certificate: 100 noisy votes to pick the class, 10,000 to certify, 99.9% confidence. The shading for the smoothed network uses only 64 votes per cell, for display.

How it's built, results and limits: How it works →