# feedback-loop

A small simulator of the recommender feedback loop: a system that only learns from the
clicks on what it chose to show, and so keeps showing what it already knows. It compares
ranking policies that break the loop (ε-greedy, UCB, Thompson sampling, a diversity
re-ranker) with pure exploitation, round by round, and measures the narrowing: topics per
feed, catalogue coverage, Gini of exposure, true feed value against the best possible,
a per-user filter-bubble index and popularity concentration. The browser version in
`web/` is a line-for-line port that draws the same random numbers, so a run on the web
page reproduces digit for digit with the command line.

numpy + matplotlib only. One 30-round run of 400 readers × 300 items takes about 2 s
in Python and about 1 s in the browser.

## Install and run

```bash
pip install -r requirements.txt           # numpy, matplotlib, pytest
python -m feedloop run --policy ucb --rounds 30 --seed 1      # out/run_ucb.json + out/run_ucb.png
python -m feedloop run --policy exploit --switch-round 15 --switch-policy ucb
python -m feedloop run --policy exploit --new-item-round 10
python -m feedloop compare --rounds 30 --seed 1               # every policy, the switch and the new item: ~50 s,
                                                              # out/compare.json + policies/switch/new_item/feed_strip.png
python -m pytest -q                                           # 14 tests, ~5 s (needs node for the parity tests)
```

`python -m feedloop run --help` lists the knobs (ε, UCB c, Thompson c, MMR λ, factors,
ridge, ALS sweeps, repeat decay, `--no-discount`, `--visitor` for the web page's ratings).

## How the simulation works

**World** (`feedloop/world.py`). 300 items in 8 topics, each with a quality in [0.35, 1], a
"style" in [−1, 1] and a generated name ("Slow trains of Hokkaido"). 400 readers, each with a
taste over topics (one favourite at 1, a second at about 0.55, a third at about 0.3, the rest
near 0) and a style preference. The true appeal of item i to reader u is
`taste[u, topic(i)] × quality(i) × (1 − 0.15 |style(i) − pref(u)|)`: the probability they click it
at the top of the feed the first time they see it.

**Clicks** (`feedloop/sim.py`). Each round every reader gets k = 10 items. The chance of a
click is appeal × position bias (1/log₂(rank + 2), as in DCG) × 0.7ⁿ for the n-th time they
have seen the item. Click coins are counter-based random numbers keyed by (round, reader,
slot), so every policy faces the same coins: a run that switches policy at round 15 is
identical to the plain run until round 14.

**Recommender** (`feedloop/recommender.py`). A 6-factor matrix factorisation with an item
bias, trained by alternating least squares on the pairs (reader, item, times shown, times
clicked). It fits the click rate on shown pairs only; an item nobody was shown has no rows,
so its bias and vector shrink to exactly zero and it is predicted to be worth nothing. By
default the recommender discounts items it already showed a reader by the same 0.7ⁿ.

**Policies** (`feedloop/policies.py`), all over the same predicted scores:
pure exploit (top k); ε-greedy (each slot random with probability ε = 0.1); UCB
(score + c·√(log N / (n + 1)) with N total impressions and n the item's, c = 0.15); Thompson
(score + c·z/√(n + 1) with one normal per reader and item, c = 0.3); MMR (greedy re-rank,
λ·score − (1 − λ)·top score if the topic is already in the feed, λ = 0.7); random.

**Metrics**, per round over the simulated readers: distinct topics per feed; share of the
catalogue shown to anyone; Gini of items' impressions; mean appeal of the feed ÷ mean appeal of
the reader's 10 best items; clicks per reader; share of the feed in the reader's favourite
topic; share of impressions on the top 1% of items; impressions of the held-back "new item".

**Random numbers** (`feedloop/rng.py`). Every number is a hash of (seed, stream, counter).
numpy draws blocks, the JavaScript draws them one by one, and they agree exactly, which is what
makes the parity test exact rather than statistical.

## Results (seed 1, 30 rounds, 400 readers, 300 items)

| policy | topics/feed 1 → 30 | coverage r30 | Gini r30 | value r30 | clicks r30 | bubble r30 | top 1% r30 |
|---|---|---|---|---|---|---|---|
| Pure exploit | 5.92 → 4.24 | 0.55 | 0.67 | 0.38 | 0.59 | 0.28 | 0.13 |
| ε-greedy | 5.95 → 4.53 | 0.97 | 0.61 | 0.39 | 0.94 | 0.35 | 0.13 |
| UCB | 5.92 → 4.24 | 0.86 | 0.70 | 0.43 | 1.15 | 0.38 | 0.17 |
| Thompson | 5.93 → 4.19 | 1.00 | 0.40 | 0.41 | 0.90 | 0.37 | 0.09 |
| MMR re-rank | 8.00 → 7.17 | 0.63 | 0.70 | 0.34 | 0.67 | 0.26 | 0.15 |
| Random | 5.97 → 6.01 | 1.00 | 0.15 | 0.20 | 0.56 | 0.13 | 0.02 |

Round 1 is a random feed for everyone (no data). In round 2 pure exploit shows only 4.3% of
the catalogue (Gini 0.97): the model is just item click rates, and everyone gets the same 13
items. Clicks under pure exploit peak at 1.44 per reader around round 7 and fall to 0.59 by
round 30 as it runs out of items it knows are good; UCB ends at 1.15 and never shows less than
86% of the catalogue in the last rounds. Thompson sampling is the most even (Gini 0.40).
Exploration does not raise per-feed topic diversity (bandit policies end near 4.2 topics, like
exploit, because they learn tastes better); only the MMR re-ranker does, at a cost in value.

Switching from exploit to UCB at round 15: coverage goes 0.48 (round 14) → 0.19 (round 15,
everyone is sent to the same never-shown items at once) → 0.88 by round 30; Gini 0.76 → 0.62.

A good new item (appeal 0.9) released at round 10: pure exploit and MMR never show it in 20
rounds (its predicted score is exactly 0); UCB shows it to all 400 readers in round 10 and
862 times by round 30; Thompson 649; ε-greedy 781 (two random showings, then the model
learns it is good); random 264.

## Files

```
feedback-loop/
├── feedloop/
│   ├── rng.py           counter-based random numbers shared with the JS
│   ├── world.py         catalogue, names, readers, true appeal
│   ├── recommender.py   ALS matrix factorisation on shown pairs
│   ├── policies.py      exploit, ε-greedy, UCB, Thompson, MMR, random
│   ├── sim.py           the loop and the metrics
│   └── cli.py           run / compare, figures
├── tests/test_loop.py   narrowing, exploration, determinism, common random numbers, cold start, JS parity
├── tests/run_js.js      runs web/feedloop.js under node for the parity test
├── web/feedloop.js      the port; web/worker.js runs it off the main thread; web/page.js the page
├── index.html           the page; how-it-works/index.html the write-up
└── out/                 compare.json and the figures
```