# pianoify

**Try it in the browser:** [williamc.me/projects/pianoify](https://williamc.me/projects/pianoify/) (`index.html` + `web/`, a JavaScript port of this tool).

Turn a song recording (MP3, WAV, FLAC, M4A…) into a **simplified two-hand piano arrangement** with chord symbols and engraved sheet music.

```
song.mp3 ──► beats & bars ──► key ──► chords ──► melody ──► arrange ──► PDF / MusicXML / MIDI / chord chart
```

## Install

```bash
pip install -r requirements.txt          # Python 3.10+, ffmpeg on PATH for MP3
# optional but recommended for real songs (better melody from the isolated vocal):
pip install torch demucs
```

## Use

```bash
python pianoify.py song.mp3 -o out/                      # standard arrangement
python pianoify.py song.mp3 --level easy                 # beginner version
python pianoify.py song.mp3 --to-key C                   # transpose to an easier key
python pianoify.py song.mp3 --transpose -2               # or by semitones
python pianoify.py song.mp3 --separate                   # use Demucs vocal/bass stems
python pianoify.py song.mp3 --bpm 72                     # fix a doubled/halved tempo
python pianoify.py song.mp3 --meter 3 --sevenths         # 3/4 time, allow 7th chords
```

Outputs in `out/`:

| file | what |
|---|---|
| `*.pdf` | engraved sheet music (treble + bass staves, chord symbols above) |
| `*.musicxml` | open/edit in MuseScore, Sibelius, Finale, Dorico, Flat.io |
| `*.mid` | playback / import into a DAW or Synthesia |
| `*.chords.txt` | four-bars-per-line chord chart |
| `*.analysis.json` | tempo, key, chords, melody and beat times (for building UIs on top) |
| `*.preview.png` | first page as an image |

## Levels

| | **standard** | **easy** |
|---|---|---|
| Right hand | melody on an 8th-note grid | melody on a quarter grid, chromatic notes snapped to the key, repeats merged |
| Left hand | voice-led triads (smallest hand movement), re-struck each bar | root + fifth |
| Chords | up to 2 per bar | 1 per bar |

## How it works

1. **Separation** — harmonic/percussive split (librosa HPSS); with `--separate`, Demucs `htdemucs` stems (vocals → melody, other+bass → chords).
2. **Beats** — librosa beat tracker on the percussive part, extended to cover the whole file.
3. **Key** — Krumhansl–Kessler profile correlation on energy-weighted chroma plus bass.
4. **Chords** — beat-synchronous CQT chroma scored against major/minor (optionally 7th) templates, with a bass-root bonus and a gentle diatonic prior; Viterbi smoothing; flat/atonal beats → N.C.
5. **Bar lines** — an HMM over position-in-bar uses chord changes, chroma flux and low-frequency onsets to find downbeats. It can re-sync, so a dropped beat or an odd 2-beat bar doesn't shift every bar line after it. The beat grid is then rebuilt with exactly N beats per bar and chords are recomputed on it.
6. **Melody** — harmonic-summation salience on a 1/3-semitone CQT (C3–C6), accompaniment suppressed by subtracting each bin's 4-second running median, Viterbi path favouring smooth motion, sub-octave correction, onsets from salience rises. Quantised to the grid, blips removed, octave leaps fixed, folded into a comfortable right-hand range.
7. **Engraving** — music21 builds the grand staff (key/time signatures, tempo, chord symbols, ties); Verovio renders it; cairosvg + pypdf assemble the PDF.

## Accuracy

On the bundled synthetic test song (known answers; `tests/`):

| | result |
|---|---|
| tempo | 95.7 vs 96 BPM |
| key | G major ✓ |
| chords | 98.5% of time correct (all 17 changes incl. a mid-bar split) |
| melody pitch | 93.4% exact |
| note onsets | 92% recall, 94% precision |

```bash
python tests/make_test_song.py test_song && python pianoify.py test_song.mp3 -o out
python tests/evaluate.py out/test_song.analysis.json test_song.truth.json
node tests/run_js.js test_song.mp3 js_out          # the browser version, under Node
python tests/evaluate.py js_out.analysis.json test_song.truth.json
```

The browser version scores 99.6% chords, 95.0% melody pitch and 96% / 89% note-start recall / precision on the same song.

Real recordings are harder than the synthetic test: chords are usually reliable; the melody line is the weakest part on dense mixes. `--separate` (Demucs) helps most there.

## Known limitations / next steps

- Tempo can come out doubled or halved (e.g. 144 vs 72 for a ballad) — pass `--bpm`.
- Assumes a constant meter; a genuinely odd bar gets stretched to fit.
- Melody is the single predominant line; harmonies, backing vocals and instrumental hooks are not separated out.
- Chord vocabulary is major/minor (+ 7ths with `--sevenths`); no sus/dim/aug/slash chords yet.
- No lyrics.
- Good upgrades: Spotify's `basic-pitch` for note transcription, a learned chord model (e.g. madmom / BTC).