Harmonica Learner
A real-time acoustic pitch tracker, adaptive difficulty trainer, and automated song-to-tabs ingestion engine.
Timeline
2026 — Present
Role
Product, iOS (Swift) & Web (SvelteKit)
Context
Personal Project
Platforms
iOS 17+ (SwiftUI) & Modern Web
Harmonica Layouts
Standard Richter C & Lee Oskar C
Links
Overview
Harmonica Learner is an acoustic mirror for the harmonica. I built it because I have been a harmonica owner since 2018 but I never really learned the ins and outs of playing it, and since it is a very small instrument that sits inside your mouth, it is completely blind... you can't see the hole positions. I wanted to create an app that could help me learn the harmonica, first with freestyles and later with music notes.
Built with Swift 5.9+, SwiftUI, and AudioKit, the app listens to your breath in real time using the device microphone. It displays precise hole numbers (1–10) and airflow guidance (Blow ↑ vs. Draw ↓), measures pitch deviations with a cents-accurate tuning needle, and requires three consecutive clean hits before advancing to the next note.
“A piano gives you 88 keys to look at. A guitar gives you frets and strings. A harmonica gives you a dark, metal box inside your mouth. You have to learn entirely through feel and sound... unless your phone can be your acoustic mirror.”
Key Accomplishments
- Real-Time Pitch Detection Engine: Integrated AudioKit and SoundpipeAudioKit (
PitchTap) with noise-floor gating, streaming reactive pitch and frequency updates into Combine publishers at 60+ FPS without dropped frames. - Pedagogical 3-Hit Progression Rule: Developed a note advancement state machine requiring 3 consecutive valid hits within the allowable tolerance window to prevent flukes and reinforce clean embouchure control.
- Adaptive Difficulty Curve (
AttemptToleranceModel): Formulated a dynamic tolerance interpolation curve starting at ±30 cents for novices and tightening down to ±15 cents over 20 attempts. - On-Device Audio Analysis (
ImportedSongAnalyzer): Engineered offline pitch extraction using Apple's Accelerate framework (vDSPFFT, Hann windowing, 11,025 Hz downsampling, 8,192-frame windows) to transcribe audio into playable harmonica tabs 100% locally and privately. - SvelteKit Companion Web Sandbox: Built an in-browser 10-hole harmonica synthesized with the Web Audio API, an interactive song-to-tabs converter demo, and a live pitch tolerance visualizer in
/web. - Liquid Glass iOS 18 Design: Designed responsive interfaces leveraging iOS 18 MeshGradients, frosted glass cards, haptic feedback pulses, and dual-column layouts for iPads and landscape orientations.
Technical Challenges & Solutions
Taming Harmonics and Reeds with AudioKit
Harmonica reeds produce dense, complex acoustic overtones and sharp attack transients. Naive autocorrelation algorithms often latch onto the second or third harmonic instead of the fundamental frequency. By configuring SoundpipeAudioKit's PitchTap, tuning amplitude gating to reject room reflections, and converting frequencies into standard scientific pitch notation and cents deviations via MIDI mathematical mapping (NoteMapper), the engine reliably isolates the primary reed frequency.
Offline Spectral Audio Analysis via Apple Accelerate
Transcribing a user-imported song on-device without draining battery or causing memory pressure required writing an efficient DSP pipeline. ImportedSongAnalyzer downsamples audio to 11,025 Hz, applies a Hann window over 8,192-frame analysis windows, and runs Discrete Fourier Transforms using Apple's Accelerate vDSP routines. Observations are clustered into continuous note events, mapped to the nearest playable harmonica holes (Standard Richter C or Lee Oskar C), merged across duplicates, and clamped to playable durations (0.1s–4.0s).
Swift to Web Audio API: Zero-Friction Web Sandbox
To allow prospective users to experience real-time harmonica feedback before installing the native app, the core audio synthesis and pitch logic was ported to SvelteKit using the Web Audio API (audio.ts). Visitors can click or tap 10 harmonica holes, toggle blow/draw states, test pitch tolerance meters, and simulate song transcription directly in their desktop or mobile browser.
Core Features
Guided Practice Mode
Real-time pitch detection with note-by-note progression. Land 3 consecutive clean hits on a target note to automatically advance, guided by hole callouts (1–10) and airflow directions (Blow ↑ / Draw ↓).
Cents-Accurate Tuner
A fluid tuning needle displaying real-time pitch deviations in cents, shifting smoothly between idle, emerald glow hit, and miss states with subtle haptic confirmation.
"Add Your Own Songs"
Import files (.mp3, .wav, .m4a, .flac) from Files or iCloud, pick from your device music library, record live acoustic audio via mic.
Freestyle Recording Mode
Unguided free play capturing both M4A audio and timestamped note events with duration and pitch data. Saved performances can be converted into silent practice targets.
On-Device Accelerate FFT
Apple Accelerate vDSP Discrete Fourier Transform analysis runs 100% locally and privately, clustering dominant frequencies into playable notes with synthesized reference previews.
SvelteKit Web Companion
Web experience featuring an in-browser 10-hole harmonica synthesized with the Web Audio API, interactive tolerance visualizer, and live song-to-tabs converter.
Technical Specifications
| Dimension | Specification |
|---|---|
| Supported Harmonica Types | 10-Hole Diatonic in Key of C (Standard Richter & Lee Oskar layouts) |
| Pitch Detection Engine | AudioKit + SoundpipeAudioKit (PitchTap real-time spectral detection) |
| Tolerance Range | Adaptive: ±30 cents (attempt 0) down to ±15 cents (attempt 20) |
| Progression Requirement | 3 consecutive valid hits per target note |
| Audio Analysis Framework | Apple Accelerate (vDSP_DFT, Hann window, 11,025 Hz downsampling) |
| Song Ingestion Methods | Files App (.mp3/.wav/.m4a/.flac), Device Music Library, Mic Recording, Song Links |
| Supported Platforms | iOS 17.0+ (iOS 18 MeshGradients) |
| Bundled Songs | 7 core tracks (Scales, chord drills, folk standards, blues riffs) |
| Privacy & Storage | 100% on-device local storage with zero telemetry/cloud lock-in |
Bundled Song Library
Harmonica Learner includes 7 core tracks designed to build fundamental breath control, single-hole isolation, and blues phrasing:
Release Milestones
v1.6
Web landing page & Polish- •Full SvelteKit web experience.
- •Searchable Practice Library sheet (HeaderView.swift) separating built-in tracks from custom recordings, with real-time search, swipe actions, and rename/delete context menus.
- •Streamlined compact header navigation, cached note-to-hole frequency lookup in HarmonicaLayout.swift, and adaptive screen heuristics.
v1.5
Song Ingestion & On-Device FFT- •"Add Your Own Songs" multi-channel ingestion engine: import from Files app (.mp3, .m4a, .wav, .aiff, .flac), Device Music Library, or ambient live microphone recording.
- •High-performance offline audio pitch analyzer powered by Apple Accelerate (vDSP FFT, Hann windowing, 11,025 Hz downsampling, 8,192-frame spectral windows).
- •Automatic mapping to nearest playable harmonica notes on Standard Richter C or Lee Oskar C with duration clamping and consecutive duplicate note merging.
- •Compliant song-link transcription seam & serverless Cloudflare Worker consuming Spotify Audio Analysis API chroma segments without audio ripping.
- •Synthesized reference audio playback providing isolated harmonica lead previews.
v1.0
Core iOS Launch- •Initial native iOS app target built with Swift 5.9+, SwiftUI, and Combine for iOS 17+.
- •Low-latency acoustic pitch detection engine powered by AudioKit, AudioKitEX, and SoundpipeAudioKit PitchTap.
- •Guided Practice Mode with 3-hit progression rule: require 3 consecutive clean hits within tolerance before automatically advancing.
- •Real-time hole and airflow visual guidance (Blow ↑ / Draw ↓ on holes 1 through 10) with cents-accurate tuning needle.
- •Adaptive pitch tolerance model smoothly tightening from ±30 cents to ±15 cents over 20 attempts.
- •Freestyle Mode: concurrent M4A audio capture and real-time note event timestamp logging, convertible into silent practice targets.
- •Harmonica layout models for Standard Richter Diatonic C and Lee Oskar Diatonic C with MIDI-standard mathematical mapping.
- •7 bundled beginner songs spanning scales, chord drills, folk melodies, and blues riffs.
Built With
