Shipping an LLM-Powered iOS App; Embeddings, performance, and other Gotchas

I tried to ship an on-device LLM & RAG functionality on my own app for iOS; the hardest parts weren't the model.