Skip to content
Back to projects

LLM & RAG · 2026

Edge4-SQL: CPU-Optimized Text-to-SQL

Phi-4-mini-instruct (3.8B) fine-tuned with QLoRA on the QuerySmith-Spider-Bird dataset (derived from Spider and BIRD), then compressed with a partial-residual quantization scheme and served with llama.cpp. The final model reaches 0.743 exact-match accuracy at 22.3 ms/token and 3.5 GB, against an FP32 baseline of 0.713 at 148 ms/token and 14.9 GB (6.6x faster, 4.2x smaller).

Tech stack

  • Phi-4-mini
  • QLoRA
  • Quantization
  • GGUF
  • llama.cpp
M

drawing ideas into systems...