September 20, 2026 · 11 min
A³ Attention: The Softmax Is Not Sacred
Transformers have used the same fixed softmax since 2017. A³ asks a simple question: what if attention could adapt its own activation?
- Research
- Transformers
- Attention
thoughts & build logs
Notes on ML engineering, attention research, and shipping AI systems that work on real hardware.
September 20, 2026 · 11 min
Transformers have used the same fixed softmax since 2017. A³ asks a simple question: what if attention could adapt its own activation?
August 5, 2026 · 9 min
A model that only lives in a notebook helps no one. Lessons from building BurnoutCompass AI end to end — from ensemble training to a deployed FastAPI backend.
June 12, 2026 · 10 min
No GPU? No problem. How I fine-tune small models with QLoRA and serve them on plain CPUs with quantization and llama.cpp.
May 28, 2026 · 9 min
I've built the same RAG loop three times now. Every time the failure was retrieval, not the model — here's what I changed after each one.
April 14, 2026 · 8 min
Instead of a user × item rating matrix, Movie-Recommender learns a vector space with a Siamese Transformer and searches it with FAISS.
March 9, 2026 · 8 min
A CNN-LSTM on phone accelerometer and gyroscope streams — and why the experiment tracker ended up mattering more than the extra layer.
February 18, 2026 · 6 min
OpenCV, a Haar Cascade and a thin FastAPI service. What a deliberately small computer-vision project taught me about picking boring tools.
January 22, 2026 · 8 min
Resume-Screening-System is a RAG pipeline with a strict output contract. Here's how I keep a local Mistral model honest over hundreds of CVs.
December 10, 2025 · 6 min
Smart Task Manager has no machine learning in it at all — FastAPI, SQLAlchemy, SQLite and a Streamlit UI. It's still the cleanest thing I ship.
drawing ideas into systems...