Personal site and technical notebook

Ilia Ilmer

Writing about algorithms, systems, implementation details, and the parts of software work worth keeping around.

Attention Matching for Key Value Compaction

Posted on Mon 01 June 2026 in posts • Tagged with Machine Learning, NLP • 2 min read

KV compaction is the quintessential step for long context conversations.
Let's review a new method for it!

Posted on Sun 23 March 2025 in posts • Tagged with C/C++, NLP, Data Structures • 1 min read

Learning BPE by Implementing it in C

Posted on Tue 04 March 2025 in posts • Tagged with Python, NLP, Transformers • 3 min read

Learning about transformers by implementing from scratch