Sebastian Raschka

LLM Architecture Gallery

A gallery that collects architecture figures from The Big LLM Architecture Comparison and related articles, with fact sheets and links back to the original sections.

Kimi K3 Architecture Overview and Notes

Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.

Key differences between Python 2.7.x and Python 3.x (2014)

Many beginning Python users are wondering with which version of Python they should start. My answer to this question is usually something along the lines…

Intro to Deep Learning

I just sat down this morning and organized all deep learning related videos I recorded in 2021. I am sure this will be a useful reference for my future…

Understanding and coding the self-attention mechanism of large language models

Step-by-step tutorial explaining scaled dot-product self-attention for large language models, with Python code that builds the mechanism from scratch.

Machine Learning with PyTorch and Scikit-Learn

Machine Learning with PyTorch and Scikit-Learn has been a long time in the making, and I am excited to finally get to talk about the release of my new book…

An introduction to parallel programming using Python's multiprocessing module