Sebastian Raschka

LLM Architecture Gallery

A gallery that collects architecture figures from The Big LLM Architecture Comparison and related articles, with fact sheets and links back to the original sections.

Kimi K3 Architecture Overview and Notes

Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.

Key differences between Python 2.7.x and Python 3.x (2014)

Many beginning Python users are wondering with which version of Python they should start. My answer to this question is usually something along the lines…

Intro to Deep Learning

I just sat down this morning and organized all deep learning related videos I recorded in 2021. I am sure this will be a useful reference for my future…

Python, Machine Learning, and Language Wars (2015)

This has really been quite a journey for me lately. And regarding the frequently asked question “Why did you choose Python for Machine Learning?” I guess it…

Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

Understanding and coding the self-attention mechanism of large language models

Step-by-step tutorial explaining scaled dot-product self-attention for large language models, with Python code that builds the mechanism from scratch.

Implementing a Principal Component Analysis In Python (2014)

Implementing Principal Component Analysis from scratch in Python: scatter matrices, eigenvectors, variance explained, and comparison with scikit-learn PCA.

Machine Learning with PyTorch and Scikit-Learn

Machine Learning with PyTorch and Scikit-Learn has been a long time in the making, and I am excited to finally get to talk about the release of my new book…

A Short Chronology of Deep Learning for Tabular Data

Occasionally, I share research papers proposing new deep learning approaches for tabular data on social media, which is typically an excellent discussion…

Optimizing LLMs from a Dataset Perspective

Practical guide to improving LLM finetuning with better instruction datasets, covering data curation, prompt-output pairs, synthetic data, and experiment ideas.

Naive Bayes and Text Classification I – Introduction and Theory

Naive Bayes classifiers, a family of classifiers that are based on the popular Bayes’ probability theorem, are known for creating simple yet well performing…

An introduction to parallel programming using Python's multiprocessing module