5 Things to Do With MicroGPT to Get the Most

Andrej Karpathy’s microGPT has distilled the GPT-like neural network architecture to its simplest possible form: a single, dependency-free Python file containing only what you need - nothing more. Because the point is clarity: dataset, tokenizer, autograd, a GPT-2-like transformer, Adam, training, inference. Its only imports are os, math, random, and argparse.

If you build LLM systems for a living or fun, microGPT is the reset you may be looking for. You get a clean artifact you can poke, break, and extend without spelunking through frameworks or kernels. Keeping the code simple and open, makes Lightning AI a natural companion when you want scale, repeatability, and shared workflows without turning a learning exercise into a mult-phase infrastructure project.

Here are five practical ways to get value out of microGPT on Lightning AI today (complete with enough specifics to actually do the thing).

1) Turn a 200-line script into a repeatable experiment

microGPT is small enough that every change feels consequential, which also means it’s easy to lose track of what you changed, whether it helped, and how to reproduce it when it does (because a run you can’t reproduce is useless).

We suggest using Lightning as the experiment wrapper around the repo: consistent environments, pinned runs, and a place your team can fork from without local setup drift.

▸ How to do it

Create a new Studio in Lightning AI and open a terminal.
Clone or paste the gist into microgpt.py, then commit it to a repo inside the Studio so it's versioned from the start.
Install nothing microGPT runs with python microgpt.py.
For each experiment, use a Lightning Run with a distinct name (e.g. baseline, bigger-context, 4-layers). This pins the environment snapshot and stdout log.
Write your hypothesis and outcome as a comment at the top of each run's script diff, not in a separate doc.

Keep the how-to in the repo itself and treat Lightning as the place you run it and compare outcomes. A good rule: if you can't reconstruct the exact run from the git diff and the log file, it didn't happen.

2) Swap the dataset without rewriting the model

Karpathy's guide emphasizes that microGPT will learn whatever statistical patterns live in the data. The training loop stays the same regardless of what you feed it. That makes it a great dataset microscope.

Put in a tiny domain dataset: product names, error strings, short code snippets, internal terminology. Then observe what changes when you adjust sequence length, vocab construction, or sampling temperature. The model is the control variable. The data is the experiment.

▸ How to do it

Find the input.txt fetch block near the top of microgpt.py. That's the only thing you need to change.
Replace it with your own input.txt: one document per line, UTF-8, at least a few hundred lines for anything interesting to emerge.
Watch the vocabulary size change. microGPT uses a character-level tokenizer, so your domain's character distribution matters more than you'd expect.
Run two versions side by side in Lightning — the original names dataset and yours — and compare the loss curves and generated samples at the same step count.

3) Stress-test your intuition about transformers

microGPT includes every piece practitioners talk about daily (tokenization, embeddings, attention, MLP, residuals, normalization, cross-entropy loss, and Adam) without hiding them behind tensor ops. Everything is explicit scalar arithmetic.

A practical exercise: pick one "obvious" change, predict what it will do to loss and sample quality, then test it. Commit your prediction as a comment before you run. Being wrong is the point: it means you've found a gap in your mental model.

▸ Experiments to try (in order of difficulty)

Change n_head from 4 → 1 or → 8. Does more heads help with a 16-dim embedding? (Spoiler: 8 heads × 2 dims each is probably worse, not better.)
Remove RMSNorm. Find the normalization calls, comment them out, and watch training destabilize at higher learning rates.
Change block_size (context window) from 10 → 32. The model needs more context to generate names — but it also trains much slower on CPU.
Change the Adam learning rate by 10× in either direction. lr=0.001 is default; try 0.0001 and 0.01 and observe whether loss oscillates or crawls.

4) Bridge from “algorithm” to “efficiency” in small steps

microGPT's explicit thesis is that production systems keep the math but change the execution: batching, vectorization, GPUs, mixed precision, better tokenizers, better kernels, distributed training. Everything else is just efficiency.

The way to internalize this is to close that gap one change at a time, not all at once. microGPT is the baseline. Each step toward a production system should be a deliberate, testable decision up an efficiency ladder (one rung at a time).

Add NumPy for vector ops. Replace the list-based dot products with np.dot. Measuring the speedup is the first step from "pure Python" to "uses a library.
Add batching. microGPT processes one token at a time. Adding a batch dimension is the single biggest conceptual jump toward production. Try implementing it manually before reaching for PyTorch.
Port to PyTorch, keeping the same architecture. Now you have GPU access through Lightning's hardware selector. The architecture should produce near-identical loss curves. If it doesn't, you've found a bug.
Enable mixed precision. Wrap your PyTorch training loop with torch.autocast and measure throughput vs. loss stability.

5) Use microgpt as an open-source teaching artifact inside your team

microGPT is "small enough to read in one sitting," which is genuinely rare. That makes it a strong internal learning tool for both new hires and experienced practitioners who can train large models but haven't recently revisited the fundamentals.

The goal isn't to replace deep reading with a one-hour demo. It's to give people a concrete codebase to anchor the conversation so that "attention" and "residual stream" and "the Adam betas" aren't just words.

▸ A lightweight team ritual that actually works

Assign one section per person. The code breaks into clean sections: data + tokenizer, autograd, embeddings, attention, MLP, training loop, inference. Each is ~25–40 lines and self-contained.
Each person prepares a 10-minute walkthrough of their section: what it does, what surprised them, what breaks if you remove it.
Each person also prepares one testable change to their section. Run it live in a Lightning Studio during the session.
Keep Karpathy's
Use Lightning as the shared workspace where everyone can run, compare, and iterate without friction. A shared Studio means no "works on my machine."

Where to start

If you haven’t read it yet, start with Karpathy’s write-up then decide which of the five paths above matches your goal: learning, benchmarking intuition, dataset exploration, or building a bridge from minimal code to scalable runs.

5 Things to Do With MicroGPT to Get the Most Value Out of It