# ARIA — Zero-to-Tiny LLM (Python)

**ARIA** is a beginner-friendly, step-by-step course that takes you from **“Hello World”** to training a **tiny decoder-only, character-level LLM** in Python. Each lesson is a single, runnable file with clear docstrings, doctests where helpful, and minimal dependencies.

> **Note:** This repository’s instructional content was **generated with the assistance of an AI language model**.

---

## What you’ll build

- A progression of tiny language models:
  - Count-based bigram model → NumPy softmax toy → PyTorch bigram NN
  - Single-head self-attention → Mini Transformer block
  - A tiny decoder-only model trained on a small corpus (e.g., Tiny Shakespeare)

---

## Who this is for

- Beginners who can run `python script.py` and have written a basic “Hello World”.
- Learners who want a **clear path** to an LLM without heavy math or large datasets.

---

## Course outline (lessons)

1. Read a Text File (with docstrings)  
2. Character Frequency Counter  
3. Train/Val Split  
4. Char Vocabulary + Encode/Decode  
5. Uniform Random Text Generator  
6. Bigram Counts Language Model  
7. Laplace Smoothing (compare w/ and w/o)  
8. Temperature & Top-k Sampling  
9. Perplexity on Validation  
10. NumPy Softmax + Cross-Entropy (toy)  
11. PyTorch Tensors 101  
12. Autograd Mini-Lab (fit *y = 2x + 3*)  
13. Char Bigram Neural LM (PyTorch)  
14. Sampling Function (PyTorch)  
15. Single-Head Self-Attention (causal mask)  
16. Mini Transformer Block (pre-LN)  
17. Tiny Decoder-Only Model (1–2 blocks)  
18. *(Optional)* Save/Load & CLI Interface

Each lesson includes: **Outcome, Files, Dependencies, Directions, Starter Code with docstrings + doctests, Run, What you learned, Troubleshooting, Mini-exercises, Next lesson.**

---

## Requirements

- **Python**: 3.10+  
- **OS**: Windows/macOS/Linux (UTF-8 locale recommended)  
- **Dependencies**:  
  - Stdlib only until Lesson 9  
  - **NumPy** for Lessons 8–10  
  - **PyTorch** (CPU is fine) from Lesson 11 onward  
- **Hardware**: CPU is enough for all lessons; tiny models, short runs

Install common deps (when needed):

```bash
pip install numpy torch --upgrade

```