Quick Start

# macOS / Linux
git clone https://github.com/TrenTorch/TrenTorch.git
cd TrenTorch
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .
tren setup
tren

# Windows (PowerShell)
git clone https://github.com/TrenTorch/TrenTorch.git
cd TrenTorch
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt
pip install -e .
tren setup
tren

tren setup will ask to add tren to your PATH: say yes, and every terminal after that just needs tren, from any directory, with no activating first.

Why TrenTorch?

Be the best engineer yourself !

We wanted to know. So we built one. Then we didn't stop at "good enough."

The Bricks 🧱

TinyTorch teaches the fundamentals. TrenTorch takes those same bricks and adds the reps: cleaner internals, sharper performance instincts, and an implementation pushed past the original spec wherever we saw the chance.

• Small enough to read in one sitting - every op traceable back to raw NumPy
• Big enough to actually flex - the real architecture real frameworks run on
• Ours - rebuilt, refactored, and hardened in our own hands

No black boxes. No import torch. Just the machinery, exposed.

What You'll Build

A complete ML framework, built from zero. No single finish line, a set of missions you clear on the way there:

🎯 Mission: Image - we teach Image

• Real computer vision on standard benchmarks
• Conv2d, pooling, and CNNs, built entirely from scratch on NumPy
• Performance that holds its own against the frameworks it's built to demystify

🎯 Mission: NLP - we teach NLP

• Tokenization, embeddings, and multi-head attention, hand-rolled
• The groundwork every language model stands on

🎯 Mission: LLM - we teach LLM

• Full GPT-style transformer blocks, not a wrapper around someone else's
• Real self-attention, real language generation

🎯 Mission: Inference - we teach Inference

• Profiling, quantization, and acceleration, so your model doesn't just train, it runs
• KV-cache and memoization for the speed that production demands

🎯 Mission: LLMOps - we teach LLMOps

• Modern optimizers with learning rate scheduling: SGD, Adam, AdamW, Lion, Muon
• Competitive benchmarking and the capstone that ties it all together

Zero PyTorch. Zero TensorFlow. Every line is ours.

Current Status

Want to explore the code? Browse the repository structure.

Adventurous? Local installation works, but bring a spotter. See the setup notes on the wiki's Getting Started page.

🏗 20 Progressive Modules

Build your framework through four progressive parts:

Each module asks one question: "Can I build this from scratch, and can I build it well?"

🏆 Historical Milestones

As you progress, you unlock recreations of landmark ML achievements, run on YOUR framework:

Not toy demos. Historically significant ML achievements, rebuilt with a framework we wrote ourselves.

Learning Philosophy

# Most courses:
import torch

model.fit(X, y)  # magic happens somewhere else

# TrenTorch:
# You implement every component
# You measure memory usage
# You optimize performance
# You own every layer of the stack

Why build your own framework?

• Deep understanding - know exactly what loss.backward() does, because you wrote it
• Systems thinking - memory, compute, and scaling stop being abstractions
• Debugging at any depth - fix problems at the model level or the tensor level
• Production instincts - the same patterns real ML systems run on

Repository Structure

TrenTorch/
├── data/
│   ├── src/                     # 💻 Curriculum source (edit here)
│   │   ├── 01_tensor/           # Module 01: Tensor operations from scratch
│   │   ├── 02_activations/      # Module 02: ReLU, Softmax activations
│   │   ├── 03_layers/           # Module 03: Linear layers, Module system
│   │   ├── 04_losses/           # Module 04: MSE, CrossEntropy losses
│   │   ├── 05_dataloader/       # Module 05: Efficient data pipelines
│   │   ├── 06_autograd/         # Module 06: Automatic differentiation
│   │   ├── 07_optimizers/       # Module 07: SGD, Adam optimizers
│   │   ├── 08_training/         # Module 08: Complete training loops
│   │   ├── 09_convolutions/     # Module 09: Conv2d, MaxPool2d, CNNs
│   │   ├── 10_tokenization/     # Module 10: Text processing
│   │   ├── 11_embeddings/       # Module 11: Token & positional embeddings
│   │   ├── 12_attention/        # Module 12: Multi-head attention
│   │   ├── 13_transformers/     # Module 13: Complete transformer blocks
│   │   ├── 14_profiling/        # Module 14: Performance analysis
│   │   ├── 15_quantization/     # Module 15: Model compression (precision reduction)
│   │   ├── 16_compression/      # Module 16: Pruning & distillation
│   │   ├── 17_acceleration/     # Module 17: Hardware optimization
│   │   ├── 18_memoization/      # Module 18: KV-cache/memoization
│   │   ├── 19_benchmarking/     # Module 19: Performance measurement
│   │   └── 20_capstone/         # Module 20: Complete ML systems
│   │
│   ├── modules/                 # 📓 Generated notebooks (learn here, stub-only)
│   │   └── ...                  # (20 module directories)
│   ├── solutions/                # 🔒 Reference implementations (maintainer/CI-only)
│   ├── datasets/                 # 🗂️ Curated training data (tinydigits, tinytalks)
│   ├── milestones/                # 🏆 Historical ML evolution - prove what you built
│   │   ├── 01_1958_perceptron/
│   │   ├── 02_1969_xor/
│   │   ├── 03_1986_mlp/
│   │   ├── 04_1998_cnn/
│   │   ├── 05_2017_transformer/
│   │   └── 06_2018_mlperf/
│   │
│   └── trentorch/                # 📦 Generated package (import from here)
│       ├── core/                  # Core ML components
│       └── ...                    # The framework you built
│
├── platforms/
│   ├── cli/                     # 🎛️ The `tren` CLI itself
│   │   ├── main.py               # Entry point
│   │   ├── core/                  # Shared plumbing: config, console, theme, runtime
│   │   ├── commands/               # Genuinely shared code only: base.py, export_utils.py, jupyter.py
│   │   ├── cli_platform/           # The CLI's own bootstrap: setup, system, package, dev tooling
│   │   ├── processes/               # The student-facing workflow: module_workflow, milestone, benchmark, olympics, convert
│   │   └── tests/                    # The CLI's own test suite
│   └── dev_tools/                # Maintainer scripts (release, fresh-install verification)
│
├── user_data/                  # 🗃️ Your own progress, milestones, benchmarks (not committed)
│
└── tests/                      # ✅ Integration, e2e, and environment tests

Key workflow: data/src/*.py → data/modules/*.ipynb (you solve it) → data/trentorch/*.py

Credit Where It's Due

TrenTorch is our implementation, built on the curriculum and foundation of TinyTorch, created by Prof. Vijay Janapa Reddi and the ML Systems Book community at Harvard University. Full respect to the original project, we just wanted to take it further.

Related educational frameworks worth knowing about:

• tinygrad - George Hotz's minimalist framework
• micrograd - Andrej Karpathy's tiny autograd
• MiniTorch - Cornell's educational framework

We're addicted to making great software that runs (we're a bit of perfectionists ourselves) and is useful to humanity. This is our shot at making TinyTorch, but adding more stuff that is currently experimental and quite famous.

Team Engineers

Recomputed nightly from real issue/PR activity via .github/workflows/update-contributors.yml. Want to show up here? Open an issue or a PR: the first-contribution bot will say hello, and this grid picks you up on the next nightly run.

Aadityansha Maintainer Reducing CPU stalls, one commit at a time. Issues: 0 · PRs: 1

maanas1234 Core Engineer Catches bugs, builds solutions and ships products Issues: 11 · PRs: 12

Rocky Principal Maintainer IIT Guwahati, Debugs autograd for fun, ships before sunrise. Issues: 9 · PRs: 73

Shivtej Gaikwad Maintainer IIT Guwahati. Shows up, ships, moves on to the next thing. Issues: 0 · PRs: 7

License

MIT License - see LICENSE for details.