Neuralearn dotAI/Building LLMs like ChatGPT from scratch and Cloud Deployment

  • $49

Building LLMs like ChatGPT from scratch and Cloud Deployment

  • Course
  • 16 Lessons

This course is a deep, hands-on engineering journey to code a complete LLM—specifically, the highly efficient and powerful Mistral 7B architecture—from scratch in PyTorch. We bridge the gap between abstract theory and practical, production-grade code. You won't just learn what Grouped-Query Attention is; you'll implement it. You won't just read about the KV Cache; you'll build it to accelerate your model's inference.

Contents

Introduction

Course Introduction
Preview
What you'll learn
Preview
Colab Notebooks
Preview

Pre-requisites

RNNs and Attention Models
Preview
How the transformer works
Preview
Difference in training and inference
Preview

Building Mistral from scratch

Global Architecture of Mistral
Preview
Tokenization
Preview
Rotary Positional Encoding (RoPE)
Preview
RoPE Practice
Group Query Attention
Sliding Window Attention
Kv-caching
Transformer Block
Full Transformer Model

Deploying Mistral to the cloud (Runpod)

Deployment