Interactive courses for engineers and tech professionals

The skills you need in the AI Era

Runtime Visuals turns the invisible internals of software into simulators you drive step by step.

No 600-page books. No 40-hour courses. A focused course takes an evening, and what you learn stays.

Explore Courses View Pricing

One subscription. Every course. From $99/year. Cancel anytime.

Courses view all →

How LLMs Actually Work

From tokenized text to trained parameters and next-token generation

next-token prediction tokenization self-attention open →

Engineering AI Agents

Bounded tool-using loops with durable state, guardrails, and evals

ai agents agent loop tool governance open →

MCP for Engineers

Build stateless servers with typed tools, live context, and secure access

mcp stateless protocol typed tools open →

System Design Fundamentals

Turn requirements into explicit, defensible architecture trade-offs

system design trade-offs capacity estimation open →

Inside Kubernetes

Run resilient workloads across scheduling, networking, storage, and failures

kubernetes pods scheduling open →

Modern Authentication

Sessions, passkeys, delegated authorization, tokens, and single sign-on

authentication oauth passkeys open →

every course: simulators you drive, a quiz that checks you, a certificate that proves it

Product tour

One lesson, five ways in.

Watch it run. A short narrated animation shows the system in motion first.

Read the model. Tight, visual explanations. Hover any part of a diagram for the detail.

Drive the simulator. Step through the real computation yourself and check every number.

Lock it in. Fast true-or-false calls on the ideas interviews actually test.

Check yourself. Match, order and choose, with instant feedback on every answer.

Prove it. Pass the final quiz and share a certificate anyone can verify.

runtimevisuals.com/course/how-llms-work-tokens-attention-generation/lesson/self-attention#video

How LLMs Actually Work

From tokenized text to trained parameters and next-token generation

Introduction

1 What an LLM Reads: Text to Tokens

2 From Embeddings to Contextual Representations

3 Self-Attention Under the Hood

4 Learning by Predicting the Next Token

5 How Loss Becomes Learning

6 From Base Model to Assistant

7 Generate One Token at a Time

8 Generation Limits and Reliability

Bonus

Get Certified

2/8 completed 8/8 completed 25% 100%

3/8

Ask AI

Video Theory Lab Gist Quiz

Self-Attention Under the Hood

The robot dropped the ball because it was round

it, after attention 0.71 ball + 0.12 robot + 0.17 rest

Resolved: it now carries the meaning of ball.

0:13 / 2:38 1×

Take the sentence: the robot dropped the ball because it was round. The vector for it is generic. Self-attention is the calculation that resolves this.

Self-Attention Under the Hood

Inside a transformer layer every token position holds a vector, and self-attention lets each position pull in information from the positions before it: score them, turn the scores into weights, and take a weighted average of what they offer.

Every Position Asks, Advertises, and Offers

Attention treats every token as playing three roles at once. Think of a code review: you look for where a variable was defined, every earlier line summarizes what it holds, and you read the line that matches.

Three learned projections of one token

Query

what am I looking for?

Key

what do I advertise?

Value

what I hand over

weights = softmax(q · k^T / √d) · output = Σ weight × v

Scoring Relevance with Scaled Dot Products

The score between two positions is the dot product of one query and one key. Vectors that point the same way give a large number; perpendicular ones give zero. Dividing by the square root of the dimension keeps the scores in a range where softmax still makes a choice.

score = dot(q_it, k_dog) / sqrt(2) # 4 / 1.41 = 2.83

Self-Attention Under the Hood

Step 3

Score one query against every key

Take the query of "it" and dot it with each key: the, dog, chased, it. A bigger number means a better match.

Step 4

The full score row

Dog's key aligns with the query and scores 2.83. "it" partially matches itself at 1.41. Every position computes a row like this, all at once.

Step 5

The causal mask

A token may only look back. Scores for later positions become −∞, so softmax gives them exactly zero weight.

3/9 4/9 5/9 Score one query against every key The full score row The causal mask

Row 4: the query of "it" against every key Row 4 filled: dog scores 2.83 Future positions masked before softmax

Attention map (scaled scores)

the (k) dog (k) chased (k) it (k) the (q) ? 0.00 ? −∞ ? −∞ ? −∞ dog (q) ? 0.00 ? 2.12 ? −∞ ? −∞ chased (q) ? 0.00 ? 3.54 ? 1.41 ? −∞ it (q) ? 0.00 ? 2.83 ? 0.00 ? 1.41

Scaled dot-product score s_4j = q₄·k_j / √d_k

Causal mask j > i → −∞ → weight 0

Self-Attention Under the Hood

The quickest prep for your next interview

Card 1 · True or false?

In self-attention, a token's query is dot-producted with every visible key, and the resulting weights decide how much of each value flows into its new representation.

Gut call: does it hold up?

Sounds right Smells wrong

Holds up. Scores come from query times key; softmax turns them into weights over the values.

Card 2 · True or false?

A token's attention weights can add up to more than 1 when several keys match it well.

Gut call: does it hold up?

Sounds right Smells wrong

Right call. Softmax normalizes every row, so the weights always sum to exactly 1.

Next: Quiz

Self-Attention Under the Hood

Match

Attention runs for the token refund in the customer requested a refund. Match each description to what it refers to.

Encodes what refund is looking for among the earlier tokens

Query vector Key vector Value vector Attention weight

Advertises what customer offers, so other tokens can judge its relevance

Query vector Key vector Value vector Attention weight

Carries the content that gets blended into the updated refund token

Query vector Key vector Value vector Attention weight

Check matches 3/3 correct

Query asks, key advertises, value is what gets mixed in. The weight itself is softmax(q · k / √d).

Get Certified: How LLMs Actually Work

Certification quiz

Passed

Questions from all 8 lessons

Runtime Visuals

Certificate of Completion

Course certification

This is to certify that

Alex Rivera

has demonstrated a working understanding of

How LLMs Actually Work

DateOct 16, 2026 Score92% IssuerRuntime Visuals

runtimevisuals.com/certificate/7f3c9a21

Copy link Link copied Print

The first lessons of every course are free.

Create free account Explore Courses

The AI Era Demands More

Become the Engineer Who Supervises AI

As AI generates more code, understanding what that code does becomes more valuable, not less. Someone must verify AI output, debug failures, and make architectural decisions.

Build Your Architectural Edge

01 Where it pays off

Understanding that shows up when it counts.

interviews

Ace your next technical interview

The "how does it work" questions senior interviews ask, answered by running the system yourself.

Browse Courses →

teams

Level up the whole team

Invite your team and see each member's progress. Per-seat billing, cancel anytime.

See team plans →

free

Start free

The first 2 lessons of every course are free, every tab included.

Create free account →

03 Pricing

One plan. Everything included.

Every course, every simulator, every certificate, and all future content. No tiers, no locked lessons, no upsells.

Individual

Full access for one developer

$15 / MONTH

Billed monthly. Cancel anytime. Switch to yearly whenever you want.

01 All deep-dive courses
02 Interactive simulators
03 Concept lessons with mental models
04 Practice quizzes per lesson
05 Reference cheatsheets
06 New courses as released

Start Learning Now

Team

For teams

Give your whole team deep technical skills

$129 / SEAT / YEAR $259 Now 50% off

You're seat #1. Add members anytime. Cancel anytime.

01 Everything in Individual
02 Centralized team billing
03 Add & remove members anytime
04 Track courses per team
05 Seats shared across your teams
06 Volume grows with your team

Start a Team

Secure checkout via Stripe. Certificates stay valid after cancellation.

faq Questions, answered

01 What exactly are the 'interactive simulators'? +

Our simulators are custom-built Vue.js applications that visualize invisible system mechanics in real-time. You can step through an event loop tick-by-tick, watch memory allocation on the heap, trace prototype chains, or see exactly how the browser renders a page. It's experiential learning that text explanations and videos simply cannot replicate.

02 Is this content suitable for beginners? +

This platform is designed for developers who already have foundational knowledge and want to go deeper. We don't teach syntax or framework basics, we teach how things work under the hood. If you're comfortable writing code but want to understand what's actually happening at runtime, this is for you.

03 How is this different from YouTube tutorials or Udemy courses? +

Most courses show you what to type. We show you why it works. Our interactive simulators let you visualize concepts like memory management, rendering pipelines, and authentication flows in ways that passive video watching cannot achieve. Plus, our content is verified for accuracy, no AI hallucinations or outdated tutorials.

04 Will AI make this knowledge obsolete? +

Quite the opposite. As AI generates more code, the ability to understand, debug, and architect systems becomes more valuable, not less. Someone needs to verify AI output, fix production issues at 3 AM, make architectural decisions, and handle security concerns. We create engineers who can effectively supervise AI, not compete with it.

05 What does each course include? +

Each course contains 5-16 micro-lessons with a mix of: (1) Concept lessons with rich diagrams and mental models, (2) Interactive lab simulators you can step through, (3) Practice quizzes to test understanding, and (4) A comprehensive cheatsheet for quick reference. No 40-hour video marathons, just focused, deep content.

06 What's included in the subscription? +

Your subscription gives you full access to all current courses and any new courses we release. This includes all interactive simulators, concept lessons, quizzes, and reference cheatsheets. We regularly add new courses on topics like JavaScript internals, browser rendering, authentication, and system design.

07 Is my payment secure? +

Absolutely. We use Stripe, a trusted payment platform, to process all transactions. Your payment information is never stored on our servers. We support credit cards and various payment methods depending on your region.

08 Can I cancel my subscription anytime? +

Yes, you can cancel your subscription at any time from your account dashboard. You'll retain access until the end of your current billing period. No questions asked, no complicated cancellation process.

09 Do you offer team or enterprise plans? +

Yes! We offer team licenses for companies who want to upskill their engineering teams. Contact us to discuss volume pricing and custom arrangements for your organization.

Still have questions?

Visit the Help Center

Start today

You don't need 40 hours to understand something deeply. You need to see it run.

One subscription. Every course, every simulator, every certificate, and everything we publish next.

Start Learning Now Browse Courses

COURSES

All included

FORMAT

Simulators

COMMITMENT

Cancel anytime