Skip to main content

Inference

Inference

2 articles tagged with “Inference”

GPU4LLMs3AI2Systems Architecture2AI Agents2CUDA2Inference2Browser Engineering1Security1Rendering1Low-Level Systems1WebAssembly1Accessibility1Networking1QUIC1Software Engineering1Benchmarks1Architecture1Governance1AI Security1LLM1Framework1QSAF1Prompt Injection1RAG1Data Governance1Cybersecurity1Virtual Memory1MMU1Page Faults1Kernel Optimization1Memory Architecture1FlashAttention1
When Your LLM Trips the MMU: Page Faults, TLB Shootdowns, and the Hidden Virtual-Memory Tax of AI Inference
LLMsGPUVirtual MemoryCUDAInferenceMMUPage FaultsSystems Architecture

When Your LLM Trips the MMU: Page Faults, TLB Shootdowns, and the Hidden Virtual-Memory Tax of AI Inference

A distinguished-architect deep dive into GPU virtual memory internals, MMU fault pipelines, TLB shootdown mechanics, page-table walks, Unified Memory/HMM coherence, ATS, and why page migration turns your p99 into a hardware problem nobody on the team budgeted for.

Hazem Ali
Hazem Ali··45 minutes read
Kernel Dynamics: The Real Bottleneck of AI
LLMsGPUKernel OptimizationMemory ArchitectureCUDAInferenceFlashAttention

Kernel Dynamics: The Real Bottleneck of AI

Why LLM inference speed is dominated by kernel execution, memory traffic, and runtime scheduling — not raw FLOPS. A deep technical guide to prefill vs decode, the Roofline model, memory walls, FlashAttention, KV cache paging, warp mechanics, and GPU pipeline design.

Hazem Ali
Hazem Ali··35 min read