Faster and Lighter Model Inference with ONNX Runtime from Cloud to Client
19:55
Faster and Lighter Model Inference with ONNX Runtime from Cloud to Client
Aug 3, 2022
Microsoftmarkdefalco
Quantization: What Everyone Gets Wrong (Accuracy Myths)
0:44
Quantization: What Everyone Gets Wrong (Accuracy Myths)
65 views3 weeks ago
YouTubeCode & Capital
ArmVision Assist – Offline Action Agent for ARM Mobile
1:17
ArmVision Assist – Offline Action Agent for ARM Mobile
13 views5 months ago
YouTubeShyam Sharma
FP32 FP16 FP8 TENSOR INT #chatgpt #llm #google #tech #ytshorts #yt #youtube #youtubeshorts
0:16
FP32 FP16 FP8 TENSOR INT #chatgpt #llm #google #tech #ytshorts #yt #youtube #youtubeshorts
61 views1 month ago
YouTubeAmit_Chopra_assruc
Google magic bullet - TurboQuant #ai #gpu #google #chips #cuda #quantization
0:41
Google magic bullet - TurboQuant #ai #gpu #google #chips #cuda #quantization
1.3K views1 month ago
YouTubeNeural AI Flair
What is Quantization in AI? How you Run Models on Your Laptop| #ai #maths #shorts #yt #quantization
2:18
What is Quantization in AI? How you Run Models on Your Laptop| #ai #maths #shorts #yt #quantization
1 month ago
YouTubeHarsh Shukla
Tikhomirov M.M. - Training of large language models - 8. Inference, quantization
1:08:05
Tikhomirov M.M. - Training of large language models - 8. Inference, quantization
218 views2 weeks ago
YouTubeteach-in
6:29
Inference Optimization: Making AI Faster & Cheaper (Latency, Throughput & GPUs)
56 views1 month ago
YouTubewecite
0:45
Quantization Explained: How LLMs Get Smaller and Faster
88 views1 month ago
YouTubeDev Alpha Lab
7:29
Model Quantization Explained 8 bit, 4 bit & Inference Optimization #genai #aigenerated
32 views2 months ago
YouTubeSmartSkale
2:36
I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x.All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework.On a 4-token prompt with 252 generated tokens:- Original: 0.76 tok/s- KV cache fp32: 27.21 tok/s- KV cache int8 (quantized): 27.29 tok/sTry it out yourself here: https://t.co/kFS9Z0fs4hIn practice:- KV caching gave us about a 35x end-to-end speedup- INT8 KV cache kept roughly the same speed as fp32 but cut KV cac
48.8K views3 weeks ago
x.comReese Chong
0:23
This is the clearest explanation of how LLM quantization works:It lets engineers compress a model by 4x and run it 2x faster without quality loss.I stumbled on this piece by Sam Rose and honestly wish I had it when I first tried to understand quantization.He breaks it down from absolute zero - bits, floats, how weights are stored - all the way to actually benchmarking quantized models. With interactive demos you can play with.Here's the core idea in 60 seconds:→ An LLM is billions of numbers (we
7K views2 weeks ago
x.comSukh Sroay
Z-Image-Turbo INT8 — AI Playground & API - deAPI.ai
3 weeks ago
deapi.ai
Model Precision and Deployment Choices in Object Detection | Mohammad Zaid posted on the topic | LinkedIn
4 views3 weeks ago
linkedin.com
Edge ML Development for i.MX Processors
Aug 21, 2024
nxp.com
41:59
Sampling Theorem Quantization and Binary Coding
7.1K viewsApr 11, 2021
YouTubeEngineering with Bingabr
9:58
SmoothQuant
4.4K viewsOct 25, 2023
YouTubeMIT HAN Lab
0:51
Quantization explained
504 views3 months ago
YouTubeChip Talks AI
9:57
What is LLM Quantization ?
3.2K viewsMar 19, 2025
YouTubeNew Machina
2:11
NVIDIA Tesla T4 Introduction to Inference
3.7K viewsApr 18, 2019
YouTubeBoston Limited
1:16:40
Lecture 30: Quantized Training
3.3K viewsOct 7, 2024
YouTubeGPU MODE
12:10
Optimize Your AI - Quantization Explained
465.1K viewsDec 28, 2024
YouTubeMatt Williams
1:36
What Is Quantization? | Decoding LLM File Names
1.3K views4 months ago
YouTubeAnaconda, Inc.
1:01
Towards Unified INT8 Training for Convolutional Neural Network
803 viewsJul 17, 2020
YouTubeComputerVisionFoundation Videos
7:14
What Are Weights in AI Models
381 views3 months ago
YouTubeCloudProInc
4:42
Optimize LLMs for faster AI inference
434 views3 months ago
YouTubeRed Hat
18:58
From FP32 to INT8: Post-Training Quantization Explained in PyTorch
928 views6 months ago
YouTubeMLWorks
57:10
Lec 30 | Quantization, Pruning & Distillation
6.8K viewsMar 23, 2025
YouTubeNPTEL IIT Delhi
6:59
Boosting Model Performance with Quantization Techniques
7 views7 months ago
YouTubeNextGen AI Explorer
1:11:43
Lecture 05 - Quantization (Part I) | MIT 6.S965
19.2K viewsSep 22, 2022
YouTubeMIT HAN Lab
See more