))
Kyle Kranen: End Points, Optimizing LLMs, GNNs, Foundation Models - AI Portfolio Podcast #011
Manage episode 445844977 series 3596668
Inhoud geleverd door Mark Moyou, PhD and Mark Moyou. Alle podcastinhoud, inclusief afleveringen, afbeeldingen en podcastbeschrijvingen, wordt rechtstreeks geüpload en geleverd door Mark Moyou, PhD and Mark Moyou of hun podcastplatformpartner. Als u denkt dat iemand uw auteursrechtelijk beschermde werk zonder uw toestemming gebruikt, kunt u het hier beschreven proces https://nl.player.fm/legal volgen.
Get 1000 free inference requests for LLMs on build.nvidia.com
Kyle Kranen, an engineering leader at NVIDIA, who is at the forefront of deep learning, real-world applications, and production. Kyle shares his expertise on optimizing large language models (LLMs) for deployment, exploring the complexities of scaling and parallelism.
📲 Kyle Kranen Socials:
LinkedIn: / kyle-kranen
Twitter: https://x.com/kranenkyle
📲 Mark Moyou, PhD Socials:
LinkedIn: / markmoyou
Twitter: / markmoyou
📗 Chapters
[00:00] Intro
[01:26] Optimizing LLMs for deployment
[10:23] Economy of Scale (Batch Size)
[13:18] Data Parallelism
[14:30] Kernels on GPUs
[18:48] Hardest part of optimizing
[22:26] Choosing hardware for LLM
[31:33] Storage and Networking - Analyzing Performance
[32:33] Minimum size of model where tensor parallel gives you advantage
[35:20] Director Level folks thinking about deploying LLM
[37:29] Kyle is working on AI foundation models
[40:38] Deploying Models with endpoints
[42:43] Fine Tuning, Deploying Loras
[45:02] SteerLM
[48:09] KV Cache
[51:43] Advice for people for deploying reasonable and large scale LLMs
[58:08] Graph Neural Networks
[01:00:04] GNNs
15 afleveringen
Delen
