https://subhash.net/tags/inference/Recent content in Inference on Subhash DasyamHugoenSun, 13 Jul 2025 23:23:34 +0400https://subhash.net/posts/mixture-of-experts-moe-specialist/Sun, 13 Jul 2025 23:00:00 +0400https://subhash.net/posts/mixture-of-experts-moe-specialist/<p>Building on our transformer story - if you haven&rsquo;t read the complete transformer guide yet, check it out first!</p> <h2 id="remember-our-transformer-story">Remember Our Transformer Story?</h2> <p>In our previous deep dive, we learned that transformers have this amazing &ldquo;deep thinking step&rdquo; (the Feed Forward Network) where they:</p>https://subhash.net/posts/complete-guide-to-llm-inference-servers/Mon, 26 May 2025 19:55:00 +0400https://subhash.net/posts/complete-guide-to-llm-inference-servers/<hr> <h2 id="introduction-why-inference-servers-matter">Introduction: Why Inference Servers Matter</h2> <p>Imagine you&rsquo;ve trained the perfect AI model that can answer any question, write code, or help with complex reasoning. But there&rsquo;s a catch: it takes 30 seconds to respond to each query, can only handle one user at a time, and requires expensive hardware that costs $50,000 per month to run.</p>