https://subhash.net/tags/moe/Recent content in MOE on Subhash DasyamHugoenSun, 13 Jul 2025 23:23:34 +0400https://subhash.net/posts/mixture-of-experts-moe-specialist/Sun, 13 Jul 2025 23:00:00 +0400https://subhash.net/posts/mixture-of-experts-moe-specialist/<p>Building on our transformer story - if you haven&rsquo;t read the complete transformer guide yet, check it out first!</p> <h2 id="remember-our-transformer-story">Remember Our Transformer Story?</h2> <p>In our previous deep dive, we learned that transformers have this amazing &ldquo;deep thinking step&rdquo; (the Feed Forward Network) where they:</p>