You already know that Speculative Decoding reduces LLM inference time, don't you? (If you don't, watch shot #14 to learn how Speculative Decoding works!) The magic behind Speculative Decoding lies in leveraging a smaller draft model. But what if you don't have a smaller model on hand? In this shot you'll learn how to achieve a 10% speedup via Speculative Decoding, by simulating a smaller model using the layers of the big model! This shot was filmed in front of a live audience 👥👥, as part of the One Shot Learning’s 2nd Anniversary Fest 🎉 (https://www.oneshotlearning.io/anniversary-fest-2024). The audience was thrilled with this festive event (just as we were). Want to know about future events? Join our Telegram community! https://t.me/one_shot_learning