Cheat codes for LLM performance: An introduction to speculative decoding
Sometimes two models really are faster than one Hands on When it comes to AI inferencing, the faster you can generate a response, the better - and over the past few weeks, we've seen a number of announcements from chip upstarts claiming mind-bogglingly high numbers....