Boffins detail new algorithms to losslessly boost AI perf by up to 2.8x
New spin on speculative decoding works with any model - now built into Transformers We all know that AI is expensive, but a new set of algorithms developed by researchers at the Weizmann Institute of Science, Intel Labs, and d-Matrix could significantly reduce the cost of serving up your favorite large language model (LLM) with just a few lines of code....