Together AI's ATLAS adaptive speculator delivers 400% inference speedup by learning from workloads in real-time

This technique (called speculative decoding) has become essential for enterprises trying to reduce inference costs and ...