by jake on (#72TSC)
Lossless data compression is an important tool for reducing the storagerequirements of the world's ever-growing data sets. Yann Collet developedthe LZ4algorithm and designed the Zstandard (or Zstd)algorithm; he came to the 2025Open Source Summit Japan in Tokyo to talk about where data compressiongoes from here. It turns out that we have reached a point wheregeneral-purpose algorithms are only going to provide limited improvement;for significant increases in compression, while keeping computation costswithin reason for data-center use, turning to format-specific techniqueswill be needed.