Many modern applications from camera arrays to federated learning depend on distributed collection, processing and communication of correlated data, requiring efficient compression techniques to minimize the induced communication overhead. While information-theoretic foundations of distributed compression are well-investigated, the impact of theory on practice has been somewhat limited. As the field of data compression is undergoing a significant transformation with the emergence of learning-based techniques, it is natural to ask whether machine learning can reap the benefits of distributed compression promised by information theory long ago.

In this talk we answer this question affirmatively by focusing on a simple distributed lossy compression setting, also known as the Wyner-Ziv problem, in which the decoder has direct access to correlated information that is unknown at the encoder. We show that for some well-studied source distributions, neural compression techniques mimic information theoretically optimal solutions such as “binning” or “grouping” in the source space as well as optimal combination of the quantization index and side information. These interpretable behaviors appear even though we neither impose a particular structure nor assume any prior knowledge about the source distributions. Binning is a widely-used tool in network information theory, and its emergence from data-driven learning can have implications beyond the setting considered in this talk.