DL-Structure

Blogs from Renjie Liao's Lab


Test-Time Steering for Lossless Text Compression via Weighted Product of Experts

When I was a child, I always wondered: if I keep compressing the same file, will it eventually shrink to nothing? Of course, the answer is no—once a file is optimally compressed by a lossless compressor, compressing it again with the same method gives a file of exactly the same size. Today I know this comes from the fundamental limits of lossless compression in information theory. But what if we use multiple compressors instead of one? If we combine them, can each remove a different part of the data’s redundancy—and how should such a combination be designed? In this blog we discussed the above questions and proposed a method called Weighted Product of Experts.

1 min read   ·   2025

Why the Exponential? From Max‑Entropy RL to the Boltzmann Distribution

This blog post explores why the exponential function appears ubiquitously across modern RL, energy-based modeling, and statistical mechanics. We examine the connection between max-entropy reinforcement learning and the Boltzmann distribution, uncovering the fundamental principles that make the exponential form inevitable and explaining what "temperature" actually does in these frameworks.

1 min read   ·   2025

A Unified Framework for Diffusion Distillation

The explosive growth in one-step and few-step diffusion models has taken the field deep into the weeds of complex notations. In this blog, we cut through the confusion by proposing a coherent set of notations that reveal the connections among these methods.

22 min read   ·   2025

On the Permutation Invariance of Graph Generative Models

This blog post discusses the permutation invariance principle of graph generative models, which has often been taken for granted in graph-related tasks. While permutation symmetry is an elegant property of graph data, there is still more to learn about its empirical implications.

20 min read   ·   2025

Conditional Generative Models for Motion Prediction

In this blog post, we discuss good engineering practices and the lessons learned—sometimes the hard way—from building conditional generative models (in particular, flow matching) for motion prediction problems.

12 min read   ·   2025

Evaluating Motion Consistency by Fréchet Video Motion Distance (FVMD)

In this blog post, we introduce a promising new metric for video generative models, Fréchet Video Motion Distance (FVMD), which focuses on the motion consistency of generated videos.

20 min read   ·   2024

Flow Matching (Part 1)

This is the beginning in a series of blog posts about flow matching and its generalizations. We start by covering normalizing flows, the foundation for flow matching methods.

21 min read   ·   2024

Flow-based Models and Flow Matching

An introduction to flow matching.

60 min read   ·   2024

Flow Matching (Part 2)

This is part two in a series of blog posts about flow matching. In this post, we dig deep into continuous normalizing flows as they form the basis for flow matching methods. We discuss the benefits but also the drawbacks of continuous normalizing flows.

15 min read   ·   2024