Control Operators for Interactive Character Animation

02/10/2025

This year at SIGGRAPH Asia we will be presenting Control Operators for Interactive Character Animation. Control Operators are a method for encoding arbitrary different inputs to Neural Networks in a way which is accessible to non-technical users. This allows users to design their own machine-learning-based character controllers, which we demonstrate on two different model types including a variation of Learned Motion Matching and a new flow-matching-based model.

Webpage • Paper • Supplementary Material • Video • Article • Code

Abstract: Neural-network-based character controllers are increasingly common and capable. However, the integration of desired control inputs such as joystick movement, motion paths, and objects in the environment, remains challenging. This is because these inputs often require custom feature engineering, specific neural network architectures, and training procedures. This renders these methods largely inaccessible to non-technical designers. To address this challenge, we introduce Control Operators, a powerful and flexible framework for specifying the control mechanisms of interactive character controllers. By breaking down the control problem into a set of simple operators, each with a semantic meaning for designers, and a corresponding neural network structure, we allow non-technical users to design control mechanisms in a way that is intuitive and can be composed together to train models that have multiple skills and control modes. We demonstrate their potential with two current state-of-the-art interactive character controllers - a Flow-Matching-based auto-regressive model, and a variation of Learned Motion Matching. We validate the approach via a user study wherein industry practitioners with varying degrees of ML and technical expertise explore the use of our system.