Deep model reuse : paving the way for efficient and generalizable AI systems

Abstract

Humans easily apply learned skills to different situations, a flexibility that AI systems still struggle to achieve. Current AI models are often confined to their training setup, leading to isolated developments and a narrow scope of application. This largely restricts the creation of flexible and general-purpose AI systems. Deep Model Reuse presents a novel solution. Imagine tapping into a vast library of pre-trained models, each a master in its specialized domain. Our approach re-purposes these existing models, extracting and transforming their knowledge for the development of novel AI systems. In this talk, we explore the essential techniques of this transformative process, highlighting the shift towards versatile and efficient AI that mirrors human cognition's adaptability. We introduce three foundational pillars of deep model reuse: understanding, composing, and refining. First, we investigate the internal behavior of neural networks—using language models as explainers and analyzing the representation space of diffusion models—to uncover how and what models have learned. Second, we develop methods to transform and compose models through weight mapping, knowledge distillation, and model dissection, enabling the creation of new capabilities by reassembling existing expertise. Third, we enhance reliability by editing model behaviors and mitigating biases, ensuring robustness in complex and dynamic environments. We demonstrate the power of this paradigm in generative AI, where model reuse leads to efficient diffusion models free from spectral bias, improved compositional understanding in video generation, and the repurposing of 2D/3D models for 3D/4D content creation. By shifting from training from scratch to intelligently reusing and recombining models, we move closer to adaptive, scalable, and human-like AI systems—ushering in a new era of sustainable and general intelligence.