Navigating the unseen : out-of-distribution generalization and detection in open environments

Abstract

In open environments, artificial intelligence (AI) models face two main types of out-of­-distribution (OOD) samples that deviate from their training data (i.e., in-distribution data): covariate-shifted OOD samples, which are consistent in semantics but differ in covariate shifts, and semantic-shifted OOD samples, which have different semantic labels. Such OOD samples can severely challenge the safety and reliability of AI sys­tems by inducing high-confidence errors. Comprising four studies, this thesis targets enhancing generalization to covariate shifts through methods like style augmentation and memory networks, and improving detection of semantic-shifted samples using strategies such as prompt tuning and adaptive negative proxies. These efforts are crucial for the reliable performance of AI models in open environments. In Chapter 1, we introduce the concepts of covariate-shifted and semantic-shifted OOD samples and review existing methods and challenges associated with OOD gen­eralization and detection. We detail the objectives, contributions, and the structure of the thesis. In Chapter 2, we introduce Exact Feature Distribution Matching (EFDM), a novel technique that advances style augmentation by integrating higher-order statis­tics for enhanced generalization to covariate-shifted OOD samples. EFDM employs empirical Cumulative Distribution Functions and a Sort-Matching technique, demon­strating superior performance over traditional methods in extensive experiments. In Chapter 3, we develop the dual memory networks to extend the generalization capabil­ities of vision-language models (VLMs) like CLIP. This strategy significantly improves performance on both in-distribution and covariate-shifted OOD samples, validated through rigorous testing across a variety of datasets. Moving forward, Chapters 4 and 5 focus on detecting semantic-shifted OOD samples. In Chapter 4, we intro­duce Label-driven Automated Prompt Tuning (LAPT) to address the limitations of manual prompt engineering in VLMs-based OOD detection. Using distribution-aware prompts and automatically collected negative training data, LAPT reduces manual effort and improves detection performance across various tasks. In Chapter 5, we fo­cus on constructing adaptive negative proxy with test images in a test-time adaption manner. This approach facilitates online mining of negative test samples, enhanc­ing the model’s ability to distinguish between in-distribution and OOD instances, as proven on standard benchmarks. In summary, this thesis contributes to the field of OOD generalization and detection by introducing innovative methods that enhance performance and reduce manual intervention. By addressing specific challenges associated with covariate and semantic shifts in OOD samples, these studies significantly improve the reliability and safety of AI systems in open environments.