Padetbench : towards benchmarking texture- and patch-based physical attacks against object detection

Abstract

Physical attacks against object detection have gained significant attention due to their practical implications. However, conducting physical experiments is time-consuming and labor-intensive, and controlling physical dynamics and cross-domain transformations in the real world is challenging, leading to inconsistent evaluations and hindering the development of robust models. To address these issues, we rigorously explore realistic simulations to benchmark physical attacks under controlled conditions. This approach ensures fairness and resolves the problem of capturing strictly aligned adversarial images, which is challenging in the real world. Our benchmark includes 23 physical attacks, 48 object detectors, comprehensive physical dynamics, and evaluation metrics. We provide end-to-end pipelines for dataset generation, detection, evaluation, and analysis. The benchmark is flexible and scalable, allowing easy integration of new objects, attacks, models, and vision tasks. Based on this benchmark, we generate comprehensive datasets and perform over 8000 evaluations, including overall assessments and detailed ablation studies. These experiments provide detailed analyses from detection and attack perspectives, highlight limitations of existing algorithms, and offer revealing insights. The code and datasets are publicly available at https://github.com/JiaweiLian/PADetBench.