FireDM : a weakly-supervised approach for massive generation of multi-scale and multi-scene fire segmentation datasets

Abstract

Data availability and quality are crucial for the development of semantic segmentation techniques. However, creating high-quality fire scene datasets in a safe and efficient manner remains an unsolved challenge. To fill this gap, we introduce FireDM, the first method to generate unlimited fire segmentation datasets at virtually no cost. FireDM takes full advantage of the combined strengths of a combination of pre-trained diffusion models (Stable Diffusion XL 1.0 and Stable Diffusion 2.1) and text-guided diffusion using ChatGPT4-Fire to generate multi-scale and detail-rich fire images. The innovative fire-decoder module in FireDM then efficiently converts the cross-attention and multi-scale feature maps obtained during diffusion into accurate segmentation masks. This process requires only about 100 images and their corresponding segmentation masks for training. In our experiments, we trained the segmentation algorithms using the large-scale segmentation dataset generated by FireDM and all publicly available fire segmentation datasets respectively, and found that the segmentation algorithms trained with the former dataset outperformed the latter by at least 5% or more in terms of IoU, accuracy, F1-score and AP. This demonstrates the capability of FireDM in expanding a limited fire segmentation dataset. Additionally, the datasets generated by FireDM, with their multiple image resolutions, can adapt to the input sizes of different segmentation algorithms, significantly reducing information loss caused by resizing the image (e.g., cropping and scaling). Finally, we have created the world's first high-quality fire segmentation dataset benchmark using FireDM. The complete code and dataset of FireDM are publicly available at https://github.com/ZhengHongtao2001/FireDM.