AI models for remote object detection are vulnerable to both physical and digital attacks, research finds

Today, our understanding and assessment of the physical characteristics of the world around us relies less and less on human intelligence and more and more on artificial intelligence.

Remote sensing (RS) technologies have become critical tools for government intelligence, environmental monitoring, autonomous transportation, urban planning and disaster management. But the vast number of images produced by remote cameras must be processed and interpreted, and these tasks are being delegated more and more to deep learning (DL) models.

DL models can process and interpret images much more quickly than humans can, and recent advances in AI have only improved this capacity over time. Despite this increase in efficiency, no study has ever attempted to assess the overall robustness and potential vulnerabilities of deep-neural-network (DNN)-based models used for object detection and image classification in RS images.

To address this issue, a team of scientists from Northwestern Polytechnical University and The Hong Kong Polytechnic University performed a review of all the existing research studies on the robustness of DL models used for object detection and classification using RS images and developed a benchmark to assess the performance of various DL model detectors (e.g., YOLO versions, RetinaNet, FreeAnchor). Critically, their analysis revealed several vulnerabilities of DL algorithms for object detection, which could be leveraged by attackers.

The team published their review in the Journal of Remote Sensing.

“We sought to address the lack of comprehensive studies on the robustness of deep learning models used in remote sensing tasks, particularly focusing on image classification and object detection. Our aim was to understand the vulnerabilities of these models to various types of noise, especially adversarial noise, and to systematically evaluate their natural and adversarial robustness,” said Shaohui Mei, professor in the School of Electronic Information at Northwestern Polytechnical University in Xi’an, China and lead author of the review paper.

More specifically, the team investigated the effects of natural noise and various attacks on model performance. The scientists used natural noise sources, including salt-pepper and random noise and rain, snow and fog, at different intensities to test the robustness of object detection and identification using DL models.

The team also tested model performance using various digital attacks to exploit vulnerabilities in models, including the Fast Gradient Sign Method (FGSM), AutoAttack, Projected Gradient Descent, Carlini & Wagner and Momentum Iterative FGSM. They also determined the effects of potential physical attacks, where a patch could be physically painted or attached to an object or object’s background to impair the DL model.

The researchers found many vulnerabilities in DL models that could be exploited by potential adversaries. “Deep learning models, despite their powerful capabilities in remote sensing applications, are susceptible to different kinds of disturbances, including adversarial attacks. It is crucial for developers and users of these technologies to be aware of these vulnerabilities and to work towards improving model robustness to ensure reliable performance in real-world conditions,” said Jiawei Lian, graduate student at the School of Electronic Information at Northwestern Polytechnical University and an author on the paper.

To help other researchers improve DL model robustness in these applications, the authors summarized the results of their analysis across various models, noise types and attacks:

• Training an adversarial attack shares many similarities with training a neural network and is affected by the same factors as model training, including training data, victim models (deep learning models used to generate adversarial attacks) and optimization strategies.
• Weak detectors, like YOLOv2, may only require the learning of limited information to successfully attack a DL model, but the attack generally won’t succeed with more robust detectors.
• Techniques such as “momentum” and “dropout” can boost the effectiveness of an attack. Investigation into training strategies and test augmentations could improve DNN model security.
• Physical attacks can be equally effective as digital attacks. Vulnerabilities in DL models must be translated into potential real-world applications, such as the attachment of a physical patch to compromise DL algorithms, that could exploit these weaknesses.
• Researchers can tease out the feature extraction mechanisms of DL models to understand how adversaries could manipulate and disrupt the process.
• The background of an object can be manipulated to impair a DL model’s ability to correctly detect and identify an object.
• Adversarial attacks using physical patches in the background of a target may be more practical than attaching patches to targets themselves.

The research team acknowledges that their analysis provides only a blueprint for improving RS DL model robustness.

“[Our] next step[s] involve further refining our benchmarking framework and conducting more extensive tests with a wider range of models and noise types. Our ultimate goal is to contribute to the development of more robust and secure DL models for RS, thereby enhancing the reliability and effectiveness of these technologies in critical applications such as environmental monitoring, disaster response, and urban planning,” said Mei.

Xiaofei Wang, Yuru Su and Mingyang Ma from the School of Electronic Information at the Northwestern Polytechnical University in Xi’an, China and Lap-Pui Chau from Department of Electrical and Electronic Engineering at The Hong Kong Polytechnic University also contributed to this research.

Provided by
Journal of Remote Sensing