We Need Robust Neural Networks
Primary Investigator:
Yung-Hsiang Lu
Daniel Merrick, Karthik Maiya, Kirthi Sivamani, Rui Wang
Abstract
Despite their recent success, deep convolutional object detection algorithms demonstrate poor detection stability on sequential image data. By looking at the detection results on neighboring frames in a video stream, it is easy to observe that these models inconsistently detect objects of interest. This problem is apparent despite there being little visual difference between the neighboring frames.
This project is focused on gaining a deep understanding of why object detection algorithms fail to consistently detect objects of interest on sequential image data. This is related to some work on adversarial attacks, which are attacks usually in the form of unnoticeable pixel perturbations to input images. These small but targeted perturbations can cause neural networks to completely misclassify the image. The fact that neural networks are not robust to small changes that are nearly imperceptible to humans introduces pressing security concerns for organizations with systems using this technology.
Our work is in its’ early stages, however, small-scale experiments have shown that specific features learned by the network are hyper-sensitive to small input perturbations. These sensitive features seem to be correlated with poor detection stability. Our methods are unique such that we are examining the sensitivity of specific features learned by the network where as other work focuses on the sensitivity of the output of the entire network. These results might suggest that introducing a regularization penalty on the gradients of the intermediary features may improve detection stability. Looking ahead, some goals we are focused on are (1) reproducing our results at large-scale, (2) gaining a deeper understanding of the types of features that tend to be more sensitive, and (3) investigating if this problem exists across many network architectures.