The Center for Education and Research in Information Assurance and Security (CERIAS)

The Center for Education and Research in
Information Assurance and Security (CERIAS)
Center for Education and Research in Information Assurance and Security

Reusing Deep Learning Models: Challenges and Directions in Trustworthy Software Engineering

Research Areas: Artificial Intelligence and Machine Learning

Principal Investigator: Jamie Davis

The development and training of deep learning models have become increasingly costly and complex. Consequently, software engineers are adopting pre-trained models (PTMs) for their downstream applications. The content, dynamics, and effective use of the PTM supply chain remain largely unexplored. This project seeks to characterize the associated engineering processes and artifacts, in order to identify and mitigate failure modes. We then develop tools, e.g. automation, to optimize the secure reuse of PTMs. We integrate research methods from human factors, mining software repositories, and machine learning. We are focused on major model registries such as HuggingFace and PyTorchHub, and interoperability infrastructure such as Pickle (pickle deserialization attacks) and the Open Neural Network eXchange (ONNX -- converter degradation challenges).

This project's goal is to accelerate secure software engineering work with PTMs. For example, we have built:

- The first typosquatting detector targeting pre-trained models, also achieving state-of-art detection performance on other registries such as NPM and PyPI

- The first secure loader for Pickle-based ML models

- The first analysis tool to detect incorrectly named ML models (eg backdoors)

Personnel

Other PIs: Yung-Hsiang Lu (Purdue) George K Thiruvathukal (Loyola University Chicago)

Students: Purvish Jajal, PhD student Nick Eliopoulos, PhD student Parth Patil, MSc student ~20 Purdue undergraduate students from several majors, through the VIP program

Representative Publications

  • Exploring Naming Conventions (and Defects) of Pre-trained Deep Learning Models in Hugging Face and Other Model Hubs.
    Jiang, Cheung, Thiruvathukal, and Davis.
    arXiv 2023.

  • AgentHub: A Research Agenda for Agent Sharing Infrastructure E Pautsch, T Singla, W Jiang, H Peng, B Hassanshahi, K Läufer, G.K. Thiruvathukal, J.C. Davis

  • Software Dependencies 2.0: An Empirical Study of Reuse and Integration of Pre-Trained Models in Open-Source Projects

    J Yasmin, W Jiang, JC Davis, Y Tian
  • AgentHub: A Research Agenda for Agent Sharing Infrastructure E Pautsch, T Singla, W Jiang, H Peng, B Hassanshahi, K Läufer, G.K. Thiruvathukal, J.C. Davis

  • Reusing Deep Learning Models: Challenges and Directions in Software Engineering.
    Davis, Jajal, Jiang, Schorlemmer, Synovic, and Thiruvathukal.
    Proceedings of the IEEE John Vincent Atanasoff Symposium on Modern Computing (JVA’23) 2023.

  • Discrepancies among Pre-trained Deep Neural Networks: A New Threat to Model Zoo Reliability.
    Montes, Peerapatanapokin, Schultz, Guo, Jiang, and Davis.
    Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering: Ideas, Visions, and Reflections track (ESEC/FSE-IVR) 2022.

  • Analysis of Failures and Risks in Deep Learning Model Converters: A Case Study in the ONNX Ecosystem.
    Jajal, Jiang, Tewari, Woo, Lu, Thiruvathukal, and Davis.
    arXiv 2023.

  • An Empirical Study of Artifacts and Security Practices in the Pre-trained Model Supply Chain.
    Jiang, Synovic, Sethi, Indarapu, Hyatt, Schorlemmer, Thiruvathukal, and Davis.
    Proceedings of the 1st ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses (SCORED) 2022.

  • An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry.
    Jiang, Synovic, Hyatt, Schorlemmer, Sethi, Lu, Thiruvathukal, and Davis.
    Proceedings of the ACM/IEEE 45th International Conference on Software Engineering (ICSE) 2023.

  • PickleBall: Secure Deserialization of Pickle-based Machine Learning Models

    AD Kellas, N Christou, W Jiang, P Li, L Simon, Y David, VP Kemerlis, ...
     
    CCS 2025

  • Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses

    M Ahmed, M Abdelmouty, M Kim, G Kandula, A Park, JC Davis
     
    SecDev'25-Poster

  • AI Safety in the Eyes of the Downstream Developer: A First Look at Concerns, Practices, and Challenges

    H Gao, M Zahedi, W Jiang, HY Lin, J Davis, C Treude

  • ConfuGuard: Using Metadata to Detect Active and Stealthy Package Confusion Attacks Accurately and at Scale

    W Jiang, B Çakar, M Lysenko, JC Davis
  •  
    Parth V Patil, Wenxin Jiang, Huiyun Peng, Daniel Lugo, Kelechi G Kalu, Josh LeBlanc, Lawrence Smith, Hyeonwoo Heo, Nathanael Aou, James C Davis
     
    SERP4IoT'25

Keywords: machine learning, reproducibility, software engineering