Nasir Memon - NYU Tandon
Students: Spring 2024, unless noted otherwise, sessions will be virtual on Zoom.
AI, Computational Imaging and the Battle for Media Integrity
Jul 28, 2021
Download:
MP4 Video Size: 716.1MB
Watch on YouTube
Abstract
Rapid progress in machine learning, computer vision and graphics leads to successive democratization of media manipulation capabilities. While convincing photo and video manipulation used to require substantial time and skill, modern editors bring (semi-) automated tools that can be used by everyone. Some of the most recent examples include manipulation of human faces, e.g., by their replacement or semantic manipulation (expression, age, etc.). At the same time, dissemination of fake news and misinformation campaigns are picking up speed which challenges trust in the society. Our media distribution platforms lack content integrity features as they were designed and optimized for the quality of (human) experience with strict bandwidth / storage constraints. Such an approach fails to recognize an increasing role of automated analysis by machine learning models, e.g, strong lossy compression applied to media assets removes imperceptible statistical traces indicative of content manipulation and is often referred to as media "laundering" process. In this talk, we explore end-to-end optimization of photo acquisition and distribution pipelines for media authentication. We show that feedback from forensic analysis can be used to optimize upstream components like the camera ISP or lossy compression codecs to support media authentication on the receiving end. Modern machine learning tools allow us to discover new approaches to the problem with surprising connections to other fields like information hiding, computational photography, lossy compression and machine learning security. To enable this line of work, we are currently developing a Tensorflow-based open source toolbox for modeling and optimization of various imaging applications (https://github.com/pkorus/neural-imaging).About the Speaker
Nasir Memon is a professor in the Departmentof Computer Science and Engineering at NYU Tandon. He is an affiliatefaculty at the computer science department in the Courant Institute ofMathematical Sciences at NYU. He introduced cyber security studiesto New York University Tandon School of Engineering ands is a founding directorof the Center for Cyber Security, New York University,and the Center for Cyber Security at New York University AbuDhabi. He is the founder of OSIRIS and CSAW, the worlds largest student run cybersecurity event. As the Associate Dean for Online Learning, helaunched the Bridge to Tandon program thatprovides pathways to Non-STEM students to Computer Science and Cyber Security Cyber Fellows program thatprovides a highly affordable, industry partnered online MS in cybersecurity to domestic students and the MS in Cyber Risk and Strategy in collaborationwith NYU Law.
He has published more than 300 papers andreceived several best paper awards and awards for excellence in teaching. Hehas been on the editorial boards of several journals, and was theEditor-In-Chief of the IEEE Transactions on Information Security and Forensics.He is an IEEE, IAPR and SPIE Fellow for his contributions to image compressionand media security and forensics. His research interests include digitalforensics, biometrics, data compression, network security and security andhuman behavior.
Ways to Watch
