Abstract
With the availability of powerful and easy-to-use media editing tools, falsifying images and videos has become widespread in the last few years. Coupled with ubiquitous social networks, this allows for the viral dissemination of fake news. This raises huge concerns on multimedia security. This scenario became even worse with the advent of deep learning. New, sophisticated methods have been proposed to accomplish manipulations that were previously unthinkable (e.g., deepfake).
This tutorial will present the most relevant methods for generation and detection of manipulated media. For generation the main techniques based on deep learning will be presented, with focus on those based on both graphics and neural network based methods such as generative adversarial networks or cutting edge neural rendering techniques. Both images and videos will be considered, but also the combination of multiple modalities including audio, and text associated with the underlying imagery. For detection the most reliable deep learning based approaches will be presented, with focus on those that enable domain generalization.
Results will be presented on challenging datasets and realistic scenarios, such as the spreading of manipulated images and videos over social networks. In addition, the robustness of such methods to adversarial attacks will be analyzed.
Short Bio
Dr. Matthias Niessner is Professor at the Technical University of Munich where he leads the Visual Computing Lab. Before, he was a Visiting Assistant Professor at Stanford University. Prof. Nießner’s research lies at the intersection of computer vision, graphics, and machine learning, where he is particularly interested in cutting-edge techniques for 3D reconstruction, semantic 3D scene understanding, video editing, and AI-driven video synthesis. In total, he has published over 70 academic publications, including 22 papers at the prestigious ACM Transactions on Graphics (SIGGRAPH / SIGGRAPH Asia) journal and 18 works at the leading vision conferences (CVPR, ECCV, ICCV); several of these works won best paper awards, including at SIGCHI’14, HPG’15, SPG’18, and the SIGGRAPH’16 Emerging Technologies Award for the best Live Demo. For his work, Prof. Nießner received several awards: he is a TUM-IAS Rudolph Moessbauer Fellow (2017 – ongoing), he won the Google Faculty Award for Machine Perception (2017), the Nvidia Professor Partnership Award (2018), as well as the prestigious ERC Starting Grant 2018. In 2019, he received the Eurographics Young Researcher Award honoring the best upcoming graphics researcher in Europe. Prof. Nießner is also a co-founder and director of Synthesia Inc., a brand-new startup backed by Marc Cuban, whose aim is to empower storytellers with cutting-edge AI-driven video synthesis.
Dr. Luisa Verdoliva is Associate Professor at University Federico II of Naples, Italy, where she leads the Multimedia Forensics Lab. In 2018 she has been visiting professor at Friedrich-Alexander-University (FAU) and in 2019-2020 she has been visiting scientist at Google AI in San Francisco. Her scientific interests are in the field of image and video processing, with main contributions in the area of multimedia forensics. She has published over 120 academic publications, including 45 journal papers. She is the Principal Investigator for University Federico II of Naples in the DISCOVER (a Data-driven Integrated Approach for Semantic Inconsistencies Verification) project funded by DARPA under the SEMAFOR program (2020-2024). She has actively contributed to the academic community through service as general co-Chair of the 2019 ACM Workshop on Information Hiding and Multimedia Security, technical Chair of the 2019 IEEE Workshop in Information Forensics and Security and area Chair of the IEEE International Conference on Image Processing since 2017. She is on the Editorial Board of IEEE Transactions on Information Forensics and Security and IEEE Signal Processing Letters. Dr. Verdoliva is Chair of the IEEE Information Forensics and Security Technical Committee. She is the recipient of a Google Faculty Award for Machine Perception (2018) and a TUM-IAS Hans Fischer Senior Fellowship (2020-2023). She has been elected to the grade of IEEE Fellow since January 1, 2021.