Reliable and Explainable Adversarial Machine Learning (RexLearn)
This page collects the research software that was developed in the context of some of the activities related to the PRIN 2017 project RexLearn, funded by the Italian Ministry of Education, University and Research (grant no. 2017TWNMH2). This software is intended to help researches to have a starting point for further research in the direction of the topics of the project.
Foveated Neural Computation
We developed a software package that allows Neural Networks to be augmented with a special class of layers which performs convolutions in function of the coordinates of the focus of attention (externally provided). This allows Neural Networks to gain robustness to Adversarial Attacks that involve the periphery of the image, and it reduces the number of spurious correlations that are developed during training. The code is written in Python and it is based on Pytorch.
https://github.com/sailab-code/foveated_neural_computation
Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams
Another activity of the project focussed on the developed of learning procedures that allows Neural Networks to promote pixel-wise representations that are robust to slight perturbations (typical of Adversarial Attacks), thats to the fulfilment of motion-coherence-based constraints. We developed a software package to perform experiments using a self-supervised loss function that we proposed in the context of the project. We also considered the case in which a human-like attention mechanism further foster the feature quality. The software package to reproduce experiment is written in Python and it is based on Pytorch.
https://github.com/sailab-code/cl_stochastic_coherence
Some further developments (ongoing)
https://github.com/sailab-code/unsupervised-learning-feature-flow
PARTIME: Scalable and Parallel Processing Over Time with Deep Neural Networks
Learning over time by motion coherence can be accelerated by distributing computations across multiple devices. We developed a software solutions that exploits CUDA Graph to dispatch operations over multiple NVIDIA graphics cards. Neural Network architectures are divided into computational blocks and the information flow is propagated over time introducing delays that allows to achieve a higher level of parallelization. The code is written in Python and it is based on Pytorch.
https://github.com/sailab-code/partime
Focus of Attention Improves Information Transfer in Visual Features
Adversarial attacks are frequently favoured by having exposed the Neural Network to information that might not be related to the task at hand but that the Network might find useful to learn unexpected correlations. We evaluated how introducing a human-like focus of attention mechanism allows the network to learn over time by transferring a significant amount of information from the input stream to the pixel-wise representations. We developed software to evaluate the amount of information that is transferred. The code is written in Python and it is based on Pytorch.
https://github.com/sailab-code/focus_of_attention_improves_info_transfer
Continual Unsupervised Learning for Optical Flow Estimation with Deep Networks
While the importance of motion is key factor to develop robust representations, learning how to predict motion still remain a challenging task that is usually approached with offline learning techniques. We studied how Neural Networks can learn to predict motion over time, by watching a video stream (a movie, for example). We developed software to evaluate this learning setting. The code is written in Python and it is based on Pytorch.
https://github.com/sailab-code/continual-of
Friendly Training
While Adversarial Machine Learning studies how to fool and improve Neural Networks and other Machine Learning models, we asked whether we could use the principles of such a research field to favour the learning process of Neural Networks. We introduced a further network that is responsible of generating simplified (instead of Adversarial) examples, with time-oriented dynamics that allows a neural classifier to progressively learn from simpler examples first. The code we developed is is written in Python and it is based on Pytorch.
https://github.com/sailab-code/friendly-training
Lagrangian Propagation in Graph Neural Networks
Diffusing information over graph-based data is a crucial process in many real-world problems. We studied how to make this process more efficient, exploiting locality, allowing Neural Networks (Graph Neural Networks) to learn to propagate information in a Lagrangian framework. The code we developed is is written in Python and it is based on Pytorch.
https://github.com/mtiezzi/lpgnn
Messing Up 3D Virtual Environments: Transferable Adversarial 3D Objects
We developed software to perform experiments in which we generated attacks to 3D models. Our software is the result of a research work that involved developing a Python interface to a 3D engine, and specific experiences oriented toward altering the texture of the rendered objects to fool a Neural Network that processed 2D snapshots of the observed object. The code we developed is is written in Python and it is based on Pytorch and Pytorch3D.
https://github.com/sailab-code/SAIFooler
SAILenv: Learning in Virtual Visual Environments Made Simple
In order to generate attacks to 3D objects, we created a Python interface to make it easy to have access to data from a 3D scene (Unity Engine). The code we developed is is written in Python. See also the SAILenv public page.
https://github.com/sailab-code/SAILenv
Logic Explained Networks
Having the capability of providing explanations of the decisions of Neural Models is crucial to develop robustness to Adversarial Attacks. We proposed (in collaboration with other research teams), a Neural Framework in which First Order Logic explanations are generated to describe the behaviour of a neural classifier. The code is written in Python.
https://github.com/pietrobarbiero/logic_explained_networks