Simone Scardapane and Aurelio Uncini (Università Sapienza)

Jan 30, 2019 – 11:00 AM

DIISM, Artificial Intelligence laboratory (room 201), Siena SI

Description

Recently, the design of flexible nonlinearities has become an important line of research in the deep learning community. In the first part of the talk we will review how to tackle this problem, both in the context of simple parameterizations of known functions (e.g., the parametric ReLU), and with the definition of more advanced, non-parametric models (e.g., the Maxout network). The second part of the talk will focus on a recent proposal, the kernel activation function, which is based on an kernel expansion of its input. We will show its core idea and some recent extensions, involving its use in the context of other types of nonlinearities, such as gates (as in LSTMs), and attention models. The talk is concluded with some open challenges and possible lines of research.