LINK TO THE PAPER https://arxiv.org/abs/1903.07534
LINK TO THE REPO https://github.com/GiuseppeMarra/lyrics

In spite of the amazing results obtained by deep learning in many applications, a real intelligent behavior of an agent acting in a complex environment is likely to require some kind of higher-level symbolic inference. Therefore, there is a clear need for the definition of a general and tight integration between low-level tasks, processing sensorial data that can be effectively elaborated using deep learning techniques, and the logic reasoning that allows humans to take decisions in complex environments.

We recently introduced LYRICS, a generic interface layer for integrating Logic and Deep Learning, which is implemented in TersorFlow (TF). LYRICS provides an input language that allows to define arbitrary First Order Logic (FOL) background knowledge. The predicates and functions of the FOL knowledge can be bound to any TF computational graph, and the formulas are converted into a set of real-valued constraints, which participate to the overall optimization problem. This allows to learn the weights of the learners, under the constraints imposed by the prior knowledge. The framework is extremely general as it imposes no restrictions in terms of which models or knowledge can be integrated.

LYRICS is an extension of Semantic Based Regularization, the earlier proposal of our group. It introduces GPU computing as well as FOL function.

The linked repository provides a bunch of easy examples, which shows how the integration of logic reasoning is straightforward. As an example, a manifold regularization program can be easily programmed by the following code:

    Points = lyr.Domain(label="Points", data=X)
    R1 = lyr.Relation("A", domains=("Points"), function=is_A)
    R2 = lyr.Relation("isClose", domains=("Points", "Points"), function=is_close)
    C = lyr.Constraint("forall p: forall q: isClose(p,q) -> (A(p) <-> A(q))")
    lyr.PointwiseConstraint(is_A, y_sup, X_sup)

 

We recently exploited LYRICS to perform image translation of human faces (paper), showing that also Generative Adversarial tasks are easily describable by high level descriptions.