TAILOR WP4 Workshop at NeSy2023

Workshop on “Benchmarks for Neural-Symbolic AI” 

July 3, 2023 – 11:00-13:00 

 

To participate:

 

Abstract

The study of Neural-Symbolic (NeSy) approaches has been a longstanding goal in the field of AI. However, the evaluation of novel ideas and frameworks in this context poses a significant challenge due to the lack of appropriate benchmarks. Existing tasks and datasets tend to exhibit a myopic approach, tailored exclusively to specific algorithms or methodologies, and limited to narrow scenarios and application domains. This limitation hinders the ability to obtain meaningful experimental comparisons and constrains the overall progress of NeSy AI. Consequently, there is an urgent need for more comprehensive and inclusive benchmarks that facilitate the testing of diverse approaches across a wide range of tasks and under varying conditions. The workshop will present some prominent examples of existing benchmarks in different sub-fields, like continual learning, NLP, knowledge graphs, and so forth, from the perspective of NeSy-based approaches. In addition, we will discuss the importance of developing new benchmarks that address some shortcomings of existing ones.

 

Program (General Chairs: Francesco Giannini and Michelangelo Diligenti)

  • 11:00-11:05 – Opening
  • 11:05-11:20 – Towards a survey and unified benchmark for neural-symbolic AI – Luc De Raedt and Robin Manhaeve
  • 11:20-11:30 – Neuro-Symbolic Continual Learning: A Novel Task with Novel Benchmarks – Emanuele Marconato
  • 11:30-11:40 – Entity Linking Benchmark in Computational Food: Combining Text Mining, Logical Reasoning, and LLM Prompting – Agnieszka Lawrynowicz
  • 11:40-11:50 – Challenges for Neuro-Symbolic Approaches: Case Study on Legal Analytics and Argument Mining – Federico Ruggeri
  • 11:50-12:00 – A neuro symbolic setup for link prediction in KGEs – Michelangelo Diligenti
  • 12:00-12:10 – Three New Benchmarks for Neuro-Symbolic Systems – Fabrizio Riguzzi
  • 12:10-12:20 – V-LoL: A Diagnostic Dataset for Visual Logical Learning – Devendra Singh Dhami
  • 12:20-13:00 – Round Table Discussion – Michelangelo Diligenti and Francesco Giannini
28 May 2023
 |  Category: Events