ARCH-COMP24 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants

58 pages•Published: October 10, 2024

Diego Manzanas Lopez, Matthias Althoff, Luis Benet, Clemens Blab, Marcelo Forets, Yuhao Jia, Taylor T Johnson, Manuel Kranzl, Tobias Ladner, Lukas Linauer, Philipp Neubauer, Sophie Neubauer, Christian Schilling, Huan Zhang and Xiangru Zhong

Abstract

This report presents the results of a friendly competition for formal verification of continuous and hybrid systems with artificial intelligence (AI) components. Specifically, machine learning (ML) components in cyber-physical systems (CPS), such as feedforward neural networks used as feedback controllers in closed-loop systems, are considered, which is a class of systems classically known as intelligent control systems, or in more modern and specific terms, neural network control systems (NNCS). We broadly refer to this category as AI and NNCS (AINNCS). The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2024. In the 8th edition of this AINNCS category at ARCH-COMP, five tools have been applied to solve 12 benchmarks, which are CORA, CROWN-Reach, GoTube, JuliaReach, and NNV. This is the year with the largest interest in the community, with two new, and three previous participants. Following last year’s trend, despite the additional challenges presented, the verification results have improved year-over-year. In terms of computation time, we can observe that the previous participants have improved as well, showing speed-ups of up to one order of magnitude, such as JuliaReach on the TORA benchmark with ReLU controller, and NNV on the TORA benchmark with both heterogeneous controllers.

In: Goran Frehse and Matthias Althoff (editors). Proceedings of the 11th Int. Workshop on Applied Verification for Continuous and Hybrid Systems, vol 103, pages 64-121.

Links:	https://easychair.org/publications/paper/WsgX
	https://doi.org/10.29007/mxld

BibTeX entry

@inproceedings{ARCH-COMP24:ARCH_COMP24_Category_Report,
  author    = {Diego Manzanas Lopez and Matthias Althoff and Luis Benet and Clemens Blab and Marcelo Forets and Yuhao Jia and Taylor T Johnson and Manuel Kranzl and Tobias Ladner and Lukas Linauer and Philipp Neubauer and Sophie Neubauer and Christian Schilling and Huan Zhang and Xiangru Zhong},
  title     = {ARCH-COMP24 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants},
  booktitle = {Proceedings of the 11th Int. Workshop on Applied Verification for Continuous and Hybrid Systems},
  editor    = {Goran Frehse and Matthias Althoff},
  series    = {EPiC Series in Computing},
  volume    = {103},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2398-7340},
  url       = {/publications/paper/WsgX},
  doi       = {10.29007/mxld},
  pages     = {64-121},
  year      = {2024}}

Download PDF Open PDF in browser