Video_8_Automated Discovery of Local Rules for Desired Collective-Level Behavior Through Reinforcement Learning.MP4
Complex global behavior patterns can emerge from very simple local interactions between many agents. However, no local interaction rules have been identified that generate some patterns observed in nature, for example the rotating balls, rotating tornadoes and the full-core rotating mills observed in fish collectives. Here we show that locally interacting agents modeled with a minimal cognitive system can produce these collective patterns. We obtained this result by using recent advances in reinforcement learning to systematically solve the inverse modeling problem: given an observed collective behavior, we automatically find a policy generating it. Our agents are modeled as processing the information from neighbor agents to choose actions with a neural network and move in an environment of simulated physics. Even though every agent is equipped with its own neural network, all agents have the same network architecture and parameter values, ensuring in this way that a single policy is responsible for the emergence of a given pattern. We find the final policies by tuning the neural network weights until the produced collective behavior approaches the desired one. By using modular neural networks with modules using a small number of inputs and outputs, we built an interpretable model of collective motion. This enabled us to analyse the policies obtained. We found a similar general structure for the four different collective patterns, not dissimilar to the one we have previously inferred from experimental zebrafish trajectories; but we also found consistent differences between policies generating the different collective pattern, for example repulsion in the vertical direction for the more three-dimensional structures of the sphere and tornado. Our results illustrate how new advances in artificial intelligence, and specifically in reinforcement learning, allow new approaches to analysis and modeling of collective behavior.
History
Usage metrics
Categories
- Classical Physics not elsewhere classified
- Biophysics
- Quantum Physics not elsewhere classified
- Physical Chemistry of Materials
- Solar System, Solar Physics, Planets and Exoplanets
- Condensed Matter Physics not elsewhere classified
- Mathematical Physics not elsewhere classified
- Applied Physics
- Tropospheric and Stratospheric Physics
- Computational Physics
- Condensed Matter Physics
- Particle Physics
- Plasma Physics
- Mesospheric, Ionospheric and Magnetospheric Physics
- High Energy Astrophysics; Cosmic Rays
- Space and Solar Physics
- Cloud Physics
- Astrophysics
- Photonics, Optoelectronics and Optical Communications
- Classical and Physical Optics
- Physical Chemistry not elsewhere classified