Creating Artificial Life with Reinforcement Learning

Introducing ReinLife , a package for simulating evolution with RL

Pre-trained algorithms battling it out

Artificial Lifeis a field that studies systems related to natural life, typically in the form of simulating evolution.

Although Evolutionary Algorithms have shown to result in interesting behavior, they focus on learning across generations whereas behavior could also be learned during one's lifetime. This is where Reinforcement Learning comes in, which learns through a reward/punishment system that allows it to learn new behavior during its lifetime. Using Reinforcement Learning, entities learn to survive, reproduce, and make sure to maximize the fitness of their kin.

In this article, I would like to guide you through to ReinLife , a package that I created which leverages RL in order to simulate evolution and create Artificial Life.

1. Reinforcement Learning vs. Evolutionary Algorithms

Before diving into the application, it is important to understand the difference between Reinforcement Learning and Evolutionary Algorithms.

Evolutionary Algorithms

Evolutionary Algorithms (EA) try to solve optimization problems by applying methods inspired by biological evolution, such as mutation, reproduction, and selection. Typically, EA starts with a population of solutions to a problem. Then, it selects the best solution and creates a new population by copying and mutating that solution. This continues until it converges to the best solution.

When we are talking about organisms, EA can be used to spawn a population of agents, let them do their thing, and only keep the best agents. After this, a new population is created with copies of the best agents that are slightly mutated. The result is behavior that is learned at the end of its lifetime .

Reinforcement Learning

Reinforcement Learning (RL) solves problems by letting agents interact with the environment and change their behavior based on the results of those interactions.

It often makes use of Neural Networks as the main solver. The input for these networks is typically the state of the environment and the output of the action it has selected. If an action led to a negative reward, it updates the network to correct for this incorrect action.

In the context of organisms, RL can be used to have agents interact with the environment and update their behavior based on the rewards they received. The result is behavior that is learned during its lifetime .

Reinforcement Learning algorithm (DQN, PPO, etc.) will be used to train the agents in the ReinLife environment. The agents will learn by interacting with the environment in contrast to the evolutionary approach.

2. The Environment

Before going explaining how Reinforcement Learning is used in ReinLife, it is important to understand how the basic environment works.