Sunday, August 27, 2017

Game AI: Non-Human Behavior Part 5

This is part 5 of a series on Game AI for Non-Human Behavior. Here's what you might have missed!
Part 5 will be a deep dive into sensory input, and resulting behaviors.

Source: Atari
In 2015 there was an article published in nature that used Atari 2600 games to explore Reinforcement Learning in AI. Reinforcement Learning is the process of allowing AI to explore different options and learn behavior through a reward system, as opposed to Supervised Learning where AI performing sub-optimal behavior is explicitly corrected. The article, "Human-level control through deep reinforcement learning," examines sensory input used to understand the surrounding environment. In their research, the sensory input used correlates to visual input (or sight), allowing the AI to look at the pixels that make up the current game state.

Source: The Hunt, Netflix
In contrast, these blind catfish living in underwater caves have adapted to survive completely without sight. When one sense fails, we rely on honing our other senses to adapt to a world without the missing sense.

As children, we learn the five senses as sight, sound, smell, taste, and touch. These are the senses we as humans understand, because these are the senses we experience. However, there are other senses present in nature that are outside our area of experience. Bats, for example, use sonar in place of sight to determine the positions of things around them.

A "sense" is defined as a system of a group of cell types that responds to specific physical phenomena, which correspond to a particular region of the brain that interpret those signals. Some plants have specialized cells which detect gravity, allowing the plants to grow upright with their roots growing down into the earth.

Even with senses we understand, such as smell, non-human creatures have far superior uses of them in many cases. Sharks can determine the direction a smell is coming from based on which nostril received the scent first.

Though the focus of reinforcement learning is on changing behavior based on rewards, it relies on information about the environment being interpreted through senses. We can see something similar happen in the classic rat maze example - the hungry rat is placed in a maze that has a piece of cheese at the end and allowed to explore the maze until it finds the cheese. The rat is then placed back at the start of the maze, and this continues until the rat manages to navigate the maze perfectly. The cheese in this example is obviously the reward, but the rat needs a way to understand its environment in order to get the reward.

Beyond just understanding the environment, the senses also allow organism to understand the creatures around them, resulting in group-based behavior like flocking. To put flocking in simple terms, we can use an algorithm that uses cohesion, alignment, and separation to simulate behavior well enough for gameplay purposes.

Each agent in the AI flock will calculate it's desired vector of movement based on cohesion (the need to stay within in the group), alignment (the need to face the same direction as the group), and separation (the need to avoid hitting other members of the group). The calculation for cohesion is to look at the position of your neighbors within a specific range, find the center of mass, and move towards that center of mass. For alignment, the agent looks at the direction each of its neighbors is facing and aligns itself to the average. For separation, the agent will check to see if it is too close to any of its neighbors, and adjust accordingly. The three of these combined into the agent's velocity vector will result in a simple flocking behavior.

However, the flock's simple behavior may be disrupted by the presence of predators, causing the agents to need to assess their environment beyond just the position and orientation of their neighbors.

Bees are another wonderful example of teamwork behavior. Pheromones are special scented chemicals that allow some organisms to communicate information with each other via scent. Bees use pheromones the share information through the hive, resulting in a hive the essentially thinks together as if it were a brain with each bee acting as a neuron. I can imagine a similar system that could be used to have a group of friendly characters so coordinated that they act as a hive of bees with the player as their queen.

If you have enjoyed this series on AI for non-human behavior, please follow and subscribe!

1 comment: