Game AI: Non-Human Behavior Part 5

This is part 5 of a series on Game AI for Non-Human Behavior. Here's what you might have missed!

• Part 1: Defining "Game AI" and "Non-Human Behavior"
• Part 2: Making Decisions, Predators and Prey
• Part 3: Weird Inspirations from Nature
• Part 4: Modes of Hunting

Part 5 will be a deep dive into sensory input, and resulting behaviors.

Source: Atari

In 2015 there was an article published in nature that used Atari 2600 games to explore Reinforcement Learning in AI. Reinforcement Learning is the process of allowing AI to explore different options and learn behavior through a reward system, as opposed to Supervised Learning where AI performing sub-optimal behavior is explicitly corrected. The article, "Human-level control through deep reinforcement learning," examines sensory input used to understand the surrounding environment. In their research, the sensory input used correlates to visual input (or sight), allowing the AI to look at the pixels that make up the current game state.

Source: The Hunt, Netflix

In contrast, these blind catfish living in underwater caves have adapted to survive completely without sight. When one sense fails, we rely on honing our other senses to adapt to a world without the missing sense.

Source: https://askabiologist.asu.edu/echolocation

As children, we learn the five senses as sight, sound, smell, taste, and touch. These are the senses we as humans understand, because these are the senses we experience. However, there are other senses present in nature that are outside our area of experience. Bats, for example, use sonar in place of sight to determine the positions of things around them.

A "sense" is defined as a system of a group of cell types that responds to specific physical phenomena, which correspond to a particular region of the brain that interpret those signals. Some plants have specialized cells which detect gravity, allowing the plants to grow upright with their roots growing down into the earth.

Source: http://www.defenders.org/sharks/basic-facts

Even with senses we understand, such as smell, non-human creatures have far superior uses of them in many cases. Sharks can determine the direction a smell is coming from based on which nostril received the scent first.

Source: nature.com/scientificamericanmind/journal/v19/n4/images/
scientificamericanmind0808-22-I1.jpg

Though the focus of reinforcement learning is on changing behavior based on rewards, it relies on information about the environment being interpreted through senses. We can see something similar happen in the classic rat maze example - the hungry rat is placed in a maze that has a piece of cheese at the end and allowed to explore the maze until it finds the cheese. The rat is then placed back at the start of the maze, and this continues until the rat manages to navigate the maze perfectly. The cheese in this example is obviously the reward, but the rat needs a way to understand its environment in order to get the reward.

Source: http://www.smithsonianmag.com/smart-news/
were-terrible-distinguishing-real-and-fake-schools-fish-180953162/

Beyond just understanding the environment, the senses also allow organism to understand the creatures around them, resulting in group-based behavior like flocking. To put flocking in simple terms, we can use an algorithm that uses cohesion, alignment, and separation to simulate behavior well enough for gameplay purposes.

Source: http://harry.me/blog/2011/02/17/neat-algorithms-flocking/

Each agent in the AI flock will calculate it's desired vector of movement based on cohesion (the need to stay within in the group), alignment (the need to face the same direction as the group), and separation (the need to avoid hitting other members of the group). The calculation for cohesion is to look at the position of your neighbors within a specific range, find the center of mass, and move towards that center of mass. For alignment, the agent looks at the direction each of its neighbors is facing and aligns itself to the average. For separation, the agent will check to see if it is too close to any of its neighbors, and adjust accordingly. The three of these combined into the agent's velocity vector will result in a simple flocking behavior.

Source: https://www.gizmodo.com.au/2014/03/
what-happens-when-you-throw-four-sharks-into-a-giant-school-of-fish/

However, the flock's simple behavior may be disrupted by the presence of predators, causing the agents to need to assess their environment beyond just the position and orientation of their neighbors.

Source: https://www.wired.com/2011/12/
the-true-hive-mind-how-honeybee-colonies-think/

Bees are another wonderful example of teamwork behavior. Pheromones are special scented chemicals that allow some organisms to communicate information with each other via scent. Bees use pheromones the share information through the hive, resulting in a hive the essentially thinks together as if it were a brain with each bee acting as a neuron. I can imagine a similar system that could be used to have a group of friendly characters so coordinated that they act as a hive of bees with the player as their queen.

If you have enjoyed this series on AI for non-human behavior, please follow and subscribe!

Game AI: Non-Human Behavior Part 2

This is part 2 in my series on non-human game AI, if you missed part 1 check it out first!

In nature, survival and reproduction are the two biggest driving factors of decision-making.Let's start with survival.

Starting with the big picture view, we know that all living things require energy to live, ie. food, and organisms have evolved different techniques based on where they get their energy from. One way to break this down is to figure out whether a creature is a predator or a prey at various points in the food web.

At the top we have organisms that are always predators, and generally nothing hunts them. But moving down the chain, there are creatures hunt, but are also hunted. This is where we can see some interesting behavior trees. Just from the high level goal of "survival" these creatures will need to make decisions about which is more important - avoiding a predator or finding food. If food is plentiful, that decision is easy, but if that creature has gone a long time with out food they may take bigger and bigger risks to find food, encroaching into areas they know to be dangers.

From a design perspective, what's interesting here is we can use this information in two very distinct ways. 1. Some games, especially hunting games and some survival games, attempt highly realistic simulated environments with a balance of creatures that exist for the player to hunt. These games can use information about predator/prey relations to generate believable content. 2. Beyond realistic simulations, in any game with enemies we can regard to player as a part of the predator/prey relation. The player wants to survive, so she must defend herself from enemies either with stealth or armor or by attacking and killing the enemies first, however most games have other objectives and the player must decide how much risk they are willing to take to accomplish those objectives.

In any scenario in games that have enemies, we can decide - do we want the player to feel like a predator here, or prey? Do we want the player to feel sneaky and clever and avoid getting caught/killed, or do we want the player to feel powerful and dominant and on the hunt? Both options create interesting dynamics, and a lot of games alternate between the two to create powerful exciting experiences.

As an example, in World of Warcraft if you encounter a high-level creature too early you will probably try to avoid it because you know it's stronger than you, but once you have leveled up you might return to fight it once you know you have a chance to defeat it.

Let's design an AI for a creature that is in the middle of a predator/prey situation. I like to think of AI decision-making systems as a sort of pro-con list:

This gives us an idea of the possible behaviors the AI might take, and what some of the factors are that determine that decision. Based on our pro-con list, we know that the primary decision we want to focus on is "stay and eat" vs. "run and hide" and we know that some of the factors include how hungry they are, how prevalent the food is, how dangerous the predator is, how close the predator is, and whether or not the predator has seen them. Now we can work on prioritizing these and converting them into a decision graph (or a behavior tree or state machine, depending on your approach).

This would obviously be different for different types of creatures, and it is also a very simplified solution - it only covers one very specific decision and two possible behaviors. Generally AI will have a lot of possible behaviors and different decision factors across the spectrum of possibilities. That is really what makes AI design such a challenging area to work in.

The complexity of the AI design depends heavily on how realistic the behaviors need to be. In theHunter:Call of the Wild the designers knew that players wanted a realistic hunting experience, and that often players would spend a lot of time watching an animal before taking a shot, so the designers had to be prepared to do extensive research on how those creatures behaved to ensure a believable experience for the player.

Most games are not held to quite such high standards of realism. Creatures in Legend of Zelda do not have parallels in real life, so they have the flexibility to be weird and wonky and still be believable.

In part 3 I'm going to get into some of the stranger behavior in nature, and how we can use it as inspiration in AI design.

Image Sources:
http://www.cbc.ca/lifestory/extras/stories/8-fascinating-baby-animal-survival-techniques
https://blog.oup.com/2013/12/future-human-plant-animal-survival-in-arctic/
 http://www.huffingtonpost.co.uk/2014/05/06/national-geographic-traveler-photo-contest-2014-entries-pictures_n_5271505.html
https://www.tes.com/lessons/Jxdjqc0dlasfEA/food-chains
http://www.gamasutra.com/blogs/KarinESkoog/20170713/301653/theHunter_Call_of_the_Wild__Designing_Believable_Simulated_Animal_AI.php
 http://www.wikihow.com/Play-World-of-Warcraft
 https://www.youtube.com/watch?v=937xoXVfs0k