Game AI: Non-Human Behavior Part 5

This is part 5 of a series on Game AI for Non-Human Behavior. Here's what you might have missed!

• Part 1: Defining "Game AI" and "Non-Human Behavior"
• Part 2: Making Decisions, Predators and Prey
• Part 3: Weird Inspirations from Nature
• Part 4: Modes of Hunting

Part 5 will be a deep dive into sensory input, and resulting behaviors.

Source: Atari

In 2015 there was an article published in nature that used Atari 2600 games to explore Reinforcement Learning in AI. Reinforcement Learning is the process of allowing AI to explore different options and learn behavior through a reward system, as opposed to Supervised Learning where AI performing sub-optimal behavior is explicitly corrected. The article, "Human-level control through deep reinforcement learning," examines sensory input used to understand the surrounding environment. In their research, the sensory input used correlates to visual input (or sight), allowing the AI to look at the pixels that make up the current game state.

Source: The Hunt, Netflix

In contrast, these blind catfish living in underwater caves have adapted to survive completely without sight. When one sense fails, we rely on honing our other senses to adapt to a world without the missing sense.

Source: https://askabiologist.asu.edu/echolocation

As children, we learn the five senses as sight, sound, smell, taste, and touch. These are the senses we as humans understand, because these are the senses we experience. However, there are other senses present in nature that are outside our area of experience. Bats, for example, use sonar in place of sight to determine the positions of things around them.

A "sense" is defined as a system of a group of cell types that responds to specific physical phenomena, which correspond to a particular region of the brain that interpret those signals. Some plants have specialized cells which detect gravity, allowing the plants to grow upright with their roots growing down into the earth.

Source: http://www.defenders.org/sharks/basic-facts

Even with senses we understand, such as smell, non-human creatures have far superior uses of them in many cases. Sharks can determine the direction a smell is coming from based on which nostril received the scent first.

Source: nature.com/scientificamericanmind/journal/v19/n4/images/
scientificamericanmind0808-22-I1.jpg

Though the focus of reinforcement learning is on changing behavior based on rewards, it relies on information about the environment being interpreted through senses. We can see something similar happen in the classic rat maze example - the hungry rat is placed in a maze that has a piece of cheese at the end and allowed to explore the maze until it finds the cheese. The rat is then placed back at the start of the maze, and this continues until the rat manages to navigate the maze perfectly. The cheese in this example is obviously the reward, but the rat needs a way to understand its environment in order to get the reward.

Source: http://www.smithsonianmag.com/smart-news/
were-terrible-distinguishing-real-and-fake-schools-fish-180953162/

Beyond just understanding the environment, the senses also allow organism to understand the creatures around them, resulting in group-based behavior like flocking. To put flocking in simple terms, we can use an algorithm that uses cohesion, alignment, and separation to simulate behavior well enough for gameplay purposes.

Source: http://harry.me/blog/2011/02/17/neat-algorithms-flocking/

Each agent in the AI flock will calculate it's desired vector of movement based on cohesion (the need to stay within in the group), alignment (the need to face the same direction as the group), and separation (the need to avoid hitting other members of the group). The calculation for cohesion is to look at the position of your neighbors within a specific range, find the center of mass, and move towards that center of mass. For alignment, the agent looks at the direction each of its neighbors is facing and aligns itself to the average. For separation, the agent will check to see if it is too close to any of its neighbors, and adjust accordingly. The three of these combined into the agent's velocity vector will result in a simple flocking behavior.

Source: https://www.gizmodo.com.au/2014/03/
what-happens-when-you-throw-four-sharks-into-a-giant-school-of-fish/

However, the flock's simple behavior may be disrupted by the presence of predators, causing the agents to need to assess their environment beyond just the position and orientation of their neighbors.

Source: https://www.wired.com/2011/12/
the-true-hive-mind-how-honeybee-colonies-think/

Bees are another wonderful example of teamwork behavior. Pheromones are special scented chemicals that allow some organisms to communicate information with each other via scent. Bees use pheromones the share information through the hive, resulting in a hive the essentially thinks together as if it were a brain with each bee acting as a neuron. I can imagine a similar system that could be used to have a group of friendly characters so coordinated that they act as a hive of bees with the player as their queen.

If you have enjoyed this series on AI for non-human behavior, please follow and subscribe!

Game AI: Non-Human Behavior Part 1

In this series I will be talking about game AI, specifically designing behaviors for non-human characters. At the end of August 2017 I will be speaking on this topic at PAX Dev, so this blog series is where I will be organizing my material and hopefully getting some feedback! So let's get started.

Game AI is one of those areas of game development that is both very technical and very design-heavy. However, it is important not to overlook the role that art and sound play in making the AI believable, understandable, and relatable. When we talk about AI in games, we are talking about anything in the game that makes decisions for itself, and specifically what is the process for making and acting on those decisions. We often use either a behavior tree, or a state machine, or some combination of the two, to determine what an AI is going to do at any given moment. The AI also needs to store and process information about its surroundings and its past actions to use in its decision-making.

I think it's safe to say that we are all humans, and so when we approach a design problem we approach it from the human perspective. When designing something that makes decisions, we instinctively design it to make decisions the way we make decisions. Obviously this works out well when designing AI for human characters, but not so well when designing for non-human characters.

There are plenty of people out there talking about game AI for human characters, and they know way more on that subject than I do, so for this talk I will not be covering anything to do with humans. When I say "non-human AI" you are probably thinking of animals, and specifically of mammals, probably most likely mammals on our size-scale like cats.

I'm going to talk about more than just cat AI, in fact I'm going to go beyond mammals or animals in general. There are a lot of options out there for non-human inspiration and design - fish, birds, insects, arachnids, plants...robots...zombies...aliens...I mean, like, real aliens...

(not an alien, this is a real Earth creature)

...anyway...

Let's talk about what constitutes AI in games. First I need to clarify, AI for games is not at the same caliber as AI in the world of computer science and machine learning. AI for games is essentially dumb AI, it is constrained to what is needed for gameplay purposes and to appear to be making informed decisions to fool the player. In fact, not only does AI in games cut corners, it also often has hindrances applied to it to make it feel more believable and natural to players. Players tend to feel cheated if the AI seems too smart, even if it is accurately acting on the information it is given.

Ok, so Game AI is not the same as real AI. Let's also talk about some definitions, just for the sake of clarity. An Agent is something that uses information to make a decision through a process of Action Selection. An Agent in a game could be a friendly NPC, an enemy, a semi-controllable simulated character, or in some cases a background character, or a companion (such as a pet). AI Agents generate responsive behavior based on a predetermined set of stimuli, to give the illusion of intelligent decision-making.

Now on to the point of this series: how do we as humans design AI for things that are not humans? There is also a larger problem here, once we manage to design something that does not make decisions the way we do, how do we make sure the player understands those characters and their decisions? Not surprisingly, a common way to make non-human characters relatable is to anthropomorphize them.

Let's avoid that approach for now, because if we are going to anthropomorphize our characters then there is no real reason they shouldn't make decisions like humans. In this series I am going to talk about some amazing things that really exist, things we can use as inspiration for our AI design. Things like flocking, hive-mind, predator/prey relations, natural selection, and aggressive plants. Stay tuned!

Food for thought: if a tree grew fast enough that we could watch it engulf something, how scary would that be?

Death's Daughter

The second team of interns at Astire Games just wrapped up their project Death's Daughter. It is an endless runner on Android and iOS where you play as the daughter of the Grim Reaper helping him reap souls.

Some cute and interesting things they did with their mechanics - the character you play is very young, and the souls weigh her down both figuratively and literally. She carries the souls in a bag on her back and drops them off at courier checkpoints, but as the bag fills with souls she moves slower and it becomes more difficult to clear obstacles. If her bag has too many souls it will tear open and she will drop them all and have to start over. One of the obstacles she faces is a black cat, which gives nine souls if she hits it - this can be good or bad for the player depending on how full your bag is!

The game went live today on Google Play and iTunes, but the interns will be continuing their internship for another week to release updates to the game based on reviews and feedback from players. We hope you will play and give feedback!