Game AI: Non-Human Behavior Part 5

This is part 5 of a series on Game AI for Non-Human Behavior. Here's what you might have missed!

• Part 1: Defining "Game AI" and "Non-Human Behavior"
• Part 2: Making Decisions, Predators and Prey
• Part 3: Weird Inspirations from Nature
• Part 4: Modes of Hunting

Part 5 will be a deep dive into sensory input, and resulting behaviors.

Source: Atari

In 2015 there was an article published in nature that used Atari 2600 games to explore Reinforcement Learning in AI. Reinforcement Learning is the process of allowing AI to explore different options and learn behavior through a reward system, as opposed to Supervised Learning where AI performing sub-optimal behavior is explicitly corrected. The article, "Human-level control through deep reinforcement learning," examines sensory input used to understand the surrounding environment. In their research, the sensory input used correlates to visual input (or sight), allowing the AI to look at the pixels that make up the current game state.

Source: The Hunt, Netflix

In contrast, these blind catfish living in underwater caves have adapted to survive completely without sight. When one sense fails, we rely on honing our other senses to adapt to a world without the missing sense.

Source: https://askabiologist.asu.edu/echolocation

As children, we learn the five senses as sight, sound, smell, taste, and touch. These are the senses we as humans understand, because these are the senses we experience. However, there are other senses present in nature that are outside our area of experience. Bats, for example, use sonar in place of sight to determine the positions of things around them.

A "sense" is defined as a system of a group of cell types that responds to specific physical phenomena, which correspond to a particular region of the brain that interpret those signals. Some plants have specialized cells which detect gravity, allowing the plants to grow upright with their roots growing down into the earth.

Source: http://www.defenders.org/sharks/basic-facts

Even with senses we understand, such as smell, non-human creatures have far superior uses of them in many cases. Sharks can determine the direction a smell is coming from based on which nostril received the scent first.

Source: nature.com/scientificamericanmind/journal/v19/n4/images/
scientificamericanmind0808-22-I1.jpg

Though the focus of reinforcement learning is on changing behavior based on rewards, it relies on information about the environment being interpreted through senses. We can see something similar happen in the classic rat maze example - the hungry rat is placed in a maze that has a piece of cheese at the end and allowed to explore the maze until it finds the cheese. The rat is then placed back at the start of the maze, and this continues until the rat manages to navigate the maze perfectly. The cheese in this example is obviously the reward, but the rat needs a way to understand its environment in order to get the reward.

Source: http://www.smithsonianmag.com/smart-news/
were-terrible-distinguishing-real-and-fake-schools-fish-180953162/

Beyond just understanding the environment, the senses also allow organism to understand the creatures around them, resulting in group-based behavior like flocking. To put flocking in simple terms, we can use an algorithm that uses cohesion, alignment, and separation to simulate behavior well enough for gameplay purposes.

Source: http://harry.me/blog/2011/02/17/neat-algorithms-flocking/

Each agent in the AI flock will calculate it's desired vector of movement based on cohesion (the need to stay within in the group), alignment (the need to face the same direction as the group), and separation (the need to avoid hitting other members of the group). The calculation for cohesion is to look at the position of your neighbors within a specific range, find the center of mass, and move towards that center of mass. For alignment, the agent looks at the direction each of its neighbors is facing and aligns itself to the average. For separation, the agent will check to see if it is too close to any of its neighbors, and adjust accordingly. The three of these combined into the agent's velocity vector will result in a simple flocking behavior.

Source: https://www.gizmodo.com.au/2014/03/
what-happens-when-you-throw-four-sharks-into-a-giant-school-of-fish/

However, the flock's simple behavior may be disrupted by the presence of predators, causing the agents to need to assess their environment beyond just the position and orientation of their neighbors.

Source: https://www.wired.com/2011/12/
the-true-hive-mind-how-honeybee-colonies-think/

Bees are another wonderful example of teamwork behavior. Pheromones are special scented chemicals that allow some organisms to communicate information with each other via scent. Bees use pheromones the share information through the hive, resulting in a hive the essentially thinks together as if it were a brain with each bee acting as a neuron. I can imagine a similar system that could be used to have a group of friendly characters so coordinated that they act as a hive of bees with the player as their queen.

If you have enjoyed this series on AI for non-human behavior, please follow and subscribe!

Game AI: Non-Human Behavior Part 4

This is part 4 of a series on Game AI for Non-Human Behavior. Here's what you might have missed!

• Part 1: Defining "Game AI" and "Non-Human Behavior"
• Part 2: Making Decisions, Predators and Prey
• Part 3: Weird Inspirations from Nature

 Today let's talk about how creatures in the wild go about hunting.

Living things need food to survive. There are different ways to get food - some plants make their own food, some animals eat plants, and some creatures hunt. Hunting generally refers to actively pursuing your food, but for the purpose of this post I'm going to expand it to include luring your food to come to you.

Source: https://www.youtube.com/watch?v=z5fOsgrAJiU

Here are a few of the tactics that predators use to catch their food:

• Speed
• Strength
• Traps / Lures
• Stealth / Camouflage
• Teamwork
• Built-in Tools (ie. claws and teeth)

The Venus Fly Trap is a fabulous example of  a lure / trap. Predatory plants have adapted this way of life because they live in environments where it is challenging to absorb nutrients from the ground using roots. So, they capture little packets of nutrients from the air in the form of insects. Predatory plants have different methods of luring prey including bright colors and sticky nectar. Once a fly is lured into the danger zone, the Venus Fly Trap has special hairs inside it's "mouth" that detect when something bumps against it. On the second bump, the trap will close. That's some interesting and clever behavior there - if it immediately started closing, the fly could still be airborne and would have an easier time escaping, so by waiting for the second bump the fly is lulled into a false sense of security thinking it can safely land. After the trap has closed, it waits for 5 more stimuli of the hairs before it begins digestion, to ensure the thing it trapped is actually an edible insect wriggling around.

Source: keywordsuggest.org/gallery/242369.html

Another clever take on luring prey is the Angler Fish, which has a glowy lure on top of it's head that attracts unsuspecting fish into its terrifying mouth. Using traps and lures can save a lot of energy, because you don't need to move around as much if your food comes to you.

Source: www.flickr.com/photos/uswildlifeimagescom/5733874585

Birds have some of the scariest built-in tools in the form of deadly talons. Talons can be used for a variety of killing purposes including dismemberment and squeezing to death, and can be used in combination with the beak to tear critters into bite-sized pieces (sometimes while the critter is still alive).

Source: http://tigers4kids.weebly.com/hunting--diet.html

Big Cats can cover the spectrum of strength, speed, stealth, and teamwork, but they are certainly not the only ones. Tigers are known for their tremendous strength which allows them to take down prey in their solitary lifestyle.

Source: http://southernafricatravel.com

Lionesses work together to bring down prey that would be much too large and fast to bring down solo. While working together, the lionesses also employ stealth to sneak up close to their prey before begging the chase.

Source: https://s-media-cache-ak0.pinimg.com/originals
/62/c0/d2/62c0d2b7d56679af09270d36e72a5f5c.jpg

Cheetah, being the fastest land animal, is a go-to example of using speed for hunting. Cheetahs can reach top speeds of 60 miles per hour, though only for short sprints. Perhaps more impressive than their top speed is how quickly they can start and stop a sprint.

Source: wideopenspaces.com/10-animals-school-humans-camouflage/

Leopards are a master of camouflage, their patterns of spots help them melt into the shadows of trees and brush.

Many of these methods of hunting can also be used by prey to avoid being hunted. Prey rely on their speed, strength, teamwork, and stealth to stay alive in a dangerous world. And all of these methods can be applied to your characters when design your AI's behavior in games.

Game AI: Non-Human Behavior Part 3

This is part 3 of a series, if you are just joining in be sure to start at part 1!

Nature is weird, and full of constant surprises.There are a lot of things in nature that we still don't understand.Take for example these thousands of circles in the Kalahari desert:

Source: Africa, Netflix

We don't know what causes them...Scientists have suggested numerous options - but ruled all of them out - including poisonous plants, insects, and magnetism. There is so much about our planet that is outside of our understanding, and so much to draw inspiration from.

Source: Bill Nye the Science Guy 

People think science is pretty cool, that's why we have a large chunk of the entertainment industry using science to draw in their audience. As game developers, we can use that fascination to our advantage, to get our players excited.

Source: Niche, Game

I've been playing a game recently called Niche where the player controls a herd of animals and tries to help them survive. The game does some really interesting things with reproduction and genetics, allowing the player to alter the chances of certain traits, and choosing which animals will mate with each other. This game does a really great job of taking elements of nature and science that people find fascinating, and then turning those elements into engaging gameplay mechanics.

Source: Factorio, Game

Another game that I think does a great job with this is Factorio. In Factorio, the player plays as a character stranded on an alien planet, and the player needs to collect resources to build a rocket to get off the planet. A lot of the resources can't be used as raw materials, so the player builds a lot of different kinds of processing plants, essentially creating an entire factory from scratch. But the alien species living on the planet does not want to see their beautiful planet destroyed by pollution and over-harvesting. The more the player builds and expands, the more of the natural planet gets destroyed, and the angrier the aliens become. The aliens will attack the player and the base, because they are defending their planet. I really like the way this game makes you think about how your actions are impacting nature, and finding more sustainable and less invasive ways to do things is more challenging but helps keep you at peace with the creatures inhabiting the planet.

I've design characters and AI for a handful of games, and I've learned a few lessons from the process. Instinctively, it might seem that character design, combat, and behavior are all distinct design challenges that can be handled independently, but I've found that when it comes to the player interacting with the characters, these three things are very tightly intertwined, and they deserve to be designed together. There are games that have characters that just make sense because their character design feels perfectly aligned with their AI behavior, and the transition from behavior to combat seems completely natural and expected for how that character is perceived. On the other hand, I'm sure we've all played games where a character feels out of place, their behavior does not make sense for what that creature is, and the transition to combat feels clunky and unexpected.

I personally have fallen into this trap, and I'd like to think that I learned from my mistakes, and hopefully you can too. The very first game I worked on where I designed the AI and combat was a co-op exergame where two players work together to defeat giant robot spiders in an underground subway system.

We spent a lot of time designing and researching the exercise portion, and not a lot of time thinking about our enemies. We knew we needed something that would be immediately recognized as evil, we needed the player to be able to attack from a distance, and we wanted the enemies to swarm the player. A couple of interesting things that came from our design was that the spiders would come out of webs in the corners, so players could predict where they would come from, and some spiders could attack from a distance by spitting sticky webs at the player to temporarily trap them.

This project was over 5 years ago. I was the lead designer on this project so I was responsible for designing engaging enemies, but it was my first time designing anything related to AI, so I treated the character design, AI behavior, and combat completely separately, because I didn't know any better. I took the easy route - enemies were mainly different in their health and how much damage they could do, with the exception of the web-spitting spiders, and the only distinction for the boss was that she was huge and much stronger than the little minions, but no interesting differences and no narrative reason why there was one enormous spider surrounded by a horde tiny spiders.

Why are they robot spiders? What made them? The spiders are pretty scary, but their creator is probably pretty terrifying. Why are they attacking the player? Blood lust? Or are they defending their nest? Perhaps seeing a mother spider with an egg sac could help shed some light on their behavior. If I could go back and do it again, I would begin by considering why are these characters here, and what is their incentive to attack the player. I would also draw inspiration from nature to come up with more interesting behavior instead of run in and attack.

Source: The Hunt, Netflix

Spiders are bizarre but highly specialized hunters, with unique features and behavior adapted to any environment they live in.

Source: The Hunt, Netflix

This is a Portia Spider hunting another spider by dropping down on it from above. The Portia Spider specializes in hunting other spiders, so rather than building her own web she goes out on the hunt.

Source: The Hunt, Netflix

 This is a Spitting Spider that shoots poison out of its mouth at its prey. And this Spitting Spider could end up being a meal for Portia.

Source: Africa, Netflix

And here is a spider that is cartwheeling down a hill to escape a predator. Spiders are an amazing example of how their actual behavior in nature is probably more profound and well-designed than anything we could come up with when brainstorming a spider-based character for a video game. This world has wonders and terrors beyond our wildest imagination.

Source: Image By Jon Richfield

 If you are concerned that spiders are just not large enough in real life to inflict the kind of terror you are looking for, then allow me to present to you the Huntsman spider which can get up to one foot in diameter, and can move nearly a meter per second. That sounds pretty terrifying to me.

In part 4 I'm going to talk about modes of hunting (and avoiding being hunted) that can be found in nature.

Game AI: Non-Human Behavior Part 2

This is part 2 in my series on non-human game AI, if you missed part 1 check it out first!

In nature, survival and reproduction are the two biggest driving factors of decision-making.Let's start with survival.

Starting with the big picture view, we know that all living things require energy to live, ie. food, and organisms have evolved different techniques based on where they get their energy from. One way to break this down is to figure out whether a creature is a predator or a prey at various points in the food web.

At the top we have organisms that are always predators, and generally nothing hunts them. But moving down the chain, there are creatures hunt, but are also hunted. This is where we can see some interesting behavior trees. Just from the high level goal of "survival" these creatures will need to make decisions about which is more important - avoiding a predator or finding food. If food is plentiful, that decision is easy, but if that creature has gone a long time with out food they may take bigger and bigger risks to find food, encroaching into areas they know to be dangers.

From a design perspective, what's interesting here is we can use this information in two very distinct ways. 1. Some games, especially hunting games and some survival games, attempt highly realistic simulated environments with a balance of creatures that exist for the player to hunt. These games can use information about predator/prey relations to generate believable content. 2. Beyond realistic simulations, in any game with enemies we can regard to player as a part of the predator/prey relation. The player wants to survive, so she must defend herself from enemies either with stealth or armor or by attacking and killing the enemies first, however most games have other objectives and the player must decide how much risk they are willing to take to accomplish those objectives.

In any scenario in games that have enemies, we can decide - do we want the player to feel like a predator here, or prey? Do we want the player to feel sneaky and clever and avoid getting caught/killed, or do we want the player to feel powerful and dominant and on the hunt? Both options create interesting dynamics, and a lot of games alternate between the two to create powerful exciting experiences.

As an example, in World of Warcraft if you encounter a high-level creature too early you will probably try to avoid it because you know it's stronger than you, but once you have leveled up you might return to fight it once you know you have a chance to defeat it.

Let's design an AI for a creature that is in the middle of a predator/prey situation. I like to think of AI decision-making systems as a sort of pro-con list:

This gives us an idea of the possible behaviors the AI might take, and what some of the factors are that determine that decision. Based on our pro-con list, we know that the primary decision we want to focus on is "stay and eat" vs. "run and hide" and we know that some of the factors include how hungry they are, how prevalent the food is, how dangerous the predator is, how close the predator is, and whether or not the predator has seen them. Now we can work on prioritizing these and converting them into a decision graph (or a behavior tree or state machine, depending on your approach).

This would obviously be different for different types of creatures, and it is also a very simplified solution - it only covers one very specific decision and two possible behaviors. Generally AI will have a lot of possible behaviors and different decision factors across the spectrum of possibilities. That is really what makes AI design such a challenging area to work in.

The complexity of the AI design depends heavily on how realistic the behaviors need to be. In theHunter:Call of the Wild the designers knew that players wanted a realistic hunting experience, and that often players would spend a lot of time watching an animal before taking a shot, so the designers had to be prepared to do extensive research on how those creatures behaved to ensure a believable experience for the player.

Most games are not held to quite such high standards of realism. Creatures in Legend of Zelda do not have parallels in real life, so they have the flexibility to be weird and wonky and still be believable.

In part 3 I'm going to get into some of the stranger behavior in nature, and how we can use it as inspiration in AI design.

Image Sources:
http://www.cbc.ca/lifestory/extras/stories/8-fascinating-baby-animal-survival-techniques
https://blog.oup.com/2013/12/future-human-plant-animal-survival-in-arctic/
 http://www.huffingtonpost.co.uk/2014/05/06/national-geographic-traveler-photo-contest-2014-entries-pictures_n_5271505.html
https://www.tes.com/lessons/Jxdjqc0dlasfEA/food-chains
http://www.gamasutra.com/blogs/KarinESkoog/20170713/301653/theHunter_Call_of_the_Wild__Designing_Believable_Simulated_Animal_AI.php
 http://www.wikihow.com/Play-World-of-Warcraft
 https://www.youtube.com/watch?v=937xoXVfs0k

Game AI: Non-Human Behavior Part 1

In this series I will be talking about game AI, specifically designing behaviors for non-human characters. At the end of August 2017 I will be speaking on this topic at PAX Dev, so this blog series is where I will be organizing my material and hopefully getting some feedback! So let's get started.

Game AI is one of those areas of game development that is both very technical and very design-heavy. However, it is important not to overlook the role that art and sound play in making the AI believable, understandable, and relatable. When we talk about AI in games, we are talking about anything in the game that makes decisions for itself, and specifically what is the process for making and acting on those decisions. We often use either a behavior tree, or a state machine, or some combination of the two, to determine what an AI is going to do at any given moment. The AI also needs to store and process information about its surroundings and its past actions to use in its decision-making.

I think it's safe to say that we are all humans, and so when we approach a design problem we approach it from the human perspective. When designing something that makes decisions, we instinctively design it to make decisions the way we make decisions. Obviously this works out well when designing AI for human characters, but not so well when designing for non-human characters.

There are plenty of people out there talking about game AI for human characters, and they know way more on that subject than I do, so for this talk I will not be covering anything to do with humans. When I say "non-human AI" you are probably thinking of animals, and specifically of mammals, probably most likely mammals on our size-scale like cats.

I'm going to talk about more than just cat AI, in fact I'm going to go beyond mammals or animals in general. There are a lot of options out there for non-human inspiration and design - fish, birds, insects, arachnids, plants...robots...zombies...aliens...I mean, like, real aliens...

(not an alien, this is a real Earth creature)

...anyway...

Let's talk about what constitutes AI in games. First I need to clarify, AI for games is not at the same caliber as AI in the world of computer science and machine learning. AI for games is essentially dumb AI, it is constrained to what is needed for gameplay purposes and to appear to be making informed decisions to fool the player. In fact, not only does AI in games cut corners, it also often has hindrances applied to it to make it feel more believable and natural to players. Players tend to feel cheated if the AI seems too smart, even if it is accurately acting on the information it is given.

Ok, so Game AI is not the same as real AI. Let's also talk about some definitions, just for the sake of clarity. An Agent is something that uses information to make a decision through a process of Action Selection. An Agent in a game could be a friendly NPC, an enemy, a semi-controllable simulated character, or in some cases a background character, or a companion (such as a pet). AI Agents generate responsive behavior based on a predetermined set of stimuli, to give the illusion of intelligent decision-making.

Now on to the point of this series: how do we as humans design AI for things that are not humans? There is also a larger problem here, once we manage to design something that does not make decisions the way we do, how do we make sure the player understands those characters and their decisions? Not surprisingly, a common way to make non-human characters relatable is to anthropomorphize them.

Let's avoid that approach for now, because if we are going to anthropomorphize our characters then there is no real reason they shouldn't make decisions like humans. In this series I am going to talk about some amazing things that really exist, things we can use as inspiration for our AI design. Things like flocking, hive-mind, predator/prey relations, natural selection, and aggressive plants. Stay tuned!

Food for thought: if a tree grew fast enough that we could watch it engulf something, how scary would that be?