Following the River Home; Video Game Semiotics

By Josh Snyder

When you first load up Sir Hammerlock’s Big Game Hunt, the third piece of downloadable content (DLC) for Borderlands 2, there is no quest marker. Walls of stone tower around you, obstructing your view. There is no one to greet you – no Hammerlock, no CL4P-TP. In this alien world, the game gives you no clues on how to proceed. Yet every gamer, within seconds of experiencing this world for the very first time, will no doubt run down alongside a flowing river, around the corner and off a cliff, into a wide open world with a rather large hunting lodge in the center. From there the story unfolds, and the player gives no thought at all as to how they arrived there.

rivers used as semiotics in video games

Follow the water to the open expanses of Skyrim.

Similarly, players of Bethesda’s 2011 hit Skyrim face a dilemma early on in their adventure. After escaping into a series of caves beneath the town of Helgen,the player, accompanied by a companion, must flee the city and the dragon attacking it. The player is lead by their companion for most of the way, until there is a fork in the path. One trails off into a dead end, the other leads you out into the wilderness of Skyrim. And again, with little thought or hesitation, players choose the correct path, following the bubbling river out of the caves and into hundreds of hours of flower collecting and butterfly hunting (in fact, ignoring the river and following the other path will reveal, to the keen observer, some hidden treasure, further emphasizing that this path is not to be commonly taken).

In each case, the developers have told gamers exactly where to go, without using so much as one tiny arrow. Follow the flowing water, and you will find your destination.

 The Power of Symbols

When the medium of film was first evolving into the art form we today take for granted, it adopted the term semiotics. Semiotics is the study of signs and sign processes. In other words, cinema created a set of symbols and signs that audiences began to understand and interpret – its own cinematic language. Want to make a character feel powerful? Shoot them from a low-angle; the character will appear to tower over the audience as the camera looks up at them, and they will be seen as strong, imposing. Want to make a character appear weak, feeble? Shoot them from a high-angle; the audience will now tower above them. The classic “damsel in distress” is almost always shot from a high-angle, because the damsel is weak, frightened and needs to be saved by our handsome, muscular hero, almost always shot from a low-angle. These techniques allow writers and directors to convey information to the audience without having to take the time to spell it out. We instantly know traits about these characters, just by the choice of cinematic symbolism employed by their creators.

Arrival of a Train

The train pulls into the station at La Ciotat. Image first appeard on

As the medium of film grew, its use of symbols not only became more complex, but more understood. As a direct result, film stories became more intricate, as less time was needed to tell the audience what was happening in each scene – the audience could decode the language as they were processing the rest of the film. There is no doubt that, without the use of semiotics, contemporary film would be as advanced as the Lumière Brothers’ now famous Arrival of a Train at La Ciotat, a film (generally considered to be one of the earliest motion pictures) simply about a train arriving at a station.

In the early days of video games, stories were just as simple. In the Atari classic Pitfall, you simply move from one side of the screen to the other, swinging from vines and jumping over alligators. There was little need of symbols to convey complex ideas to the player – just avoid the monster trying to eat you.

But just like film, the medium of video games changed. In the original Metroid, players were introduced to a new alien world, and for many, this was their first experience with atmosphere in a game. Yes, we know to avoid those spikey-things on the wall, but the world felt so cold, so empty. Everything was an obstacle ready to kill you.

Video games were becoming more and more complex.

Complex Games Demand Sophistication

In The Legend of Zelda: Ocarina of Time, the main protagonist, Link, is a helpless, lazy kid. At the start of the game, he has absolutely no chance of vanquishing the evil that plagues Hyrule. Unprepared, he is thrown out into an open world. Volcanoes erupt in the distance, creatures lurk at the bottom of giant lakes. Villagers are themselves helpless, watching as their world ends with no resistance. Yet Link is able to build up an arsenal of weapons and skills that allow him to stand up and face this evil threat. Of course, he is successful, and the game ends on a rather optimistic note.

Navi annoys

Navi annoys players in Ocarina of Time. Image first appeared on

But how did Link get to this point? Despite some amazing level design and an engrossing plot, players often found themselves wandering the vast fields of Hyrule, with little clue where to proceed to next. In these moments, a tiny glowing orb would appear, circling around  Link’s body, impossible not to notice.

“Hey, listen!”

These words still haunt a generation of gamers. For everything Ocarina of Time did right, it got one key element horribly wrong – your guide, Navi.

Navi was a fairy that followed you everywhere. Ancient crypts filled with zombies. Underwater palaces teeming with carnivorous fish. Everywhere. And when Navi felt that you were lost, unsure of where to go, it would shoot up and across the screen, quickly following with a one-two punch of irritating and annoying dialogue.

It was easy to simply say that Navi was annoying because of the sound it made when it (seemingly) constantly appeared before you. But what made Navi so annoying was that it was the only way the game could communicate to the player where to go, which town to head to next, which path to take. Navi was your guide, your giant, glowing arrow, and gamers were right to feel like it was a poorly implemented concept. We expect our games to communicate to us, and when one did so on such a basic level, it felt wrong, out of place.

Clearly, there is a need for our games to not only talk to us, but to do so intelligently. And much like movies, that language needs to be easily understood, without getting in the way of the rest of the elements that make a video game an engrossing experience.

Settling an Old Debate

At the beginning of this current generation, the main discussion on where video games would take us was not centered on powerful processors or frame rates. It was whether or not these games could be considered works of art. Just like cinema lovers in the 1940s and 50s, the contemporary gamer had found themselves fighting an uphill battle to declare that the medium is indeed an art. Many arguments have been presented, but what always struck me was how these arguments simply compared video games to movies. Shadow of the Colossus, the game most often cited as being an example of art in the medium, was labeled as such because of its expert use of cinematography. Yes, this was an important and well done aspect of the game. And it is true that all great games make excellent use of editing, cinematography and dialogue. But there was little attempt made by defenders of the industry to highlight what video games did that was specific to the medium – a key component to defining any medium as art. Cinematography, editing, dialogue – all of these aspects are common in even the most basic film. Gamers needed to present unique aspects of video games for their arguments to be strong. Because of these weak arguments, I always found myself on the side that said video games were not art. Not because I truly believed that, but because no argument presented itself to make me feel otherwise.

And then I stood atop a cliff face, overlooking a hunting lodge in the distance, as water rushed past me and off the edge into the world below.

A Technique to Call Their Own

As I rushed to Sir Hammerlock’s hunting lodge, the waterfall long behind me, it occurred to me that cinema once had the problem of being too closely related to photography. Critics of film said it wasn’t an art form because it had no elements that were unique to it as a medium – everything artistic about it was available in photography. Today, those critics’ views are so off the mark that we laugh them off. But film was able to prove that it was an art form through semiotics. And here I was, listening to developer Gearbox’s patented humorous dialogue, which I would had never experienced if that river hadn’t told me where to go.

The unique aspect of video games is the ability of the player to control their view of the world. In film the viewer plays a passive role – we can only venture where the camera takes us via editing and cinematography. In video games, we can turn the camera, zoom in and out, move our avatars throughout the world at any speed and pacing we want. This is unique to this medium – no other art form has to deal with the viewer/player spinning the world around, looking for holes or cracks, or new possible adventures otherwise unseen.

I believe it is here where we can begin to establish video games as an art form. Developers have a language, a short hand to tell us how to navigate their complex worlds. And without any thought, we see these symbols and we process them, allowing us to run off and enjoy these worlds. The issue currently facing gamers is that, although this language exists, it isn’t entirely fleshed out, and some developers fail to employ it when crafting their games. The language needs to be established and remain consistent for it to truly have an impact on how we process video games.

The Power of Symbols in Video Games

Where the idea of semiotics in video games gets interesting is in how developers can provide us with this language organically. It seems obvious now, but the first time the player ventures into a cave in Skyrim, the water is there guiding them where to go. You can still get turned around and lost, in a way that any great adventure will allow you to, but at any moment, if the player has had enough and wants to leave, they can just follow the water out.

go this way

Arrows light the way in Halo: Combat Evolved.

As games become more sophisticated, so will the way in which these symbols are conveyed to the player. Unfortunately, examples like Navi from Ocarina of Time still exist today. One of the most famous examples is Halo: Combat Evolved. When the game was first released, the nav point system, which puts an icon on your screen, pointing you toward your objective (and even including the distance that objective was from your current position) was heralded as a revelation. But simplistic and obvious signs were implemented elsewhere – as you explored Covenant bases, the pattern on the floor just happened to be in the shape of an arrow, conveniently pointing you toward your object. These methods of guiding the player may have seemed revolutionary for their time, but today they seem heavy handed, too obvious. Sadly, the Halo franchise has continued to implement these design choices, even as recently as Halo 4.

What the industry needs more of are games like Mirror’s Edge. A first person platformer, the game was, from top to bottom, a radical experiment, especially for a publisher such as E.A., known mostly for their “safe bets” (i.e. Madden, Battlefield; sure-fire hits that also do not seek to advance the medium in any discernable way). The gameplay relied on aspects of parkour (freestyle running), and encouraged the player to run full speed across rooftops and down fire escapes. Players were tasked with navigating complex environments, all while police chased them, doing their best to gun them down. How did the player know where to jump to without given a chance to analyze the environment? In a mostly white city, your path was always marked with red paint. Water pipes were red, doors were red, ramps were red. And the great thing is that this color palette fit in with the aesthetic of the world – it wasn’t immediately obvious that players had to “follow the red,” but without having to dissect and analyze the game, players figured it out intuitively, and in no time were leaping from rooftop to rooftop.

I imagine that the future of gaming will mean a future where these symbols are well known, they are varied in appearance and implementation, and can convey a whole host of information to the player, all in an instant. Perhaps, if the developer isn’t so concerned about how the player will navigate their world, they can focus on character development and dialogue. Only when video games have their own fully developed semiotics will they be able to surpass film, music and literature as truly the most immersive, personable art form available.