ARQuake – Communications of the ACM

With the advent of commercially available affordable wearable computers and head-mounted displays (HMDs), it is possible to develop augmented reality entertainment applications suitable for an outdoor environment. We extended an existing desktop game and developed it into the ARQuake system [4], one of the first systems that allows users to play augmented reality games outdoors—allowing them to move in the physical world, and at the same time experience computer-generated graphical monsters and objects.

The game we extended was Quake from id Software (see www.idsoftware.com), a first-person-perspective, shoot-em-up game initially released in 1996. In Quake, the player runs around a virtual world, shooting at monsters, collecting objects, and completing objectives. The game is desktop-based, with the user interacting with it using a monitor, keyboard, and mouse. Although the game is relatively old, the graphics engine is very powerful and runs on a wide range of computing hardware. Recently, id Software released the source code for Quake, making it possible for enthusiasts and researchers to modify the game for their own purposes.

Augmented reality (AR) is the process of overlaying and aligning computer-generated images over a user’s view of the physical world. Using a transparent HMD placed on the user’s head, an internal half-silvered mirror combines images from an LCD display with the user’s vision of the world (see Figure 1). By combining this display technology with a wearable computer, it is possible for the user to walk outdoors and visualize graphical objects that are not normally visible. A comprehensive survey article [1] discusses most aspects of this research area.

The aim of the ARQuake project was to construct a first-person-perspective game with the following attributes:

The game is played in the physical world, with the user able to freely move about the world.
The view is determined solely by the orientation and position of the user’s head.
The game is experienced as augmented reality using a transparent HMD.
The game is controlled using easy to understand real-life props and metaphors.

We wanted to build a system that allowed the user to play the game in a natural way. The user can see the game monsters at their virtual locations, and use real-life props such as a plastic gun with simulated recoil to shoot at the monsters.

Our experience with test subjects has shown they find the game very natural, since the haptic feedback gun operates in a way people are used to from other experiences, such as movies. Moving and looking around in the game world is simple, the user just has to walk in the appropriate direction for the required distance, or look in the correct direction. To shoot the weapon, the user presses the trigger on the gun prop. The mobile nature of the game prohibits the use of traditional desktop input devices such as a mouse and keyboard.

As previously mentioned, the user’s position and orientation are updated by physically walking around, and we developed Quake worlds where the user was not required to swim or fly. Also, we had to carefully chose monsters that would not be too powerful for the user to play against—in traditional Quake the user has superhuman powers and therefore fighting these monsters is easier. In order to be visible outdoors to the user, we had to modify the skin textures of the monsters, which are normally quite dark and not visible on a transparent HMD. We chose seven different types of monsters, with our choices restricted to land-based creatures in order to enhance playability.

In order to walk around the campus and play a Quake game, a complete model of all the buildings was created and entered as a Quake map. The buildings were all created as solid black objects, so they do not render to the display—in real life we have the actual buildings to provide both visible and haptic feedback on their locations. By having the buildings modeled in this way, this becomes one of our rendering techniques, and any monsters that are located around a corner or behind a building will not be visible until the user would be able to see them in real life. Items such as secret or locked doors are not used in our maps as they can not be synchronized between the real and virtual worlds. The campus model contained 30 buildings and was approximately 350 meters × 450 meters in size. Figure 2 shows an example of ARQuake being played outdoors, taken with a camera through the HMD. A monster with adjusted colors is shown in front of the user, and another is in the distance to the right.

Quake Modifications

Desktop-based Quake uses the keyboard or mouse in order to move the player through the world, and is relatively controlled. By pressing an arrow key or the mouse, the user moves in the specified direction, but when walking around outdoors, the player’s current position must be matched against that inside Quake. To achieve this, we modified the Quake game so it would accept absolute tracker information for the user’s position and orientation. We used a TCM2 digital compass for orientation and a Garmin GPS for position.

Our previous research into AR has led to the development of the Tinmith software system [3]. Components of this system were designed to handle trackers for AR systems, and so we reused some software modules as well as the communications infrastructure. Apart from just trackers, other interfaces were also written to control weapons firing, and also to allow external programs to monitor Quake’s progress by sending out UDP status packets as well. Using these interfaces, the gun trigger can fire off Quake weapons, and the haptic feedback can be driven by the Quake status packets.

It should be possible to play ARQuake in most locations, assuming reasonably accurate maps have been created. The simplest example would be to play in a large open field, with no obstacles and just monsters—in this case problems such as position inaccuracies will be less noticeable because the user has no reference objects. In our example, we used our university campus buildings to provide occluding objects, but due to the accuracy of our tracking devices, the alignment is not perfect—monsters appear to walk through walls or pop out of thin air.

Hardware

The ARQuake system previously discussed [4] runs on inexpensive and easy to acquire hardware, as the software does not require much resources. We use a Toshiba Pentium-233 laptop running Linux, driving an I-Glasses color PAL display, along with a Precision Navigation TCM2-80 orientation sensor, and Garmin GPS12XL with DGPS for positioning. Recently, we have upgraded our backpack system (shown in Figure 3) to use much newer and more accurate hardware, which is used for a variety of other AR user interface research in [2] and [3].

To support the ARQuake game, a haptic feedback gun was constructed. A toy gun was used as the base, with a solenoid and large weight attached used to generate feedback when the user fires the weapon or is hit by a monster. Appropriate interfacing hardware was also built to allow this gun to connect to the laptop. Currently, the gun does not have any tracking devices so it cannot be used by the user as a pointing device, the aim is made by adjusting the head orientation.

Conclusion

We have presented a new and novel entertainment application that uses AR and is based on an existing freely available game engine. We use physical props as well as real-world motion of the user to interact with the game engine so that the rendered display approximately aligns with that of the real world. The entertainment industry is a major driver of technological progress in the computer industry, and currently there are only a small number of games designed for AR systems, ARQuake being one of the first.