Mar 9, 2020

Multiplayer in Augmented Reality

Frame 1 1 1 - Multiplayer in Augmented Reality

Augmented reality apps and games are a fun way to spend time. Game developers, using AR-technologies, try to immerse users into the game as much as possible, supplementing the physical world with virtual objects. Thus, we can get a new gaming experience by interacting at once on two levels: virtual and real. Surely many gamers would like to share this joy with friends and play together. However, for creators of applications, multiplayer is a rather difficult option to implement. We will talk about the difficulties of implementing multiplayer in AR in this article.

Frame 2 16 1 1024x341 - Multiplayer in Augmented Reality

A single world map for all users

First, the AR device determines the world map for the gamer using the familiar SLAM. Initially, each player sees only his own world map. SLAM approximately determines the distance between objects, then to refine the map.

As soon as a new player is connected, the first user’s devices need to be calibrated to determine the new location relative to space and objects. So, you need to collect control points from Sparse maps in a single format. After, this data should be transferred to the cloud or between devices. This isn’t difficult to implement from a technical point of view, however, there are many small things need to be taken into account for the multiplayer to work correctly.

Frame 2 15 2 1024x341 - Multiplayer in Augmented Reality

Devices need to make sure that players are nearby. And if the players continue the previously saved session, the devices re-determine the coordinates of the world. The difficulty here is that GPS on different devices can easily give different data.

Back to the transfer of Sparse cards. There is currently no standard way to transmit them. Each framework creates its own way, which is in no way suitable for other devices. For example, ARKit has an ARWorldMap for this, which allows you to save the AR map of the world and restore display data after closing the application.

But even with the correct coordinate system and formats, devices can recognize distances on a 3D map in different ways. Depending on the manufacturer, camera angle, accelerometer and sensor calibration, the map on different devices will be larger or smaller.

Relocalization

As for the technical hurdles for multiplayer in AR games, relocalization is one of the most difficult moments. When a new user joins the game, he needs to find himself on the map: calculate his map, calculate the location, determine the control points. After that, the most complicated process takes place, which is called relocalization — when the player’s map is compared at control points with a common map to combine the worlds.

Frame 2 18 1 1024x341 - Multiplayer in Augmented Reality

Difficulties with relocalization appear when players are facing each other, which happens quite often in the game. At this point, the devices cannot determine any control points. To prevent this, many AR games ask players to stand next to each other at the beginning of the session. This is a pretty good decision, but it limits the freedom of the players.

Copying SLAM cards for all devices restricting user visibility are not the only methods of relocalization:

  1. Compass and GPS. Just rely on the GPS of both players’ devices and use latitude and longitude as a coordinate system. This is the easiest method, and, of course, not the most effective. The same object can be located at a completely different distance for each player. Such a UX may not appeal to users.
  2. Markers. Although this is a fairly outdated tracking method, it works fine for some projects. For example, they are suitable for AR installations in museums or board games with augmented reality. Since all users look for example at the game field, this approach solves several problems at once: there is no need to transmit and synchronize anything, and the game field can serve as a convenient coordinate system.
  3. Randomly place objects or relocalize the user. This method allows the device to simply scan the surrounding stead and objects instead of looking for familiar control points. If the application is designed in this way, no one will notice that the device is lost and recreates the world anew.

Real-time changes

In AR games and applications, users interact with objects, move them or, for example, shoot. And all this should be reflected in each of the users in real-time. In this case, the device calculates its position and transfers this data to others. All these actions need to be synchronized.

There are two working methods for this: transfer data via the Internet to the clouds or transfer it between users’ devices. So for ARCore, clouds are used, and for ARKit — the MultipeerConnectivity framework.

Frame 2 17 1 1024x341 - Multiplayer in Augmented Reality

Transmitting changes through the cloud allows you to keep large amounts of information and do complex calculations thanks to its high speed. Also, theoretically, this method allows you to simultaneously work for an unlimited number of users. Of the minuses, there can be noted problems with the players’ privacy and the fact that the method requires a constant stable Internet connection.

MultipeerConnectivity is a structure that detects services provided by nearby devices and communicates with them through the transmission of data based on messages, streaming data, and resources, such as files. Multipeer Connectivity transfers maps and changes without an internet connection with fewer errors. Also, this method is much cheaper than the cloud. However, he has enough disadvantages: small cards, low speed, multiplayer works for a maximum of six devices.

Wrap it up

As you can see, there are many options for adding multiplayer to your game or application. Although now the multiplayer in games isn’t ideal and corporations still have work to do, because play with friends or other users — this is what people really expect from AR. This is worth paying attention to, especially from a business point of view.