Exploring AI and New Uses of Augmented Reality

This semester I started my internship at PreviewLabs’s Belgian office. I’m a student at the University of Ghent studying MSc in Information Engineering Technology. At PreviewLabs, I am working on my Master thesis titled “Evaluation of technologies concerning Rapid Game Prototyping“.

As the title implies I will be exploring and evaluating some interesting technologies for their usability in PreviewLabs’s prototyping endeavors.

They asked me for a picture of my face and upper body, so here is a picture of my face and upper body.

As part of the internship, we looked at different technologies to see if there would be anything new that we could discover and prototype, contributing to PreviewLabs’s experience. Three different cases were selected.

Case 1: Using Augmented Reality and SLAM to See What’s Going on in Virtual Reality

Augmented reality (AR) is a very active field which has progressed quite a lot in recent years. PreviewLabs’s experience with this medium started in 2011 with a dart board prototype on the Google Nexus One smartphone, while with the more recent evolutions in this medium, we explored various uses of the Microsoft HoloLens which augments your vision in a very natural way.

SLAM, or simultaneous locating and mapping, constitutes a way of simultaneously building a map of your surroundings and determining your location within that map. This technology has used in head-mounted AR devices such as the Microsoft HoloLens and the Magic Leap, but also on iOS and Android smartphones through technologies called ARKit and ARCore.

For this case, the idea is to use a phone or tablet powered by SLAM as a window, looking into a virtual world and thus allowing us to spectate an ongoing game or VR experience. This can make it easier to create more compelling videos of our work, or allow our clients to better understand what people are experiencing in VR when they’re testing one of our prototypes.

More on this will be released in an upcoming blog post in which I’ll elaborate more on the details and how certain difficulties were overcome.

Case 2: Using Neural Networks for Artificial Intelligence in a Fighting Game

Neural networks are used in the realm of artificial intelligence (AI). Vaguely inspired by how a brain works, they allow software to make decisions in a way that has been figured out by the software itself. While the conception of neural networks dates back to the early 50s, they are currently quite the hype as they’re becoming more and more useful as a consequence of the evolving processing power. After all, large neural networks run a lot of calculations when executing normally, and even more when being trained….

While neural networks are often mentioned together with AI, not all applications of AI are powered by neural networks, especially in games. Instead, most computer controlled adversaries in games will react in ways that have been entirely defined by the developer of the game. However, when using a neural network, instead of telling the computer controlled adversary or non-player character (NPC) what it has to do, we tell it what it can do and we reward it when it makes correct decisions.

As part of this case, we applied neural networks to a basic fighting game to see what we can learn about it. More on this will be shared an upcoming blog post in which I will discuss the process of creating the prototype. The process of training my NPCs and seeing them come to life was really fun, so be sure to look out for that one!

Neural Network Learning

By having the AI play multiple games against itself the learning process can be accelerated.

Case 3: A Shared Mobile AR Experience

This case is all about exploring the possibilities of multiplayer games in AR within the same room. While PreviewLabs has already done this kind of shared space AR in a HoloLens prototype, on mobile phones and tablets it’s quite a different beast. One of the key challenges in general is that each player lives in their own coordinate space. Things that are already possible (but not straight forward) such as detecting collisions between virtual objects and their environment, suddenly become a bigger challenge – especially on mobile. After all, each mobile device interprets the area around it in a slightly different way, resulting in inaccuracies which may make certain types of gameplay impossible. Understanding the limitations of the technology is key when PreviewLabs advises their clients on key decisions in their projects.

The Buggy Blasters prototype previously developed at PreviewLabs featured a shared experience on Microsoft Hololens – working online as well as when you’re playing with a friend in the same room.

Personally I’ve had a warm welcome here at PreviewLabs. The atmosphere is very friendly: we crack jokes, we drink coffee, we eat together, we play together. I’m here most of the work week, which means I get constant feedback and they can see the progress I make. Their help has been invaluable to my work, and without it I wouldn’t have gotten as far as I have. There’s only a few weeks left for me here at PreviewLabs, and I will definitely enjoy them!

Leave a comment

Your email address will not be published. Required fields are marked *