Devlog #2: Lost on the way to navigation

August 9th, 2018 | Heri

 Hi Everyone,

 

Today I would like to talk about one of our biggest challenges: the navigation system. Make it short, the navigation is one of the key component of our app since it makes the link between the supermarket, the products and users profiles.. We’re not willing to use a 2d map since it’s been done many times and it’s not interactive compared to an AR navigation. Right now, we saw prototypes but no complete software. We wanted to try our chance at finding a solution too !

 

Native solution with only ARKit/ARCore

After seeing some concepts like Findme (https://github.com/mmoaay/Findme) we planned to use a pure SLAM solution. The idea was that we let ARCore/ARKit do the object positioning on the ground while we use a fixed starting point. While the solution proved to work in the short term, it asked many questions like “How do we do if we don’t start at the same starting point, without even the same orientation?”. We got to the conclusion that this solution wouldn’t be viable because we can’t make sure that after checking a product, the user starts the experience at the same place with the exact same orientation. This solution proved to give more headache than a simple solution so we had to give it up.

 

Good: Good object positioning

Bad: Impossible to know where you start the navigation from

 

Use another technology and use an AR layer

In order to solve the position detection problem, we tried to put an AR layer over an already existing indoor positioning technology. We found a SDK called IndoorAtlas specialized in indoor positioning. They achieve good results indoors on a 2D map with correct camera orientation so we supposed it wouldn’t be too hard to display something in AR with it. IndoorAtlas provides a unity plugin so it was pretty easy to setup. We also found out that we had to dig into some GPS calculations to correctly move the map around the user. Since we don’t have control of the AR camera we can’t position our camera on the map, but the trick is to move the map around the user instead. After multiple attempts at mapping a supermarket we finally managed to get a prototype working. The issue? IndoorAtlas relies on magnetic field to work and the location data need to be processed online before coming back to the phone. As a result there’s a lag between different positions update and sometimes the position difference is so big that even by smoothly translating the result is weird on the screen. IndoorAtlas is aware of the problem and they developed an offline solution, unfortunately the Unity plugin for this version is not available yet. Furthermore, after improving calculation methods and refining the models, the best result we could get was around 1 meter accuracy, which is good for large venues but not good enough for supermarkets where dozen of products are stocked within one shelf.

 

Good: Able to find yourself in supermarket with 1m accuracy

Bad: Update time too long, need a long time to correctly calibrate and AR mode creates jumps that damage the user experience

 

 And now? Let's go for the AR Cloud!

We are now still looking for a way to navigate indoor. After exploring solutions and checking what is achievable with the current technologies, we decided to turn towards the point cloud. Mainly inspired by Place Note and the research done by Kudan, the idea will be to map the space and create a cloud of points. Once the space have been mapped, the phone will use the ARKit/Arcore point cloud datas coupled with machine learning to locate its position in the cloud of points and then find space. Basically we want to do a simultaneous localization and mapping (SLAM) system using ARCore/ARKit. People will argue that these two SDK’s already provide SLAM but as we said before they don’t provide space/scene recognition (ARCore has plans to add anchors but the availability of this feature in China is still to be checked). Fortunately, other startups (6D.AI and Fantasmo) are working on tools to create your own point cloud and we are in discussion with them to establish a partnership.

 

This were just our first attempts in the previous months and now that we're focusing on the AR cloud solution, we should be able to reveal more about it soon!