ALTstage is the world’s first technology that brings AR together with live events and video content. It came to life using Azure Kinect hardware and volumetric videos.
Its main goal is to let the watchers enjoy the three-dimensional views of performers, whether at home or live. Just with the help of their mobile phones.
Due to the Coronavirus outbreak, more live performances are being broadcasted than ever. However, the two-dimensionality of videos quickly became an issue. Videos take away the thrill of the theater as they just can’t convey the beauty and grace of a performer at work.
We were tasked with developing an app that would solve this problem with AR technology. The app was required to allow viewers to see performers in motion in augmented reality.
Our clients wanted the audience to have three-dimensional views of performers, whether at home or live. The app was required to be compatible with modern gadgets and also suitable for low-budget projects.
The ALTstage app is the first step in a broad project. Our clients’ end goal is to develop a big ALTstage platform that will be used by performing artists around the world.
As always, the first step for the WeAR Studio team during any project is getting a clear idea of what our client hopes to achieve. We worked hand-in-hand with the Ukrainian Platform of Contemporary Dance and the German Post Theater from start to finish to ensure that we achieved the best results.
After understanding the project, we proceeded to a Chromakey studio to record a volumetric video. We then processed it to procure a material in which the performers were taken out of the background. That way, we were able to add any background we desired.
We got this project done using the Azure Kinect hardware. The team had to use a 12-megapixel camera along with an infrared sensor.
We recorded 2D images with the camcorder and measured the distance between the device and the pixels of the image with the sensor. That way, we were able to transform the video from 2D to 3D.
We developed a software environment that synchronizes AR with the performance on a screen. Two devices were needed for this; a laptop for playing the video and a mobile device reproducing the augmented reality.
The synchronization in space was done with the aid of markers. They transmit the details of the desired location and size of the AR object to the app via the camera of a smartphone or tablet.
However, time synchronization is a new technology created by the ALTstage team. The principle behind it is simple: the laptop’s video player and the ALTstage app receive instructions from a server to play video and AR. This way, the performers in the video and AR can interact.
This process was all about making the technology easy to use. The importance of this was to allow viewers to enjoy live events instead of concentrating on the complexity of the technology.
Identifying the goals of the project
Converting 2D images to 3D
Developing time synchronization
Mastering volumetric video technology
Combining two modes (live and home) in one AR application
Building a foundation for the ALTstage platform
This technology isn’t restricted to the art industry alone. As long as your event has an audience, you can give them spectacular views with ALTstage.
After the presentation is held in a few cities across Ukraine, the ALTstage platform will be fully launched so that anyone can use it. What this implies is that performing artists around the globe will be allowed to create their projects on the ALTstage platform.