You are not logged in yet.

It can be said with certainty that augmented reality has been struggling to find opportunities.

When Nintendo 3DS was launched in 2011, it was bundled with a series of marker cards that can be used for AR-driven mini-games, and was pre-installed with the AR title Face Raiders. It can see you launch a tennis ball at the flying robot. The face comes from a photo taken using 3DS. camera.

In the years before that, I remember magazine covers, pages, and flyers as markers for AR mini-games or dioramas to promote larger non-AR games. You can take it a step further-after all, what is EyeToy: just play a series of augmented reality mini-games in the living room? These are concepts that have gradually expanded over the years, from Sony's Invizimals to Microsoft's many promises to HoloLens.

In 2016, the launch of Pokémon Go sparked a wave of AR photos featuring Pikachu and close friends. These photos were imposed on real-world landscapes, which at the time felt like the beginning of a new wave of augmented reality. Of course, the novelty of this visual technique faded away, and developers turned to focus on recreating Niantic's location-based gameplay, rather than its AR capabilities.

At the time, Niantic's mobile phenomenon did feel like a step forward in technology, but five years later, it is hard to say that augmented reality has not yet become a foothold for making games or any other type of major technology. Experience. The repeated attempts to make AR happen show that the technology must be promising, so what is holding it back?

Gabriel Brostow, Niantic’s chief research scientist, told GamesIndustry.biz: “The AR project has been completed and released for more than 20 years, but [it] only really started five or six years ago.” “There’s a reason why we say it’s really here now. Like when Skype appeared-there were video calls before that, but suddenly my elderly parents could do it without a lot of guidance. I think this is where we are now."

We are talking with Brostow at the recent launch of Niantic's Lightship platform, a set of development tools that can help other studios use the company's knowledge of augmented reality and the latest developments in its technology.

One of them is semantic segmentation: the ability of computers and even smartphones to analyze camera information and recognize aspects such as sky, ground, grass, trees, and solid objects. Brostow said that the computer vision community has known about this technology for several years. It could even be used on computers and laptops three years ago, but it was later condensed to smartphones. If nothing else, Pokémon Go proves it: Augmented reality is likely to take off only when it is provided in a smart device and used in multiple locations, not just at home.

"It's absolutely fair," Brosto said. "AR really should be a wide church. [EyeToy], that's AR, and it's connected to your console, which is great. But being able to carry it with you, that's a new generation."

Semantic segmentation has a broader impact than simply recognizing your surroundings; it also supports occlusion, so objects on the screen may be obscured by objects in the real world, or even interact with them. Niantic Lightship’s demonstration showed mobile users throwing virtual balls in a real-world environment, and then the balls bounced off objects such as lamp posts and benches.

"We have a way to enjoy technology, we have good game designers, but we can't do all of this. And we know the industry will only continue to grow, so why try to kill it?"

At first glance, this is something that the human brain takes for granted, but when you realize the meaning of it, the virtual world can interact with the physical world, which opens up more possibilities for AR games. Without it, augmented reality is just the digital layer on the video feed. Pikachu is not really in your living room; he is on your phone, it looks like he is just sitting on your sofa, whether you have placed the phone correctly. But with semantic segmentation and occlusion, the interaction opportunities in AR increase dramatically. It also helps keep AR away from new things and makes it an integral part of the experience.

"If you had to make AR experiences before, you would focus on flying things so that they don't have to interact with any surface," Brosto said. "Humans will accept this. But the contact with the real world... This is completely similar to the digital special effects in movies. You can have completely excessive special effects-for example, Jar-Jar Binks. But there are other special effects that are very subtle So that you don't notice them, such as the robot boy in artificial intelligence: artificial intelligence never blinks.

"This is the next frontier of AR, so let's bring it to the outdoors and people."

Phil Stuart, the founder and CEO of British studio Preloaded, is one of the developers who have tried Lightship. He said that augmented reality is all about the world. He added: "The ability to understand the world allows us to create more contextual experiences. Knowing that I am looking at the sky, the grass... When the camera knows what it is looking at and can create a digital experience" This reflects this, It is very powerful. It has a certain degree of intelligence, makes people feel credible, and really eliminates the suspense of doubt. This is the opportunity we have. It is too early, but it is the beginning of a very exciting journey. "

Lightship was launched last month and is currently free for all developers-although multiplayer game studios with more than 50,000 users will face undisclosed fees in six months to compensate for the impact on Niantic's servers. The revenue model will be fully announced later, but the company is currently eager to let more developers try AR.

Some examples shown at launch include partnerships with Historic Royal Palaces (users can plant and water flowers in the moat around the Tower of London), American PGA (players can start a round of AR golf in any open space), and Koche La Valley Music and Art Festival (the app allows you to see a giant butterfly fall on the rainbow-colored Spectra Tower). Plans to establish more partnerships with the Science Museum Group, Japanese manga publisher Shueisha, and health app Tripp.

"Niantic will stay in the game and always make games. We may also make other things, but games are our source"

The fact that Niantic provides its technology to such a wide range of companies for free (at least for now) is strange. Since the launch of Pokémon Go, many other developers have tried to build similar games around IP, such as Jurassic Park, Ghostbusters, Minecraft, and The Witcher, but none of them have reached the level of success enjoyed by Niantic's flagship games. Why reveal the secret weapon behind this and future projects?

"When I joined Niantic, I asked a very similar question," Brosto admitted. "They announced on my first day [as early as 2018] that they would open this platform, and I thought,'...what?!' I was gradually convinced.

"Niantic is a startup — a large startup with 700 people now — we have too many projects, we can’t take on all the cool and creative things. And it’s not a zero-sum game. The projects seen here come from the selected teams, not because they have no competition, but because they are interested and savvy enough to try the most cutting-edge technology. They will do a good job of using it.

"We have a way to enjoy technology, we have good game designers, but we can't do all of this. And we know the industry will only continue to grow, so why try to kill it?"

The growth in the AR field is the key driver here. As we have determined, augmented reality has been working hard to achieve the expected adoption among developers and users in 2016. Brostow even hinted that there may be "another generation of'oh, now this is real AR'". He has not mentioned yet, he pointed out the potential of AR headsets-he told us that Niantic is collaborating on this.

Of course, as a developer, Stuart is full of expectations for the future of AR: "When I first started making games, it was to create the most compelling experience on the screen. With AR, we have the opportunity to pass through our screens. Looking back at our AR is a blurred line between the digital world and the physical world-if you look at and play in the real world, is it a screen activity? This is a different type of experience.

Being able to play real-time AR multiplayer games at any location, smartphones can detect the type of terrain, this will be a big step forward

"For me, this is an important moment because it is the attraction of the screen and the reality of our world. I think this is a very powerful thing. Pokémon Go has done this to a certain extent, but we still It’s too early and we are still trapped by the screen size. Really, this is a stepping stone to the realization of hands-free AR glasses. All these are precursors, and it is the prelude to the main events that are coming in two or three years."

He acknowledged that AR will continue to be "a subset of the large mobile game [market]", but insists that it "is a powerful, interesting and will continue to grow." He was particularly excited about the multiplayer technology provided by Lightship, pointing out the effect of Pokémon Go in bringing people-even strangers-together to participate in the game, and added that real-time multiplayer is a particularly big opportunity.

He also stated that, as demonstrated by the series of partnerships that Niantic has established, AR’s promise goes far beyond gaming: “Pokémon Go and Pikmin will be the foundational parts of the platform, but the opportunity they are trying to pursue is about watching These new types of users, new behaviors across different verticals. The way I look at it, how Niantic uses Pokemon Go and similar games to change people’s behavior and bring it to new industries such as retail, concerts, art, and culture ?"

This is certainly in line with Niantic's ambitions displayed at the press conference, although Brostow is keen to emphasize that the company will not deviate too far from its roots.

"Niantic will stay in the game and always make the game," he concluded. "We may also make other things, but games are our source. They are the perfect challenge-if you want to understand a programming language, if you want to understand a platform, use it to write games. You have to do logic, Graphics, network-you have to know it, because you can't give up any part of the technology in the game. This is the ultimate proof of your technology."

Sign up for the mobile newsletter and get the best GamesIndustry.biz in your inbox.

"We have a common vision to build a community around shared experiences," the studio said

The Pokémon Go developers aim to create a "evolving AR map of the world"

Production Assistant-PC/Console and Mobile Device-UK, Southwest-Open to juniors and graduates

Simplified Chinese localization QA senior tester

There are no comments for this article. Why not be the first to post?

Need an account? Register now.

Latest articles from our team

"The Next Frontier of AR": Why Niantic shares its technology with Lightship

Roblox criticized the lack of child protection measures, similar to stock market collectibles

Record sales shortage: Consumers face a frustrating Christmas | Viewpoint

Christopher Dring Game B2B Director

Xbox is the game publisher of 2021 | Viewpoint

Tell a generation of unknown stories in Windrush Tales

20-year-old Miniclip: "I think the console experience is a complete failure for us"

Jeaneane Falkler appointed as President of Technicolor Games | Work Overview

March 3, 2022-March 5, 2022

September 22, 2022-September 25, 2022

Peter Bond commented on

Gary Bowser pleads guilty to change in Team Xecuter piracy case

Ron Dippold commented

Ubisoft’s head of human resources: “people lost their trust” during the reporting process

Fernando Brischetto commented on

Do the old game consoles stand up today? | Business this week