What the dawn of the 5G era means for immersive technologies

The dawn of the 5G era will see a step-change in mobile network speed, reliability, and capacity.

Many technologies will see performance enhancements as a result, but perhaps none will be so transformative as those set to shape VR and AR.

In this article, we look at how 5G will change the game for immersive technologies, allowing us to do things which are impossible today.

What is 5G?

The next generation of mobile internet connectivity, 5G networks promise faster speeds and more reliable connections on smartphones and other devices than we could have ever dreamed of.

How fast? Well, there have been claims that 5G could eventually offer speeds of 20Gbps.

That would mean you could download an ultra-high definition 4K movie in the same amount of time it takes to read this sentence.

We’re a way off those sorts of speed right now, but there’s no doubt 5G is going to be much faster than anything we’re used to today. 

Combining cutting-edge network technology with the power of the cloud has the potential to revolutionize virtual and augmented reality, Foundry Engineering Manager Jon Attfield says: “I think the real difference 5G will make is that you can start to think about putting processing in the cloud. 

That’s because on making the connection, the latency on the roundtrip of data is much lower than 4G. 

All the heavy lifting,  all the rendering, can be done in the cloud, and we can actually just stream images down to the device in real-time, because we have both the bandwidth and the latency to do that at that point. It's not possible with 4G because the round trip on the latency is just too high.”

Lower latency, more immersion

Latency - the time lag between a request and the response - could see a 10X decrease on 5G networks. 

It would mean that web pages will load completely in one second; the days of waiting several seconds will be gone.

For VR and AR, lower latency will make for far more immersive experiences. This is because your device will respond to your movement in real-time. 

As Jon explains: “The trick with any immersive technology is the smoothness of the experience. Anything that judders or lags breaks that immersion.”

5G will effectively open up a highway between the collosal computing power of the cloud and the mobile devices in our pockets. 

This will allow us to do a lot more realistic rendering for augmented reality experiences, he continues: “The challenge with AR is that the digital object needs to appear as if it’s really in the environment. 

And to do that, you need to quite a lot of heavyweight rendering - you need to take the light from the current scene, you need to do some processing on that so you can match it roughly in the renderer, then you need to actually render the scene - and it needs to reasonably high-res, because otherwise it doesn't fit in with the image on the camera. 

Achieving that on a mobile device is really hard. Even if you’ve got a device capable of that sort of rendering - say an iPhone 10 or more - it will kill your battery life. You could run the application for 15 minutes and your battery would be dead. 

Now you can do all that heavyweight processing up in the cloud, the device just becomes a receiver for the imagery: it just does that final composition of that render into the camera image, which doesn’t need a lot of power.

So all of a sudden, you can have real-time experiences that last longer than 15 minutes.”

Powerful VR and AR, anywhere

The implications of this for immersive experiences are huge. 

Currently, high-quality VR headsets require a powerful PC to run them. That means your enjoyment of VR is confined to a specific location. 

With the arrival of 5G pushing the storage, power consumption, and processing power away from the PC and into the edge cloud, all you would need for these experiences would be a headset (for VR) or a phone (for AR): “The big problem with VR right now is that it’s almost always tethered”, Jon says.

“But you can break that now, because you can put a mobile reception device in the headset, do all your rendering up in the cloud, and just stream it down anywhere there’s a 5G network signal.

“It no longer has to be in a room, you don’t have to be on a wireless network, you could just be out in a field, having a VR experience, anywhere you can get 5G reception.”

5G could also enable higher fidelity of graphics to be streamed onto VR headsets, Jon explains: “Let's say we’re a bit further down the line, in the future, and we’ve got 4K screens in our VR headsets. Transferring that much data across is not possible in the existing technology. But you could actually stream that image data across on 5G.

And certainly in terms of the accuracy of the render, it allows for that higher fidelity, because again you can push that up to the cloud and get those images streaming down.”

At the foot of the mountain

So how long will it take for 5G to really start to have an impact VR and AR? 

Ultimately, that depends on how long it takes for widespread 5G network coverage to become available, and Jon believes this will all come down to the level of investment put in: “If you look at 4G, signal across most of the cities in the UK is pretty good now, but outside of the cities, it can be patchy. 

It’s going to be a very similar thing for 5G. The difference is that 5G antennas are short range, which means you’ll have to have more of them to cover a similar area. 

I think you’ll have good 5G coverage in the centre of London pretty soon - whether you’ll see it in the wilds of Derbyshire? That could be 10 years, 15 years.

I don’t think you'll ever see full 5G coverage across rural areas, just because of the density of antennas - it’s just not going to be feasible. 

So you’ll just fall back onto the 4G and 3G service, as you do these days.”

He also predicts that, at least in the short term, 5G will prohibitively expensive for ordinary consumers: “You remember in the early days of 4G, you were paying £40 a month for a gig of data? It’s going to be like that. 

So I think that will limit the initial take-up. It will likely be a business technology initially, because it’s going to be expensive. But eventually it will migrate down to the rest of us, and we’ll get to have fun with it. 

I think realistically, we’re looking at a 5-10 years lifecycle: you’ll see the first ones in the next two years, it’ll be a reasonably affordable tech in 5, and pervasive in 10.”

The change is already happening

In the US, 5G has been slowly rolling out since late 2018 when service provider AT&T launched its 5G service. Since then, other carriers have joined the fray, including Verizon and Sprint. 

In the UK, it’s telco EE leading the charge. 

As Jon correctly surmised, coverage is currently limited and costs are somewhat eye-watering: the lowest-priced deal is £54 a month plus a one-off £170 fee for a compatible handset (note, that only buys you 10GB of data a month, which you’re likely to chew through fairly quickly).

And the technology is still very much in the trial stages, with EE having brought 5G to this year’s Glastonbury Festival, giving festival-goers an early chance to experience the technology at the event.

What’s undeniable though is that 5G is here, and slowly but surely, it will seep out across the country. 

Tantalizingly, technology companies already appear to be making moves anticipating the 5G revolution: Google’s Stadia platform (think the Netflix of gaming) was surely conceived with an eye on the lightning fast speeds the new generation of connectivity will enable.  

Similarly Niantic, the mobile game developer behind Pokemon Go, have an eye on the possibilities 5G holds for the AR games and experiences they develop. 

At the Mobile World Congress in Barcelona, CEO John Hanke showcased a video of a prototype Niantic has developed, codenamed ‘Neon’, that allows multiple people in the same place at the same time to play an augmented reality game. 

Players can shoot at each other, duck and dodge, and pick up virtual reality items, with each player’s phone showing them the game’s graphics superimposed on the real world. 

But the game depends on highly responsive wireless connections for all the phones -  connections unavailable on today’s 4G LTE networks.

“We’re really pushing the boundaries of what we can do on today’s networks,” Hanke said. “We need 5G to deliver the kinds of experiences that we are imagining.”

What’s clear from these developments in 5G is that we’re at the beginning of a step-change - one which will undoubtedly play a big part in the rise of immersive technology in the years to come.