Through the Mirror World
Mirror worlds are becoming an increasingly popular topic with people in the tech industry, especially those working in the virtual and augmented reality spaces. Kevin Kelly from Wired Magazine wrote an article on the topic in 2019. In his article, Kelly paints a picture where an exact digital replica of the world, which he calls Mirrorworld, will be seen through augmented reality (AR) glasses. And he’s right, that will certainly happen. However, it’s not the full picture; there are other mirror worlds on their way.
Before diving into details, we need to give this expansive topic some shape. For example, we’re dealing with terms including “virtual reality”, “augmented reality”, “augmented virtuality”, “mixed reality”, “digital twins”, “mirror worlds” and “metaverse” to name a few. How the heck are we supposed to sort through all of that?
The simplest way I know how to break things down is from two main vantage points: 1) what I am looking through, and 2) what I am looking at. What I am looking through could be my own eyes, a computer/device screen, AR glasses/goggles or virtual reality (VR) goggles. And, yes, the terms ‘augmented reality’ and ‘virtual reality’ just mean what device you have on your face at the time. Regarding what I am looking at, I divide this into things that are real and things that are fictitious for sake of simplicity.
* For the purpose of this article, I will treat holographic worlds the same as augmented reality even though, technically, you are looking through your own eyes when viewing holographic imagery.
As you can see, mirror worlds can be viewed by a variety of devices. This is one of the reasons why it’s so easy to get things mixed up.
Metaverse
Another thing mixing things up and adding to the confusion, perhaps even more so, is the term “metaverse.” Cathy Hackl, a Metaverse futurist, recently wrote an article about it that was published on Forbes.com. She does a good job laying out various examples of how we might interact with this metaverse in the future. She describes the metaverse as “the concept of a future iteration of the internet, made up of persistent, shared, 3D virtual spaces linked into a perceived virtual universe.” It’s a good article and helps promote the VR/AR industry. But there’s a general problem: the concepts of this future metaverse are so “meta” that they don’t help us back here in the present.
I think the main problem is the metaverse concept is overly idealistic and does not account for the economic factors at play. For example, the ideal metaverse calls for open standards, which are typically developed by open consortiums that are funded by well-established companies who have a stake in the standard. We can’t expect startups to tackle this sort of thing. We all need to eat and pay for the rooves over our heads. Further, given the nascent stage VR and AR are at, it will take a long time to create well-established companies in the industry, never mind forming a metaverse standards consortium. The concepts could easily become obsolete before you even start writing them down. For this reason, I won’t refer to the metaverse moving forward. However, I will say this, I’m sure metaverse will continue to be used in pop culture and by lay people for the general reference of diving into some future device and exploring virtual worlds of their choosing because it sounds cool. I’ll leave it at that.
Digital Twins
Let’s get back to mirror worlds. As it sounds, a mirror world is a digital reflection of our real world. The term was first brought about by Yale professor, David Gelernter, in his 1991 book of the same title. In Mirror Worlds, Gelernter offers a wide array of subjects under the topic ranging from information machines, to software architecture, to an obtuse analogy of a dollhouse. On the whole, I relate Gelernter’s view of mirror worlds akin to the concept of smart cities today. A digital world where you can get real-time answers to questions about what’s happening in your area of interest. This could be anything from energy consumption of a building, to tracking city water usage in real-time, or maybe a traffic simulation of a proposed new bridge. This “answering a question” characteristic leads us to digital twins.
Digital twins are [originally] replicas of real pieces of equipment for the purpose of monitoring, analysing and fixing/improving the actual operating equipment. The first example of this was NASA replicating the Apollo 13 vehicle on Earth to figure out what was happening in space. Another example is aircraft engine manufacturers who maintain real-time digital 3D models of their engines flying around the world. With these models, they can see how the engine is performing, analyse discrepancies and fine-tune performance. The meaning is now expanding to whole cities and eventually to the whole planet.
A couple of years ago I discussed using VR with people investigating smart-city technologies at cities across Canada. They generally indicated VR would not add much value to their smart city. 3D would add some value but a 2D map for a smart city went a very long way in fulfilling their needs. (Surprisingly true for other industries as well.) This was an unexpected but intriguing response. It reinforced a point I learned when training Canadian Air Force personnel on the new radar system for their maritime patrol aircraft.
There are two forms of digital models (or simulations in this case): 1) simulations for answers, and 2) simulations for show. And they do not naturally converge as one might think. For airborne surveillance operators, the simulation my team built didn’t need to answer questions. All it had to do was make the operator feel like they are in an aircraft operating the radar system. Conversely, when I submitted a proposal for the simulation of the Arctic to the Government of Canada, the proposed simulation had to provide insight into things like surveillance coverage and ship detection rates for example. This clearly falls into the simulation-for-answers bucket.
Lifelike Worlds
Simulations-for-show are meant to make the user feel like they are in some other place. Some people refer to this as immersion or presence. These include training simulations, flight simulator games and virtual tourism to name a few. When building a simulation-for-show, you tend to not worry about everything being super accurate. After all, the point is to trick the user into believing they are in that environment. This is what video games do all the time; sacrifice fidelity to the real-world in favour of the game being engaging and fun.
When a simulation-for-show represents the real world, it, too, can be called a mirror world. But it’s not the same as a digital twin. For example, when someone makes a digital twin of a door on a building, the scanned model is static; nothing moves. This makes it difficult to enter that building in a simulation-for-show. One could argue that technicians/artists could animate the scanned 3D door to swing open but that workflow is cumbersome compared to building a 3D door from scratch using current video game workflows. This is even more true when we get into natural environments. Does a digital twin of a waterfall make sense? The answer is obviously no in this case.
I call these lifelike worlds because they closely resemble the real world. Also, they generally seem to come to life; things move around inside these worlds. This is an important distinction. For example, virtual globes like Google Earth could be considered lifelike worlds as well because they resemble the real Earth very closely. However, they don’t feel lifelike to me because nothing moves. They’re not interactive, and I interact with the real world ALL the time during my day.
Augmented Worlds
In the context of a city, digital twinning can mean, among other things, scanning all the buildings, roads, traffic lights, mailboxes, etc. and making a complete, accurate and precise 3D model out of all the scans. This enables us to do things we couldn’t before. For example, with a smartphone that has access to this scanned model, the device can localize your position and viewing angle precisely from a GPS signal combined with what it sees through the camera.
This precise location and view will be very useful especially in the context of augmented reality. You could be walking down the street and your AR glasses would recognize your surroundings perfectly giving them the ability to overlay virtual objects around you. For example, you could walk by a restaurant and the menu with prices could appear right before your eyes. These augmented worlds are obviously a form of mirror worlds but they do not fit well into digital twins nor lifelike worlds. They seem to be in a category of their own.
Mirror Worlds
Let’s come back up to the 10,000-foot level. We have augmented worlds, digital twins and lifelike worlds, each having their own applications, attributes and technology driving them.
Augmented worlds use AR devices like your smartphone or AR goggles/glasses. They are useful to augment your surroundings with things that are interesting or valuable to you.
Digital twins, on the other hand, primarily use modeling and simulation technologies to answer questions. Visualizations in AR and VR are not necessary to the process but they might be useful to view the results.
And lifelike worlds are interactive and make you feel like you are somewhere else in the world for the purpose of entertainment, training/learning or some other interesting or valuable activity.
I hope this helps you navigate through the mirror worlds that are coming soon to a reality near you.
