The Real Metaverse is Earth

Posted by Christopher Mitchell, Ph.D..

Downtown New York City in Geopipe

Imagine a future where in one massive virtual world, you can do anything you could do in the real world, and more. You can play games set in the real world that are so immersive you don't have to suspend disbelief. You can walk through the streets of the country you've always wanted to see on vacation, but can't find the time to visit. Soldiers, first responders, and bus drivers can be trained in virtual environments that match reality so closely, they already know exactly what they need to do when they're ready to go out into the field. Machines like autonomous cars and delivery drones can be trained for millions of hours in digital cities that precisely match their real world counterparts - down to wind, rain, snow, downed trees, and errant pedestrians.

Now imagine that this all takes place in the same virtual world, and just like reality, the rest of humanity is in the virtual world around you.

This is the Metaverse: the massive future digital version of reality that first inspired Geopipe. The true Metaverse is a massively multi-use environment that replicates our real world, built on top of a richly labeled digital twin of the Earth. Layered together with experiences that understand the objects in that environment to simulate a living world, this true Metaverse is far bigger, infinitely more useful than any imaginary world or patchwork Metaverse could be.

Let's break down the three pieces of the unified Metaverse: an environment that can be used for any application, digital twins of our reality, and a way to interact with these twins.

The true Metaverse is a massively multi-use environment that replicates our real world, built on top of a richly labeled digital twin of the Earth.

First, the Metaverse must be multi-use or multi-world: calling a small, single-use (virtual) world a "metaverse" would ignore the "meta" part of the portmanteau ("meta universe"). It's not accurate to apply the "metaverse" label to virtual environments like "Liberty City" in Grand Theft Auto, or a digital twin of a New York City neighborhood created and used by an architectural firm for their renderings, or an "HD Map" of San Francisco purpose-built to train autonomous vehicles. As the New York Times asserts, these certainly are in "the neighborhood of metaversality"1, but a true metaverse is a multipurpose virtual space, not just a single-use space2. Compare and contrast to our own real world: we don't use our reality only for attending meetings, or playing paintball, or for Drivers' Ed. Instead, these all co-exist in a large-scale world that can accommodate all of these simultaneously.

Geopipe's virtual Washington Square Park

Just like New York City's real version, the Multiverse's Washington Square Park can simultaneously be a social gathering place, a music venue, a hub of activism, and the setting for a skateboarding game.

Unfortunately, the real world is limited by space-time and by logistics: you can't do all of these at the same time in the same place in the real world. The laws of physics forbid a cohort of trainee locomotive engineers from traveling the same route in different trains simultaneously. Logistics make it impossible to drill first responders on responding to an emergency on a highway or in an airport on the same weekday morning that hundreds of thousands of travelers are trying to make their way to their offices (if we ever go back to real offices). In virtual space, you have the option of letting ten people share a conference room to have a meeting together, or simultaneously, a different set of four people could share that same conference room together - just in a different copy of the world. And therein lies a tension, and an opportunity, in defining the metaverse: is it one unified environment (or set of linked environments) with an authoritative copy, just like the real world? Where a change one person makes to a location is a change that every single other person can see, and if you're in a particular virtual location in those worlds, anyone in the same location can see you? Or can it be the same world, copied over into different versions that can be independently navigated and modified: a multiverse of metaverses, if you will?

The most useful, true Metaverse replicates this real world as closely as possible, fractally copying every building, every tree, every window, every traffic light, and perhaps even every blade of grass.

The power of the Metaverse is that it can be either and both. It's likely we'll end up with an authoritative Metaverse, the "main" twin of the Earth in which most of virtual life happens, and on-demand copies branched or forked out when you need to use or re-use a particular area for a specific purpose. You can demolish and rebuild the building you're working in, rerouting traffic and planning construction while simultaneously hosting an international conference. You have infinite opportunities to simulate different courses of action and anticipate the potential challenges and resources needed.

So how does this Metaverse actually match our Earth? One more component is necessary: Digital Twins. Digital twins of the real world are a vital prerequisite for building the Metaverse (not just a metaverse). Virtual copies of imaginary places, like Fortnite maps that could be used for concerts one day and battle royale fights the next, are an important step towards realizing a metaverse. But in ten years, it is the Metaverse writ large where we'll spend most of our virtual time, because the Metaverse will mirror the real world. Most of our work and much of our play already requires and involves the real world, so logically the most useful, true Metaverse replicates this real world as closely as possible, fractally copying every building, every tree, every window, every traffic light, and perhaps even every blade of grass. For that actual environment, you will need a whole-Earth digital twin, and at Geopipe, that's exactly what we're creating.

Human-generated maps like OpenStreetMap are effective at capturing the broad detail of popular urban spaces, thanks to the time investments of their many volunteers. But the deeply detailed "Matrix" of the Metaverse can only be properly created - and then kept up-to-date - by tasking computer intelligence (AI!) with rapidly understanding the real world and turning it into digital twins. One existing way to automate this without AI is to create 3D pictures of the world called "photogrammetry" (think Google Earth, Bing Maps, etc., or what drones typically create) that look good from far away. But get up close and they fall apart, because there's no intelligence or labeling about what's actually in the world.

Semantic data in a rich digital twin of reality

The foundation of the Metaverse is rich digital twins of reality labeled with exactly the objects and materials in the world. Knowing the difference between types of objects and surfaces (left), being able to individually identify every unique object (like one of millions of windows, center), and as game engine-ready geometry (right) makes the Metaverse interactive and simulate-able.

At Geopipe, we create digital twins of what's in the world, not just how it looks. From our earliest inspiration from trying to turn cities into game levels, we knew that the labels and the metadata that describe the world are a vital part of making useful, interactive digital twins. We teach algorithms to parse photographs, laser scans, maps, and other raw data about the world into recipes that describe every object, from type (building, tree, road, ...) to shape and size to material. From these recipes, we can then generate rich digital twins of reality that not only look correct, but can be simulated and interacted with. The metadata of these recipes is embedded in the twins, so that the trees can change as seasons change, doors can open, water can flow, and traffic lights can change colors. And we can do it fast: today, we can reproduce entire metropolitan areas in less than a day. By relying on the speed of a highly distributed machine learning system instead of humans, we can scale to cover ever more of the Earth and dive deeper into replicating smaller and smaller details of the world.

Golden hour on a New York City street

Finally, to build the Metaverse, you need to layer that whole-Earth digital twin with some way to interact with it. David Weir-McCall of Epic Games recently expressed a thesis very close to my own, that digital twins are important environments for the most useful Metaverse(s)3. It is gratifying to see that Epic is embracing some of the infrastructure necessary to support the Metaverse built on our reality. Epic's Unreal Engine and Unity Technologies' Unity game engine both are widely used as "game tech" supporting games, training simulations for humans and autonomous vehicles, visualizations and VR experiences for architecture and real estate, even developing traffic control systems for drones and letting medical students examine virtual patients. Building the true Metaverse requires combining a richly detailed and labeled digital twin with a simulation or game engine that allows humans and machines to interact with, visualize, and simulate the real world: that Metaverse not only looks like and contains what reality contains, it also behaves like reality.

Science fiction may cast the Metaverse as dystopian at least as often as it represents it as utopian, but in the real world, the Metaverse of and overlaid with our physical reality enables countless applications from the freeing to the life-saving to the entertaining to the mundane. Humans and machines will be trained to perfection in the Metaverse before seamlessly transitioning to the real world. Disability will no longer restrict anyone from paying a visit anywhere and to any person on Earth. Environments for games, movies, and stories will be more detailed, much larger, and more populated than today's gamers and consumers can imagine, let alone access. Cities will be planned for the next day, the next year, the next century in comprehensive virtual detail before a single shovel of dirt is moved.

The Metaverse will never replace the unique and singular experience of being present with others4, in the flesh, as humans. Nor is it supposed to. But it will give us the power to imagine, create and share virtual worlds that will make 'real life' richer, safer, more joyful and more vibrant.

References:
  1. "Are We in the Metaverse Yet?": https://www.nytimes.com/2021/07/10/style/metaverse-virtual-worlds.html
  2. "The Metaverse is Coming": https://www.wired.co.uk/article/metaverse
  3. "Digital twins will act as the “foundation of the Metaverse": https://www.dezeen.com/2021/07/23/digital-twins-metaverse-david-weir-mccall-epic-games/
  4. "The metaverse is the future of our reality. Mark Zuckerberg and its other evangelists are planning for it.": https://www.washingtonpost.com/opinions/2021/07/28/will-metaverse-connect-us-or-just-remind-us-how-far-apart-we-remain/