I'm Martin Edström, a National Geographic Explorer and photographer - and I use 3D, VR and AR to tell stories in new digital ways. AMA!

What's the holy grail in your line of work? Like what do you want to discover or capture next?

Very good question, that I've actually asked myself recently. For me it's becoming more and more about impact - the impact that the story/experience can have. So while I definitely have some destinations and/or sites that I would loooove to capture in 3D and/or for a VR experience, I find myself choosing projects a bit more carefully to see if they can contribute something. That's why it was a blast to work on the Stonehenge piece, which might seem like a "small" projects in one sense (I mean for some people it's just some old stones...) - but that offers a chance to be part of spreading more knowledge on such an ancient site and the new scientific discoveries around it. Projects that help us connect to our shared past, and to bring history alive. Same thing with the Everest and Ganges VR series, that gave me a chance to help tell stories about climate change and the problem of plastic. So I think my holy grail for a VR story would be defined as A) something that taps into bigger and more profound stories about our planet and B) that get appreciated enough that it becomes a standard story playthrough in classrooms around the world as VR gets bigger in the education space. I can't imagine more 'impact' than that.

But as for 'discoveries' as in your questions, I would of course love to capture more stunning places on the planet. Preferably inaccessible ones, like caves and hard-to-reach-places, so we can really offer people to experience something digitally that would be impossible for everyone to see in real life.

I'm working as a ranger at a glacier and feel very fortunate to have such a visible indicator of climate change to talk with people about. I really appreciate your answer about impact and would love to explore some vr glacier caves since ours aren't so accessable anymore.

This is exactly what I'm talking about - something that's incredibly tangible once you visit in person, but it's a trip that most people will never do in their lifetime. I would love to connect about your work as a 'glacier ranger' and see if that's something we can capture - as I'm currently doing research for a VR project where that would be incredibly interesting to show to people.

Hi Martin! What do you prefer to order, kebabtallrik or a kebabrulle?

Hej där! Hmm... Svaret blir nog ändå falafelrulle. Gärna på Burgerian på Östgötagatan.

Hi Martin! How did you end up in your line of work and do you have any tips for aspiring photographers?

I was a 'regular' photographer for about 8-9 years, doing still photography and more breaking news-style reporting. But I got a bit tired of seeing my images as part of the vast stream of other images online, and didn't feel I could 'get through' to people properly. This was especially true about my work with social issues like the refugee crisis in Europe - I did so many reporting trips in Europe that deeply affected me, but I didn't feel like those feelings could properly translate to people through still images.

So I started experimenting, and tried putting 360-photography images together like virtual tours - but with a reporting side to it. Like virtual newspaper stories. One thing led to another, and I realized this was a format that I could get more traction with - people reacted way more to these digital stories than to my images. I realized I couldn't reach as many people (as you need more time to interact with a virtual story than with just images), but I could reach fewer people with more impact. And to me that felt important, and I've pursued that very feeling ever since.

My tips for aspiring photographers is to find your niche. Try and experiment, and it's fine to take several years to dabble in many different photographic styles etc (don't quit your day job before you do!). But eventually also find something that you are good at and can pursue with a bit more expertise. That's where you will become valuable, to yourself and to others. Get good at that, start small and step carefully - and you have the beginning steps of a career in photography.

Are these going to be experiences that can translate to VR?

Yes, I'm quite sure they will be! It's not my decision really (as it's all up to my editors and decision makers at National Geographic) but I believe these assets we've made from Stonehenge may well make their way into larger experiences than just AR on instagram. I'm definitely rooting for it, and will do all I can to bring stuff like this to you in VR! It's a pretty cool place to visit in person, and walking among the stones to see scratch-marks and details from thousands of years ago is definitely something I would love to make possible in VR. It looks brilliant, and almost feels like you're there for real.

what VR cameras do you use? and what gear in general?

Good question, with lots of answers! Me and my team generally tailor cameras and gear for each project, since the requirements and logistics require so many different solutions... At this point, I've used everything from our own jerry-rigged solutions (where we strapped together GoPro cameras back in 2013) to the off-the-shelf big brand solutions that are tailored for studio and field use. So I'm quite comfortable in telling you that I have tried *almost* every VR camera out there at this point, hah! It also has to do with whether we're going for monoscopic versus stereoscopic VR - and right now I'm mostly a fan of the monoscopic 360-video cameras because they are easier to work with and offer a smoother experience for beginner end users.

Hi Martin!

I've wondered for a long time about how VR and AR can be made more accessible as learning tools, particularly for educational institutions and nonprofits. What resources are there to help get 3D learning tools in the hands of teachers and students? How can these tools be used broadly in classtooms, libraries, museums, etc, and what advantages do these media have over traditional media for teaching both kids and adults?

I'm a firm believer that XR (so meaning both VR, AR and whatever other other formats we will see in the future) is the new frontier in education. As I've worked with VR storytelling since 2012, I've seen many of our projects put to use in classrooms - and we've even tailored some of our experiences completely for the educational space. And the reactions are just overwhelming. I've seen kids cry when coming out of VR headsets, simply because they got to meet a kid in a refugee camp. I've seen kids recite facts from a 360-video that they probably wouldn't have remembered from reading a text, simply because they got so engaged in the spatial sense of virtual reality.

A well-made narrative VR story put to use to convey message and meaning about a subject just transcends reading a text-book in so many ways. Even if we *definitely* still need text-books to learn science in school, there are so many aspects that come to life better in XR. You get to experience a story up close, you get to meet people you wouldn't otherwise have met and feel 'like you're there'.

I definitely think it will take some time for VR to become a household product, but I have high hopes that it doesn't take that long to become a classroom product. Learning by doing is such an important aspect of education, and VR helps us do that by transporting kids to places, stories and lessons that wouldn't be possible otherwise.

At the current stage in VR history though, there aren't quite enough experiences out there tailored to curriculums and classrooms for it to break through just yet. But we're working hard to be a part of turning that tide. My colleague Carl-Fredrik is also deep into the XR education space, and I think he will post a few resource tips.

From Carl-Fredrik:

Awesome question, VR for education is one of its strongest capabilities and I’m super excited to see that tech evolve further. I’m going to plug an awesome group that has a very active community specifically for education that I highly recommend you check out: Educators in VR https://educatorsinvr.com/ Their resource page has a lot of information on how to get VR up and running for classrooms and a very active Discord server filled with teachers and developers.

They primarily use a software tool, or website, called Mozilla Hubs. It’s a type of meeting app designed fully for use in browsers with 3D and VR. The link you create from that software can be opened in a VR headset and allows the user to jump into a world, much like how Minecraft and videogames work into a virtual meeting. The world can be either created from a template or with some work be created by hand. The link works on browsers as well, so a rudimentary laptop connected to the internet can work just as well and at the same time as a Quest 2, although slightly less immersive.

Within these hubs you can further have immersive links, or play videos or have full lectures with tools to manage users or students (like muting, or establishing raising their hands for questions and the like). Super cool and it’s just the beginning and one example for education. Please check in with that community to delve further!

Any chance you will be releasing the assets in any way? It would be very nice to be able to plop Stonehenge into an UE5 project.

What are you using for your development HMDs?

From Carl-Fredrik:

I echo your sentiment with using models UE5, the capabilities of the nanite tech they’ve added is incredibly exciting for very complex models made from photogrammetry. Though there’s still some limitations to keep in mind, specifically loading textures. Would we have used Stonehenge in its full resolution from the 7k pictures we took we’d be looking at hundreds of 16k textures for one object.

For developing in HMDs we’re primarily working with WebXR to make our experiences available to as many people as possible. Every tour we make uses one link that dynamically changes depending on what hardware you use. On a desktop it opens in a browser, on a smartphone it uses your sensors, and on a headset it will open into a VR experience.

For fully 3D experiences we mainly develop towards the Oculus Quest 2 and Pico Neo 3 with Unreal Engine as our primary engine.

I will take it with the 2 terabytes of files 😃 Thank you for the response. Follow up if I catch you, are you using any lidar?

From Carl-Fredrik:

We do use LIDAR and it’s amazing for rapidly scanning large areas. The detail will never be quite as good as manual photogrammetry with a DSLR camera but the neat thing is that it can be used in conjunction with cameras. I’m working with some LIDAR data right now for a future project. It gave me the ability to scan an entire underground complex in just 2 days, but also allowed me time to focus on one room and fully capture it with a camera and then add that along with the LIDAR data. So I’ll end up with an incredibly accurate, laser-measured, model with the high detail and texture data from a camera.

I’m looking forward to LIDAR scans being more accessible for everyday use, as they can be prohibitively expensive if you use a high end device. The moden iPhone 12/13 Pro and iPad have an excellent small LIDAR scanner and great apps to use them with, though they’re still on the very high $$$ end of the smartphone market.

Check out the app Polycam for the iPhone (and android now too! and “Canvas: LiDAR 3D” for the iPad pro.

Just wait until the day like in Prometheus where you can toss a few drones into the air and they'll map an area completely. Well we are almost there now.

From Carl-Fredrik:

When we were scanning with the LIDAR in the underground complex we were joking about that exact scenario…spooked ourselves pretty quickly thinking about Xenomorphs when we were all alone in a huge complex deep below the earth

Hi Martin! What would you say is the most challenging part when filming places for VR? Thanks!

When filming 360-video, the largest and often most daunting challenge is to not end up in the shot yourself! This might not have occured to everyone, but when you film with a 360-camera in all directions you literally have to run out of the shot every single time you press 'record'. Sometimes this means just hiding behind a corner, if we're filming indoors, but outdoors and in vast spaces it can mean you have to run pretty far! I remember this problem especially well from filming close to Mt Everest on the large mountain plateus of the Nepalese Himalayas - where we had to run several hundred meters for some shots, simply because we didn't have any big boulders or similar to hide behind in the place we wanted to film.

And this really sucks as a director and filmmaker. Of most of our cameras have preview-systems where we can look at the footage as we are filming from a remote screen (so we can hide and then watch from afar) but it's really challenging to leave the camera and not be "in control" in the middle of the capture like you are with regular video cameras.

Sometimes, my solution has even been to sit underneath the camera, basically squeezing myself in between the tripod legs, just so I can be there in the center as the scene unfolds around me.

From Carl-Fredrik:

I’ll chime in on this question as well with how you film for 3D scanning and photogrammetry. Much like with 360 photos you have to be sure to not capture anything you don’t want in your model. Thankfully you don’t have to go run and hide from the camera as with 360 though.

However you will look like the absolute worst photographer in the world when you take pictures for photogrammetry. You want to take as many pictures as possible in a very specific format and style. As a good rule of thumb you want your picture to cover at least 70-80% of what was in the previous picture and you want to go along the silhouette of the object from as many angles as possible. So this means you take a picture, you take a step to the left, take a picture, take a step to the left, over and over again. Then you raise or lower the camera bit…and repeat the entire process. Oh and the weather has to be awful for it to really shine. Preferably overcast so there’s no sharp shadows that would otherwise look cool in a normal photo.

For Stonehenge we took around 7000 pictures over 3 days. That includes drone photos (and the same rule applies for drones) and pictures taken with a ladder. Our team went around Stonehenge probably 6 times and then they went in and did the same for every stone for detail work.

After taking all those pictures you then have to do a specific set of post-processing on the pictures, again you'll look like the worst photographic editor in the world. All those cool shadows, contrasts and highlights have to be removed to make the pictures as flat and boring as possible. This gives the photogrammetry software more detail to work with but also means the 3D model will end up looking “unlit”, which means it has no shadows on it. The idea behind that is that all the lighting can then be created in 3D, no matter where the lightsource is.

How's your week been?

It's good so far! I had leg day this morning at the gym so I can just about feel my muscle soreness flaring up... How was yours?

How do you balance the safety aspect of your job with the reward / mission? Are there ever any times where you maybe found yourself second guessing a risk you put yourself into? I'd imagine that's always a struggle and easy to lose sight of.

I've never felt insecure on expeditions I've planned myself. But then again, some people might not be comfortable with the things I've gone through when climbing or packrafting on expeditions in remote places. But in general, I think I'm more safe on my expeditions than people would guess from seeing the photos.

In my risk assessments for overseas trips, the main risk is always traffic. Traffic is always worse than the occupational hazards of our expedition work. So what I'm most scared of, really, is to end up as a traffic accident statistic when traveling in a country where seatbelts aren't the norm.

How did you start your career with Natgeo?

Easiest answer is: I didn't! Someone else did... I find this to be true with most colleagues in the National Geographic community, where it's hard to break in - but rather someone has to introduce you. In my case, I had the fortune that someone involved with the National Geographic Society caught wind of my early (early!) work with 360-photography and virtual tours. He had seen one of my earliest projects, where I had created like a 'virtual refugee camp' from the cast refugee camp Zaatari in Jordan during the first year of the war in Syria. It was kind of a VR journalism prototype, but in 2012 before the big VR craze started. I was just experimenting at this point, but he told me he hadn't seen any digital story quite like it - and encouraged me to pitch another project. At that point, I already wanted to make a story about the largest cave in the world - Son Doong in Vietnam - because I saw what a great virtual tour it would be and what an interesting conservation story it represented. So suddenly, I was pitching that as a project for National Geographic - and to my great surprise (I almost shit my pants) they said yes. And we pulled it off, to great reviews. Since then National Geographic has been a large part of my career, and I've been fortunate enough to now get assignments for this kind of stuff. Even typing this out, I almost can't believe my luck.

In one of your replies, you mentioned that some of your "bucket list" captures would be stunning yet inaccessible places that are difficult, if not impossible, for normal people to reach, and I think that's really awesome.

Do you think it'll be more of a challenge to also do your homework on the cultural/historical impacts of those locations once you get there?

The Everests, Tokyos and Stonehenges of the world have been the focus of extensive research for centuries, and there are lots of field experts for those locations. Some of the more "off-the-beaten-path" locales might not have as many resources to pull from, so I'm curious about how you may be planning to do some of the fact-finding.

Very good question, I appreciate it. Taps into lots of the complex issues around reporting and journalism, which is a larger part of my work than I think most people realize. As a photographer, especially working with National Geographic, you're often spending just as much time - or more - on research and writing than you are taking photos.

The best example in my career would be the first story I captured for NatGeo, from the largest cave in the world - Son Doong in Vietnam. At the time when we captured it in January 2015, it had just recently been discovered (in 2012). So facts were there, but scarce. So in a project like that, I spent lots of time getting details right from the scientists and cave researchers working to map out and describe the cave academically. At the end of the project, once the photos were all edited and ready to publish, I spent weeks crafting the texts that ended up in the final virtual tour-style experience. They might seem like short texts (they are) but they involved defining key facts about the actual cave and other cave-related facts that I had no previous knowledge about (as I'm not a scientist).

So in other projects, and in future ones like you ask about, my main strategy is to always involve key people on site. There is always someone who knows more about a place than me, unless I was telling the story of my own apartment (probably not even then). So for any overseas project, I always want to make a local partner part of the project - usually a main part. For the recent expeditions we've done in Kyrgyzstan, using drones and 3D models to find new caves in the Tian Shan mountain range, we involved one of Kyrgyzstan's top geologists Dr. Alexey Dudashvili as a main project member. Heck, he was our official expedition leader - as he was the main 'brain' behind the subject matter on that expedition. For sure, me and my team handled the drones, the tech, the 3D modeling and many of the important aspects that made that expedition happen and that gave us attention and funding - but he's the driving force for exploration in that space. And while expeditions usually haven't been conducted in this way before, where often the photographer rides in as the 'key person' to lead exploration in a place where other people have lived for generations - I think we need to lead the change in that.

So you place a finger on a very important aspect of any documentary endeavour (be it 3D, photography, filmmaking or whatever) which is research and local participation. And for me, that will always be about finding the people who are already experts (usually people who live there or are part of that field) and simply make them your expedition leader. Then I'm just happy to take the photos and show them to the world :)

Hey, what inspired you to come up with this idea?

Which idea do you refer to? Happy to answer.

Where do you see the future of AR heading, and what has been your favorite project so far?

Answer from Carl-Fredrik:

Cool question! I’ll check in with some AR thoughts:

AR is incredibly exciting right now as it feels a bit like how 3D used to be “back in the day”. There’s a lot of limitations to make it run smoothly on, mostly, smartphones. So you have to optimize the heck out of your 3D models and content, like with the Stonehenge model I first started out with a ridiculously detailed several billion number of triangles on the 3d model. Triangles are a common way to keep track of how complex a model is. The more triangles equals more detail model but also equals more work for the computer or smartphone to run.

So I’m excited to see the technology hardware develop for AR in two ways. First and foremost to be able to run much more complex models with richer detail and more information. This could allow more animation, greater detail on the model. For example, on the Stonehenge model it would be fantastic to let the user more clearly see more of the glyphs literally carved into the stone itself.

And secondly it would be amazing to see AR filters jump out of the usual smartphone and into our worlds. Probably in the form of smartglasses, maybe later as projections from drones if we’re really leaping forward into the imagination.

To see these models or filters, in full scale, somewhere out in the world and not have to peer through a looking glass, would truly unlock the potential of AR!

I agree with Carl-Fredrik!

My favorite project has definitely been the Stonhenge AR project that's currently on the natgeo instagram. It was such a blast to work with the Immersive team at National Geographic to craft that experience, and to see what more we can build from the scans.

But apart from cultural heritage stories, and making people 'visit' places like Stonehenge in a digital way, I'm excited about the everyday uses for AR.

I can't believe there's still not a well-made functioning pair of AR glasses out there. Imagine when I can put on my glasses, walk down the street, and get walking directions in a seemless way integrated into my visual field. But I am also dreading ads in this space - when you walk by a grocery store and suddenly you see a big "SALE" pop-up in your way. But I'm generally excited for the everyday help we will all be able to get from AR, just like we (very recently) recently got help from things like maps and messaging apps in our smartphones.

How did you get into the career and then the idea of vr/ar?

I replied on another thread above about my career, and how I ended up heading deeper into the whole XR space.

How'd you become a National Geographic explorer? I remember that was something I wanted to do many years ago before I put the idea to rest and it seemed like something that was impossibly difficult to get into.

So what was the process in becoming one?

Hi Poopy! Great name and good question. I answered the same thing above, but I'll just say that you shouldn't give up on it. It's difficult for sure, but depending on your line of work the journey to work with National Geographic might not be as hard as you think. If you work with something that might not seem interesting to you, it might be to others. Either from a visual storytelling side, from a scientific point of view or even from a digital programming point of view. I have no idea what you work with, but engaging with the idea of working with NatGeo shouldn't feel like an impossibility! The main trick is to be really good at something, and make that your way in.

Okay I have a few feel free to answer any one you like;) Have you done Underwater 3D models jet? How good is the resolution? What is the main purpose of the 3D modelling: Preservation or Research?

From Carl-Fredrik:

Interesting! I haven’t personally captured anything underwater but I know that it has been done, one example I know of is https://thehydro.us/photogrammetry

Like you said I believe resolution is a bit of an issue as the camera of course has to be water proof, and since photogrammetry works by detecting details in the picture the water has to be very clear and very still. Lots of little moving dust particles will seriously slow down the algorithms as it tries to figure out what is actually part of a static model or just some dust in front of the lens. There’s also an interesting color aspect when taking pictures underwater. Everything will take on a huge color shift as the light bends in the water turning it a dark blue/green. So there has to be extra care in the post production to either restore the color to their “true” color, or leave the specific hue shift. It can depend a bit on what the purpose is as you mention.

The main purpose of 3D modeling for us I would say is storytelling. Which would fall more into the category of preservation. Certainly it doesn’t mean changing or distorting a model, but for example sometimes you would need physical markers and other objects placed into the world to help get an accurate scan. These I would edit out in post-production, this technically means the object is distorted but it preserves the story of the object.

We haven’t done modeling specifically for research as they often focus more on technical aspects and require specific technology that won’t necessarily add to the story or adds information for the end user. But we have been in contact with researchers who have been very interested in the models and data we’ve captured.

Would you rather make public all of your internet searches or all of the photos on your phone?

Ooooh, interesting. I really hope this is not about you intending to hack me and do either - but if you do you will see lots of idiotic searches about medical conditions and six million photos of my 1,5 year old son.

How much data is one of your models?

They can quickly snowball in size depending on how you interpret the question. When creating a 3D scan you end up with tons and tons of files created along the way to the 3D model.

You have the source images from the cameras during the capture process. Then you post-process these images and create edited images of almost every single photo of the source images.

Then you create a project file for photogrammetry, and then you create a couple of different versions of the 3D model to fine tune some details. Etc, etc.

Finally you export out a 3D model with a whole bunch of textures to show all it’s colors, reflection and hidden details. All things in total was roughly 1 terabyte of data. But the final 3d model for the AR filter had to correspond to the limitations of Spark AR which is used to create official Instagram filters. So that was a full challenge in itself, keeping as much information as possible from 1 terabyte….down to around 10 megabytes AR optimized 3D model.