Augmenting the realities of work

Imagine an integrated workplace with 3D visualizations that augment presentations, interactive and accelerated onboarding, and controlled training simulations. This is the future of immersive technology that global head of Immersive Technology Research at JPMorgan Chase, Blair MacIntyre is working to build. Augmented reality (AR) and virtual reality (VR) technologies can blend physical and digital dimensions together and infuse new innovations and efficiencies into business and customer experiences.

These technologies can offer newer ways of collaborating over distance both synchronously and asynchronously than we can get with the traditional work technologies that we use right now," says MacIntyre. It's these new ways to collaborate, ways of using the environment and space in new and interesting ways that will hopefully offer new value and change the way we work."

Many enterprises are integrating VR into business practices like video conference calls. But having some participants in a virtual world and some sidelined creates imbalances in the employee experience. MacIntyre's team is looking for ways to use AR/VR technologies that can be additive, like 3D data visualizations that enhance financial forecasting within a bank, not ones that overhaul entire experiences.

Although the potential of AR/VR is quickly evolving, it's unlikely that customers' interactions or workplace environments will be entirely moved to the virtual world anytime soon. Rather, MacIntyre's immersive technology research looks to infuse efficiencies into existing practices.

It's thinking about how the technologies integrate and how we can add value where there is value and not trying to replace everything we do with these technologies," MacIntyre says.

AI can help remove some of the tedium from immersive technologies that have made them impractical for widespread enterprise use in the past. Using VR technology in the workplace may prohibit taking notes and having access to traditional input devices and files. AI tools can take and transcribe notes and fill in any other gaps to help remove that friction and eliminate redundancies.

Connected Internet of things (IoT) devices are also key to enabling AR/VR technologies. To create a valuable immersive experience, MacIntyre says, it's imperative to know as much about the surrounding world of the user as well as their needs, habits, and preferences.

If we can figure out more ways of enabling people to work together in a distributed way, we can start enabling more people to participate meaningfully in a wider variety of jobs," says MacIntyre.

This episode of Business Lab is produced in association with JPMorgan Chase.

Laurel: From MIT Technology Review, I'm Laurel Ruma, and this is Business Lab, the show that helps business leaders make sense of new technologies coming out of the lab and into the marketplace.

Our topic today is emerging technologies, specifically, immersive technologies like augmented and virtual reality. Keeping up with technology trends may be a challenge for most enterprises, but it's a critical way to think about future possibilities from product to customer service to employee experience. Augmented and virtual realities aren't necessarily new, but when it comes to applying them beyond gaming, it's a brave new world.

Two words for you: emerging realities.

My guest is Blair MacIntyre, who is the global head of Immersive Technology Research at JPMorgan Chase.

This podcast is produced in association with JPMorgan Chase.

Welcome, Blair.

Blair MacIntyre: Thank you. It's great to be here.

Laurel: Well, let's do a little bit of context setting. Your career has been focused on researching and exploring immersive technology, including software and design tools, privacy and ethics, and game and experience design. So what brought you to JPMorgan Chase, and could you describe your current role?

Blair: So before joining the firm, I had spent the last 23 years as a professor at Georgia Tech and Northeastern University. During that time, as you say, I explored a lot of ways that we can both create things with these technologies, immersive technologies and also, what they might be useful for and what the impacts on people in society and how we experience life are. But as these technologies have become more real, moved out of the lab, starting to see real products from real companies, we have this opportunity to actually see how they might be useful in practice and to have, for me, an impact on how these technologies will be deployed and used that goes beyond the traditional impact that professors might have. So beyond writing papers, beyond teaching students. That's what brought me to the firm, and so my current role is, really, to explore that, to understand all the different ways that immersive technology could impact the firm and its customers. Right? So we think about not just customer-facing and not just products, but also employees and their experience as well.

Laurel: That's really interesting. So why does JPMorgan Chase have a dedicated immersive technology focus in its global technology applied research division, and what are the primary goals of your team's research within finance and large enterprises as a whole?

Blair: That's a great question. So JPMorgan Chase has a fairly wide variety of research going on within the company. There's large efforts in AI/ML, in quantum computing, blockchain. So they're interested in looking at all of the range of new technologies and how they might impact the firm and our customers, and immersive technologies represent one of those technologies that could over time have a relatively large impact, I think, especially on the employee experience and how we interact with our customers. So they really want to have a group of people focusing on, really, looking both in the near and long term, and thinking about how we can leverage the technology now and how we might be able to leverage it down the road, and not just how we can, but what we should not do. Right? So we're interested in understanding of these applications that are being proposed or people are imagining could be used. Which ones actually have value to the company, and which ones may not actually have value in practice?

Laurel: So when people think of immersive technologies like augmented reality and virtual reality, AR and VR, many think of headsets or smartphone apps for gaming and retail shopping experiences. Could you give an overview of the state of immersive technology today and what use cases you find to be the most innovative and interesting in your research?

Blair: So, as you say, I think many people think about smartphones, and we've seen, at least in movies and TV shows, head mounts of various kinds. The market, I would divide it right now into the two parts, the handheld phone and tablet experience. So you can do augmented reality now, and that really translates to we take the camera feed, and we can overlay computer graphics on it to do things like see what something you might want to buy looks like in your living room or do, in an enterprise situation, remote maintenance assistance where I can take my phone, point it at a piece of technology, and a remote expert could draw on it or help me do something with it.

There's the phone-based things, and we carry these things in our pockets all the time, and they're relatively cheap. So there's a lot of opportunities when it's appropriate to use those, but the big downside of those devices is that you have to hold them in your hands, so if you wanted to try to put information all around you, you would have to hold the device up and look around, which is uncomfortable and awkward. So that is where the head mount displays come in.

So either virtual reality displays which, right now, many of us think about computer games and education as use cases in the consumer world or augmented reality displays. These sorts of displays now let us do the same kind of things we might do with our phones, but we can do it without our hands having to hold something so we can be doing whatever work it was we wanted to do, right? Repairing the equipment, taking notes, working with things in the world around us, and we can have information spread all around us, which I think is the big advantage of head mounts.

So many of the things people imagine when they think about augmented reality in particular involve this serendipitous access to information. I'm walking into a conference room, and I see sort of my notes and information about the people I'm meeting there and the materials from our last meeting, whatever it is, or I'm walking down the street, and I see advertising or other kinds of, say, tourism information, but those things only work if the device is out of mind. If I can put it on, and then go about my life, I'm not going to walk into a conference room, and hold up a phone, and look at everybody through it.

So that, I think, is the big difference. You could implement the same sorts of applications on both the handheld devices and the head-worn devices, but the two different form factors are going to make very different applications appropriate for those two sorts of technologies.

On the virtual reality side, we're at the point now where the displays we can buy are light enough and comfortable enough that we could wear them for half an hour, a couple hours without discomfort. So a lot of the applications that people imagine there, I think the most popular things that people have done research on and that I see having a near-term impact in the enterprise are immersive training applications where you can get into a situation rather than, say, watching a video or a little click-through presentation as part of your annual training. You could really be in an experience and hopefully learn more from it. So I think those sorts of experiences where we're totally immersed and focused is where virtual reality comes in.

The big thing that I think is most exciting about head-worn displays in particular where we can wear them while we're doing work as opposed to just having these ephemeral experiences with a phone is the opportunity to do things together, to collaborate. So I might want to look at a map on a table and see a bunch of data floating above the map, but it would be better if you and our other colleagues were around the table with me, and we can all see the same things, or if we want to take a training experience, I could be in there getting my training experience, but maybe someone else is joining me and being able to both offer feedback or guidance and so on.

Essentially, when I think about these technologies, I think about the parallels to how we do work regularly, right? We generally collaborate with people. We might grab a colleague and have them look at our laptop to show them something. I might send someone something on my phone, and then we can talk about it. So much of what we do involves interactions with other people and with the data that we are doing our job with that anything we do with these immersive technologies is really going to have to mimic that and give us the ability to do our real work in these immersive spaces with the people that we normally work with.

Laurel: Well, speaking of working with people, how can the scale of an institution like JPMorgan Chase help propel this research forward in immersive technology, and what opportunities does it provide that are otherwise limited in a traditional university or startup research environment?

Blair: I think it comes down to a few different things. On one hand, we have the access to people who are really doing the things that we want to build technologies to help with. Right? So if I wanted to look how I could use immersive visualization of data to help people in human resources do planning or help people who are doing financial modeling look at the data in new and interesting ways, now I could actually do the research in conjunction with the real people who do that work. Right? So I've already and I've been at the firm for a little over a year, and many conversations we've had were either we've had an idea or somebody has come to us with an idea. Through the course of the conversations, relatively quickly, we hone in on things that are much more sophisticated, much more powerful than what we might have thought of at a university where we didn't have that sort of direct access to people doing the work.

On the other hand, if we actually build something, we can actually test it with the same people, which is an amazing opportunity. Right? When I go to a conference, we're going to put 20 people who actually represent the real users of those systems. So, for me, that's where I think the big opportunity of doing research in an enterprise is, is building solutions for the real people of that enterprise and being able to test it with those people.

Laurel: Recent years have actually changed what customers and employees expect from enterprises as well, like omnichannel retail experiences. So immersive technologies can be used to bridge gaps between physical and virtual environments as you were saying earlier. What are the different opportunities that AR and VR can offer enterprises, and how can these technologies be used to improve employee and customer experience?

Blair: So I alluded back to some of that in previous answers. I think the biggest opportunities have to do with how employees within the organization can do new things together, can interact, and also how companies can interact with customers. Now, we're not going to move all of our interactions with our customers into the virtual world, or the metaverse, or whatever you want to call it nowadays anytime soon. Right? But I think there are opportunities for customers who are interested in those technologies, and comfortable with them, and excited by them to get new kinds of experiences and new ways of interacting with our firm or other firms than you could get with webpages and in-person meetings.

The other big opportunity I think is as we move to a more hybrid work environment and a distributed work environment, so a company like JPMorgan Chase is huge and spread around the world. We have over 300,000 employees now in most countries around the world. There might be groups of people, but they're connected together through video right now. These technologies, I think, can offer new ways of collaborating over distance both synchronously and asynchronously than we can get with the traditional work technologies that we use right now. So it's those new ways to collaborate, ways of using the environment and space in new and interesting ways that is going to, hopefully, offer new value and change the way we work.

Laurel: Yeah, and staying on that topic, we can't really have a discussion about technology without talking about AI which is another evolving, increasingly popular technology. So that's being used by many enterprises to reduce redundancies and automate repetitive tasks. In this way, how can immersive technology provide value to people in their everyday work with the help of AI?

Blair: So I think the big opportunity that AI brings to immersive technologies is helping ease a lot of the tedium and burden that may have prevented these technologies from being practical in the past, and this could happen in a variety of ways. When I'm in a virtual reality experience, I don't have access to a keyboard, I don't have access to traditional input devices, I don't have necessarily the same sorts of access to my files, and so on. With a lot of the new AI technologies that are coming around, I can start relying on the computer to take notes. I can have new ways of pulling up information that I otherwise wouldn't have access to. So, I think AI reducing the friction of using these technologies is a huge opportunity, and the research community is actively looking at that because friction has been one of the big problems with these technologies up till now.

Laurel: So, other than AI, what are other emerging technologies that can aid in immersive technology research and development?

Blair: So, aside from AI, if we step back and look at all of the emerging technologies as a whole and how they complement each other, I think we can see new opportunities. So, in our research, we work closely with people doing computer vision and other sort of sensing research to understand the world. We work closely with people looking at internet of things and connected devices because at a 10,000-foot level, all of these technologies are based on the idea of understanding, sensing the world, understanding what people are doing in it, understanding what people's needs might be, and then somehow providing information to them or actuating things in the world, displaying stuff on walls or displays.

From that viewpoint, immersive technologies are primarily one way of displaying things in a new and interesting way and getting input from people, knowing what people want to do, allowing them to interact with data. But in order to do that, they need to know as much about the world around the user as possible, the structure of it, but also, who's there, what we are doing, and so on. So all of these other technologies, especially the Internet of things (IoT) and other forms and ways of sensing what's happening in the world are very complimentary and together can create new sorts of experiences that neither could do alone.

Laurel: So what are some of the challenges, but also, possible opportunities in your research that contrast the future potential of AR and VR to where the technology is today?

Blair: So I think one of the big limitations of technology today is that most of the experiences are very siloed and disconnected from everything else we do. During the pandemic, many of us experimented with how we could have conferences online in various ways, right? A lot of companies, small companies and larger companies, started looking at how you could create immersive meetings and big group experiences using virtual reality technology, but all of those experiences that people created were these closed systems that you couldn't bring things into. So one of the things we're really interested in is how we stop thinking about creating new kinds of experiences and new ways of doing things, and instead think about how do we add these technologies to our existing work practices to enhance them in some way.

So, for example. Right now, we do video meetings. It would be more interesting for some people to be able to join those meetings, say, in VR. Companies have experimented with that, but most of the experiments that people are doing assume that everyone is going to move into virtual reality, or we're going to bring, say, the people in as a little video wall on the side of a big virtual reality room, making them second class citizens.

I'm really interested and my team is interested in how we can start incorporating technologies like this while keeping everyone a first-class participant in these meetings. As one example, a lot of the systems that large enterprises build, and we're no different, are web-based right now. So if, let's say, I have a system to do financial forecasting, you could imagine there's a bunch of those at a bank, and it's a web-based system, I'm really interested in how do we add the ability for people to go into a virtual reality or augmented reality experience, say, a 3D visualization of some kind of data at the moment they want to do it, do the work that they want to do, invite colleagues in to discuss things, and then go back to the work as it was always done on a desktop web browser.

So that idea of thinking of these technologies as a capability, a feature instead of a new whole application and way of doing things permeates all the work we're doing. When I look down the road at where this can go, I see in, say, let's say, two to five years, I see people with displays maybe sitting on their desk. They have their tablet and their phone, and they might also have another display or two sitting there. They're doing their work, and at different times, they might be in a video chat, they might pick up a head mount and put it on to do different things, but it's all integrated. I'm really interested in how we connect these together and reduce friction. Right? If it takes you four or five minutes to move your work into a VR experience, nobody is going to do it because it just is too problematic. So it's that. It's thinking about how the technologies integrate and how we can add value where there is value and not trying to replace everything we do with these technologies.

Laurel: So to stay on that future focus, how do you foresee the immersive technology landscape entirely evolving over the next decade, and how will your research enable those changes?

Blair: So, at some level, it's really hard to answer that question. Right? So if I think back 10 years to where immersive technologies were, it would have been inconceivable for us to imagine the videos that are coming out. So, at some level, I can say, Well, I have no idea where we're going to be in 10 years." On the other hand, it's pretty safe to imagine the kinds of technologies that we're experimenting with now just getting better, and more comfortable, and more easy to integrate into work. So I think the landscape is going to evolve in the near term to be more amenable to work.

Especially for augmented reality, the threshold that these devices would have to get to such that a lot of people would be willing to wear them all the time while they're walking down the street, playing sports, doing whatever, that's a very high bar because it has to be small, it has to be light, it has to be cheap, it has to have a battery that lasts all day, etcetera, etcetera. On the other hand, in the enterprise, in any business situation, it's easy to imagine the scenario I described. It's sitting on my desk, I pick it up, I put it on, I take it off.

In the medium term after that, I think we will see more consumer applications as people start solving more of the problems that are preventing people from wearing these devices for longer periods of time. Right? It's not just size, and battery power, and comfort, it's also things like optics. Right? A lot of people - not a lot, but say, let's say 10%, 15% of people might experience headaches, or nausea, or other kinds of discomfort when they wear a VR display as they're currently built, and a lot of that has to do with the fact that the optics that you're looking at when you're putting this display are built in a way that makes it hard to comfortably focus at objects at different distances away from you without getting into the nitty-gritty details. For many of us, that's fine. We can deal with the slight problems. But for some people, it's problematic.

So as we figure out how to solve problems like that, more people can wear them, and more people can use them. I think that's a really critical issue for not just consumers, but for the enterprise because if we think about a future where more of our business applications and the kind of way we work are done with technologies like this, these technologies have to be accessible to everybody. Right? If that 10% or 15% of people get headaches and feel nauseous wearing this device, you've now disenfranchised a pretty significant portion of your workforce, but I think those can be solved, and so we need to be thinking about how we can enable everybody to use them.

On the other hand, technologies like this can enfranchise more people, where right now, working remotely, working in a distributed sense is hard. For many kinds of work, it's difficult to do remotely. If we can figure out more ways of enabling people to work together in a distributed way, we can start enabling more people to participate meaningfully in a wider variety of jobs.

Laurel: Blair, that was fantastic. It's so interesting. I really appreciate your perspective and sharing it here with us on the Business Lab.

Blair: It was great to be here. I enjoyed talking to you.

Laurel: That was Blair MacIntyre, the global head of Immersive Technology Research at JPMorgan Chase, who I spoke with from Cambridge, Massachusetts, the home of MIT and MIT Technology Review.

That's it for this episode of Business Lab. I'm your host, Laurel Ruma. I'm the global director of Insights, the custom publishing division of MIT Technology Review. We were founded in 1899 at the Massachusetts Institute of Technology, and you can find us in print, on the web, and at events each year around the world. For more information about us and the show, please check out our website at technologyreview.com.

This show is available wherever you get your podcasts. If you enjoyed this episode, we hope you'll take a moment to rate and review us. Business Lab is a production of MIT Technology Review. This episode was produced by Giro Studios. Thanks for listening.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review's editorial staff.

This podcast is for informational purposes only and it is not intended as legal, tax, financial, investment, accounting or regulatory advice. Opinions expressed herein are the personal views of the individual(s) and do not represent the views of JPMorgan Chase & Co. The accuracy of any statements, linked resources, reported findings or quotations are not the responsibility of JPMorgan Chase & Co.