[00:00.000 --> 00:05.620] So, we were asked by the XR Village to come and just give a little talk about the work [00:05.620 --> 00:10.400] that we've been doing at the Idaho National Lab using some XR equipment. [00:12.200 --> 00:19.200] So, behind the scenes of our daily lives, there's critical infrastructure that's supplying us. [00:21.360 --> 00:25.520] You know, all of the necessary services, you know, water treatment, [00:25.520 --> 00:31.240] um, our electricity, and all of that infrastructure needs to be secured. [00:31.240 --> 00:39.380] But as, you know, working in the security kind of aspect, we know that these systems [00:40.280 --> 00:46.740] definitely aren't as secure as we'd like them to be. And so, [00:46.740 --> 00:53.620] um, part of the work that we've been doing at the National Lab in partnership with the [00:53.620 --> 01:04.100] Seller Group is to work on securing those systems and helping, um, you know, smaller [01:04.100 --> 01:08.600] cities to know that their systems may not be secure, that there are vulnerabilities. [01:08.780 --> 01:15.040] And we've been using XR as a component of that, um, of that outreach. [01:15.580 --> 01:21.540] So, my name is Colton Heaps, and I've been working with the Idaho National Laboratory [01:21.540 --> 01:29.020] for about two years. Um, it's been really exciting to be able to work with, in a laboratory where, [01:29.020 --> 01:35.480] um, they have such unique infrastructure. So, for those of you who don't know, the Idaho National [01:35.480 --> 01:43.800] Lab is a research laboratory that's focused on nuclear research. And so, um, we have, you know, [01:43.800 --> 01:52.680] couple of different reactors out on our, our campus. And with that, you know, um, micro grids [01:52.680 --> 02:00.900] and a lot of different infrastructure that enables us some unique cyber security research. And so, [02:00.900 --> 02:11.840] um, I will kind of start out by describing the group that I work with, with digital engineering [02:11.840 --> 02:20.660] and some terms that I'll be sharing and introduce the, the seller group and then, um, get into how [02:20.660 --> 02:25.960] we've been using mixed reality and, uh, virtual reality for that, that work. [02:27.920 --> 02:34.560] So, what is digital engineering? Digital engineering focuses on supplying data to [02:34.560 --> 02:40.780] researchers regardless of the platform that they're performing research on. So, in, you know, [02:40.780 --> 02:47.640] times past, we had, um, researchers passing around documents. Uh, they'd have sacred word [02:47.640 --> 02:52.640] documents that had all the information for the projects. Um, that is not a good way to do [02:52.640 --> 02:58.160] engineering. So, we're trying to centralize all of that data no matter where it comes from and [02:58.160 --> 03:07.220] enable, um, the different platforms to be able to reach out and use that data. So, um, digital [03:07.220 --> 03:12.900] engineering helps to provide semi-autonomous design, autonomous operations, and real time [03:12.900 --> 03:20.160] anomaly detection. It integrates the threads of data, visualizations, AI and ML and physics models [03:20.160 --> 03:29.460] into a cohesive digital twin. And so, a digital twin is, it's a buzz word. Um, so, we're trying, [03:29.460 --> 03:34.640] it gets thrown around a lot in industry, but we're trying to define that at the national lab. [03:34.640 --> 03:42.300] And so, I like to compare it to, to the iPhone. So, we all know, you know, the impact that the [03:42.300 --> 03:49.940] iPhone had on our world. And, um, because of the iPhone, you know, led to other smartphones, [03:49.940 --> 03:57.520] rising popularity, and that truly changed our lives. But the iPhone itself is made up of a [03:57.520 --> 04:02.580] bunch of smaller technologies that already existed. And those technologies themselves [04:02.580 --> 04:07.100] weren't necessarily revolutionary, but the combination of all of those made something [04:07.100 --> 04:19.380] special. And so, that is, um, how I see digital twins. And, um, it's the combination of all of [04:19.380 --> 04:23.980] these different technologies that have existed in the past, but the combination of all of those [04:23.980 --> 04:32.060] together makes it something special. So, what is a digital twin? Like I mentioned, the digital twin [04:32.060 --> 04:38.260] is a merging of all these different technologies. The connection of integrated data, sensors and [04:38.260 --> 04:42.860] instrumentation from physical assets. So, taking that realtime data from the systems and feeding [04:42.860 --> 04:53.780] it back into visualizations and AI ML algorithms. The AI and an online monitoring and advanced [04:53.780 --> 04:59.600] visualization with mixed reality and virtual reality. So, that's kind of the INL's definition [04:59.600 --> 05:06.380] of the digital twin. And a digital twin mirrors the physical model while using realtime [05:06.380 --> 05:16.440] bidirectional communication to perform, you know, this magic. Enabling the user to take in data [05:16.440 --> 05:26.200] more efficiently. Giving the operator more data in a way that they can understand. And so, put [05:26.200 --> 05:30.940] that into context of the work that we do at the lab. Like I said, nuclear research and working [05:30.940 --> 05:42.040] with these nuclear systems. We need to be able to have safe, small modular reactors in the future. [05:42.220 --> 05:51.900] And they need to be able to have autonomous corrections. So, broadcasting into the future [05:51.900 --> 05:56.860] to see if these things are safe or not. And then if they're, you know, going into a hazardous [05:56.860 --> 06:05.780] direction, then they need to be able to fix themselves. But we've been using it with the [06:05.780 --> 06:12.740] cyber security group at the INL, specifically the CELR group. So, CELR stands for control [06:12.740 --> 06:21.840] environment laboratory resource. And so, CELR is sponsored by DHS's cyber security and [06:21.840 --> 06:28.980] infrastructure security agency or CISA. CELR is a lab scale environment with sector specific [06:28.980 --> 06:36.680] platforms with operational physical components as well as operational IT and OT environments with [06:36.680 --> 06:44.040] real controllers and human machine interfaces to operate the physical components. And so, [06:44.040 --> 06:50.900] there are about six of these training platforms that CELR has provided to the public. And each [06:50.900 --> 06:58.980] of these platforms have unique HMIs, PLCs and protocols that are used. And so, it takes the [06:58.980 --> 07:07.400] breadth of, you know, manufacturer's hardware and protocols that are used in industry and [07:07.400 --> 07:15.180] displays them here as a resource publicly available for, you know, international use so that [07:15.720 --> 07:23.640] anybody can gain more experience in securing this critical infrastructure. So, the environment [07:23.640 --> 07:29.760] is used to train cyber incident responders and expose them that actual networks and malicious [07:30.600 --> 07:35.740] activities are similar to what they would see in the real world. Support tool evaluation to [07:35.740 --> 07:42.580] determine effectiveness in operations and research mitigation measures that may counter those [07:42.580 --> 07:50.600] effects. CELR's capabilities have matured to include six sector specific platforms with six [07:50.600 --> 07:59.320] different controllers, HMIs and network protocols. This allows for diverse options of to fully [07:59.320 --> 08:07.620] represent a broader range of the nation's infrastructure. And it's definitely a valuable [08:07.620 --> 08:19.740] resource that is being provided and helps to supply this resource to smaller cities that may [08:19.740 --> 08:29.200] not have the funds to put towards cyber security and securing their infrastructure so they can [08:29.200 --> 08:54.020] take advantage of these resources. Okay. So, in the CELR group, while building these systems, [08:54.020 --> 09:02.620] they started to reach out to digital engineering just as a way to visualize their, you know, [09:02.620 --> 09:08.600] their plans in their new work space. And so, they were planning to build, they had one existing [09:08.600 --> 09:13.820] skid, I believe, and they were planning to build more in the space. And so, they wanted to use [09:13.820 --> 09:20.040] some of our XR capabilities that we've been using across the lab to visualize these components in [09:20.040 --> 09:33.140] their facility and maximize the efficiency of the space. And so, it's maybe not a very exciting way [09:33.140 --> 09:38.800] of using mixed reality as it's, you know, seems like everybody's using that now, but it's very [09:38.800 --> 09:48.540] valuable and saves so much time and money. And so, we were able to use some virtual reality goggles [09:48.540 --> 09:57.600] and take the existing CAD models as geometries and import them as meshes into our visualization [09:57.600 --> 10:06.520] software and mainly we use Unity for this. Because of how easy they've made it to work with a [10:06.520 --> 10:16.540] variety of hardware. And so, with those CAD models and the BIM model from their facility, so BIM is [10:16.540 --> 10:25.900] Building Information Management Model. And they took both of those models and put them into one [10:25.900 --> 10:32.900] so they could move each of their training testbeds around the facility, maximize the use of the space [10:34.320 --> 10:41.740] and also in the design of the physical testbeds, they were able to see them to scale in front of [10:41.740 --> 10:47.260] them, interact with the different components, make sure everything was, you know, would work just [10:47.260 --> 10:58.300] how they expected. And so, as that partnership grew with Cellar, we started working a little [10:58.300 --> 11:08.720] bit more in public engagement. Which kind of led into us realizing the, how valuable extended [11:08.720 --> 11:21.350] reality headsets were in this work. So, with the new testbed designs, they had, you know, an [11:21.350 --> 11:30.950] interface similar to this where they were able to have a portion of the HMI in the virtual world [11:32.010 --> 11:38.330] and they could click buttons to toggle on different effects, just run through in software the [11:38.330 --> 11:57.600] iterations of how things would look and ultimately save time. And so, all of these testbeds, they [11:57.600 --> 12:03.540] represent physical critical infrastructure. So, they have the same OT, IT components that are [12:03.540 --> 12:12.100] used in, you know, saying like this is just a laboratory space. And so, to help to increase [12:12.100 --> 12:16.620] engagement, they have different effects, like that smoke that you can see in the bottom left [12:16.620 --> 12:28.740] corner. So, when the system, you know, maybe the HMI is vulnerable to a certain DLL injection, then [12:28.740 --> 12:36.800] that can cause some actual physical damage to the system. Maybe, you know, turning off some of the [12:36.800 --> 12:46.220] fans in a lab space or reversing fans in a fume hood causing gas to go into the lab space. [12:47.300 --> 12:59.650] So, working with this critical infrastructure has physical effects. So, you know, it's not [12:59.650 --> 13:06.110] only some database or something that goes down, but it can actually cause harm to a system and [13:06.110 --> 13:16.860] people. So, then we worked in public engagement during the first sort of half a year that we were [13:16.860 --> 13:22.700] working with Cellar. And as you can tell by the ugly carpet on the ground and the pictures on [13:22.700 --> 13:29.920] the slide, we're here at DEF CON. So, we had the opportunity to come to DEF CON as well as an [13:29.920 --> 13:40.520] array of different conferences around the nation. So, on the left is a picture of a small test bed [13:40.520 --> 13:49.940] that they developed for use in conferences. Because the full test beds are like 6 feet tall. [13:50.460 --> 13:56.020] And it would just be a pain to pack those around to conferences. So, they developed a smaller [13:56.020 --> 14:03.880] version which still has real IT components. They can, you know, share that with the public. [14:04.500 --> 14:13.280] But then in mixed reality and on the let's see, on the left, [14:15.420 --> 14:22.200] that man is wearing a Microsoft HoloLens. So, that's a mixed reality system. [14:22.740 --> 14:29.240] And what he's seeing is being shown in the middle there. So, because we aren't constrained to any [14:29.240 --> 14:36.200] sort of space in virtual or mixed reality, we can make it as representative as we want to. [14:36.520 --> 14:42.560] And so, there's a demo that they brought. [14:44.780 --> 14:49.000] Hello, hello. There's a small demo that they had brought. [14:50.880 --> 14:57.540] Emulates an electrical substation. And so, [14:57.540 --> 15:05.640] in the visualization, we modeled out, you know, a representative substation. And then we were also [15:05.640 --> 15:12.820] able to expand it out to show more of a suburb or community. And then in the attack, we can show [15:12.820 --> 15:26.900] how it affects the entire community losing power. So, at the bottom left, you have the physical [15:26.900 --> 15:32.640] skid in the environment, the lab space that they're housed in. So, they just have some computers [15:32.640 --> 15:42.300] with webcams that are monitoring these systems. And then anybody can log in and see the attacks [15:42.300 --> 15:48.620] and vulnerabilities that are on these systems and then learn how to secure against those attacks. [15:50.000 --> 15:57.820] And then the webcam shows a kind of difficult to see view of any effects that might happen if that [15:57.820 --> 16:10.300] critical infrastructure is attacked. And so, in the virtual space, again, we modeled out [16:10.300 --> 16:17.260] their systems to be able to show others resources that are available. And then we've also tried to [16:17.260 --> 16:24.300] make a more representative view of each of these systems. And so, the idea is that using [16:24.300 --> 16:32.000] digital engineering and digital twins, we're going to mix together these physical assets [16:32.540 --> 16:41.380] with the visualizations. And so, as somebody is working, you know, going through the training [16:41.380 --> 16:49.340] online, then that data is being relayed to a visualization that they can use with their team. [16:49.340 --> 16:58.260] You know, maybe it's a school or, you know, the workers, the cyber team of that city. [16:58.320 --> 17:03.620] And they can have a more engaging training as they go through it. [17:04.540 --> 17:15.280] And as well as just relaying state of the system, we merge all of that to a central data [17:15.280 --> 17:21.740] warehouse that we can then distribute data between the different systems, whether they need to do, [17:21.740 --> 17:38.670] you know, analytics on the data afterwards. Okay. So, some of the other projects that we have, [17:40.650 --> 17:49.290] I apologize. So, just to speak of some of the successes that we've had with this system. So, [17:49.290 --> 17:56.610] it was developed to be remote during COVID, because COVID sucks. And we wanted to still [17:56.610 --> 18:04.790] be able to help people. And so, they were able to see the visualizations of what would happen, [18:04.790 --> 18:09.890] be familiar with those things, have an engaging training. But then, [18:11.270 --> 18:17.570] Cellar was still able to provide that training on real hardware, not emulated hardware, [18:18.810 --> 18:27.210] in a remote setting. And so, one of the groups was a local municipality, [18:27.210 --> 18:32.330] and they were able to remote into the environment and train using this [18:35.110 --> 18:41.950] using their lab space. And while they were going through the trainings, you know, [18:41.950 --> 18:47.310] learning how to secure these systems that they needed to work on in their own municipality, [18:48.410 --> 18:55.630] their facility was physically hit. And so, since they were remotely training, [18:55.630 --> 19:01.730] you know, not having to travel out to Idaho, they were able to respond to that incident [19:02.690 --> 19:17.730] and use the training immediately. So, that's a big win. And so, the idea, again, [19:17.730 --> 19:23.990] is to be able to reach out, help as many people who maybe don't have the funding towards these [19:23.990 --> 19:33.310] trainings, to put towards cybersecurity research. And so, using mixed reality, [19:33.630 --> 19:40.670] we don't have to refill the tanks for these trainings. We don't have to physically do any [19:40.670 --> 19:45.390] maintenance. We can just run through training after training after training without worrying [19:45.390 --> 19:56.090] if it's going to break. And so, that saves us money as well. Lowers the cost of the resource. [19:56.150 --> 20:02.250] And then enables us to just send out these VR goggles or mixed reality goggles to [20:03.630 --> 20:10.010] whoever is signed up for the service. And they can still have that engagement. [20:11.370 --> 20:20.830] Okay. So, what's next? So, at the right image, might be a little hard to distinguish, but that's [20:21.390 --> 20:27.970] a router with some signals coming off of it. And I'm using some quotes right now. [20:28.270 --> 20:34.490] Because it's still theoretical. But the research is being done to leverage extended reality [20:34.490 --> 20:40.410] capabilities to visualize wireless radio frequency signals. And the goal is to be able to [20:40.410 --> 20:47.890] see the radio frequency signals in an environment, identify that it might be abnormal, and then [20:50.630 --> 20:57.090] using the geolocation support of these systems, be able to pinpoint the source of the signals. [21:00.670 --> 21:09.450] And so, second research effort focuses on expanding the services to train remotely with [21:09.450 --> 21:14.710] realtime visualization. Enable cyber incident responders to actually conduct their network [21:14.710 --> 21:19.470] analysis or hunt for malicious activities on their laptops while then being able to see the [21:19.470 --> 21:28.130] representations of the infrastructure behind them as they work. So, it really has been [21:31.110 --> 21:38.990] although these digital twins are still in early stages, and we still have a lot that we can do to [21:38.990 --> 21:51.070] improve the use of extended reality headsets in this work, it's already helped us enormously. [21:54.550 --> 22:05.110] And so, just to I guess just speak about a little bit about the some of the other work that we've [22:05.110 --> 22:12.610] been using these headsets for in the lab space. So, apart from cyber security, it's been [22:12.610 --> 22:21.090] instrumental in designer views between these different projects at the lab space. And so, [22:23.540 --> 22:29.990] we'll have multiple engineers come in to a visualization and they can view, you know, [22:29.990 --> 22:36.610] the new micro reactor that they're building in front of them to scale. And they can walk around, [22:36.610 --> 22:45.390] make notes on the on the object, move different components, and talk to each other while it's [22:45.390 --> 22:55.550] going on with data being relayed through our own lab built multiplayer server. And so, that [22:55.550 --> 23:02.170] has really, you know, saved the national lab millions of dollars for each project that we've [23:02.170 --> 23:08.790] been involved in because they've been able to catch mistakes early on before the project was [23:08.790 --> 23:17.050] actually built physically. And then share just one more one more project that we've been working [23:17.050 --> 23:25.310] on by creating a full digital twin of a nuclear reactor. So, it's a small reactor. It's about [23:25.310 --> 23:33.850] five watts used for testing. But we were able to create a digital twin, a high level digital twin [23:33.850 --> 23:45.610] that incorporated AI predictive analysis on the system. And so, it would predict 15 minutes in [23:45.610 --> 23:51.670] the future using the data that it's learned off of. Predict 15 minutes in the future if the system [23:51.670 --> 24:00.410] was going to be passing some sort of a temperature threshold or threshold on the the neutron flux. [24:00.410 --> 24:09.810] And because of this, it would then warn the operator and take autonomous control of the [24:09.810 --> 24:17.030] system bringing it back into its operating range. And a big portion of that is being [24:17.030 --> 24:24.010] able to give the operator a view into what's being what's being learned through the AI [24:24.750 --> 24:30.330] and how the digital twin is actually running. And so, we've given them two interfaces between [24:30.330 --> 24:42.010] the mixed reality headset and a web display. And it's truly been valuable and has been [24:42.010 --> 24:49.210] integral for the research that needs to be done for these smaller modular reactors. [24:49.950 --> 24:58.230] So, I will just open it up to any questions if there are any. And seems to just be a microphone [24:58.230 --> 25:06.870] right here. And then, if not, we can go and rest. So, there's a there's a microphone right [25:06.870 --> 25:14.110] there if you'd like to if you had a question. Which reactor were you modeling? Out of curiosity, [25:14.110 --> 25:24.810] was it the ATR or no? I'll just give you this microphone. Which reactor were you [25:24.810 --> 25:31.750] modeling? Was that the ATR or a different one? So, you know a little bit about the lab. Nice. [25:31.890 --> 25:38.930] So, his question was, which reactor were we modeling in the digital twin? And if it was the [25:38.930 --> 25:46.850] ATR or advanced test reactor at the Idaho National Lab? And so, this reactor was actually a university [25:46.850 --> 25:54.250] collaboration between the Idaho National Lab and Idaho State University. And so, it was using their [25:54.250 --> 26:11.280] AGN 201 reactor. AGN 201. Okay. Thank you. All right. I'll try this. Okay. You were saying that [26:11.280 --> 26:15.040] millions of dollars were saved because they were able to catch stuff early. I was just curious, [26:15.040 --> 26:20.720] do you have like one example? Like just an anecdote? Yeah. Great question. Thank you. So, [26:21.520 --> 26:27.260] his question again was, I had mentioned that we'd saved millions of dollars on construction on [26:27.260 --> 26:34.260] these projects using extended reality hardware. And he asked if there were any examples I could [26:34.260 --> 26:42.340] share. So, there's some glove boxes that we've been developing to be able to process some different [26:42.340 --> 26:51.260] materials. And they were able to, you know, again, we were loading the CAD models from the engineers [26:51.260 --> 26:58.680] into extended reality environments. And then they were able to visualize the glove boxes within the [26:58.680 --> 27:04.620] space that they were to be built. And they could go around, have design reviews with each other. [27:05.860 --> 27:11.280] They noticed that ergonomics on the system was not very good for all of their operators. [27:11.480 --> 27:17.120] And so, they had, you know, people put their arms into the virtual glove boxes. And they found that [27:17.120 --> 27:22.240] they were going to have to build some platforms for some people to stand on to be able to use [27:22.240 --> 27:26.300] the system. And then that just kind of defeats the whole purpose of building this multimillion [27:26.300 --> 27:32.080] dollar system, right? And so, they were able to lower things down so that everybody could use the [27:32.080 --> 27:42.040] system. On the same project, they were able to move the glove box around in parts, in the parts [27:42.040 --> 27:46.780] that it was going to be built into. And check if they were even able to fit it through the doorways [27:46.780 --> 27:52.380] of the facility. And so, they were able to determine that there would be some collisions. And [27:52.380 --> 28:00.200] they broke a couple of the pieces into even smaller sizes. And then clearances between doorways [28:00.200 --> 28:06.440] has always been, seems to always be an issue on whatever project we're on. So, just escape routes [28:06.440 --> 28:11.240] and things, they need to have a certain, you know, clearance between the doorways. And they were [28:11.240 --> 28:18.100] able to find those and make adjustments. And so, that's a smaller project. Some of the other [28:18.100 --> 28:24.260] larger projects that we've been on, you know, if they were able, if they were to go into construction [28:24.260 --> 28:30.540] at all on those, before they caught these design changes that needed to be made, then it would have [28:31.320 --> 28:35.540] cost millions and millions of dollars. So, yeah, thank you. [28:40.080 --> 28:44.080] Can you say a little bit, can people hear me or not? [28:44.080 --> 28:45.400] You might need to get closer. [28:45.400 --> 28:50.500] Yeah, hello. Can you say just a little bit about why virtual reality, as you've described it, [28:50.500 --> 28:55.600] is actually better than traditional, you know, on the flat monitor, you know, which has been used [28:55.600 --> 29:00.200] for many, many years now to catch many of the same sorts of mistakes. You know, what's, you know, [29:00.200 --> 29:05.660] I noticed that actually that your glove box example is probably a good example. But do you have any [29:05.660 --> 29:13.120] thoughts or why is this better than the traditional, you know, flat screen CAD? [29:13.340 --> 29:22.040] Yeah. So his question is, how does mixed reality differ from using just normal 2D CAD software [29:22.940 --> 29:29.860] and how it's been beneficial? And so how I see it is that [29:31.200 --> 29:38.320] with CAD, you need to be familiar with the system to fully understand everything that's going on. [29:38.320 --> 29:45.300] And so sharing with stakeholders, especially, it's hard for engineers to portray exactly what's [29:45.300 --> 29:52.240] going on. But in mixed reality, it's there. It's just there virtually, and they can walk around [29:52.240 --> 30:00.060] the system and have a discussion. And, you know, it removes the need for interpretation [30:00.060 --> 30:20.350] of these CAD models or blueprints. Thank you. Hey, man. Are the tools for VR development [30:20.350 --> 30:28.910] catching up or caught up to the same tools you would use for a flat screen or is there [30:28.910 --> 30:35.190] a difference? That's an interesting question. If the VR tools are caught up to the same tools [30:35.190 --> 30:41.970] that we'd be using in traditional CAD modeling. And so we're not actually doing any modeling [30:41.970 --> 30:51.870] inside of these, you know, virtual reality systems. Traditional CAD on computers is still superior. [30:53.370 --> 31:04.530] You have more precise controls. And it just it allows them to go in greater depth on the design [31:04.530 --> 31:13.810] as well. And so we don't do any of the design. But the software is catching up a lot. And already [31:13.810 --> 31:18.210] in the, you know, past couple of years that I've been working with these systems, they've matured [31:18.310 --> 31:26.190] a lot to where we can import the CAD into the virtual reality. And then if they make any updates [31:26.190 --> 31:33.310] to those systems, then it's getting to the point where we can just update it in our virtual system [31:33.310 --> 31:38.030] as well. So all of the interactions and things that we've developed as well can transition to [31:38.030 --> 31:47.070] the new model. And so I'd say that it's maturing, but it's not quite as mature as we want it to be. [31:47.070 --> 31:59.840] Thank you. Okay. Well, let's go rest. Thank you all.