WEBVTT 00:00.000 --> 00:10.400 So, we were asked by the XR Village to come and just give a little talk about the work that we've been doing at the Idaho National Lab using some XR equipment. 00:12.200 --> 00:19.200 So, behind the scenes of our daily lives, there's critical infrastructure that's supplying us. 00:21.360 --> 00:31.240 You know, all of the necessary services, you know, water treatment, um, our electricity, and all of that infrastructure needs to be secured. 00:31.240 --> 00:44.640 But as, you know, working in the security kind of aspect, we know that these systems definitely aren't as secure as we'd like them to be. 00:44.740 --> 01:08.600 And so, um, part of the work that we've been doing at the National Lab in partnership with the Seller Group is to work on securing those systems and helping, um, you know, smaller cities to know that their systems may not be secure, that there are vulnerabilities. 01:08.780 --> 01:15.040 And we've been using XR as a component of that, um, of that outreach. 01:15.580 --> 01:23.320 So, my name is Colton Heaps, and I've been working with the Idaho National Laboratory for about two years. 01:23.940 --> 01:32.820 Um, it's been really exciting to be able to work with, in a laboratory where, um, they have such unique infrastructure. 01:32.820 --> 01:41.220 So, for those of you who don't know, the Idaho National Lab is a research laboratory that's focused on nuclear research. 01:41.220 --> 01:47.220 And so, um, we have, you know, couple of different reactors out on our, our campus. 01:47.380 --> 01:58.880 And with that, you know, um, micro grids and a lot of different infrastructure that enables us some unique cyber security research. 01:58.880 --> 02:24.740 And so, um, I will kind of start out by describing the group that I work with, with digital engineering and some terms that I'll be sharing and introduce the, the seller group and then, um, get into how we've been using mixed reality and, uh, virtual reality for that, 02:24.740 --> 02:25.960 that work. 02:27.920 --> 02:30.940 So, what is digital engineering? 02:30.980 --> 02:38.600 Digital engineering focuses on supplying data to researchers regardless of the platform that they're performing research on. 02:38.740 --> 02:45.420 So, in, you know, times past, we had, um, researchers passing around documents. 02:45.820 --> 02:50.660 Uh, they'd have sacred word documents that had all the information for the projects. 02:50.880 --> 02:53.340 Um, that is not a good way to do engineering. 02:53.480 --> 03:02.480 So, we're trying to centralize all of that data no matter where it comes from and enable, um, the different platforms to be able to reach out and use that data. 03:03.960 --> 03:14.360 So, um, digital engineering helps to provide semi-autonomous design, autonomous operations, and real time anomaly detection. 03:15.200 --> 03:22.240 It integrates the threads of data, visualizations, AI and ML and physics models into a cohesive digital twin. 03:22.620 --> 03:27.340 And so, a digital twin is, it's a buzz word. 03:27.820 --> 03:34.640 Um, so, we're trying, it gets thrown around a lot in industry, but we're trying to define that at the national lab. 03:34.640 --> 03:38.580 And so, I like to compare it to, to the iPhone. 03:38.960 --> 03:44.160 So, we all know, you know, the impact that the iPhone had on our world. 03:44.400 --> 03:53.580 And, um, because of the iPhone, you know, led to other smartphones, rising popularity, and that truly changed our lives. 03:53.700 --> 04:00.680 But the iPhone itself is made up of a bunch of smaller technologies that already existed. 04:00.800 --> 04:08.080 And those technologies themselves weren't necessarily revolutionary, but the combination of all of those made something special. 04:08.080 --> 04:12.540 And so, that is, um, how I see digital twins. 04:13.060 --> 04:25.960 And, um, it's the combination of all of these different technologies that have existed in the past, but the combination of all of those together makes it something special. 04:27.380 --> 04:29.620 So, what is a digital twin? 04:30.140 --> 04:34.800 Like I mentioned, the digital twin is a merging of all these different technologies. 04:35.140 --> 04:40.540 The connection of integrated data, sensors and instrumentation from physical assets. 04:40.540 --> 04:48.840 So, taking that realtime data from the systems and feeding it back into visualizations and AI ML algorithms. 04:49.860 --> 04:56.940 The AI and an online monitoring and advanced visualization with mixed reality and virtual reality. 04:57.280 --> 05:01.460 So, that's kind of the INL's definition of the digital twin. 05:02.740 --> 05:11.780 And a digital twin mirrors the physical model while using realtime bidirectional communication to perform, you know, this magic. 05:12.780 --> 05:18.020 Enabling the user to take in data more efficiently. 05:18.180 --> 05:23.600 Giving the operator more data in a way that they can understand. 05:24.520 --> 05:28.820 And so, put that into context of the work that we do at the lab. 05:28.820 --> 05:32.660 Like I said, nuclear research and working with these nuclear systems. 05:36.280 --> 05:42.040 We need to be able to have safe, small modular reactors in the future. 05:42.220 --> 05:49.080 And they need to be able to have autonomous corrections. 05:49.920 --> 05:54.120 So, broadcasting into the future to see if these things are safe or not. 05:54.120 --> 05:59.620 And then if they're, you know, going into a hazardous direction, then they need to be able to fix themselves. 06:01.460 --> 06:09.440 But we've been using it with the cyber security group at the INL, specifically the CELR group. 06:11.000 --> 06:15.820 So, CELR stands for control environment laboratory resource. 06:17.380 --> 06:25.280 And so, CELR is sponsored by DHS's cyber security and infrastructure security agency or CISA. 06:25.360 --> 06:42.580 CELR is a lab scale environment with sector specific platforms with operational physical components as well as operational IT and OT environments with real controllers and human machine interfaces to operate the physical components. 06:43.040 --> 06:50.340 And so, there are about six of these training platforms that CELR has provided to the public. 06:50.340 --> 06:56.780 And each of these platforms have unique HMIs, PLCs and protocols that are used. 06:56.780 --> 07:21.640 And so, it takes the breadth of, you know, manufacturer's hardware and protocols that are used in industry and displays them here as a resource publicly available for, you know, international use so that anybody can gain more experience in securing this critical infrastructure. 07:22.280 --> 07:33.840 So, the environment is used to train cyber incident responders and expose them that actual networks and malicious activities are similar to what they would see in the real world. 07:34.140 --> 07:43.780 Support tool evaluation to determine effectiveness in operations and research mitigation measures that may counter those effects. 07:44.920 --> 07:54.780 CELR's capabilities have matured to include six sector specific platforms with six different controllers, HMIs and network protocols. 07:55.080 --> 08:02.740 This allows for diverse options of to fully represent a broader range of the nation's infrastructure. 08:04.260 --> 08:31.560 And it's definitely a valuable resource that is being provided and helps to supply this resource to smaller cities that may not have the funds to put towards cyber security and securing their infrastructure so they can take advantage of these resources. 08:43.580 --> 08:45.420 Okay. 08:45.720 --> 09:05.240 So, in the CELR group, while building these systems, they started to reach out to digital engineering just as a way to visualize their, you know, their plans in their new work space. 09:05.320 --> 09:11.900 And so, they were planning to build, they had one existing skid, I believe, and they were planning to build more in the space. 09:11.960 --> 09:23.960 And so, they wanted to use some of our XR capabilities that we've been using across the lab to visualize these components in their facility and maximize the efficiency of the space. 09:25.460 --> 09:43.320 And so, it's maybe not a very exciting way of using mixed reality as it's, you know, seems like everybody's using that now, but it's very valuable and saves so much time and money. 09:43.580 --> 10:00.740 And so, we were able to use some virtual reality goggles and take the existing CAD models as geometries and import them as meshes into our visualization software and mainly we use Unity for this. 10:02.000 --> 10:07.720 Because of how easy they've made it to work with a variety of hardware. 10:10.040 --> 10:19.160 And so, with those CAD models and the BIM model from their facility, so BIM is Building Information Management Model. 10:19.360 --> 10:42.420 And they took both of those models and put them into one so they could move each of their training testbeds around the facility, maximize the use of the space and also in the design of the physical testbeds, they were able to see them to scale in front of them, 10:42.420 --> 10:48.940 interact with the different components, make sure everything was, you know, would work just how they expected. 10:53.540 --> 11:00.120 And so, as that partnership grew with Cellar, we started working a little bit more in public engagement. 11:02.460 --> 11:13.820 Which kind of led into us realizing the, how valuable extended reality headsets were in this work. 11:16.670 --> 11:43.010 So, with the new testbed designs, they had, you know, an interface similar to this where they were able to have a portion of the HMI in the virtual world and they could click buttons to toggle on different effects, just run through in software the iterations of how things would look and ultimately save time. 11:55.140 --> 12:00.680 And so, all of these testbeds, they represent physical critical infrastructure. 12:00.680 --> 12:09.860 So, they have the same OT, IT components that are used in, you know, saying like this is just a laboratory space. 12:10.280 --> 12:17.600 And so, to help to increase engagement, they have different effects, like that smoke that you can see in the bottom left corner. 12:19.340 --> 12:33.740 So, when the system, you know, maybe the HMI is vulnerable to a certain DLL injection, then that can cause some actual physical damage to the system. 12:33.740 --> 12:46.220 Maybe, you know, turning off some of the fans in a lab space or reversing fans in a fume hood causing gas to go into the lab space. 12:47.300 --> 12:54.850 So, working with this critical infrastructure has physical effects. 12:56.850 --> 13:06.990 So, you know, it's not only some database or something that goes down, but it can actually cause harm to a system and people. 13:11.040 --> 13:18.300 So, then we worked in public engagement during the first sort of half a year that we were working with Cellar. 13:18.660 --> 13:25.140 And as you can tell by the ugly carpet on the ground and the pictures on the slide, we're here at DEF CON. 13:25.640 --> 13:33.680 So, we had the opportunity to come to DEF CON as well as an array of different conferences around the nation. 13:35.080 --> 13:44.300 So, on the left is a picture of a small test bed that they developed for use in conferences. 13:44.800 --> 13:49.940 Because the full test beds are like 6 feet tall. 13:50.460 --> 13:54.740 And it would just be a pain to pack those around to conferences. 13:54.740 --> 14:00.540 So, they developed a smaller version which still has real IT components. 14:00.540 --> 14:03.880 They can, you know, share that with the public. 14:04.500 --> 14:19.800 But then in mixed reality and on the let's see, on the left, that man is wearing a Microsoft HoloLens. 14:19.900 --> 14:22.200 So, that's a mixed reality system. 14:22.740 --> 14:26.820 And what he's seeing is being shown in the middle there. 14:27.340 --> 14:36.200 So, because we aren't constrained to any sort of space in virtual or mixed reality, we can make it as representative as we want to. 14:36.520 --> 14:42.560 And so, there's a demo that they brought. 14:44.780 --> 14:46.300 Hello, hello. 14:46.620 --> 14:49.000 There's a small demo that they had brought. 14:50.880 --> 14:54.900 Emulates an electrical substation. 14:55.560 --> 15:04.740 And so, in the visualization, we modeled out, you know, a representative substation. 15:04.740 --> 15:10.300 And then we were also able to expand it out to show more of a suburb or community. 15:10.440 --> 15:17.260 And then in the attack, we can show how it affects the entire community losing power. 15:22.760 --> 15:31.160 So, at the bottom left, you have the physical skid in the environment, the lab space that they're housed in. 15:31.280 --> 15:36.500 So, they just have some computers with webcams that are monitoring these systems. 15:36.660 --> 15:48.620 And then anybody can log in and see the attacks and vulnerabilities that are on these systems and then learn how to secure against those attacks. 15:50.000 --> 16:05.220 And then the webcam shows a kind of difficult to see view of any effects that might happen if that critical infrastructure is attacked. 16:06.680 --> 16:14.280 And so, in the virtual space, again, we modeled out their systems to be able to show others resources that are available. 16:14.400 --> 16:20.560 And then we've also tried to make a more representative view of each of these systems. 16:20.840 --> 16:35.120 And so, the idea is that using digital engineering and digital twins, we're going to mix together these physical assets with the visualizations. 16:35.780 --> 16:49.340 And so, as somebody is working, you know, going through the training online, then that data is being relayed to a visualization that they can use with their team. 16:49.340 --> 16:58.260 You know, maybe it's a school or, you know, the workers, the cyber team of that city. 16:58.320 --> 17:03.620 And they can have a more engaging training as they go through it. 17:04.540 --> 17:25.340 And as well as just relaying state of the system, we merge all of that to a central data warehouse that we can then distribute data between the different systems, whether they need to do, you know, analytics on the data afterwards. 17:33.310 --> 17:34.210 Okay. 17:34.210 --> 17:41.670 So, some of the other projects that we have, I apologize. 17:41.670 --> 17:48.710 So, just to speak of some of the successes that we've had with this system. 17:48.710 --> 17:54.450 So, it was developed to be remote during COVID, because COVID sucks. 17:54.670 --> 17:58.290 And we wanted to still be able to help people. 18:00.170 --> 18:08.850 And so, they were able to see the visualizations of what would happen, be familiar with those things, have an engaging training. 18:08.910 --> 18:20.650 But then, Cellar was still able to provide that training on real hardware, not emulated hardware, in a remote setting. 18:20.650 --> 18:37.230 And so, one of the groups was a local municipality, and they were able to remote into the environment and train using this using their lab space. 18:37.990 --> 18:51.330 And while they were going through the trainings, you know, learning how to secure these systems that they needed to work on in their own municipality, their facility was physically hit. 18:51.410 --> 19:06.150 And so, since they were remotely training, you know, not having to travel out to Idaho, they were able to respond to that incident and use the training immediately. 19:06.790 --> 19:11.470 So, that's a big win. 19:14.430 --> 19:28.430 And so, the idea, again, is to be able to reach out, help as many people who maybe don't have the funding towards these trainings, to put towards cybersecurity research. 19:29.670 --> 19:38.150 And so, using mixed reality, we don't have to refill the tanks for these trainings. 19:38.150 --> 19:41.710 We don't have to physically do any maintenance. 19:41.830 --> 19:46.910 We can just run through training after training after training without worrying if it's going to break. 19:47.890 --> 19:53.490 And so, that saves us money as well. 19:53.490 --> 19:56.090 Lowers the cost of the resource. 19:56.150 --> 20:06.090 And then enables us to just send out these VR goggles or mixed reality goggles to whoever is signed up for the service. 20:06.090 --> 20:10.010 And they can still have that engagement. 20:11.370 --> 20:12.210 Okay. 20:12.210 --> 20:13.650 So, what's next? 20:15.310 --> 20:26.010 So, at the right image, might be a little hard to distinguish, but that's a router with some signals coming off of it. 20:26.010 --> 20:27.970 And I'm using some quotes right now. 20:28.270 --> 20:30.710 Because it's still theoretical. 20:30.710 --> 20:39.270 But the research is being done to leverage extended reality capabilities to visualize wireless radio frequency signals. 20:39.270 --> 20:57.090 And the goal is to be able to see the radio frequency signals in an environment, identify that it might be abnormal, and then using the geolocation support of these systems, be able to pinpoint the source of the signals. 21:00.670 --> 21:11.330 And so, second research effort focuses on expanding the services to train remotely with realtime visualization. 21:11.370 --> 21:24.770 Enable cyber incident responders to actually conduct their network analysis or hunt for malicious activities on their laptops while then being able to see the representations of the infrastructure behind them as they work. 21:26.270 --> 21:51.070 So, it really has been although these digital twins are still in early stages, and we still have a lot that we can do to improve the use of extended reality headsets in this work, it's already helped us enormously. 21:54.550 --> 22:09.210 And so, just to I guess just speak about a little bit about the some of the other work that we've been using these headsets for in the lab space. 22:09.210 --> 22:19.250 So, apart from cyber security, it's been instrumental in designer views between these different projects at the lab space. 22:19.530 --> 22:34.010 And so, we'll have multiple engineers come in to a visualization and they can view, you know, the new micro reactor that they're building in front of them to scale. 22:34.210 --> 22:53.550 And they can walk around, make notes on the on the object, move different components, and talk to each other while it's going on with data being relayed through our own lab built multiplayer server. 22:55.010 --> 23:10.490 And so, that has really, you know, saved the national lab millions of dollars for each project that we've been involved in because they've been able to catch mistakes early on before the project was actually built physically. 23:11.290 --> 23:22.610 And then share just one more one more project that we've been working on by creating a full digital twin of a nuclear reactor. 23:22.610 --> 23:25.050 So, it's a small reactor. 23:25.050 --> 23:27.490 It's about five watts used for testing. 23:27.510 --> 23:41.610 But we were able to create a digital twin, a high level digital twin that incorporated AI predictive analysis on the system. 23:41.750 --> 23:48.790 And so, it would predict 15 minutes in the future using the data that it's learned off of. 23:48.790 --> 24:00.410 Predict 15 minutes in the future if the system was going to be passing some sort of a temperature threshold or threshold on the the neutron flux. 24:00.410 --> 24:13.010 And because of this, it would then warn the operator and take autonomous control of the system bringing it back into its operating range. 24:13.990 --> 24:27.910 And a big portion of that is being able to give the operator a view into what's being what's being learned through the AI and how the digital twin is actually running. 24:27.910 --> 24:34.150 And so, we've given them two interfaces between the mixed reality headset and a web display. 24:34.390 --> 24:49.210 And it's truly been valuable and has been integral for the research that needs to be done for these smaller modular reactors. 24:49.950 --> 24:56.310 So, I will just open it up to any questions if there are any. 24:56.310 --> 24:59.250 And seems to just be a microphone right here. 24:59.730 --> 25:02.410 And then, if not, we can go and rest. 25:05.010 --> 25:09.890 So, there's a there's a microphone right there if you'd like to if you had a question. 25:11.490 --> 25:13.430 Which reactor were you modeling? 25:13.430 --> 25:15.950 Out of curiosity, was it the ATR or no? 25:20.330 --> 25:22.170 I'll just give you this microphone. 25:23.610 --> 25:25.230 Which reactor were you modeling? 25:25.230 --> 25:27.430 Was that the ATR or a different one? 25:28.610 --> 25:30.650 So, you know a little bit about the lab. 25:30.650 --> 25:31.750 Nice. 25:31.890 --> 25:37.870 So, his question was, which reactor were we modeling in the digital twin? 25:37.870 --> 25:42.650 And if it was the ATR or advanced test reactor at the Idaho National Lab? 25:42.870 --> 25:52.950 And so, this reactor was actually a university collaboration between the Idaho National Lab and Idaho State University. 25:53.090 --> 25:56.110 And so, it was using their AGN 201 reactor. 26:01.070 --> 26:02.810 AGN 201. 26:06.800 --> 26:07.440 Okay. 26:07.440 --> 26:08.320 Thank you. 26:08.540 --> 26:08.860 All right. 26:08.860 --> 26:09.500 I'll try this. 26:09.500 --> 26:10.040 Okay. 26:10.140 --> 26:14.300 You were saying that millions of dollars were saved because they were able to catch stuff early. 26:14.300 --> 26:16.100 I was just curious, do you have like one example? 26:16.100 --> 26:17.240 Like just an anecdote? 26:17.660 --> 26:18.120 Yeah. 26:18.120 --> 26:18.760 Great question. 26:18.760 --> 26:19.520 Thank you. 26:19.820 --> 26:31.100 So, his question again was, I had mentioned that we'd saved millions of dollars on construction on these projects using extended reality hardware. 26:31.420 --> 26:35.160 And he asked if there were any examples I could share. 26:35.360 --> 26:43.560 So, there's some glove boxes that we've been developing to be able to process some different materials. 26:44.060 --> 26:55.160 And they were able to, you know, again, we were loading the CAD models from the engineers into extended reality environments. 26:55.200 --> 27:00.820 And then they were able to visualize the glove boxes within the space that they were to be built. 27:01.060 --> 27:04.620 And they could go around, have design reviews with each other. 27:05.860 --> 27:11.280 They noticed that ergonomics on the system was not very good for all of their operators. 27:11.480 --> 27:15.400 And so, they had, you know, people put their arms into the virtual glove boxes. 27:15.500 --> 27:23.080 And they found that they were going to have to build some platforms for some people to stand on to be able to use the system. 27:23.080 --> 27:28.260 And then that just kind of defeats the whole purpose of building this multimillion dollar system, right? 27:28.420 --> 27:32.900 And so, they were able to lower things down so that everybody could use the system. 27:34.100 --> 27:43.720 On the same project, they were able to move the glove box around in parts, in the parts that it was going to be built into. 27:43.720 --> 27:47.960 And check if they were even able to fit it through the doorways of the facility. 27:48.320 --> 27:51.680 And so, they were able to determine that there would be some collisions. 27:52.000 --> 27:56.460 And they broke a couple of the pieces into even smaller sizes. 27:57.720 --> 28:05.040 And then clearances between doorways has always been, seems to always be an issue on whatever project we're on. 28:05.140 --> 28:10.580 So, just escape routes and things, they need to have a certain, you know, clearance between the doorways. 28:10.580 --> 28:13.280 And they were able to find those and make adjustments. 28:14.000 --> 28:17.520 And so, that's a smaller project. 28:17.520 --> 28:33.300 Some of the other larger projects that we've been on, you know, if they were able, if they were to go into construction at all on those, before they caught these design changes that needed to be made, then it would have cost millions and millions of dollars. 28:33.300 --> 28:35.540 So, yeah, thank you. 28:40.080 --> 28:44.080 Can you say a little bit, can people hear me or not? 28:44.080 --> 28:45.400 You might need to get closer. 28:45.400 --> 28:46.240 Yeah, hello. 28:46.300 --> 28:58.960 Can you say just a little bit about why virtual reality, as you've described it, is actually better than traditional, you know, on the flat monitor, you know, which has been used for many, many years now to catch many of the same sorts of mistakes. 28:58.960 --> 29:05.060 You know, what's, you know, I noticed that actually that your glove box example is probably a good example. 29:05.060 --> 29:13.120 But do you have any thoughts or why is this better than the traditional, you know, flat screen CAD? 29:13.340 --> 29:14.740 Yeah. 29:15.060 --> 29:26.640 So his question is, how does mixed reality differ from using just normal 2D CAD software and how it's been beneficial? 29:26.640 --> 29:38.320 And so how I see it is that with CAD, you need to be familiar with the system to fully understand everything that's going on. 29:38.320 --> 29:46.080 And so sharing with stakeholders, especially, it's hard for engineers to portray exactly what's going on. 29:46.080 --> 29:49.480 But in mixed reality, it's there. 29:49.480 --> 29:55.720 It's just there virtually, and they can walk around the system and have a discussion. 29:56.020 --> 30:04.800 And, you know, it removes the need for interpretation of these CAD models or blueprints. 30:06.240 --> 30:07.740 Thank you. 30:15.810 --> 30:17.150 Hey, man. 30:17.650 --> 30:28.910 Are the tools for VR development catching up or caught up to the same tools you would use for a flat screen or is there a difference ? 30:28.910 --> 30:30.690 That's an interesting question. 30:31.510 --> 30:39.330 If the VR tools are caught up to the same tools that we'd be using in traditional CAD modeling. 30:39.370 --> 30:47.770 And so we're not actually doing any modeling inside of these, you know, virtual reality systems. 30:47.870 --> 30:51.870 Traditional CAD on computers is still superior. 30:53.370 --> 30:56.210 You have more precise controls. 30:56.490 --> 31:05.430 And it just it allows them to go in greater depth on the design as well. 31:05.430 --> 31:08.890 And so we don't do any of the design. 31:08.890 --> 31:13.230 But the software is catching up a lot. 31:13.230 --> 31:24.110 And already in the, you know, past couple of years that I've been working with these systems, they've matured a lot to where we can import the CAD into the virtual reality. 31:24.110 --> 31:34.010 And then if they make any updates to those systems, then it's getting to the point where we can just update it in our virtual system as well. 31:34.010 --> 31:39.150 So all of the interactions and things that we've developed as well can transition to the new model. 31:39.550 --> 31:47.070 And so I'd say that it's maturing, but it's not quite as mature as we want it to be. 31:47.070 --> 31:48.810 Thank you. 31:53.860 --> 31:55.540 Okay. 31:56.400 --> 31:58.380 Well, let's go rest. 31:58.380 --> 31:59.840 Thank you all.