[00:00.770 --> 00:07.390] All right, well welcome folks. It's my pleasure and honor to introduce some great friends of mine. [00:07.390 --> 00:12.110] They are all hackers, and we have our moderator today. I'll do a brief introduction for him, [00:12.110 --> 00:17.430] but first and foremost, welcome to the biohacking panel. How independent security researchers, [00:17.430 --> 00:22.930] vis-a-vis hackers, work with MDMs, or that notorious medical device manufacturer. This [00:22.930 --> 00:28.330] is the bad, the ugly, and the great, also affectionately known as bug. My name is the [00:28.330 --> 00:32.710] Red Dragon 1949. You know me as the volunteer coordinator. But more importantly, I'd like to [00:32.710 --> 00:37.130] introduce you to the panelists. First and foremost, ladies first, we have Veronica, [00:37.130 --> 00:43.290] or Bea Schmidt. Yesterday was her birthday, so happy birthday Bea. She is a former federal law [00:43.290 --> 00:50.130] enforcement in South Africa. She has done forensic investigations. She is a cardiac implant patient [00:50.130 --> 00:55.890] three times, and she is doing some great forensic work for one of the MDMs that will be showcased [00:55.890 --> 01:02.430] here at the Biohacking Village Medtronic. Welcome Bea. Second, we have Ms. Natalie Shuva. [01:02.430 --> 01:09.290] She is the CEO for Sternum out of Tel Aviv, Israel. She comes to us from the UDIN 8200 of the [01:09.290 --> 01:14.490] Israeli Defense Force. She can do things remotely with a mobile phone and the applications that are [01:14.490 --> 01:19.870] fascinating and yet scary at the same time, so welcome Natalie. And then last but not least, [01:19.870 --> 01:25.970] we have Mr. Peter Morgan out of San Diego, California. Mr. Morgan is quite, [01:25.970 --> 01:31.110] how shall we say, effective at basically taking a medical device down to its very [01:31.110 --> 01:35.630] fundamental binary level and doing things with it that would surprise you, not only from a [01:35.630 --> 01:41.070] software and a firmware level, but certainly from a vulnerability exposure. He has notable [01:41.070 --> 01:44.990] common vulnerabilities and exposures to his name associated with some Medtronic devices, [01:44.990 --> 01:50.790] so welcome Peter. And then last but not least, our moderator, Mr. Kyle Erickson. He's the Director [01:50.790 --> 01:55.670] for Product Security Engineering with the Cardiac Rhythm Heart Failure Business Division [01:55.670 --> 02:01.530] at Medtronic out of Minneapolis, Minnesota. He has extensive breach and incident response [02:01.530 --> 02:07.610] experience with one of the largest healthcare organizations here in the Minnesota area as well [02:07.610 --> 02:11.930] as globally, so he has faced some of these individuals on the panel that he's going to [02:11.930 --> 02:16.130] be moderating. So welcome, Mr. Erickson, and I will turn it over to you. [02:18.030 --> 02:23.910] Thank you, Red Dragon, for those great introductions. I have the luxury to be here on the panel with [02:23.910 --> 02:29.570] Pete, Natalie, and Vi, and I'm really excited to dig into their story and learn how they've [02:29.570 --> 02:34.390] been working with medical device manufacturers, what medical device manufacturers can do better, [02:34.390 --> 02:39.810] and what we're working on to improve in the industry. So I want to start out with an origin [02:39.810 --> 02:45.630] story, and I'm going to start with Natalie. Natalie, what makes you a hacker, and how did [02:45.630 --> 02:53.170] you really get involved in the security movement around medical devices? Yeah, well, I actually [02:53.170 --> 03:00.810] don't refer to myself as a hacker, but I can tell you basically my story. I grew up with four big [03:00.810 --> 03:08.910] brothers and started my undergraduate degree at the age of 14, computer science. It wasn't just [03:08.910 --> 03:13.890] weird because I was 14, it was also weird because nobody in my family ever went to college or [03:13.890 --> 03:20.910] something like that, so I was like the black sheep in the family. And once I graduated, then I just [03:21.550 --> 03:28.730] started my period in the Israeli Unit A200, one of the elite technological units there, [03:28.730 --> 03:34.850] and this is where I got exposed to cybersecurity, low-level development, and research and [03:34.850 --> 03:42.730] exploitation in general. We did some significant things in the unit, handled the most advanced [03:42.730 --> 03:51.690] techniques and technology, and after the period of time there, I wanted to keep using [03:51.690 --> 03:59.350] cybersecurity to create impact. So during my time in Cetebrite, I did a lot of vulnerability research [03:59.350 --> 04:04.810] and exploitation, mainly on Linux kernel and Android platforms, and I didn't consider it [04:05.670 --> 04:15.290] as being a hacker, but mainly to provide evidence to stop pedophiles and human trafficking and stuff [04:15.290 --> 04:19.930] like that, because using the vulnerabilities and exploitation, we were able to extract [04:20.630 --> 04:26.790] forensics and basically evidence from the mobile devices, including encrypted ones. [04:26.790 --> 04:35.730] So for me, it was the passion and the impact that such tools can create. And the medical device was [04:35.730 --> 04:43.730] actually not a coincidence. I was always dreaming and thinking about becoming a doctor. So when I [04:43.730 --> 04:50.250] finished my master's in computer science, I figured that this is like my last opportunity to go into [04:50.250 --> 04:56.490] the medical industry. I thought about starting medical school, but with the knowledge I had [04:56.490 --> 05:02.970] in cybersecurity, I met with a few friends and entrepreneurs, and one of them is my partner Boaz, [05:02.970 --> 05:09.710] and we started thinking about medical devices and cybersecurity. And when I saw that remote care [05:09.710 --> 05:16.090] and remote monitoring could have such a tremendous effect on our lives, and they were not [05:16.090 --> 05:23.470] properly secure, this was something that attracted me tremendously, because I figured that creating [05:23.470 --> 05:30.250] the right security to these devices could really improve patients' lives and improve quality of [05:30.250 --> 05:38.750] care and treatment. And I saw how the knowledge that we have from the cybersecurity industry [05:38.750 --> 05:46.270] could leverage this kind of technology. And this is what actually attracted me to Cofan External. [05:47.230 --> 05:51.830] That's excellent. Obviously, you may not call yourself a hacker, but you have the tactics, [05:51.830 --> 06:00.530] techniques, and capabilities. Correct. So, Vee, I'd like to go to you. And obviously, we know you have a unique story. [06:00.530 --> 06:06.730] You're not only a hacker, a researcher, a goon, but you have implanted a medical device and refer to [06:06.730 --> 06:10.730] yourself as a cyborg. So why don't you just give us a little bit more about your background and how [06:10.730 --> 06:15.510] you've been getting more and more involved with medical device manufacturers in the industry. [06:17.210 --> 06:23.330] Yeah, so I wasn't a very good hacker when I was a child. And I basically got caught changing [06:23.330 --> 06:29.450] records at school, because I felt everyone should have a good year. Unfortunately, that led me down [06:29.610 --> 06:37.070] a path that my parents weren't very proud of. But going back at the age of 18, I joined the [06:37.070 --> 06:41.650] Special Emergency Learning Unit, which is a law enforcement agency that deals with cross-border [06:41.650 --> 06:48.410] human trafficking, child pornography, some really nasty things. We actually made use of [06:48.410 --> 06:54.070] Celebrite. So I'm very familiar with what Nati's referring to. And let me tell you, if you want to [06:54.070 --> 07:02.250] get a physical image, those things that they develop are key. But at 19, I also had about [07:02.250 --> 07:08.190] approximately two weeks left to live when I got my device implanted. And I know the quality of [07:08.190 --> 07:14.130] life that it gave me. But since then, I was obsessed with what my device does, for the pure [07:14.130 --> 07:20.950] fact that I believed in the saying, trust but verify. But I could never verify what was keeping [07:20.950 --> 07:27.030] me alive. And that was the biggest, greatest frustration. I reached out to you guys about [07:27.030 --> 07:36.490] three years ago, and I was met with the ugly part of Medtronic, sadly. I got met with the legal team. [07:37.350 --> 07:42.970] And I felt that the conversation wasn't going very well. I remember these words, and [07:42.970 --> 07:47.210] Tara, that used to work with you, knows this well. I said to them, well, if you want to bring your [07:47.210 --> 07:52.950] lawyers to the table, I can do the same, and then we can have a conversation. But that's where it [07:52.950 --> 08:00.710] ended. And in walks Biohack Village three years ago. Nina, being forever the master planner, put [08:00.710 --> 08:06.770] me on the path for Bill. But then I came to severe loggerheads last year, because he could [08:06.770 --> 08:13.670] not confirm to me that he was from Medtronic, even though I knew he was. And we went at it, [08:13.670 --> 08:20.530] and we had our conversation. And fast forward a couple of months, I'm now finding my feet [08:20.530 --> 08:26.710] within Medtronic and doing research into early detection. Because I didn't want to focus on the [08:26.710 --> 08:33.170] problem. I wanted to find the problem and then solve it. So it's not just about whether the [08:33.170 --> 08:37.770] device can be hacked. We all know everything can be hacked. But it's actually about having [08:37.770 --> 08:43.730] some tangible evidence to deal with, to early detect and to mitigate it. For me, literally [08:43.730 --> 08:49.570] close to my heart is the embedded implanted devices that are often misunderstood, forgotten [08:49.570 --> 08:56.990] about. And these are the things that are connected to people, flesh and bones. And I just decided to [08:56.990 --> 09:02.230] take this on and put my story out there and say, you know, these things are cool, but they're also [09:02.230 --> 09:06.890] scary. But that doesn't mean we need to go burn the internet. It just means we need to do more [09:06.890 --> 09:14.570] to make it safer. Absolutely. And we're glad that we can make some inroads there. I know it's been [09:14.910 --> 09:20.590] a journey, to say the least. Pete, you've had a similar journey. I'd love to just kind of hear [09:20.590 --> 09:25.850] your origin story and what got you interested in medical device security. And not only do you have [09:25.850 --> 09:30.350] vulnerability discovery on your side, you've also looked at solutions and improving the problem. So [09:30.350 --> 09:37.650] give us your background. Sure. So I started out as a kid learning how to hack video games and [09:37.650 --> 09:45.150] change them before the advent of client mutual authentication. And that really got me hooked. [09:45.430 --> 09:50.930] And as I grew up learning about software development, networking, reverse engineering, [09:51.650 --> 09:58.050] exploitation, going forward into things like software-defined radio and actually understanding [09:58.050 --> 10:05.010] RF better, it was kind of like I picked up these skills along the way. And as I was able to [10:05.010 --> 10:09.990] escalate in a career path and get better jobs, I finally got into a spot where I was able to [10:09.990 --> 10:16.190] affect what direction our team was going. And previously, I'd spent some time at a hedge fund [10:16.950 --> 10:20.610] kind of protecting a billionaire, making sure their billionaire is still worth billions. [10:21.030 --> 10:24.810] And there wasn't a ton of job satisfaction out of that other than the interest of [10:24.810 --> 10:32.890] how that works. And I got a project at doing a small medical device in a consulting relationship. [10:32.890 --> 10:40.410] And it was really neat because the problem set was really interesting from like the requirements [10:40.410 --> 10:44.450] that how these devices operate, the conditions they have to operate in, the battery life, [10:44.450 --> 10:48.750] processing power, all this kind of stuff gives really interesting constraints. But also, [10:48.750 --> 10:53.130] when you make improvements to something like that, it has a direct impact on people's lives. [10:54.310 --> 10:58.490] And it was a lot more interesting to work on those types of problems. [10:58.490 --> 11:04.930] That's when I got kind of started just doing some research on some of the Medtronic gear and [11:04.930 --> 11:13.190] was able to publish some vulnerabilities after maybe an ugly or bad initial interaction that [11:13.190 --> 11:18.390] then turned quite positive. And then one of the things I'm most proud of, I think, is [11:19.930 --> 11:26.490] having the patience to work through that process with Medtronic and then end up in a productive [11:26.490 --> 11:32.430] working relationship where now I do get to work on solutions and help with fixing things, [11:32.430 --> 11:36.890] not just identifying issues, sending a vulnerability report, and walking away. [11:36.930 --> 11:42.090] So that's been really rewarding. Absolutely. And we thank all three of you. You've all been [11:42.090 --> 11:48.150] influential in the journey here as MDMs have progressed and adapted and understood that [11:48.150 --> 11:53.410] there's a problem but also looked to solutions. So Natalie, I want to come back to you. I have [11:53.550 --> 11:58.210] a specific question and it's more around military-grade technology and techniques. [11:58.210 --> 12:03.330] How do you think that can really help secure medical devices and the infrastructure and [12:03.330 --> 12:11.710] hospital systems that they reside in? Yeah. Well, I think many times in the history, [12:11.710 --> 12:18.770] we saw that the military developed one of the cutting-edge technologies in many areas [12:18.770 --> 12:23.850] in the industry. And as part of it, I think that security techniques are also [12:24.450 --> 12:29.770] well-developed within the military. And when you talk about military-grade security, [12:29.770 --> 12:35.170] I think this is what we want for our medical devices, right? Because you don't want your [12:35.170 --> 12:42.930] security camera to be protected the same way as your pacemaker. I think medical devices do require [12:42.930 --> 12:53.210] some higher extent of security and deeper level of security. And I think what we see in the [12:53.210 --> 13:00.930] military is wide range of attack scenarios and attack techniques and exploitation techniques. [13:00.930 --> 13:06.810] And you realize that for me, at least, securing medical devices just by doing vulnerability [13:06.810 --> 13:14.210] patching or penetration testing or avoiding hard-coded passwords, it was not enough because [13:14.850 --> 13:21.710] I know and I'm familiar with how you can bypass those things pretty easily. And I think that [13:21.710 --> 13:29.350] taking some of the knowledge and basically military-grade techniques can help you understand [13:29.350 --> 13:36.550] the kind of holistic and comprehensive security that these devices deserve. And one of the things [13:36.550 --> 13:44.270] that I think was significant in our way of thinking is that even if you secure the MDMs code [13:44.790 --> 13:50.390] or doing some vulnerability research or pen testing, it's not enough because those devices [13:50.390 --> 13:56.830] contain third-party code as well. And Ripple 20 is a great example, right? We see a TCPIP [13:57.630 --> 14:03.890] library affected millions of devices, power grids, medical devices, and most solutions today [14:03.890 --> 14:09.890] just can't handle third-party vulnerabilities. You can't really protect against vulnerabilities [14:09.890 --> 14:14.590] that are within code that you didn't develop and you don't have the source code to. [14:14.590 --> 14:21.350] And I mention in it because that in the military, we handled a lot with binary code and with embedded [14:21.350 --> 14:27.830] systems and with the ability to reverse engineer other people's code. And actually, we are [14:27.830 --> 14:33.650] leveraging these kinds of techniques in our philosophy to embed protections and security [14:33.650 --> 14:42.570] into these closed-source components. Because in my view, the MDM has to be responsible for [14:42.570 --> 14:47.890] the security of the device regardless of the components within the device. So, we have to [14:47.890 --> 14:54.730] give controls to the MDMs to secure these third parties. And I think these binary-level capabilities [14:55.370 --> 15:01.630] are well-established in the military and can be used to holistically secure medical [15:01.630 --> 15:09.030] devices. And this is what we try to do. Excellent. I just want to say, Natalie, [15:09.030 --> 15:15.030] as a patient and as someone that's been shouting about this stuff very loudly, [15:15.330 --> 15:21.630] it's like you're serenading my ears at the moment. You know, I'm all for what you are saying. [15:21.630 --> 15:28.130] It is solely the MDM's responsibility. They have to take ownership and responsibility [15:28.130 --> 15:35.850] of the tick label, right? As a patient... I agree with you. But on the other hand, [15:35.850 --> 15:41.430] I must say it's not an easy task, right? Because the MDM, it's very difficult to secure third [15:41.430 --> 15:47.030] parties. And there is no way to avoid implementing third parties within your device. So, until [15:47.030 --> 15:54.410] someone wouldn't take the responsibility to create such controls for MDMs and to create [15:54.410 --> 16:01.530] the security products that they can use to basically protect their devices, we really [16:01.530 --> 16:11.050] can't blame them. But we do need to ask them to use the cutting-edge tools to secure their devices. [16:11.050 --> 16:17.390] This is indeed something that I believe in. Yeah, it's embedding the security culture in how [16:17.390 --> 16:23.650] we look at devices, right? But, you know, Medtronic is a good example from this learning [16:23.650 --> 16:27.610] from lessons learned, right? They've come from a place where they were in a really bad position [16:27.610 --> 16:32.390] to where they are now, which is never an easy thing taking that ownership and responsibility [16:32.390 --> 16:39.570] saying, hey, we need to do more. We need help. We can't do it on our own. I often say it takes [16:39.730 --> 16:46.110] a stronger person to say, hey, I now need help and go to people that can help him versus having [16:46.110 --> 16:51.790] the persona of, I know everything, we are a big manufacturer, we can do it on our own. [16:51.910 --> 16:56.210] Completely agree with you. [16:57.770 --> 17:01.510] That's great. I want to pull on that thread a little bit, Vy, and just talk a little bit more [17:01.510 --> 17:06.370] about your patient-hacker relationship and how being a patient maybe changes your perspective [17:07.210 --> 17:13.330] on device security and how the more you've learned about how devices are built and a little bit more [17:13.330 --> 17:18.130] under the hood, how that's also changed your perspective. Can you talk a little bit about that? [17:18.710 --> 17:23.490] Yeah, sure. So, that brings back a memory of mine. I think it was two years ago, February. [17:23.630 --> 17:31.210] I was actually in ICU where I had some issues. I had some device issues and I had to be [17:31.210 --> 17:38.910] resuscitated. But I was lying in an ICU connected to 15 different kinds of machines doing infusion, [17:38.910 --> 17:45.490] monitoring. I had a specific external monitor monitoring my pacemaker at that time. [17:45.990 --> 17:51.170] And I was offered guest Wi-Fi, obviously, because hospitals want us to feel safe and [17:51.170 --> 17:57.670] connected to the world. And soon realizing that these systems weren't segmented and that I could [17:57.670 --> 18:05.730] potentially may have or may not have access to the central station that had my vitals on it. [18:05.730 --> 18:12.330] But just realizing from a perspective of a patient how interconnected these systems are. [18:12.350 --> 18:19.750] And the more that I watch and just observe the world or even attending talks, I realize that [18:19.750 --> 18:25.190] the word medical device is exclusively used for anything in healthcare and hospital and anything [18:25.190 --> 18:31.530] that a manufacturer makes, whether it's a Windows 10 machine or versus a complex embedded system. [18:32.010 --> 18:40.230] And I think until we start understanding each system's unique, I say language, linguistics or, [18:40.230 --> 18:45.090] you know, when you have someone that you're observing, you try and, you know, observe them and [18:45.090 --> 18:51.790] learn what they are, how they talk, all those kind of things. But we don't do that with devices. [18:51.790 --> 18:55.730] And I think I appreciate what Peter does and what Nathalie does because we reverse [18:55.730 --> 19:01.810] engineer things to understand how they work, not only at a, you know, firmware level, but at a [19:01.810 --> 19:09.770] file system level, at a binary level. And as a patient, I wanted to know what keeps me alive. [19:09.770 --> 19:16.350] I wanted to know what libraries were used, what hardware was used. And I did that through OSINT. [19:16.350 --> 19:20.450] I literally Googled manuals and from manuals I pivoted to get more information because [19:20.450 --> 19:24.490] the manufacturer wasn't disclosing it to me even as a patient. [19:24.490 --> 19:29.910] But I wanted to know that I can trust my device because I've seen that these devices are [19:29.910 --> 19:36.590] misunderstood and implemented in a way where we build bridges or walls around our networks, [19:36.590 --> 19:42.610] but we're so soft in the middle, but we never consider the malicious insider, [19:42.610 --> 19:48.730] which is a big thing because I've realized that the threat from hospitals might come from within, [19:48.730 --> 19:52.890] especially with everything with COVID-19 becoming borderless. [19:52.890 --> 19:58.050] I don't know where the border stops or the perimeter starts and stops anymore. [19:58.910 --> 20:06.030] So I think it's just, we need to take the time to understand the devices, learn their language, [20:06.030 --> 20:09.610] and understand their constraints because they're all different. [20:10.130 --> 20:15.050] I always say, you know, beautifully broken, wonderfully flawed, [20:15.050 --> 20:19.610] meaning we can have the most secure and perfect device, there might still be a flaw [20:19.610 --> 20:24.250] and it still might be broken, but that doesn't take away from the beautiful [20:24.250 --> 20:30.350] synergy that goes into these devices. I know I geek out because I spoke to one [20:30.350 --> 20:34.910] of the engineers and learned what my functions on my pacemaker and ICE does. [20:35.730 --> 20:38.790] And let's just say it was like a moment where I sat back and said, [20:38.790 --> 20:43.830] this is such a simple, complicated device, but I trust it. [20:43.830 --> 20:50.290] And it's the weirdest thing, but I geeked out knowing that the maths behind it is so wonderful [20:50.290 --> 20:55.990] that in a nanosecond it raises my heart rate up. So I'm actually getting emotional. [20:55.990 --> 21:02.170] Excuse me, but yeah, it's a good thing when you realize that you can trust what's in you, [21:02.170 --> 21:05.610] right? I can trust that it's going to do what it needs to do. [21:05.610 --> 21:13.710] I must mention just regarding one thing you said about getting the hardware specifications, [21:13.710 --> 21:20.770] the software specifications, being exposed to how the device is being built. [21:20.770 --> 21:27.090] There are also downsides into publishing this information, meaning bad actors [21:29.110 --> 21:34.630] having this information very accessible, having them being able to see what kind of third parties [21:34.630 --> 21:39.150] and start looking for vulnerabilities or understand the specs and extract the firmware [21:39.990 --> 21:47.890] very easily. You want to make it as hard as possible. So I do think that we need to [21:47.890 --> 21:56.230] keep some sort of balance between what information we put public and what information we maybe [21:56.230 --> 22:02.990] expose just under certain terms. So I understand you as a patient wanting to know as much as you [22:02.990 --> 22:09.050] can about your implantable device, but imagine that bad actors could have access to this [22:09.050 --> 22:19.450] information as well. Sorry, Peter. So just to clarify, the reason that I said I OSINT this [22:19.450 --> 22:26.590] was before I spoke to them and then I realized that I could get all the information. That's [22:26.590 --> 22:32.890] when I went to the MDM, right? So I'm all for that certain things should be on the internet. [22:32.990 --> 22:38.570] And this was one of the first things I raised with them. So when I meant as a patient, I went [22:38.570 --> 22:44.430] to them, I was concerned. I was petrified because I managed to get technical manuals [22:45.010 --> 22:51.150] from technical manuals jump to hardware manuals, which led me to more information that I [22:51.150 --> 23:00.230] seriously wanted to find in an OSINT exercise. So yeah, that's what happened. And that's when [23:00.230 --> 23:05.170] the legal team slapped down on me and like, where did you get this information? And I'm like, [23:05.170 --> 23:11.950] your website. Yeah, we had the same experience where they asked, where did you get the devices? [23:11.950 --> 23:16.690] And we're like, eBay. We know about that. Yeah. [23:18.470 --> 23:24.910] I think that underscores the point of, I mean, I can understand the request to have a balance, but [23:24.910 --> 23:34.970] the minute that comes out, you can't take it back away. And I would caution the idea that [23:37.590 --> 23:43.750] hardware designers, software designers that rely on the concept of anonymity and hiding [23:43.750 --> 23:50.670] specifications of the device may make worse decisions predicated on the idea that no one's [23:50.670 --> 23:55.330] going to know what the hardware is or what the, you know, what radio chip they used, [23:55.330 --> 24:00.570] that kind of thing, right? That's an exercise in time. And when you think about the lifetime of [24:00.570 --> 24:07.770] these devices, I just caution that concept because once that comes out, we can't take it back away. [24:09.050 --> 24:12.030] Yeah, well, I mean... Go ahead, Vi. [24:12.730 --> 24:18.930] Sorry. So Peter, I think you and I, I mean, I didn't have to go buy my device. I just got it [24:18.930 --> 24:25.250] and I got the doctor. I'm like, hey, hey, hey, I would like my device, please. Can I keep it? [24:25.730 --> 24:30.650] And they gave it to me. I mean, your story, of course, is way cooler. I mean, I didn't have to [24:30.650 --> 24:36.830] buy it. Yeah. Talk a little bit more about that. Talk a little bit more about that story. [24:37.110 --> 24:43.210] So I'm a little bit of an empathic person. So like awkward situations tend to land [24:43.210 --> 24:48.630] strongly on me, I guess you might say. And amidst doing some of the work I've done, [24:50.230 --> 24:56.250] I've taken apart and reversed the majority of the home monitor device. But I needed to see live [24:56.250 --> 25:01.890] communications to understand what some of the implications of that were. So I tried to get my [25:01.890 --> 25:07.310] hands on some implantables and I think it was like between 30 and 40 morgues and coroner's [25:07.310 --> 25:13.690] offices I called trying to get my hands on before basically anyone that did cremation, [25:13.690 --> 25:18.360] right, because they remove the devices before cremation. And it was some really, [25:18.360 --> 25:24.320] awkward conversations. I got yelled at a few times pretty aggressively. Understandably so, [25:24.320 --> 25:29.660] they didn't see what I was trying to do. But yeah, that was a little bit awkward. [25:30.460 --> 25:35.380] Yeah, you should seriously try going to a cardiologist's office and they Googled you [25:35.380 --> 25:41.320] and they've seen one of your talks and you get specifically watched doing everything. [25:41.960 --> 25:47.480] Yeah, that's a different patient perspective. I'm officially banned from all major hospitals in my [25:47.480 --> 25:53.820] hometown from having a laptop, or even getting Wi-Fi access. I think they've grown tired of me [25:53.820 --> 25:59.240] saying, hey, we should fix this. Right? Or me spending time fixing it for them and saying, [25:59.240 --> 26:05.320] hey, I fixed this for you. You know, here's the credentials go, you know, go wild. It's just [26:05.320 --> 26:12.260] that whole culture of when you say to someone, but we can fix this for you. It's not an insult. [26:12.260 --> 26:16.840] It's genuinely people, researchers wanting to make things better. [26:17.160 --> 26:23.640] And perhaps not having the social... I don't sometimes have the social cues like you would [26:23.640 --> 26:30.440] say Pete. I can totally imagine having those conversations with coroners. Like, you know, [26:30.440 --> 26:35.520] I can just... I think I wanted to be a fly on the wall for that. [26:37.020 --> 26:41.700] So as a medical device manufacturer, I think I'm internalizing two takeaways. [26:41.700 --> 26:47.380] Don't rely on security by obscurity, but also make sure you keep operation security in mind, [26:47.380 --> 26:53.840] and you're not just posting manuals and details that can be found through open source intelligence [26:53.840 --> 26:59.900] gathering. So two takeaways right there. So I want to switch gears and go to you, Pete. [26:59.900 --> 27:03.780] You know, I know we talked a lot about your vulnerability discovery. Obviously, we hinted [27:03.780 --> 27:08.880] at a few of the CVEs and things that you brought to Medtronic. But how do you see the testing [27:08.880 --> 27:14.000] landscape changing for devices? And what should medical device manufacturers do on their own [27:14.000 --> 27:18.260] to learn from what researchers have done in the testing space? [27:18.800 --> 27:25.460] A great question. Well, I think the testing is going to have to follow the path of development. [27:25.900 --> 27:31.480] And one of the major things I see is very important for the medical device community right now [27:32.620 --> 27:37.720] is there's a lot of devices that have update functionality, but there's a very, [27:37.720 --> 27:43.500] very small percentage that actually use it. And I think there's an opportunity right now to fix that [27:43.500 --> 27:49.000] problem before it gets out of hand. Secure update is one of those problems that if you don't design [27:49.000 --> 27:55.180] that in early on, you can kind of lose control of what it is you're trying to work with. [27:55.480 --> 28:01.340] It's a perfectly ripe target for attackers, especially when it comes to low power, questionably [28:01.340 --> 28:07.700] connected, embedded devices. So I would stress as far as something to develop on, [28:07.700 --> 28:12.620] work on secure update and get that working immediately so it can be used. The more [28:12.620 --> 28:17.660] confidence you have in that part of the tech stack, the more able you're going to be to [28:17.660 --> 28:23.100] respond to things like security incidents or updates because of some binary or open source [28:23.100 --> 28:30.020] package that comes in. And that kind of, I think, will drive a lot more of the [28:31.160 --> 28:36.860] modifications to testing that's going to happen. As these devices, we've seen a trend now for [28:36.860 --> 28:44.900] devices that used to use homegrown protocols and bespoke radio communication systems [28:45.650 --> 28:51.340] to communicate, largely shifting over to using things like Bluetooth and something that's a lot [28:51.340 --> 28:58.120] more standardized but also tends to work with a smartphone. And as that happens, we're seeing [28:58.260 --> 29:05.100] a shift into a lot more open source library usage in these products. That particular thing [29:05.100 --> 29:10.520] is a problem near and dear to my heart. We're working on a tool to basically audit the open [29:10.520 --> 29:19.480] source ecosystem and mine for malware, mine for backdoors. The vulnerability space, I think, [29:19.480 --> 29:23.440] is kind of, well, people are looking at that pretty aggressively right now, but no one's [29:23.440 --> 29:27.800] approaching it from the concept of who are the authors of these packages and how they work [29:27.800 --> 29:34.080] together. This is going to land very important on the medical device manufacturer. When you think [29:34.080 --> 29:39.260] about, I spent a little bit of time in the defense sector myself, and when you think about how [29:39.260 --> 29:44.960] nation-states approach problems, there's a tactical and a long-term strategic goal to some of these [29:44.960 --> 29:52.340] things. And compromising the GitHub credentials of a developer that writes a package that's three [29:52.340 --> 29:56.520] levels upstream from something that goes into a medical device is a great way to weigh in weight [29:57.100 --> 30:02.800] for access into an entire fleet of these things. And again, you think about that kind of problem [30:02.800 --> 30:08.340] without a good secure update mechanism, and now you have something that is irreversible. [30:09.000 --> 30:16.240] So I see this as the trend of development changes with medical devices. [30:16.740 --> 30:21.620] The testing requirements are going to change as well. I would love to see a universe where [30:21.620 --> 30:28.600] when a device is designed, I mean, we have automated testing through like fuzzing of [30:28.600 --> 30:33.820] the protocol, right? I mean, we have knowledge of different aspects of this that we can bolster and [30:33.820 --> 30:39.260] use things like computers to help us, and have that built into the design such that when, [30:39.260 --> 30:43.480] you know, before release, there's all these different tiers that have happened [30:44.220 --> 30:53.140] aside from just someone manually reviewing source code. That is still a strong improvement over [30:53.740 --> 30:59.640] requiring people to buy these devices or call corners offices and take them apart, [30:59.640 --> 31:06.500] dump firmware, reverse engineer it, and try to find bugs. The more people we can get involved [31:06.500 --> 31:12.000] with this at a productive level, I think, is going to be more valuable too. And that's, [31:12.000 --> 31:15.680] I think, a large challenge of the medical device manufacturer. How do you [31:17.020 --> 31:22.440] adjust the signal-to-noise ratio such that you're getting competent, quality people [31:22.440 --> 31:27.680] that really want to help without, you know, wading through the deluge of [31:28.220 --> 31:31.500] things that may happen if you just bug-bounty it, let's say. [31:32.100 --> 31:36.080] Yeah, I think there's like a diverse approach to this. I think, you know, in a prior life, [31:36.080 --> 31:41.200] you were at a boutique shop and, you know, we've used all different types of testing companies, [31:41.200 --> 31:45.300] we've internalized some of this, but, you know, even finding those niche boutique shops or those [31:45.300 --> 31:49.940] individuals that can really get at certain elements have really helped us, I know. [31:50.760 --> 32:02.740] I can, if I may jump in, as a person who researched many platforms that are highly [32:03.540 --> 32:11.360] secured, highly being researched, highly being fuzzed and pen-tested, including Linux kernel [32:11.360 --> 32:19.060] and iPhone devices, I can tell you that it's impossible to put a device out there without [32:19.060 --> 32:25.360] vulnerabilities. And once it's out there and you have enough researchers, some of them will find [32:25.360 --> 32:31.480] the vulnerabilities. And if we'll take a look at the long-term strategy and take a look at what [32:31.480 --> 32:39.940] the traditional IT security world has gone through, we have endpoint security and we have network [32:39.940 --> 32:46.580] security. And the endpoint security is something sitting on the device in real time, capable of [32:46.580 --> 32:53.040] preventing and detecting attacks, regardless if the device is vulnerable, regardless what type of [32:53.040 --> 33:01.600] attack it is, and regardless of, you know, secure design or where the vulnerability is. Is it in the [33:01.600 --> 33:10.740] OTA or improper input validation, etc. So for me, if you ask about what's the right approach, [33:10.740 --> 33:16.840] the right approach would be sustainable. Meaning I think secure design is cool, I think penetration [33:16.840 --> 33:25.120] testing is cool, but it will never be sufficient. You will never have a 100% vulnerability-free [33:25.120 --> 33:34.400] device, especially when you use third-party code. So I don't know how much static analysis or [33:34.400 --> 33:39.500] vulnerability assessment tools you use, but they are not very good at finding vulnerabilities in [33:39.500 --> 33:45.520] binary code. They don't. And source code is not the only thing that goes into your device, not to [33:45.520 --> 33:50.920] mention hardware components that have software within them. So your Wi-Fi module or Bluetooth [33:50.920 --> 33:58.480] module, it contains vulnerabilities as well. You have vulnerabilities on the chip level. So in my [33:58.480 --> 34:05.340] view and in my opinion, only something that's operating in real time, capable of verifying the [34:05.340 --> 34:13.680] integrity of the device, capable of preventing damage, data theft, or basically preventing attacks. [34:13.680 --> 34:18.760] And on the other end, also monitor the device, meaning you need to be alerted if something [34:18.760 --> 34:26.320] bad happens or maybe happens. This is the approach that I believe in more, because [34:27.280 --> 34:33.820] I think this is why endpoint security has been so strong for many years now in the traditional [34:33.820 --> 34:44.900] IT space. And I think this is where IoT and IOMT should go to. And obviously there are challenges [34:44.900 --> 34:52.300] in installing something on those devices. They are low resource. They are real-time operating [34:52.300 --> 34:59.620] systems. They are diversified. They are extremely different from traditional servers or PCs. [34:59.620 --> 35:05.200] But this is where innovation comes to the picture. I think we can solve this problem. I even think [35:05.200 --> 35:12.180] that we've solved part of the problem for sure. But I don't think that this challenge should [35:12.180 --> 35:18.920] keep us from putting something on a device to protect it. We should just do a breakthrough [35:18.920 --> 35:28.120] in how we see endpoints for IoT devices. Yeah, so I totally agree with what everyone's saying. [35:28.120 --> 35:34.740] Something dawned on me the other day. If we look at the longevity of medical devices, for example, [35:34.740 --> 35:40.840] could be a decade. So if you look at just some of the statistics, 600,000 medical devices [35:40.840 --> 35:47.760] implanted yearly in the US, those will last 10 years. So if you start adding the numbers up, [35:47.760 --> 35:53.800] you're dealing with an ocean of legacy devices that at one point in their lifetime will be [35:53.800 --> 36:01.340] vulnerable. And it's not technology that you can rip out of a body or, you know, some people only [36:01.340 --> 36:06.320] can have that monitor depending on what device you are talking about. But those things are built [36:06.320 --> 36:12.240] to last for a long time. So any solution will have to be a multidisciplinary approach, eating it from [36:12.240 --> 36:18.740] all sides, but need to grow with the device, need to be updatable, need to be changeable, [36:19.340 --> 36:24.220] and basically grow with the times when the device necessarily can't. [36:26.000 --> 36:31.940] I'm glad you brought that up, Vy. So I'm interested and I'll go kind of back around the panel and [36:31.940 --> 36:36.840] start with Pete. How do we approach this legacy device problem? I think Vy already teased out [36:36.840 --> 36:44.000] the updatability and the forecasting of the life cycle, but we have a lot of legacy debt right now [36:44.000 --> 36:48.720] that we're dealing with. So what are some ways that we can work through that? Pete, I'll start with you. [36:50.040 --> 36:57.740] That's a super challenging problem. I think there has to be some preparation for [36:59.540 --> 37:05.940] a worst-case scenario. And worst-case scenario being, think about the attack surface. [37:06.660 --> 37:15.660] If that case is disabling radio connectivity on some of these things, [37:15.660 --> 37:19.300] working with some of the medical device manufacturers, come to learn [37:20.100 --> 37:24.640] this isn't a software change for a website that can be pushed into the CICD pipeline and [37:24.640 --> 37:32.120] show up in 10 minutes. This takes legal and regulatory guidance and preparation and [37:32.120 --> 37:39.620] submission to the FDA, and that can take a long time. And my suggestion is prepare for those [37:39.620 --> 37:47.200] scenarios now. Get ready for, how do we do this if we have to? Because the reality of [37:48.700 --> 37:53.780] explanting devices from hundreds of thousands or millions of people is completely untenable. [37:54.160 --> 37:59.660] So how do you get... and I realize that maybe there's going to be an inherent trade-off in [38:00.100 --> 38:06.460] the quality of care provided, and that's probably a situation you get to only [38:06.460 --> 38:12.920] as a worst-worst-case scenario. These devices keep people alive, and that we want to [38:13.660 --> 38:17.720] keep enabling that, because like, you know, as V mentioned, these are things that [38:18.500 --> 38:24.200] some people wouldn't be here without those things, and we don't want to instill the wrong level of [38:24.200 --> 38:31.540] fear. But planning for those scenarios now so that if those need to be enacted, I think is [38:31.540 --> 38:36.440] one of the only things you can do for the legacy devices that don't have an update mechanism. [38:37.420 --> 38:43.460] If they do, I mean, like setting up the testing for this now and being ready for it, [38:44.700 --> 38:48.700] it's an inevitability. It's a when, not if. [38:50.700 --> 38:57.220] I think what we are doing together with Medtronic, I think Medtronic is a true [38:57.220 --> 39:06.540] pioneer in this support to legacy devices, as what we are doing together, installing protection [39:06.540 --> 39:13.860] into a five-year-old device, I think is five years old, enhancing the security infield, [39:13.860 --> 39:22.700] getting visibility, getting protection into a really old medical device using the OTA. [39:22.700 --> 39:32.740] And I think it's a true innovative play, and I think that I truly support this kind of [39:33.740 --> 39:44.360] acts, and I think that I have a lot of respect to Medtronic in being a pioneer in doing such a thing. [39:45.920 --> 39:51.540] So I think we can learn from it. I think we can provide enhancements to existing device flits [39:52.220 --> 39:59.560] of course with the right controls and the right MDM to take it to the next level and to move [39:59.560 --> 40:12.460] forward. But it's a good example what we are doing together, and I hope that we could pave the way. [40:14.180 --> 40:19.060] Absolutely. Baby steps in the journey, but exciting stuff. Vi, I know you have an opinion on this. [40:22.020 --> 40:26.040] I always have. You're good. [40:29.310 --> 40:32.070] You're on mute. Back on mute, Vi. [40:32.450 --> 40:36.650] Oh goodness, I'm having audio issues today. It's like the audio day of a howl. [40:38.150 --> 40:44.870] So my feelings on this is we can't even start facing the legacy problems until we [40:44.870 --> 40:50.210] stop making stupid decisions and choices going forward, right? We have to now nip this in the [40:50.210 --> 40:57.930] bud and say, going forward, we're going to do better. I haven't looked into a lot of other [40:57.930 --> 41:01.910] things. I've Googled it. I have read up on it. I'm very excited about what you guys are doing [41:01.910 --> 41:08.430] with Medtronic. That certainly is a step in the right direction, but visibility is key, right? [41:08.870 --> 41:13.350] Like Pete and I have had discussions where everyone's like, we don't know if the ICD or [41:13.350 --> 41:18.570] pacemaker has been attacked. Well, do you even have the data to know that that's happened? [41:18.570 --> 41:23.170] So we want to deal with legacy. We need to start getting visibility moving forward because [41:23.170 --> 41:29.030] otherwise we're adding this amount of... another year goes past. Now we have another group of [41:29.030 --> 41:34.970] legacy devices that's going to haunt us for 10 years, right? So get it right going forward [41:35.430 --> 41:41.530] and then start dealing, finding solutions for those devices that you don't have visibility on. [41:41.530 --> 41:47.870] Find ways to get eyes on your products and your data because where you have a blind spot [41:47.870 --> 41:52.290] is most likely where you're going to be attacked. And it's not like a nation state actor is going to [41:52.290 --> 41:57.990] say, you know what? I have owned your product, right? Researchers have the courtesy to come, [41:57.990 --> 42:03.530] you know, have a discussion. Very bad people looking to do bad, bad things aren't going to [42:03.530 --> 42:08.790] have that discussion with you. You're not going to know they're there. And if you need one patient [42:08.790 --> 42:16.350] to have your device and they get attacked and they ultimately pay with their lives, [42:16.350 --> 42:20.570] I can tell you, you're going to have a string of patients wanting to explode their devices, [42:20.570 --> 42:26.150] which is why something I don't want to have happen. It scares me because I look at where [42:26.150 --> 42:33.330] I was at 19, where I am now at 33, celebrating my birthday, having two kids, having a master's, [42:33.330 --> 42:38.310] thanks to a device that kept me living, right? Otherwise I would be dead. It's as easy and as [42:38.310 --> 42:45.190] plain as that. It added years to my life I was not supposed to have. Take that technology away [42:45.190 --> 42:53.570] and as a human race, we go backwards. So patients need MDMs. Whether people in security want to see [42:53.570 --> 43:00.630] them as an adversary, there are people like me that rely on them to develop the technology to [43:00.630 --> 43:07.290] keep us alive, right? It's important to understand that it's a love-hate relationship. You know, [43:07.290 --> 43:11.850] we're not all going to do it right, but if we can have the conversations and we can fight it out, [43:11.850 --> 43:21.130] we can find the solutions that are super cool and super techy and super new, we should try them, [43:21.130 --> 43:27.690] right? What I think Sternum is doing, what Pete's doing, is hitting the same problem from a [43:27.690 --> 43:34.050] different side. I'm concentrating more on how do we detect and log these things? How do I investigate [43:34.050 --> 43:40.390] it? How can I, you know, have eyes on the prize sooner? That's my passion point. But we're all [43:40.390 --> 43:45.890] effectively covering this from a different side, which is good, because all sides are vulnerable [43:47.050 --> 43:48.650] and all sides are needed. [43:50.250 --> 43:55.190] And I think you're bringing it all together, right? Like, how can we get better visibility? [43:55.430 --> 43:59.250] And, you know, Natalie talked a little bit about this, but I'd like you to go into more detail. [43:59.250 --> 44:06.010] What are your thoughts around proactive controls in kind of the post-market, which is a term we [44:06.010 --> 44:10.950] just use for detection, logging, monitoring. I know we've thrown surveillance out there, [44:10.950 --> 44:16.610] but to be more in line with the community, it's incident response, it's logging, detection, [44:16.610 --> 44:20.350] that capability. So how do you see proactive controls fitting into that model? [44:21.210 --> 44:29.630] Yeah, it's a great question. And I think I want to learn from other industries, which already [44:29.630 --> 44:37.110] developed into this direction. So you don't see devices today within enterprise or in cloud [44:37.110 --> 44:43.550] infrastructures that are not monitored actively, that do not pop up alerts when something suspicious [44:43.550 --> 44:52.450] is happening. And I see the medical space behaving the exact same way. I think with solutions, [44:52.910 --> 44:57.450] I won't mention ours, but I think there are many solutions that can do that, [44:57.450 --> 45:01.690] provide monitoring and visibility into the device itself. And then I think that [45:02.190 --> 45:09.150] this kind of information and incident response should go into two main personas. One is the [45:09.150 --> 45:16.090] product security in the MDM. Monitoring is distributed device fleets. Understanding if [45:16.090 --> 45:22.470] somebody is researching this device, selling the device on eBay, maybe trying to hack into the [45:22.920 --> 45:28.630] information. These kinds of things can be visible with the right controls. And of course, [45:28.630 --> 45:35.590] there is a technological challenge of supporting old devices, old code base, which didn't come with [45:35.590 --> 45:41.830] visibility inside without logging, without monitoring, but that can be fixed. It can be [45:41.830 --> 45:48.890] installed post-market, and it's even easier to install it pre-market. The other persona is the [45:48.890 --> 45:55.310] hospital, the clinic, those devices that if they come with built-in protection and monitoring, [45:55.570 --> 46:01.930] they can be the eyes of the hospital for a beginning of an attack attempt. Imagine an [46:01.930 --> 46:07.650] attack attempt just a month ago, there was a ransomware attack on hospital starting for a [46:07.650 --> 46:14.090] remote patient monitor. So the hacker penetrates to the internal network through the vulnerable [46:14.090 --> 46:20.270] IoT device. So imagine this device transmits information. Imagine this device is capable of [46:20.270 --> 46:26.170] detecting the attack attempt and even preventing it. Not only you prevented the attack, the hospital [46:27.050 --> 46:34.810] that moment become aware that is under attack and can act accordingly to secure other devices, [46:34.810 --> 46:40.330] maybe unpatched devices. So you don't have to have all the environment protected, but as you [46:40.330 --> 46:46.730] have more devices that are transmitting, visible, protected, you have more visibility into your [46:46.730 --> 46:58.170] entire enterprise or hospital and can respond accordingly. So I think both personas benefit [46:58.170 --> 47:05.370] from this kind of visibility. And I think moving forward, we can use these kind of tools to embed [47:05.370 --> 47:13.770] security and visibility. And I'm making a difference between the two, because the kind of [47:13.770 --> 47:20.950] solution that I see in mind is first a deterministic solution to prevent attacks in real time, prevent [47:20.950 --> 47:27.230] the damage, because in medical devices, there is no time to waste. A successful attack could lead to [47:27.550 --> 47:33.930] a little resource, and you don't have enough time to detect the attack and respond in a week or two [47:33.930 --> 47:40.530] weeks. Sometimes in enterprise, it takes six months to respond to a security incident. You don't have [47:40.530 --> 47:46.090] this kind of time in medical devices, so you have to block the attack in real time. The other aspect [47:46.090 --> 47:52.750] is really to become aware. And to become aware means to get alerted when something is behaving [47:52.750 --> 47:58.910] differently or suspiciously, or if there were a prevented attack attempt. And once you are aware [47:58.910 --> 48:06.510] that this kind of attack happened, let's say in the US, and you have devices that you can't update [48:06.510 --> 48:12.150] in another country with the same vulnerability, because you know what the vulnerability at this [48:12.150 --> 48:18.530] point, because you have visibility into it, you're capable of providing fast response to alert your [48:18.530 --> 48:25.910] patients, to alert the hospital, to update the devices with a patch. So I think it's an essential [48:25.910 --> 48:34.930] thing that we need to embed into the medical space, and we're on it. [48:36.090 --> 48:40.470] I love that, because I think we're starting our journey to improve our visibility, improve our [48:40.470 --> 48:44.790] prevention in near real time, improve our detection capabilities. But we've thought [48:44.790 --> 48:49.490] into the future and how we need to better partner with healthcare delivery organizations, [48:49.490 --> 48:53.990] hospital systems, and provide some of that information, right, to the example you gave [48:53.990 --> 48:59.270] about the ransomware outbreak. And just, you know, having dealt with ransomware myself in a [48:59.270 --> 49:04.010] past life, you know, obviously, time is of the essence. And, you know, that's something that [49:04.010 --> 49:08.650] we'd love to be able to help out with as an MDM in the future using these techniques and tools [49:08.650 --> 49:15.530] that we're talking about today. So V, I want to switch gears a little bit. We're talking about [49:15.530 --> 49:20.450] all different types of aspects of, you know, security controls and capabilities to give back, [49:20.450 --> 49:25.290] but you have a unique forensics background. How does that make you approach device security [49:25.290 --> 49:29.750] differently than, say, you know, the traditional vulnerability disclosure scenarios? [49:31.310 --> 49:37.150] Yeah, so I think forensics is this unique focus on scientific processes. I think that's the one [49:37.150 --> 49:43.430] thing that was hammered into my head is we need to follow a scientific process. So you need to [49:44.130 --> 49:49.050] look at it from every aspect. But I was doing forensics for eight years straight, [49:49.050 --> 49:54.930] solely focusing on that and realizing that I wasn't quite understanding all sides of it. [49:55.030 --> 50:01.450] So I always say, we have a saying in forensics, we find the truth. So we reconstruct what's [50:01.450 --> 50:07.770] happened. So I decided to actually start re-achieving, so learning how to attack. [50:07.770 --> 50:14.030] And then I learned how to defend. And then I learned how to investigate. And taking that, [50:14.030 --> 50:21.270] I decided to start applying that type of mindset of taking a multidisciplinary view to things, [50:21.270 --> 50:28.070] saying, OK, well, I want to know if we're being attacked. I'm kind of tired of having, you know, [50:28.070 --> 50:33.910] it told me that we don't have any data that supports any attack. And now knowing that [50:34.990 --> 50:42.010] we don't have the data actually available. So to have more visibility, we can start seeing [50:42.010 --> 50:48.250] whether devices are actually currently being attacked. Because we don't know. I spoke to [50:48.250 --> 50:54.590] multiple coroners around the world through law enforcement agency contacts. And the question I [50:54.590 --> 51:00.350] said to them, if you have a cardiac patient that has an implantable, right, they've passed away, [51:00.350 --> 51:04.230] doesn't necessarily mean that it's, you know, suspicious circumstances. But do you, [51:04.230 --> 51:11.990] do you, you know, get a program, pull the data, look for the time of death, look of any strange, [51:11.990 --> 51:18.750] the device itself. And I realized that, yeah, the devices get sent back to the manufacturer that [51:18.750 --> 51:24.850] does basically a memory review to see that everything is like it should be. But people [51:24.850 --> 51:30.810] forget that programmers can be compromised and something can happen from a programmer to a device. [51:31.230 --> 51:37.970] And we don't leverage the information that we have. So the important thing for me is we have [51:37.970 --> 51:45.910] no ability at the moment, unless a researcher comes to an NDM, or someone makes a grand gesture [51:45.910 --> 51:55.170] that they've hacked a device, unless it causes chaos, or becomes public, somehow, we don't know. [51:55.270 --> 52:00.010] And that makes me uneasy as a forensics person, because forensics, we want to know, [52:00.010 --> 52:04.630] we want to know the bits and bytes of how something functions. I mean, the LaCroix [52:04.630 --> 52:09.990] principle is a big thing that I live my life by. So everything that comes into connection with each [52:09.990 --> 52:16.110] other leaves a trace evidence. I want to know what it is. But if we don't have visibility, [52:16.810 --> 52:21.610] right, it's there, but we don't have the capabilities to make it visual. [52:21.770 --> 52:28.330] So I want to learn more of what we have, what we need, and what we need to do to kick ass to get [52:28.330 --> 52:36.290] there. So that's my approach to what I'm doing with Medtronic is I'm assessing what you have, [52:36.910 --> 52:44.630] what we need, and holding ass to get there. And I'm seeing it come together with the various [52:44.630 --> 52:49.690] aspects that you guys have brought together, you know, covering it from different angles. [52:49.910 --> 52:54.490] And the forensics in me is like saying, we can apply forensics in development. [52:54.490 --> 52:59.390] You know, we can apply forensics pre-market, post-market. It doesn't necessarily mean [52:59.390 --> 53:04.990] it's an investigation, but it's creating the evidence that we need for the future. [53:05.370 --> 53:09.070] That's creating the data now for when we do need it. [53:09.890 --> 53:15.630] You know, I'm always drawn to the example of Microsoft and how they used other data sets [53:15.630 --> 53:21.410] to detect a large Windows vulnerability. And I think really was mining data in Windows [53:21.410 --> 53:26.650] event reporting. And I know that a big piece of what we're thinking about is we have a lot of [53:26.650 --> 53:31.250] remote monitoring information. We know what is above the line and what's below the line, [53:31.250 --> 53:36.550] and we're trying to develop that anomaly detection there. And it can't only be done [53:37.170 --> 53:42.910] with that type of detection. That's one leg in the stool. But as we look at on-device agents, [53:42.910 --> 53:47.430] as we look at other ways that we can pick this up with our fleet distribution infrastructure, [53:47.430 --> 53:52.890] I think that really has to come together to get this full picture. And all of you have hit on [53:52.890 --> 53:58.850] that element. And I think it's not to discount testing or threat modeling. All of that has to [53:58.850 --> 54:03.750] happen as well. But we really want to emphasize this prevention in real time and then this [54:03.750 --> 54:09.050] detection of the current devices and fleet that are legacy and are out there live today. And what [54:09.050 --> 54:12.530] can we learn from that? And I just, I really think that's important. And all of you have been bringing [54:12.650 --> 54:18.030] a different element of that to this discussion. And it's helping us, I think, learn as a medical [54:18.030 --> 54:27.250] device manufacturer how to handle this situation. Yeah, I'm always... Sorry. I'm sorry. I always [54:27.250 --> 54:33.810] say data doesn't lie, right? There's a saying, let's follow the money or follow the evidence. [54:34.410 --> 54:39.970] But we have so much data, we're just not mining it. We're not learning. I think it's one of your [54:39.970 --> 54:47.150] members, Sasha, that said, we need to learn to listen, right? So we need to learn what the [54:47.150 --> 54:53.990] data is telling us. And I think that's the big thing. We have all this data, but we're not [54:53.990 --> 55:01.650] listening to it. And once we start intuitively tuning in and understanding the data, those [55:01.650 --> 55:07.830] numbers don't lie. Then actually, I think an MDM can only really do threat analysis. [55:08.610 --> 55:15.750] Because if you don't know what you have, how do you know what to protect? Because then you're [55:15.750 --> 55:19.850] always on the back foot, right? You need to get ahead of the problem. You need to know where [55:19.850 --> 55:25.330] you're most vulnerable at. You shouldn't wait for that to become public knowledge. [55:28.070 --> 55:33.910] So I want to shift gears one more time here. We hear the concept of zero trust in medical [55:33.910 --> 55:41.510] devices. And I believe each of you are on the same topic. How do you think that concept can [55:41.510 --> 55:45.830] be applied from an MDM perspective or from a medical device perspective? And I'll throw that [55:45.830 --> 55:56.530] out to the group. Whoever wants to latch onto that one can start us off. Okay, I'll roll with it. [55:56.530 --> 56:02.410] I was waiting for you, V. Pete, I know you got some insight on this, too. Go ahead. [56:02.410 --> 56:08.430] I love zero trust. It makes me excited because for once we turn security on its head and say, [56:08.430 --> 56:13.730] you know, we're not going to trust but verify. That's not going to be... we're just not going [56:13.730 --> 56:20.690] to trust, right? Because the nice thing about zero trust is you can actually understand devices. [56:21.050 --> 56:28.550] But the ideal thing is now we don't have a perimeter. We harden everything. We probe a [56:28.550 --> 56:33.650] device and if it meets the specific standards, that device is allowed to have access to the [56:33.650 --> 56:39.730] minimum it needs access to, right? It's like a patient monitor doesn't have to be on the Wi-Fi [56:39.730 --> 56:46.170] and doesn't have to link to the accounting system or, you know, patient records. We have this thing [56:46.170 --> 56:53.290] trying to make everything accessible but too accessible. We take it to the extremes. We either [56:53.290 --> 57:01.210] make it too secure and unusable or we go to the other way. But zero trust has really big [57:01.210 --> 57:08.670] plus points for me because now we can attend to device manufacturers independently, right? We're [57:08.670 --> 57:14.190] not going to have the same controls across the board because everyone builds their devices [57:14.190 --> 57:20.490] different. You can understand the device and apply the appropriate rules, privilege, and access. [57:20.490 --> 57:28.310] So I think this is a really good solution as we move to a perimeterless remote monitoring and [57:28.310 --> 57:35.610] medical service. I mean, I had my first video consultation today which is something very [57:35.610 --> 57:43.910] strange but yet it felt normal and the perimeter has just been broken. COVID-19 came in, [57:43.910 --> 57:49.390] swept healthcare off its feet and we are facing new territory we'll be having to build [57:49.390 --> 57:55.990] while trying to catch up. And this virus has crippled healthcare and I think security should [57:55.990 --> 58:03.810] never put such a strain on healthcare that we cripple it more. We should be making it safer [58:03.810 --> 58:10.650] and easier to use. And that's our responsibility as security people in this field. It's the [58:10.650 --> 58:18.230] responsibility of the MDM. But zero trust is the ability to do catered security based on a device, [58:18.230 --> 58:24.670] not necessarily the same rule for everyone. So I'm very excited about it. [58:25.070 --> 58:35.170] So I agree with you and I have a slightly different interpretation of zero trust in the [58:35.170 --> 58:43.910] concept of medical devices. For me, zero trust basically means we talked earlier about third [58:43.910 --> 58:52.310] parties and we talk about embedding communication models within the code and communicating with the [58:52.310 --> 58:59.430] patient's mobile app and the patient's maybe home router, you know, each medical device with its [58:59.430 --> 59:05.550] communication. And I think zero trust should really mean that we can't trust any of those. [59:05.550 --> 59:10.330] We can't trust any of the third party, open source, closed source, embedded. We can't even [59:10.330 --> 59:17.350] trust the operating system that a medical device use, whether it's commercial one, Linux based. [59:17.350 --> 59:26.470] We should be expecting vulnerabilities throughout the entire supply chain. The code, the hardware, [59:26.950 --> 59:32.390] the software within the hardware. And of course, we can't trust that the patient didn't install a [59:32.390 --> 59:39.470] malicious application on his mobile, right? Like the last thing we want to trust is the human being [59:39.470 --> 59:45.070] not making mistakes. So for me, the concept of zero trust basically means that the kind of [59:45.070 --> 59:51.090] security that we choose to implement should take all of this under consideration and to make sure [59:51.090 --> 59:57.470] that we keep the integrity of maybe not the entire device, maybe it's impossible, but essentially [59:58.110 --> 01:00:04.850] the critical operation of the device should remain secure regardless of all the things that I [01:00:04.850 --> 01:00:10.690] mentioned. Like if we talk about the insulin pump, so you want the functionality of injecting [01:00:10.690 --> 01:00:18.470] insulin will be the most protected functionality of the device. Same goes for personal health [01:00:18.470 --> 01:00:25.010] information on the device itself. So for me, zero trust basically means taking this [01:00:25.010 --> 01:00:28.870] into consideration when designing a security solution. [01:00:30.710 --> 01:00:37.630] Exactly. I like what Natalie said. I think it's the difference in design. [01:00:38.070 --> 01:00:43.710] And some of you might be, I talk about this all the time, every time someone will listen, but [01:00:45.990 --> 01:00:55.090] I look at it as a concept of, I think a great example is like cars. When these systems were [01:00:55.090 --> 01:01:01.930] built to allow a computing interface in a car, they weren't connected to anything. [01:01:02.610 --> 01:01:06.910] And they designed according to that. And they scaffolded this over years. [01:01:06.910 --> 01:01:10.450] And then all of a sudden, someone came along and tacked on internet connections and Bluetooth and [01:01:10.450 --> 01:01:17.670] Wi-Fi. What you should do is go back and redesign everything to consider that new approach. [01:01:17.670 --> 01:01:28.070] The zero trust concept to me means we think about it from the initial idea stage of defending it [01:01:28.070 --> 01:01:32.970] in every place it can be. And it's an analogy to what I think Natalie and V were saying about [01:01:34.370 --> 01:01:40.650] no one operational environment is any more or less secure than any other one, [01:01:40.650 --> 01:01:45.910] because they're all the same. And building to consider that [01:01:46.630 --> 01:01:50.910] takes concerted effort from the start, from the very, very start. [01:01:51.290 --> 01:01:57.710] And I think approaching the problem from that way is another one of those shifts that [01:01:58.310 --> 01:02:04.530] we have to consider from legacy devices into what is going to be the new normal for this. [01:02:06.870 --> 01:02:14.170] Everyone's going to have to get there. The quicker a manufacturer of any kind of product [01:02:14.170 --> 01:02:18.590] can start considering that, the easier their life is going to be in the long run. [01:02:20.950 --> 01:02:28.650] I must add just one thing about that, because I think we are repeating a lot of [01:02:28.650 --> 01:02:35.190] think about it in advance. You have to design it. I must mention the other side. Security [01:02:35.190 --> 01:02:43.930] has always been the last part to be incorporated into things. Everybody would like to go to [01:02:43.930 --> 01:02:48.930] market faster. Everybody likes to just build their devices, build their technology. Nobody [01:02:48.930 --> 01:02:54.850] likes to think about security. And I'm saying it's okay, meaning I don't want to say you have [01:02:54.850 --> 01:03:00.150] to think about it in advance. You don't. History teaches us that nobody thinks about everything [01:03:00.150 --> 01:03:07.830] in advance, and it's okay. And security comes last, always. So for me, as someone who's looking [01:03:07.830 --> 01:03:14.670] for solutions, and not just looking to emphasize the problem, I'm looking for solutions that can [01:03:14.670 --> 01:03:22.850] be developed and installed, even if you didn't think about security. And I have to say, again, [01:03:22.850 --> 01:03:29.530] it's the best thing to think about it in advance, but I'm just saying it's okay if you didn't. [01:03:30.350 --> 01:03:36.830] Nobody holds you responsible. I will hold you responsible only if you didn't try to fix it [01:03:37.330 --> 01:03:43.190] later on. Oh, dude, I'm sorry. I have to disagree there. Like, I want to kick some, [01:03:43.190 --> 01:03:49.070] you know, I sometimes want to kick an Indian's ass and say, what the hell were you thinking? [01:03:52.910 --> 01:03:59.330] That's what's happened there. It depends. It's like you have how could it pass for like one, [01:03:59.330 --> 01:04:04.250] two, three, four, then this is the place where you can think, what the hell were you thinking? [01:04:04.250 --> 01:04:11.190] I'm just saying, you know, the more advanced stuff, it's harder to think about. [01:04:11.890 --> 01:04:19.130] Yeah, so it's a twofold problem. We're dealing with legacy, right? I call it the legacy, [01:04:19.130 --> 01:04:26.110] because that's what it's written in, that joke for the evening. So the problem is we have, [01:04:26.110 --> 01:04:32.950] we have to stop developing legacy problems, okay? So we need to address it ahead of time. [01:04:32.950 --> 01:04:37.270] But okay, we have to face the fact that previously, we didn't address it ahead of the [01:04:37.270 --> 01:04:42.450] time. And there's nothing you can do. Can't pull these things back on, tell me you're going to cut [01:04:42.450 --> 01:04:48.790] me open, take my device back, give me a new one. I'm not going to let that happen. So we have to [01:04:48.790 --> 01:04:55.130] deal with that. It's a twofold problem. We have to be realistic, meaning I think [01:04:55.130 --> 01:05:00.930] you are completely correct. But realistically looking ahead, I don't think we're going to [01:05:00.930 --> 01:05:06.110] be thinking about it in advance in the years to come, because new problems will appear. [01:05:06.350 --> 01:05:14.870] And like in the 50 years prior to where we stand today, companies didn't think about it in advance. [01:05:14.870 --> 01:05:19.770] So we can teach that, we can say that, but to be realistic, I don't think it's going to happen. [01:05:19.770 --> 01:05:25.190] People will only handle the problem once it bites them in the ass, basically. [01:05:25.830 --> 01:05:32.630] That's human nature, right? I can't disagree with you because history often repeats itself. [01:05:32.630 --> 01:05:38.530] I think what's interesting is we're covering it like a triangle from different sides, [01:05:38.530 --> 01:05:43.890] which is very interesting. I'm concerned with, we have these devices that are diversified. [01:05:43.890 --> 01:05:50.230] How do we include this on a healthcare network, right? Naki's product literally deals with legacy [01:05:50.230 --> 01:05:56.930] devices and new devices. And Pete's solution covers it from a different angle. So it's this [01:05:56.930 --> 01:06:03.090] like diversified opinion that's doing the same thing, but each in a different way. [01:06:03.470 --> 01:06:08.250] And this is why I think researching is important because we all think differently. [01:06:09.370 --> 01:06:13.610] So it's having, yeah, it's a big problem. It's massive. [01:06:14.410 --> 01:06:17.770] I think, yeah, I agree with what you're saying. We're all kind of coming at it from a little bit [01:06:17.770 --> 01:06:21.690] of a different perspective. And I do agree, Natalie, that there's going to be new attacks [01:06:21.690 --> 01:06:26.650] we haven't thought of today that come out in the next year, three years, five years. [01:06:27.610 --> 01:06:33.630] But we're talking about medical devices today. And I think we absolutely have to require [01:06:34.890 --> 01:06:45.170] better work on the design. We got here because of the evolutionary walk of how medical device [01:06:45.170 --> 01:06:53.230] manufacturers built businesses to help patients. And computers came part of that. And here we are. [01:06:53.690 --> 01:07:01.230] How we go forward, though, can be changed. And I think the idea that we'd be accepting to someone [01:07:02.190 --> 01:07:09.470] not working hard at that right now is dangerous. I agree that we need to have things on the other [01:07:09.470 --> 01:07:14.570] side to prepare for the inevitable, the new things that come out that we haven't thought of. [01:07:16.810 --> 01:07:22.730] The next thing that shakes the industry, so to speak. And we want to be resilient to that. [01:07:22.730 --> 01:07:29.730] I completely agree. But in one case, when I was looking at a medical device, [01:07:29.730 --> 01:07:38.530] I then started reverse engineering an e-bike. And the e-bike was orders of magnitude better. [01:07:38.630 --> 01:07:43.930] Because the e-bike wanted to prevent someone from bypassing the speed controller [01:07:44.930 --> 01:07:51.570] and going too fast on the bike that it wasn't regulated for. And they did an incredible job [01:07:51.570 --> 01:07:56.150] for that. But seeing the stark contrast between those two, the difference there was they designed [01:07:56.150 --> 01:08:00.910] it to be that way. And I know I harp on that a lot, but I think that's an important point. [01:08:00.970 --> 01:08:04.770] I think we can all agree that there's some fundamentals that we need to align with. [01:08:04.770 --> 01:08:11.290] I think this is why we want this diverse experience, background, opinions, because it [01:08:11.290 --> 01:08:16.410] helps us grow in the industry. And we have time for one quick final question here, and I'll go [01:08:16.410 --> 01:08:23.230] around the panel and start out with you, Pete. How can researchers, smaller shops, [01:08:23.230 --> 01:08:27.070] those attending the Biohacking Village, get more involved with device makers? [01:08:29.730 --> 01:08:35.130] The way I'd put it was eBay and calling. I think there would be a lot that could be [01:08:35.130 --> 01:08:44.130] improved if there's a way to get your hands on research devices to start learning this space. [01:08:44.130 --> 01:08:48.550] One of the challenges I think newer people to working in medical device research are going to [01:08:48.550 --> 01:08:59.390] run into is becoming aware of the monstrosity that is an MDM. And something like when you find a bug [01:08:59.390 --> 01:09:07.010] and you suggest a fix for it, the realities at scale of making that happen. So on one hand, [01:09:07.010 --> 01:09:15.910] I would say it would be wonderful if MDMs would start exposing programs for researchers to apply [01:09:15.910 --> 01:09:21.490] to and be selected to start working on this stuff. And on the other hand, I think there'd be a great [01:09:21.490 --> 01:09:28.430] spot for education on the MDM side to show what is it we're going to do. We need to do a hypothetical [01:09:28.430 --> 01:09:34.030] patch to this hypothetical device. What does it actually look like? The legal ramifications, [01:09:34.030 --> 01:09:39.890] the regulatory ramifications, because without that information, a lot of security researchers [01:09:39.890 --> 01:09:45.570] I think will be frustrated at the timeline it takes because they don't have any sort of [01:09:45.570 --> 01:09:49.330] understanding of what's really happening. Especially coming from a place of like [01:09:50.030 --> 01:09:55.430] the hacker ones and the bug crowds where that's a pretty smooth process when you're looking at [01:09:55.430 --> 01:10:00.230] web properties, right? And it's a very, very different industry. It's a very different [01:10:00.230 --> 01:10:03.450] threat model. It's a very different process you have to go through. And I think there's [01:10:03.450 --> 01:10:08.330] an education opportunity for that. Natalie, I know you've had your journey with medical [01:10:08.330 --> 01:10:12.370] device manufacturers. Same question to you. How do you think that researchers in smaller [01:10:12.370 --> 01:10:19.550] shops can get more involved in making a difference? So I completely agree with Peter, [01:10:19.550 --> 01:10:32.130] and I think that bug bounties programs and exposing the inner processes that happen within [01:10:32.130 --> 01:10:37.850] medical device development, which is extremely different from traditional development with [01:10:37.850 --> 01:10:45.290] different regulations and different constraints could help getting more engaged. I think assigning [01:10:45.490 --> 01:10:51.290] a person or a unit responsible for that is also super important. In some of the medical devices, [01:10:51.290 --> 01:10:56.290] you just don't know who is the right person to approach. It's not the CISO, right? The CISO is [01:10:56.290 --> 01:11:01.990] not concerned about medical device security. It's concerned about the network. So I think [01:11:02.830 --> 01:11:11.610] getting this better approach is good. Transparency is good. Basically saying, [01:11:11.610 --> 01:11:18.050] we want you involved. We're okay with it. We're not going to send you some terrifying legal [01:11:18.050 --> 01:11:28.710] documents if you find something. This can be helpful. And I think the research community, [01:11:28.710 --> 01:11:37.190] maybe thinking about it in a different direction, can also help educate. Meaning I think that what [01:11:37.190 --> 01:11:44.050] we sometimes do, we go to medical devices, we do some exploitation workshops, we explain [01:11:44.050 --> 01:11:50.570] what vulnerability means, how to exploit vulnerabilities, how to write code with [01:11:50.570 --> 01:11:55.930] security in mind to avoid vulnerabilities. We had this experience with Medtronic as well. It was [01:11:55.930 --> 01:12:04.450] amazing. It was super satisfying for us. And I believe that it was also interesting for medical [01:12:04.450 --> 01:12:11.130] device engineers and software developers to suddenly see the other side of the code and the [01:12:11.130 --> 01:12:17.870] other side of what can be done with the code and how devices are being exploited can also raise [01:12:17.870 --> 01:12:25.250] awareness and can also help them understand what the researchers are doing and why they're doing [01:12:25.250 --> 01:12:35.990] And this could really be an open-minded and open-hearted conversation, which we had, and [01:12:35.990 --> 01:12:44.450] I think is a great example. And it's not very common. I mean, device manufacturers, OEMs, [01:12:45.410 --> 01:12:52.090] some of them do not respond well when researchers approach them with vulnerabilities. And I [01:12:52.090 --> 01:12:59.190] think positioning yourself as an MDM saying, we respect that, we want to work with you [01:12:59.190 --> 01:13:02.530] is amazing, and it's the right approach. [01:13:03.770 --> 01:13:06.690] Last but not least, Vi, you get the final word. [01:13:07.530 --> 01:13:14.050] Oh, awesome. I'm going to do a call out to all my fellow biohackers, my friends, my goons, [01:13:14.050 --> 01:13:20.230] my family. Biggest thing I've learned that the constraints with dealing with vulnerabilities [01:13:20.230 --> 01:13:27.710] from an MDM's perspective is when you do a proof of concept, okay, approach this like a report, [01:13:27.710 --> 01:13:37.250] approach it like scientifically, give as much information, as much code as you can for the MDM. [01:13:38.130 --> 01:13:46.330] They need to verify what you have said. And if your document doesn't consist of information that is [01:13:46.330 --> 01:13:52.710] comprehensive and easy to follow, they're going to have to reverse engineer things themselves. [01:13:52.710 --> 01:13:58.430] And then the 30 days that you expect to get back from them, it's going to take them 60 to 90 days [01:13:58.430 --> 01:14:05.610] just to figure out whether it's applicable to their product. So if you come with a problem, [01:14:05.610 --> 01:14:11.390] come with a solution, right? Let's not just focus on what we've broken, let's try and fix it. [01:14:11.390 --> 01:14:17.610] So my challenge to all my hacker friends are, let's not just try to break things, let's try and solve things. [01:14:18.230 --> 01:14:24.990] For me, the term hacker is obviously sometimes associated with cyber criminal. And for me, it's a passion point [01:14:24.990 --> 01:14:31.130] because hacking is what I do. I'm not a criminal. I try and figure out what's wrong so that I can fix it [01:14:31.690 --> 01:14:38.150] using alternate ways and means of thinking. That's what a hacker does. That is what a hacker is for me. [01:14:38.730 --> 01:14:45.430] But we're also scientists and in science we need to be thorough. We need to do the approach better. [01:14:45.530 --> 01:14:52.370] And for an MDM, don't be such hard asses. We're trying to help. I promise you we're not your enemy, [01:14:52.930 --> 01:14:58.770] right? Just be a little bit nicer. I have a conversation. Buy someone a coffee. If you see [01:14:58.770 --> 01:15:05.770] someone at Biohack Village, I promise you I haven't had a hacker that bites unless asked to bite. [01:15:05.770 --> 01:15:11.170] Right? We don't like having teeth. We like having tech conversations and we like geeking out with [01:15:11.170 --> 01:15:18.350] you. But it comes from both ways. We both need to open our hearts and open our minds and realize [01:15:18.350 --> 01:15:23.310] we're working towards the same ideal. And working against each other is not going to get us there [01:15:23.310 --> 01:15:29.590] any sooner. And from a patient's side, keep on doing the research. Make this stuff better. [01:15:30.310 --> 01:15:35.570] Because I think either party needs each other. It's a trifecta. It's the medical manufacturer. [01:15:36.130 --> 01:15:41.730] And it's the FDA or the regulatory body, depending on where you're from. [01:15:42.190 --> 01:15:45.870] Those are the things that are going to make the future better. And those are basic engineering [01:15:45.870 --> 01:15:49.910] practices. A triangle is the strongest thing you can enforce something with. [01:15:49.990 --> 01:15:55.110] So let's enforce this with teamwork. That's all I have to say. V out. [01:15:55.150 --> 01:16:01.850] With that, I want to thank the panel. Drop the mic. Pete, Natalie, V, we're good to go. [01:16:01.950 --> 01:16:04.990] Have a good DEF CON Biohacking Village, everyone. Thank you. [01:16:04.990 --> 01:16:06.670] Thank you very much.