1 00:00:03,070 --> 00:00:04,010 Okay. 2 00:00:04,290 --> 00:00:05,490 Sorry. 3 00:00:12,280 --> 00:00:13,160 Sorry. 4 00:00:13,160 --> 00:00:14,880 I just figured it out. 5 00:00:15,560 --> 00:00:18,060 The audio should now be working. 6 00:00:18,920 --> 00:00:20,000 Seriously. 7 00:00:20,000 --> 00:00:20,620 Okay. 8 00:00:20,620 --> 00:00:20,920 Are we... 9 00:00:22,080 --> 00:00:24,540 So, on Twitch chat, I am so sorry. 10 00:00:24,540 --> 00:00:26,980 We should have this all sorted now. 11 00:00:26,980 --> 00:00:31,280 I found out what caused the feedback loop. 12 00:00:31,280 --> 00:00:32,100 Okay. 13 00:00:33,420 --> 00:00:40,520 So, on Twitch chat, can you please give us a thumbs up? 14 00:00:40,540 --> 00:00:41,320 Okay. 15 00:00:42,640 --> 00:00:45,000 Thumbs up if the audio is not working for everyone. 16 00:00:45,000 --> 00:00:48,460 So, you can hear Will, and it's not doing the weird Hall of Mirrors thing. 17 00:00:49,080 --> 00:00:50,040 Yay. 18 00:00:51,660 --> 00:00:54,140 I'm terrified to open Twitch at this point. 19 00:00:54,140 --> 00:00:55,100 Yeah. 20 00:00:56,040 --> 00:00:57,000 Cool. 21 00:00:57,940 --> 00:00:59,540 I'm muting myself now. 22 00:00:59,540 --> 00:01:00,900 The audio is now working. 23 00:01:00,900 --> 00:01:02,700 I am adding everyone's names. 24 00:01:02,700 --> 00:01:05,600 So, I am so sorry for everyone. 25 00:01:06,880 --> 00:01:10,660 This time, if it doesn't work, I'll off myself with a sword in the background. 26 00:01:11,760 --> 00:01:12,720 Okay. 27 00:01:13,000 --> 00:01:13,620 Okay. 28 00:01:13,620 --> 00:01:14,780 So, have at it, Will. 29 00:01:14,780 --> 00:01:16,300 It's what the people want. 30 00:01:17,300 --> 00:01:19,840 Rich, whoever I am this week. 31 00:01:20,060 --> 00:01:20,940 Okay. 32 00:01:21,780 --> 00:01:23,000 Well, last time. 33 00:01:23,280 --> 00:01:24,720 Last time through. 34 00:01:24,820 --> 00:01:32,980 So, again, welcome, everyone, to the Hype in AI panel, where I'm going to talk about does AI in InfoSec actually live up to the hype? 35 00:01:32,980 --> 00:01:36,440 I'm not going to pitch the topic again, because I'm tired of it. 36 00:01:36,440 --> 00:01:39,820 So , Ariel, please introduce yourself one last time. 37 00:01:39,820 --> 00:01:41,260 All right. 38 00:01:41,260 --> 00:01:41,860 Hello. 39 00:01:41,860 --> 00:01:43,160 My name is Ariel Hirt-Voss. 40 00:01:43,160 --> 00:01:48,620 I'm a research scientist at OpenAI, where I work on misuse and abuse of the AI technologies that we develop there. 41 00:01:48,620 --> 00:01:55,560 I'm also a PhD student at Harvard University, and I've been working on a book for No Starch Press on how to do red teaming for AI systems, specifically. 42 00:01:55,960 --> 00:01:58,080 And that's most of what I do. 43 00:01:59,120 --> 00:01:59,900 Cool. 44 00:01:59,960 --> 00:02:01,420 My name is Rich Huerang. 45 00:02:01,420 --> 00:02:05,140 I am a data scientist at Sophos. 46 00:02:05,240 --> 00:02:11,280 I have been working at the intersection of security, machine learning, and privacy for about 10 years. 47 00:02:11,280 --> 00:02:14,580 I have a PhD from the University of California at Santa Barbara. 48 00:02:14,980 --> 00:02:15,660 Will? 49 00:02:18,850 --> 00:02:20,010 My name is Will. 50 00:02:20,010 --> 00:02:29,950 I am a network operator, and I work on building tools, AI tools for red teaming and attacking machine learning systems. 51 00:02:30,810 --> 00:02:31,870 Very cool. 52 00:02:31,870 --> 00:02:32,370 Masuda? 53 00:02:33,470 --> 00:02:34,610 My name is Masuda. 54 00:02:34,610 --> 00:02:36,750 I'm a data scientist at Rapid7. 55 00:02:36,870 --> 00:02:43,730 I think a lot about incident detection, anomaly detection, and I'm a relative newcomer to security. 56 00:02:43,730 --> 00:02:51,290 I have a PhD in physics and spent some time in quantum computing research before formally switching over to data science and now security. 57 00:02:51,970 --> 00:02:53,010 Awesome. 58 00:02:53,150 --> 00:02:54,190 Rob? 59 00:02:55,270 --> 00:02:56,410 I'm Rob Brandon. 60 00:02:56,410 --> 00:03:00,550 Been working in the tech and security field for probably about 25 years now. 61 00:03:00,550 --> 00:03:04,210 Currently working for Booz Allen Hamilton as a researcher and threat hunter. 62 00:03:04,410 --> 00:03:06,850 And about eight years ago, I was doing some sock work. 63 00:03:06,850 --> 00:03:18,850 I got a little bit of a exposure to this machine learning stuff and went back to school and got a PhD from the University of Maryland, and been working in the intersection ever since. 64 00:03:19,870 --> 00:03:20,870 Awesome. 65 00:03:20,930 --> 00:03:23,390 And last, but certainly not least, John. 66 00:03:23,990 --> 00:03:24,970 Hey, everyone. 67 00:03:24,970 --> 00:03:27,150 I'm John Seymour, aka DeltaZero. 68 00:03:27,150 --> 00:03:32,270 I work as a data scientist at Salesforce, where I do intrusion detection on the DNR team. 69 00:03:32,570 --> 00:03:42,590 I also like to just kind of put my hands in everything, so I've touched a lot of, you know, offensive security and data science and all sorts of things. 70 00:03:43,030 --> 00:03:46,550 But yeah, mostly currently do intrusion detection. 71 00:03:48,230 --> 00:03:49,270 Awesome. 72 00:03:49,490 --> 00:03:53,650 So again, thanks to everyone for being here and taking part in this discussion. 73 00:03:53,710 --> 00:03:59,550 I kind of want to kick it off with a very sort of like high-level first question. 74 00:03:59,910 --> 00:04:04,070 So machine learning has actually been used in a wide range of in-physic problems now. 75 00:04:04,070 --> 00:04:09,130 So we've seen it in malware detection, where it's probably sort of the biggest success story. 76 00:04:09,130 --> 00:04:13,010 But then spam blocking is probably the oldest field it's been used in. 77 00:04:13,010 --> 00:04:18,690 And then other stuff like network traffic classification, analyst alert ranking, and triage. 78 00:04:18,770 --> 00:04:31,810 So what, if anything, do we get as sort of a value-add above and beyond sort of like the old-school tried-and-true signature-based or rule-based techniques for security? 79 00:04:31,810 --> 00:04:39,430 And so what I'm kind of hoping we can dig into here is like, what are the strengths and weaknesses of AI or ML or whatever you want to call it? 80 00:04:39,430 --> 00:04:49,250 And are there classes of problems that we should always be using ML for or maybe like should just completely give up and never waste time and money? 81 00:04:49,250 --> 00:04:57,990 And are there specific areas where if a vendor says, oh yes, we can solve this problem with AI, right, that immediately sets off a red flag in your head that they might not know what they're talking about? 82 00:04:58,490 --> 00:05:00,730 So open question, please dive in. 83 00:05:02,890 --> 00:05:04,690 Or I can call on somebody. 84 00:05:06,310 --> 00:05:17,290 So for intrusion detection, because this is what I spent the most time thinking about, I think the class of problems really depends strongly on what the data makes available. 85 00:05:17,690 --> 00:05:26,710 And just as an example for how I've approached the problem, when I first started, I thought, you know, we're awash in data, everything's digital, records are there for everything. 86 00:05:27,310 --> 00:05:35,450 And I viewed my role as analogous to the people that study volcanoes by looking at very small earthquake signatures, and have gotten really good at predicting these eruptions. 87 00:05:35,450 --> 00:05:37,350 And I thought, yeah, that's what I'm going to do. 88 00:05:37,350 --> 00:05:38,690 Can anything be more glamorous? 89 00:05:38,690 --> 00:05:43,970 I'm going to really understand these signatures and use them to predict, you know, incoming attacks. 90 00:05:43,970 --> 00:05:49,110 I very quickly realized it's hard to get a labeled data set of incidents. 91 00:05:49,310 --> 00:05:52,010 People don't want to share when they've been compromised. 92 00:05:53,070 --> 00:05:54,130 And so that's a problem. 93 00:05:54,130 --> 00:05:59,830 But even if I were to get a library of these attacks, attackers behave very differently. 94 00:05:59,830 --> 00:06:04,410 So I'd have to get a representative set of different attacker behavior. 95 00:06:04,410 --> 00:06:06,410 And also, they're opportunistic. 96 00:06:06,410 --> 00:06:13,310 So it really depends on the very specifics of network configurations and how different companies, you know, set up their assets. 97 00:06:13,530 --> 00:06:22,350 So really, now I'm talking about simulating different types of incidents over many different types of attackers over tons of different environments. 98 00:06:23,150 --> 00:06:27,910 So that very quickly put me off the classification use case, let's say. 99 00:06:28,710 --> 00:06:32,270 But there's still a lot of data. 100 00:06:32,270 --> 00:06:33,910 And so I think anomaly detection is great. 101 00:06:33,910 --> 00:06:35,970 We can baseline a lot of things. 102 00:06:36,830 --> 00:06:40,730 The volume-based baselining is maybe straightforward and not that interesting. 103 00:06:40,730 --> 00:06:48,490 But I think there's an art in defining what are the attributes that we expect to look really regular in reference to particular intrusions. 104 00:06:48,490 --> 00:06:51,750 So I think that's a really useful way to go. 105 00:06:51,750 --> 00:06:59,070 I'm very suspicious if someone were to tell me I've got a method to, you know, precisely detect a compromise for you. 106 00:06:59,070 --> 00:07:03,430 So you're mostly thinking about network traffic in that case? 107 00:07:03,990 --> 00:07:09,070 Network traffic, let's say endpoint data, you know, logs from endpoint data. 108 00:07:10,250 --> 00:07:11,370 Cool. 109 00:07:11,370 --> 00:07:12,690 Anyone else? 110 00:07:12,690 --> 00:07:20,730 I guess, Will, I'd really be interested to hear your input on this, since you work on the other side of the problem a lot of the time. 111 00:07:20,730 --> 00:07:21,470 Yeah, yeah. 112 00:07:21,470 --> 00:07:24,470 I don't come at it from a machine learning area. 113 00:07:25,410 --> 00:07:31,110 I'd say the nice bit is scale and building context around an event. 114 00:07:31,110 --> 00:07:32,730 I think that's really helpful. 115 00:07:34,010 --> 00:07:41,050 You know, I think the people who have been doing incident response for a long time, you know, they have the knowledge. 116 00:07:41,050 --> 00:07:49,510 They just don't necessarily have all the pieces to find, you know, a direction. 117 00:07:49,510 --> 00:08:00,190 So, you know, we always, I think as data scientists, or as the data is coming across, it's often incomplete, where you don't know what you need to solve the problem. 118 00:08:00,190 --> 00:08:06,930 So you only have, you know, half of the information that you actually need, and the rest is kind of fuzzy. 119 00:08:06,930 --> 00:08:10,250 But you know, maybe a direction that you're trying to go with it. 120 00:08:10,530 --> 00:08:13,290 So I think machine learning can help pull out those insights. 121 00:08:13,330 --> 00:08:19,650 But you know, I think it requires that still like a human, a human touch, a human direction. 122 00:08:19,650 --> 00:08:23,530 I think that is probably pretty limited as we go forward in time. 123 00:08:23,530 --> 00:08:31,230 But currently, where it's at it, you know, it's still pretty ineffective in networks for detections and things. 124 00:08:31,850 --> 00:08:39,630 And you potentially the other use cases. 125 00:08:40,130 --> 00:08:48,690 So proactive, secure configuration, it's like, out of this machine learning thing, the only thing we've got is just more alerts. 126 00:08:48,930 --> 00:08:53,590 I think that that may remain, you know, and people can't necessarily trust them. 127 00:08:53,870 --> 00:09:04,690 But I do think it's going forward in time, you know, it's going to be a very, very different use case, especially as you get data scientists with data sets that, you know, they start to understand. 128 00:09:05,910 --> 00:09:08,330 And, you know, we're all on the same playing field. 129 00:09:08,330 --> 00:09:15,350 So, you know, once we're on a Windows host, for example, everybody has access to the same information, more or less. 130 00:09:15,450 --> 00:09:21,570 And it's really just about how you're viewing that information inside of a given context, I think. 131 00:09:23,070 --> 00:09:34,070 Yeah, so one, one thing that I think kind of popped out from both those answers was, there's a lot of like human tuning and human feature extraction that's going on in there. 132 00:09:34,070 --> 00:09:43,010 So the, you do see a lot of marketing literature about, oh, just, you know, dump raw bytes in and detections, magical detections come out the other end. 133 00:09:43,010 --> 00:09:45,470 So any, any thoughts on claims like that? 134 00:09:45,470 --> 00:09:50,210 Do you really need a, like, is the human element, like mandatory still at this point? 135 00:09:50,210 --> 00:09:52,490 Or is there any path? 136 00:09:52,490 --> 00:09:53,950 No, I don't think it's mandatory. 137 00:09:54,370 --> 00:10:01,910 So for example, any given user is going to operate on a host, you know, within some range. 138 00:10:02,230 --> 00:10:04,530 So I think the anomaly detection is really nice. 139 00:10:04,530 --> 00:10:20,450 So let's say you have 1000 users, 998 of those users are going to come in, log in at 8am every day, they're going to visit the same things, you know, then you have that 2% that are, you know, doing something that they're not supposed to be doing or whatever it is. 140 00:10:20,570 --> 00:10:28,650 So I don't think you necessarily need to label data, but you, you need to baseline your network. 141 00:10:28,650 --> 00:10:37,470 And I think the issue for machine learning is when you try and scale it across, let's say 10 different companies, that's going to be really difficult. 142 00:10:37,530 --> 00:10:50,250 So but I think machine learning is such that you could, you could build a network for each individual network, it's just going to take time. 143 00:10:50,250 --> 00:10:54,550 But from a vendor's perspective, like that doesn't scale as well. 144 00:10:54,550 --> 00:10:58,510 And so I think it gets a little more difficult in that way. 145 00:11:00,770 --> 00:11:11,870 I kind of disagree with the anomaly detection approach, just because every network is just full of anomalies when you start going to look for anomalies. 146 00:11:11,870 --> 00:11:19,070 And while it may just be 2%, that 2% is still going to be the source of, you know, to, you know, millions and millions of alerts. 147 00:11:20,290 --> 00:11:21,250 Yeah, yeah, right. 148 00:11:21,250 --> 00:11:31,430 I should say, in specific cases, like, let's say log on, log on times, so, or if, you know, host where users are logging on. 149 00:11:31,430 --> 00:11:33,170 So yeah, you're absolutely right. 150 00:11:33,170 --> 00:11:35,250 I think anomalies are everywhere. 151 00:11:35,250 --> 00:11:44,210 But in terms of malicious events, or potentially malicious events, I think you could cut it down to a little bit where you can get some high fidelity. 152 00:11:44,230 --> 00:11:57,750 You can narrow it down, but there's still going to be a huge number of false positives, especially when you look at so many networks, the admins are doing things that look like malware, just because they have this tool that somebody put together 20 years ago, 153 00:11:57,750 --> 00:12:00,270 and nobody knows how the whole thing works anymore. 154 00:12:00,630 --> 00:12:10,050 And I think that's one of the interesting things about looking at network security in this space is when you get down to it, most networks aren't really planned and designed. 155 00:12:10,470 --> 00:12:15,890 They're more like a organism that's kind of grown and developed over the past 20 years. 156 00:12:16,690 --> 00:12:21,090 And I think that that's the type of thing that machine learning is really useful for. 157 00:12:21,090 --> 00:12:38,410 If you approach your environment as a space to be analyzed, rather than something that's designed, then machine learning techniques can be really useful for helping you weed out what's interesting and what's not. 158 00:12:39,230 --> 00:12:54,010 I totally agree that anomaly detection is rife with false positives, but I think that's where the human interaction becomes really important, because what the machine learning algorithm can do is find things that are mathematically anomalous. 159 00:12:54,130 --> 00:13:12,850 And then we have to go back to someone who has some knowledge of the network and say, oh, but these are network backups that happen, you know, at 10am every Tuesday, throw those out, and like all of these things that add contextual information, I think those can help denoise quite significantly. 160 00:13:13,090 --> 00:13:19,810 And there are ways to automate them as well, but those are add-ons to the actual algorithm. 161 00:13:20,310 --> 00:13:30,910 Okay, but I think that gets right into where one of the things I see as a big problem in this space, at least from the analyst perspective. 162 00:13:31,770 --> 00:13:44,210 I've seen countless cases where you'll have a researcher come in and say, yeah, I've got this great anomaly detection method, you know, let's go ahead and get some of the domain experts to look at the results. 163 00:13:44,430 --> 00:13:50,690 And the domain experts are already overworked, you know, nobody wants to spend their time tuning your anomaly detection system. 164 00:13:50,950 --> 00:13:59,190 And I think that's one of the key downsides and reasons that those systems just aren't practical from an applied perspective, you know. 165 00:14:00,050 --> 00:14:04,310 You don't have the people to give you the feedback on that. 166 00:14:04,510 --> 00:14:18,550 And I think that's, you know, and I hate to say it, but I think in a lot of cases that's why the only way to really do machine learning and AI successfully in this field is to spend the time acquiring both skill sets. 167 00:14:20,030 --> 00:14:27,270 I actually really strongly agree with you on that point of actually acquiring both skill sets being important. 168 00:14:27,410 --> 00:14:49,880 But one of the pieces that I do actually slightly disagree with you on is, um, well, I guess more I see just the ability to sort of identify costs between false positives and false negatives and be able to tweak that really easily is a huge powerful tool I see for at least statistical thinking about alerts and events, 169 00:14:49,880 --> 00:14:50,640 right. 170 00:14:50,660 --> 00:15:03,640 And, like, I do see, you know, like Suricata or whatever rule set you have, like, we see a ton of, you know, false positives, even in rule-based alerts, right, too. 171 00:15:03,660 --> 00:15:06,440 Because, as you say, like, networks are noisy. 172 00:15:07,760 --> 00:15:31,280 So I think one of the benefits that you can sort of think of is more data science is like a layer on top to sort of help control, sort of, you know, even filter down further, like, as you're saying, like, you know, filter out the benign alerts, the non-anomalous sort of alerts that you get for rule-based, 173 00:15:31,280 --> 00:15:32,940 at least as a starting point. 174 00:15:33,920 --> 00:15:48,340 Yeah, I mean, I think Brian Kovar had a really good point at a Canvas a couple years ago where he was saying a lot of the benefit you get out of machine learning isn't in detecting things, it's in throwing away the things that you don't want to look at. 175 00:15:51,770 --> 00:16:06,250 I'll also say, like, one of the things that we've generally had success with in our place is sort of, so, like, we have a lot of different rules for detecting, for example, C2 toolkits like Empire, things like that, right? 176 00:16:06,850 --> 00:16:16,610 And one of the things that, you know, when we went ahead and actually, like, tested out Empire was that it's very config-driven, right? 177 00:16:16,610 --> 00:16:23,650 You can change things like, you know, easily change things like how often you beacon out to the C2 host or things like that. 178 00:16:23,690 --> 00:16:28,090 And a lot of these things are sort of things that you would build into a rule, right? 179 00:16:28,090 --> 00:16:40,710 So, if, for example, you're taking a Let's Encrypt certificate as you're, like, you know, one of your main things that you build into your rules, like, you can easily change things like the certificate subject to evade, right? 180 00:16:41,310 --> 00:16:55,570 Some of the things that we've actually found a lot of use for is taking these rules and sort of trying to generalize them just a bit more to where they'll capture, like, a large number of variants of, you know, Empire something. 181 00:16:55,570 --> 00:17:08,550 So, not just specifically targeted to this particular certificate subject or structure, but, like, other variants as well for things that are easily modifiable by the attacker. 182 00:17:11,890 --> 00:17:12,490 Cool. 183 00:17:12,490 --> 00:17:18,850 I think a good point, kind of, about that is that you need to have a plan for how you're going to deal with the alerts. 184 00:17:18,850 --> 00:17:26,010 And that's kind of a key thing for anything, you know, you have to have a plan for what are you going to actually do with this model. 185 00:17:26,390 --> 00:17:28,350 Yeah, 100% agree with that. 186 00:17:29,530 --> 00:17:35,570 Yeah, I want to jump in really quick and give Ariel a chance to chime in if you've got anything. 187 00:17:35,790 --> 00:17:37,950 I don't have anything significantly different. 188 00:17:38,870 --> 00:17:55,370 I firmly agree with the necessity to have both skill sets of understanding security and also understanding the data science, especially when you, if you don't understand the underlying data of what it is you're working with, you're going to have a really hard time having any sort of alerts or whatever that you're trying to build that actually tells you anything useful. 189 00:17:55,690 --> 00:17:56,930 That's all I would add. 190 00:18:00,550 --> 00:18:01,630 Yeah, for sure. 191 00:18:02,490 --> 00:18:19,990 I think, so, we've kind of been maybe a little bit in the weeds of, like, different, yeah, like, you need to be really, like, focused on sort of the technical details of, how do I tune my anomaly detection system, or what do I know about sort of the security rules that I'm building behind this? 192 00:18:19,990 --> 00:18:32,230 I guess maybe part of what contributes to all of the hype is the fact that, like, this is, like, a really technical, really specialized discipline that not a lot of people understand. 193 00:18:32,450 --> 00:18:39,010 And so that sort of, A, makes it look like magic to people that maybe don't really understand it as well. 194 00:18:39,010 --> 00:18:47,510 And B, makes it really easy to oversell and to hype up because whoever's doing the marketing and the hyping can be pretty sure that the other person doesn't understand that. 195 00:18:47,630 --> 00:19:06,970 So, as people who are sort of, like, at the coalface, right, working, like, on these specific problems, what can we do to sort of, like, help communicate what the capabilities of these machine learning systems are and how can we sort of, like, demystify them and cut through the hype and make it give people, 196 00:19:06,970 --> 00:19:14,130 like, a good idea of, yes, they actually do do really good things and you can do really powerful, useful things with them, but no, they're not magic. 197 00:19:14,690 --> 00:19:17,910 Like, how do we sort of, like, bridge that communications gap? 198 00:19:20,390 --> 00:19:25,590 I'm going to say it's, I mean, InfoSec is ripe for it. 199 00:19:25,590 --> 00:19:30,570 Like, every year, like, the black hat vendor, you know, two years ago, it was all about application whitelisting. 200 00:19:30,750 --> 00:19:32,250 And now you don't hear about it anymore. 201 00:19:32,250 --> 00:19:33,950 All you hear about is machine learning. 202 00:19:34,750 --> 00:19:36,810 Maybe two years, it'll be something else. 203 00:19:37,070 --> 00:19:41,870 But I think it's, it's, and marketing is just, is just that. 204 00:19:41,870 --> 00:19:45,270 It's, you know, it's designed to make everything look better than it is. 205 00:19:45,470 --> 00:20:00,490 But I think in terms of practitioners, or, you know, if you're working with a company or at a company, it's, you know, saying, yeah, the hype is real, because this is, it can be really powerful, but it's, you're right, it's not going to do your dishwasher. 206 00:20:00,650 --> 00:20:07,110 It, you know, it can assist in the things you have, but it's the ultimate garbage in garbage out. 207 00:20:07,930 --> 00:20:17,170 So, you know, like, it's like the first, the first thing you find out when you install a SIM in your network is not anything security related, it's how much stuff is misconfigured. 208 00:20:17,690 --> 00:20:20,970 And I think the same is true for, for machine learning. 209 00:20:20,970 --> 00:20:23,610 It's like, once you put it in, it's going to take a little bit. 210 00:20:23,610 --> 00:20:43,330 But if you continue to work and feed and spend that time up front, like some of the best organizations, from the security perspective that we work with, are just ones that have just put the legwork in, you know, they've gone through and they've handpicked the DLLs that are allowed to run on every host, 211 00:20:43,330 --> 00:20:49,350 you know, and they haven't just plugged in a box, pat themselves on the back and, you know, gone got a beer. 212 00:20:49,690 --> 00:21:00,530 So I think it's like, doesn't matter what you're installing, I think the legwork needs to be done with it is obviously misconstrued that AI will solve your problem. 213 00:21:00,530 --> 00:21:01,790 Maybe we'll get there one day. 214 00:21:01,790 --> 00:21:09,070 But I think, you know, for now, it's still going to take some legwork, which is to be expected. 215 00:21:11,390 --> 00:21:22,550 I think some of the hype also comes from, I mean, we've interacted with the results of AI in different contexts, Netflix recommendations, Amazon recommendations. 216 00:21:22,890 --> 00:21:31,130 And as we think about how to do that for security, it's natural to say, you know what, just give me a dashboard with recommendations for what to do next. 217 00:21:31,130 --> 00:21:32,350 And it'll be AI powered. 218 00:21:32,350 --> 00:21:35,370 And it'll be just as disruptive as, you know, Netflix. 219 00:21:36,730 --> 00:21:49,470 And I think security professionals are much more skeptical of recommendations that they would get from that kind of dashboard than I as a consumer would be about what recommendations Amazon gives me for my clothing purchases. 220 00:21:50,310 --> 00:22:01,370 So I think one way I'm trying to think about this hype or talk about this hype is to just think about, like, our users are using the output of this AI very, very differently. 221 00:22:03,050 --> 00:22:06,810 I've heard it referred to as high stakes versus low stakes AI. 222 00:22:06,810 --> 00:22:11,110 In low stakes, you know, if my email is spam, I click it away. 223 00:22:11,110 --> 00:22:12,290 It's not a big deal. 224 00:22:12,290 --> 00:22:22,030 But in security, if our predictions are wrong, someone can be compromised or we could be taking someone's attention greatly away from the things they ought to be looking at instead. 225 00:22:22,030 --> 00:22:29,630 So they're intrinsically interacting with our work very differently than if we were, you know, correspondingly in e-commerce or something. 226 00:22:30,450 --> 00:22:40,990 And I feel like that's a conversation I'd like to have with folks in marketing or product management as they're trying to brainstorm how to talk about this publicly. 227 00:22:41,490 --> 00:22:49,410 I feel like the way that users use this is more specific to security, let's say. 228 00:22:50,910 --> 00:22:59,830 Actually, what's funny is there's two companies I can think of who their marketing departments talk about how they don't use ML in their security products. 229 00:23:00,070 --> 00:23:05,950 So in terms of, like, the skeptical and the skepticism that exists, these companies actually market that they don't use ML. 230 00:23:07,690 --> 00:23:09,410 How well does that work for them? 231 00:23:09,530 --> 00:23:10,190 Do you know? 232 00:23:10,850 --> 00:23:12,430 I guess they're still in business, so... 233 00:23:12,430 --> 00:23:13,230 Well, that's good. 234 00:23:13,230 --> 00:23:14,110 Okay. 235 00:23:14,270 --> 00:23:19,290 I do know one of them, and they are really, really solid. 236 00:23:20,210 --> 00:23:21,850 They are a solid company. 237 00:23:22,950 --> 00:23:23,590 That's cool. 238 00:23:23,590 --> 00:23:25,070 Or a solid product, I should say. 239 00:23:25,650 --> 00:23:30,390 Yeah, I do wonder when we're going to see companies going back to good old-fashioned AI. 240 00:23:31,210 --> 00:23:34,750 You could argue that signatures are just good old-fashioned AI, right? 241 00:23:34,750 --> 00:23:35,450 Rule-based. 242 00:23:39,630 --> 00:23:44,970 John, any thoughts on bridging the hype gap or bringing hype back to reality? 243 00:23:45,770 --> 00:23:48,750 Um, well, yeah, like... 244 00:23:51,050 --> 00:23:53,310 I don't really have much to add. 245 00:23:53,310 --> 00:23:59,030 I actually agree with both Will and Vasudha's comments that they've made. 246 00:23:59,030 --> 00:24:03,230 Like, it's... yeah. 247 00:24:03,630 --> 00:24:05,530 Marketing is marketing. 248 00:24:05,990 --> 00:24:11,550 It's really good to focus on sort of what we're actually able to do well. 249 00:24:11,550 --> 00:24:17,810 And I think one of the things that we data scientists can actually do a lot better on is communicating failures. 250 00:24:17,810 --> 00:24:29,390 And I have started to see, you know, conferences like Kamla's, as well as others, sort of welcome, sort of, hey, we tried this thing. 251 00:24:29,390 --> 00:24:30,090 It didn't work. 252 00:24:30,090 --> 00:24:31,370 Here's what we did. 253 00:24:31,370 --> 00:24:35,410 Don't do what we did, basically, sort of talks. 254 00:24:35,410 --> 00:24:46,010 And I think sort of focusing on, even as data scientists, being able to very clearly articulate what we were able to do, what we weren't able to do. 255 00:24:46,390 --> 00:24:50,250 I think that's actually very important for bridging the gap long term. 256 00:24:55,190 --> 00:24:57,030 I definitely agree with that. 257 00:24:57,830 --> 00:25:10,330 I think that given that a lot of the data science that we see in InfoSec kind of comes from academia, it sort of ends up with a lot of the same incentives that academia has for publishing stuff that you want it to look good. 258 00:25:10,330 --> 00:25:17,810 You don't really want to share your failures because you don't really get any sort of, you don't get anything out of that as an academic, unfortunately. 259 00:25:18,350 --> 00:25:25,650 And I think sort of taking a lot of this security data science out of that context and putting it more in, like, here's some practical stuff. 260 00:25:25,650 --> 00:25:31,030 Let's talk about what works and what doesn't work is exactly kind of the direction we should be taking things. 261 00:25:34,890 --> 00:25:35,690 Cool. 262 00:25:35,910 --> 00:25:44,630 Yeah, I want to loop back a little bit to something Vasudha said a second ago, where she referred to high stakes and low stakes decisions that are made. 263 00:25:45,210 --> 00:26:01,990 So yeah, I think it's obvious to me at least that a lot of the decisions that get made when you're talking about using ML or artificial intelligence or whatever you want to call it this week for InfoSec and security in general, those tend to be higher stakes decisions. 264 00:26:01,990 --> 00:26:18,830 And one of the things that I've seen people push a lot more for in recent past maybe two years or so is when you do have high stakes decisions, having some notion of explainability or interpretability that comes along with those models. 265 00:26:19,050 --> 00:26:29,930 So I kind of wonder, and this is partly driven by my own experience using ML to detect malware, those models tend to be almost impossible to explain. 266 00:26:29,930 --> 00:26:41,870 So how big of a, how important is this do you think to be able to explain the decisions that a model makes, be it in a detection, a classification, or an anomaly detection context? 267 00:26:41,950 --> 00:26:57,970 And how can we get from where we are now, where everything's about gradient boosted decision trees, or big complicated neural networks that are almost impenetrable, to something that's maybe more explicable and more directly, something you can directly interrogate? 268 00:26:59,920 --> 00:27:01,660 I think it's everything. 269 00:27:01,660 --> 00:27:23,100 I think it's worth sacrificing a little bit of accuracy even, to get a simpler model that pulls out future importances, or that you can easily explain what feature in your data led to this prediction, or led to the the presentation of this particular record for review. 270 00:27:23,900 --> 00:27:46,060 I also think, this is my opinion only, I think human-in-the-loop is super important, and that if it's a high stakes decision, it's much better received as an augmentation that an analyst can take into account, rather than being something that would be fully automated and trusted to make a decision on its own. 271 00:27:48,670 --> 00:27:51,150 Yeah, I 100% agree with that. 272 00:27:51,150 --> 00:28:04,850 Like a lot, so especially like if you're sort of starting a data science modeling, like a branch in detection or response or something, your analysts aren't going to trust your data science models at all, right? 273 00:28:05,090 --> 00:28:21,210 So really, like starting with something that's very clear and obvious, like, you know, this machine has, is supposed to be fired walled off from the rest of the network, and it's uploading hundreds of thousands of gigabytes, you know, to the outside world, 274 00:28:21,210 --> 00:28:27,050 like those sorts of things are going to start, you know, building the trust as you go. 275 00:28:27,890 --> 00:28:33,630 I'm also a really, really big proponent of the Google rules of ML that they do. 276 00:28:33,630 --> 00:28:44,610 And one of the things that they actually really, really recommend is starting with, you know, whenever you're building like a product or a machine learning based system, starting with something that's easily baselineable. 277 00:28:44,610 --> 00:28:51,110 And then that allows you to actually improve upon it and actually be able to measure like how much you've improved. 278 00:28:51,110 --> 00:28:58,170 When you have sort of these black box models, you lose a lot in sort of as you're trying to improve the model. 279 00:28:58,430 --> 00:29:06,550 You spend a lot of effort trying to actually figure out whether you've actually improved or how you can actually improve it further. 280 00:29:08,030 --> 00:29:11,230 I think organizationally, like buy-in is pretty huge. 281 00:29:11,230 --> 00:29:15,510 So as explainable as you can make it. 282 00:29:15,510 --> 00:29:20,510 And you know, like analysts, security analysts, you know, routiners are like, they're naturally curious. 283 00:29:20,510 --> 00:29:34,050 So if you're showing them something that is, how did you get this, like, they're gonna, they're gonna go and look at it and figure out and they, you know, even if you're right, if it's not as accurate, or it's not as effective, they'll figure out how to, 284 00:29:34,050 --> 00:29:48,750 to make it that way, or they'll at least figure out, you know, what inputs, what, what inputs require what outputs, and then you can start making it a little more complicated or sophisticated or whatever. 285 00:29:48,750 --> 00:29:56,850 I think we often, you know, it's like we often are misguided in the fact that we think complexity is sophistication. 286 00:29:57,430 --> 00:30:02,970 When in reality, I think a lot of times simplicity is, is actually sophistication. 287 00:30:02,990 --> 00:30:04,430 It's much cleaner. 288 00:30:05,630 --> 00:30:18,190 So I think we're kind of falling into the trap that people tend to fall into here, where we're thinking that there has to be one silver bullet system that generates everything you need to know to make a decision. 289 00:30:18,310 --> 00:30:31,630 In a lot of cases, I don't think explainability is necessarily that important, as long as you can go grab more information around whatever you're looking at to do a deeper analytic and provide more information to the analyst. 290 00:30:31,630 --> 00:30:39,110 Like if you've got a binary, there's no reason that you have to rely solely on the machine learning model to make the decision of whether it's good and bad. 291 00:30:39,270 --> 00:30:44,490 But you can use that as a tipper to do some more in-depth analytics on the binary. 292 00:30:44,490 --> 00:30:57,210 You've got, you know, start running, throwing it through some sandboxes, you know, do more expensive computation that you maybe couldn't, you can't do if you, you may not be able to do it for every single binary, but you know, you could do it for a few hundred an hour. 293 00:30:57,690 --> 00:31:12,450 You know, you know, in most other cases, there's additional context that you could automatically collect and provide to the analyst to make their decision on whether they should do something with the alert or not, rather than just relying on whatever the output of the model is. 294 00:31:12,490 --> 00:31:13,610 And that's just a good point. 295 00:31:13,610 --> 00:31:17,310 I think, like, I know Microsoft, the Windows Defender does that. 296 00:31:17,530 --> 00:31:23,130 So anything overtly malicious gets blocked on the host and anything that they're unsure about gets sent to a cloud. 297 00:31:26,900 --> 00:31:30,140 I think that sometimes explainability can be a little bit of red herring. 298 00:31:30,140 --> 00:31:31,760 In general, I think that it's good. 299 00:31:31,760 --> 00:31:48,920 I mean, for all the reasons stated above about building trust and, and having baselines for understanding how you can improve, but when you have, when you have like a human judge making a decision about whether or not someone's guilty, you're, there's no part of the process whereby you go back to the judge and you say, 300 00:31:48,920 --> 00:31:53,060 what are the pieces of evidence that convinced you that this person is guilty? 301 00:31:53,060 --> 00:32:00,600 And sometimes they can explain why and sometimes they can't, but humans are very good about trying to come up with stories to explain what the decisions that we make. 302 00:32:00,740 --> 00:32:14,640 So to some extent, I kind of view it as a little bit I don't know, naive to try to come up with like purely explainable reasons for why things are the way they are. 303 00:32:14,640 --> 00:32:19,280 Because oftentimes you can't really come up with like a true reason for why something is the way that it is. 304 00:32:23,220 --> 00:32:24,120 Cool. 305 00:32:24,260 --> 00:32:34,700 So I have kind of a twofer that I want to, I guess, point at specifically at Ariel and Will for initial responses. 306 00:32:34,700 --> 00:32:41,080 So the first bit is when we implement ML into these security systems, right? 307 00:32:41,080 --> 00:32:44,740 We bake it in, we make it a big, sometimes a critical part. 308 00:32:45,140 --> 00:32:55,220 And this is for Ariel, are we kind of opening up new vulnerabilities or new risks that we need to be worried about, right? 309 00:32:55,220 --> 00:33:01,600 Is it, anytime you add a new component, you at least change the attack surface. 310 00:33:01,640 --> 00:33:15,460 So do we really know quite what we're getting into as far as introducing new elements to an attack surface when we start putting ML into important decision-making points in these security systems? 311 00:33:15,620 --> 00:33:17,920 And then I'll come back to you, Will, for the second bit in a sec. 312 00:33:19,280 --> 00:33:20,960 My answer would be yes. 313 00:33:21,600 --> 00:33:27,400 But I think that a lot of that can be mitigated by making sure that you still have human in the loop. 314 00:33:27,700 --> 00:33:34,980 I kind of view a lot of these AI tools as sort of like centaur chess where you have an AI system that assists a human. 315 00:33:34,980 --> 00:33:42,360 I don't really anticipate there being a situation which you'd want a system to fully be automated to do everything because you're always going to run into issues. 316 00:33:42,360 --> 00:33:52,860 And like we discussed earlier about interpretability, if you don't have ways of understanding what's going on or like baselines or anything, you're going to be kind of for a lot of this stuff. 317 00:33:53,440 --> 00:34:05,820 That said, there has been a lot of good research around trying to understand what these potential vulnerabilities are, but they're all, at least for the most part, most of them have been developed inside laboratory conditions, which makes them very hard to translate it into the real world. 318 00:34:07,780 --> 00:34:09,000 Yeah, yeah. 319 00:34:09,080 --> 00:34:14,760 There's a lot of messed up stuff that's going to go down, but I don't think it's going to be as bad as maybe some people think it will be. 320 00:34:15,880 --> 00:34:16,500 I don't know. 321 00:34:16,500 --> 00:34:21,420 We are kind of clear on how poorly machine learning works for a lot of things and how well it works for other things. 322 00:34:21,420 --> 00:34:31,460 And I think there's a decent amount of healthy skepticism about machine learning at this point to sort of be some level of protection for us as we go forward adding it to more things. 323 00:34:34,180 --> 00:34:35,400 Okay, cool. 324 00:34:35,400 --> 00:34:36,140 Thanks. 325 00:34:36,720 --> 00:34:42,580 And then I'm going to kick the second part over to Will, and then I'd kind of like to hear from everyone else on the same topics. 326 00:34:43,040 --> 00:34:52,980 So do you see a risk when we make models interpretable or explainable and we say, okay, I think this thing was bad because of X. 327 00:34:54,020 --> 00:35:04,160 Do you think that that is making them necessarily easier to evade or bypass, or is that a controllable risk, or does it not really matter to you? 328 00:35:04,160 --> 00:35:05,300 That doesn't really matter to me. 329 00:35:05,300 --> 00:35:10,640 So I think when you have to sell something, making it explainable is nice. 330 00:35:11,200 --> 00:35:13,140 Like that's what you're trying to do. 331 00:35:13,140 --> 00:35:18,540 You need to make it sellable, but I don't care if I'm using a I don't care. 332 00:35:18,540 --> 00:35:20,480 I don't have to explain to anybody. 333 00:35:20,480 --> 00:35:23,120 I'm not trying to sell it to somebody, right? 334 00:35:23,120 --> 00:35:31,460 But if you want ML to take hold in an organization, you're trying to sell a product and no one's going to trust it, then you need to make it explainable. 335 00:35:31,460 --> 00:35:33,820 But for everybody else, it doesn't matter. 336 00:35:34,060 --> 00:35:37,160 Like it can be as black box as you want. 337 00:35:37,680 --> 00:35:39,780 And I think that's an important distinction. 338 00:35:40,200 --> 00:35:46,480 So it's like the... it needs to be explainable if you're trying to sell it. 339 00:35:47,740 --> 00:35:51,680 You know, but even internally, it can be useful. 340 00:35:51,940 --> 00:35:53,600 An explainable model is more useful. 341 00:35:53,620 --> 00:35:57,700 I don't necessarily think it's easier to attack or what. 342 00:35:57,700 --> 00:36:13,140 And it would be difficult to assume like from an attacker's perspective, like I generally always assume that it's going to be a black box and that you're not going to get 100% of the model, but you might be able to get some sort of, you know, some copycat, 343 00:36:13,140 --> 00:36:16,820 something that helps you explain the model that you're looking at. 344 00:36:17,980 --> 00:36:29,140 So yeah, explainability, my opinion is more about making it palatable to deploy into an organization. 345 00:36:30,740 --> 00:36:31,520 Okay. 346 00:36:31,520 --> 00:36:31,860 Yeah. 347 00:36:31,860 --> 00:36:37,140 So I find it... it's interesting because there's, I think, two different threads on the explainability that people are picking up on. 348 00:36:37,140 --> 00:36:42,480 First is building trust to be able to get people to adopt the models. 349 00:36:42,480 --> 00:36:52,080 And second is reducing risk by giving analysts who are seeing these alerts more context to kind of work with them. 350 00:36:52,080 --> 00:37:10,040 So, I mean, do those seem like those are... I guess what I'm saying is like pitching it as like we're building trust almost seems it's a little bit cynical, like a sales job, whereas we need to be able to explain it because these are sort of critical, 351 00:37:10,040 --> 00:37:14,940 maybe even life and death decisions, seems like a much higher ethical bar. 352 00:37:14,940 --> 00:37:17,400 So where does the balance between those two fall? 353 00:37:17,400 --> 00:37:21,960 Or are they necessarily even in conflict, right? 354 00:37:23,160 --> 00:37:25,580 I would add a third column to that. 355 00:37:25,580 --> 00:37:45,700 I would say putting aside the issue of having to convince someone else to trust these results for my own quality control, I look to explainability to convince myself that I haven't overfit really badly or that I'm not worried about data or that I haven't had data leakage happen or something. 356 00:37:45,700 --> 00:37:56,780 So just going back and evaluating why the model made the decision that it did gives me better confidence that it's picking up on real signal and not something spurious. 357 00:37:59,100 --> 00:38:07,480 Yeah, I also think explainability also depends on how much domain expertise the person that's consuming the results of the model has. 358 00:38:09,060 --> 00:38:10,380 Yeah, that's true. 359 00:38:12,800 --> 00:38:33,860 I'm always going to come back to the malware because that's what has the most research on it, but I mean, if you've got a malware analyst looking at a binary that he was delivered by a neural network said that, hey, this is malicious, that malware analyst is going to make a decision based on their own domain expertise, 360 00:38:33,860 --> 00:38:38,800 regardless of what the model says it made its decision on. 361 00:38:38,800 --> 00:38:41,900 I think that's the case for a lot of other things as well. 362 00:38:43,320 --> 00:38:58,120 But that leads us to the problem of the people that get the most value out of the explainability are also the ones that are least qualified to make the decision in the first place. 363 00:39:00,200 --> 00:39:14,860 Okay, so I'm going to reframe that a little bit more positively and say maybe it's a way to maybe help turn less experienced analysts and to help them perform like more experienced analysts maybe, hopefully. 364 00:39:15,100 --> 00:39:16,960 Or is that too optimistic? 365 00:39:17,460 --> 00:39:25,400 I don't think it's optimistic, but you still have the chicken and the egg problem of they have to understand the results enough. 366 00:39:25,580 --> 00:39:31,960 And if they have that understanding of the domain, they may not derive as much value from the explainability. 367 00:39:32,160 --> 00:39:36,600 Yeah, actually, I'm gonna there's a product called Artemis by Endgame. 368 00:39:36,600 --> 00:39:39,320 And it's like a talk to your sim product. 369 00:39:39,660 --> 00:39:43,960 And it's, you know, they were like, this is everything machine learning that you ever wanted. 370 00:39:43,960 --> 00:39:45,720 I don't know what their marketing is. 371 00:39:45,720 --> 00:39:57,120 But it's basically they haven't, they've only made information retrieval easier, but they it still requires an analyst that knows what they need to be looking for. 372 00:39:57,580 --> 00:40:05,780 So it actually, we already had a language to retrieve information from a sim, you know, whatever, OS query, whatever it might be, we didn't need machine learning to do that. 373 00:40:06,400 --> 00:40:17,800 What we needed machine learning to do is potentially add some context around a particular data point, not to say like, hey, you know, like a chatbot on top of your sim, we didn't need that. 374 00:40:19,960 --> 00:40:37,700 Um, I, pardon me if I'm wrong, or misunderstanding, Rob, but I thought your point was more that basically, the domain expert is going to know what features basically would be most helpful in determining which whether this binary was malicious or benign. 375 00:40:37,700 --> 00:40:51,360 And so it's kind of not the right thing to put that on the data science to try to, you know, to be the ones to actually explain why the model chose a particular, you know, feature or something in its decision. 376 00:40:51,380 --> 00:40:53,280 Am I correct on that or? 377 00:40:53,780 --> 00:41:02,440 Partly, what I'm saying is that the model, the models explainability isn't necessarily going to be helpful for the people that need it. 378 00:41:04,320 --> 00:41:09,240 You know, it can maybe say this, this particular segment of the binary is what I thought is bad. 379 00:41:09,620 --> 00:41:23,020 But then you need the expertise to know whether that means anything, you know, it could just be a packer, it could just be, you know, this particular thing is using get proc address to load things for non malicious reasons. 380 00:41:23,020 --> 00:41:34,080 And there's, there's a lot of context around things where, if you already know it, then you don't need the AI to tell you that this is why this is bad. 381 00:41:34,340 --> 00:41:40,540 And if you need the AI to tell you that this is bad, you may not understand it well enough to know whether the AI is wrong or not. 382 00:41:44,070 --> 00:41:51,370 Yeah, sensing a lot of very healthy skepticism of ML from all quarters here, which warms my dark little heart. 383 00:41:52,190 --> 00:41:57,050 So one more, one more question to kind of carry us out to the top of the hour. 384 00:41:57,510 --> 00:42:00,710 So we've touched on anomaly detection. 385 00:42:00,710 --> 00:42:04,050 I think that's people, I think there's a little bit of a network focus here. 386 00:42:04,050 --> 00:42:07,990 So that's what a lot of people are sort of talking about as a main tool. 387 00:42:07,990 --> 00:42:14,430 There's also binary classification for things like static file artifacts, be it malware detection or maldoct detection. 388 00:42:15,210 --> 00:42:22,850 So these are sort of maybe the, among the simpler forms of ML, right? 389 00:42:22,850 --> 00:42:43,310 And it doesn't get into things like, it hasn't like, we haven't seen, as far as I know, any serious, like, successes as far as using more complicated forms like reinforcement learning or, you know, like active, active learning with like really good human-in-the-loop integrations in the InfoSec space yet. 390 00:42:43,390 --> 00:42:56,770 So just sort of among the vast menu of new, of existing techniques that really haven't made a dent in InfoSec, where do you think, where do people think that the field is going next? 391 00:42:56,770 --> 00:43:02,850 Like what's going to be the next, the next big breakthrough as far as applying machine learning to security problems? 392 00:43:06,220 --> 00:43:10,040 I think that reinforcement learning is coming. 393 00:43:10,100 --> 00:43:16,800 I think it's going to have its moment at some point, but I think that that's going to depend very heavily on the kind of data that we can get into these kind of systems. 394 00:43:17,280 --> 00:43:29,720 I've seen a lot of people trying to use it for adversarial simulation, but haven't really had good data that really allows it to do a good job that's better than what we already have with Cobalt Strike and some of those other things. 395 00:43:30,400 --> 00:43:40,360 But I have seen some very convincing sort of research in the RL space around trying to use RL for like code generation kind of stuff. 396 00:43:40,360 --> 00:43:48,140 And that gives me some, I guess, I don't know if it's hope because there's also like a negative side to any sort of adversary simulation using RL. 397 00:43:48,620 --> 00:43:50,580 But I think it'll definitely be interesting. 398 00:43:51,160 --> 00:43:51,260 Yeah. 399 00:43:51,260 --> 00:43:55,920 If it is a good adversarial simulation, it's also probably a decent adversary. 400 00:43:55,920 --> 00:43:56,160 Yeah. 401 00:43:56,160 --> 00:44:04,540 I think reinforcement learning is the next, like there's a lot of attack simulation stuff that's being talked about. 402 00:44:04,540 --> 00:44:11,060 I know MITRE has some talks where they're discussing reinforcement learning for attack simulation. 403 00:44:11,060 --> 00:44:21,020 I think for actual attackers, the difficult part is you can't get on a network and learn the network and then attack it, right? 404 00:44:21,020 --> 00:44:25,520 You need to be learning from all of your general experience. 405 00:44:25,520 --> 00:44:36,000 But I think from a defensive standpoint, reinforcement learning could be, you know, because you can iterate through your network infinitely, but it's just matching what is useful. 406 00:44:36,000 --> 00:44:39,560 I think, I hate, I shouldn't say that. 407 00:44:39,560 --> 00:44:47,240 Adversary simulation is a bit of a, it's a bit of a buzzword. 408 00:44:47,540 --> 00:44:49,400 I think it's not a buzzword. 409 00:44:49,400 --> 00:44:56,960 How do I, it's like you, from the offensive side for the last three years, they've just been dropping techniques left and right. 410 00:44:56,960 --> 00:45:05,080 And so it's like, in some ways we call it adversary simulation because the techniques that we dropped are getting picked up by adversaries. 411 00:45:05,560 --> 00:45:09,500 And so it's like, we're emulating ourselves via a proxy. 412 00:45:10,300 --> 00:45:14,900 But it's because we wanted to do the research, but it hasn't been really helpful. 413 00:45:14,900 --> 00:45:22,460 But if I think of the value, you know, that the public research has generated in the defensive space, it's insane. 414 00:45:22,460 --> 00:45:27,940 You know, like there are entire products that are based off a GitHub project. 415 00:45:28,120 --> 00:45:36,180 So, you know, you can't say it's terrible, but I think the adversary simulation piece is a bit overdone. 416 00:45:36,180 --> 00:45:53,840 I wish organizations would spend more time in their own networks, just figuring out where their data is and doing some of the simple stuff like network segmentation and less time focusing on, you know, whatever APT is they think is going to come after them next. 417 00:45:53,880 --> 00:45:56,320 I think that's probably more useful. 418 00:45:56,500 --> 00:45:59,180 But reinforcement learning would be my answer. 419 00:46:00,560 --> 00:46:14,960 Yeah, I'm fairly skeptical of reinforcement learning, just because the problems in this space don't lend themselves to easily defining a target for the learning algorithm to test itself against. 420 00:46:15,240 --> 00:46:32,240 You know, from the defensive side, it's really hard to say, you know, we can't easily define what secure means in any meaningful sense to really be able to teach an agent, whether it's in a secure state or not, to determine, you know, what actions it should be taking. 421 00:46:32,380 --> 00:46:41,660 From the attack perspective, you know, like you said, the attacker can't just throw a agent on to go learn the network, because that's going to be super noisy. 422 00:46:41,660 --> 00:46:54,500 And, you know, hopefully the defenders will notice that that's going on as it's probing everything and everything so my answer to what's the next big thing is, I don't think there's going to be a next big thing. 423 00:46:54,960 --> 00:47:20,300 I think the real positive impact or the real kind of game changing impact is going to be smaller, you know, people applying machine learning to solve small problems tactically in a lot more places, you know, it's going to be the sock analyst who sits down and just makes a naive Bayes classifier to triage the AV alerts they've got, 424 00:47:20,300 --> 00:47:26,060 you know, it's going to be the small projects like that, widely distributed that are going to really make the difference. 425 00:47:27,600 --> 00:47:35,940 Okay, so we've got a couple votes for reinforcement learning, one vote for mass democratization of machine learning. 426 00:47:36,020 --> 00:47:38,540 Vasudha, you gotta pick for us. 427 00:47:38,540 --> 00:47:42,560 I'll pile on to the small projects done with ML. 428 00:47:42,720 --> 00:47:44,880 But I will add on to that. 429 00:47:45,440 --> 00:47:49,280 I'm really excited about graph theoretic approaches for that. 430 00:47:49,580 --> 00:47:58,920 Because I think as we think about very like, maybe making classifications on very specific endpoints, that that's a machine learning problem that I think we can do well. 431 00:47:58,940 --> 00:48:17,280 But to go from that to bridge from that to talking about an intrusion means understanding network effects, adding in other data sets to endpoint logs, it's not possible to do that using local, like single node properties. 432 00:48:17,280 --> 00:48:24,160 And so I'm, I'm kind of waiting to see what the what the network graph picture will do for us. 433 00:48:26,160 --> 00:48:27,360 Very cool. 434 00:48:27,360 --> 00:48:32,280 So I know we're kind of at the top of the hour, but we started a little bit late. 435 00:48:32,280 --> 00:48:35,240 Can people stay for another couple minutes? 436 00:48:35,240 --> 00:48:38,820 I want to try and take a couple of questions from the Discord, if that's okay. 437 00:48:39,720 --> 00:48:40,500 Awesome. 438 00:48:41,140 --> 00:48:41,600 Cool. 439 00:48:41,600 --> 00:48:45,200 I'm just gonna, so if I don't call your question out, please don't be offended. 440 00:48:45,200 --> 00:48:47,160 I'm just kind of scrolling through. 441 00:48:47,900 --> 00:49:00,020 So the first one that jumps out at me, FS Montenegro says, my perennial question is, what are the questions and practices that we should use to probe claims of a vendor's ML quality? 442 00:49:00,020 --> 00:49:02,460 I particularly like that one. 443 00:49:02,460 --> 00:49:03,760 Anyone want to take a stab? 444 00:49:03,760 --> 00:49:04,380 Yeah. 445 00:49:04,800 --> 00:49:06,780 How do you call BS? 446 00:49:12,250 --> 00:49:16,990 I'm going to put... I guess, I guess I can start right. 447 00:49:16,990 --> 00:49:25,170 So there's, there's a lot of different, you know, pitfalls in, you know, data science and, and performing machine learning. 448 00:49:25,170 --> 00:49:36,230 And one of the things I would like to see more of from, you know, skeptics or people trying to evaluate these models is just the simple questions like, how did you split your train and test? 449 00:49:36,430 --> 00:49:41,350 Like, did you, did you, you know, consider data leakage in sort of the time domain, right? 450 00:49:41,350 --> 00:49:44,850 Like, are you learning from future samples in your past samples? 451 00:49:45,010 --> 00:49:46,650 Those sorts of questions. 452 00:49:46,650 --> 00:49:50,530 I don't have much experience on the actual buying product side though. 453 00:49:50,530 --> 00:49:56,390 So I probably, that that's about as far as my, my opinion on the subject. 454 00:49:58,690 --> 00:49:59,830 I'm going to go with... 455 00:50:04,170 --> 00:50:05,830 Let's go with Will then Rob. 456 00:50:05,830 --> 00:50:06,670 Oh, sorry. 457 00:50:07,250 --> 00:50:13,130 I was just going to say, you know, from an attacker's perspective, these things are being deployed, but they haven't really been tested. 458 00:50:13,550 --> 00:50:22,950 So I think attackers are ultimately going to be the ones that decide whether or not ML is a viable solution for network defense. 459 00:50:23,510 --> 00:50:29,870 And we just, from an attack perspective, there's just not many people looking at it currently. 460 00:50:29,890 --> 00:50:33,870 So I would say that's probably a good way. 461 00:50:33,870 --> 00:50:35,570 Eventually it'll just happen, right? 462 00:50:35,570 --> 00:50:37,090 Attackers will have to adapt. 463 00:50:37,090 --> 00:50:42,390 And so I think that's going to be a good, a good litmus test for it. 464 00:50:45,450 --> 00:50:46,410 Rob? 465 00:50:47,390 --> 00:51:00,310 One of the first things I look at when I'm looking at a ML product is whether the data that they are purporting to operate on is even reasonably possible to make a decision on. 466 00:51:00,510 --> 00:51:13,450 And a lot of the products out there are saying, you know, we're going to look at an email, we're going to look at a phishing email and, you know, we'll predict what the person had for breakfast and, you know, all kinds of other claims that just don't really follow from the data that they have available. 467 00:51:14,490 --> 00:51:20,610 So I think, you know, to me, that's really one of the big things I look at when I'm evaluating an ML system. 468 00:51:24,370 --> 00:51:24,950 Cool. 469 00:51:24,950 --> 00:51:25,830 Good answers. 470 00:51:25,990 --> 00:51:26,510 Anyone else? 471 00:51:28,730 --> 00:51:30,850 Plus one for Rob's answer. 472 00:51:30,850 --> 00:51:32,690 I think the data tells all, really. 473 00:51:32,910 --> 00:51:33,530 Yeah. 474 00:51:35,490 --> 00:51:36,590 Okay, cool. 475 00:51:37,330 --> 00:51:39,130 From... I'm going to get your name wrong, I'm sorry. 476 00:51:39,250 --> 00:51:40,830 Scudzi, there's a question. 477 00:51:40,830 --> 00:51:46,410 Is there any focus in these vendor tools that provide good introspection into how the models are making inferences? 478 00:51:46,410 --> 00:51:46,670 Yeah. 479 00:51:46,670 --> 00:51:50,410 So do we have explainable ML for security out there? 480 00:51:50,410 --> 00:51:52,490 Does anyone know of anything offhand? 481 00:51:58,250 --> 00:52:03,210 I hesitate to... It kind of depends on what you mean by explainable. 482 00:52:05,230 --> 00:52:07,710 Okay, that's... Yes. 483 00:52:07,870 --> 00:52:10,230 If you look at the... 484 00:52:13,270 --> 00:52:17,210 I hate it when I forget the name of the paper I was an author on. 485 00:52:17,590 --> 00:52:17,590 But 486 00:52:20,870 --> 00:52:24,070 eating binary is one that I did with Ed Raff. 487 00:52:24,070 --> 00:52:35,850 You know, one of the big things we looked at for the explainability was we took a look at what parts of the binary was the model keying in on, you know, for making its decision. 488 00:52:36,070 --> 00:52:52,810 You know, I think there's been some work in things like that, but not necessarily explainability in terms that we can write an English sentence that will tell you why the model made its decisions and the things that the model has learned. 489 00:52:54,490 --> 00:52:59,930 Yeah, I mean, I know there are a number of companies that are beginning to push... 490 00:53:00,690 --> 00:53:06,910 I think explainability is probably a bridge too far, but at least they're working on enriching context, right? 491 00:53:06,910 --> 00:53:12,290 I contractually oblige to mention that Sophos has done a lot of research in that area. 492 00:53:13,190 --> 00:53:14,350 But yeah, I agree. 493 00:53:14,350 --> 00:53:19,230 I don't think there's anything that is going for specific explainability. 494 00:53:20,930 --> 00:53:22,590 I'm browsing through Discord. 495 00:53:22,590 --> 00:53:23,270 Okay. 496 00:53:23,270 --> 00:53:25,890 One dad pants revolution. 497 00:53:25,890 --> 00:53:26,730 Awesome. 498 00:53:26,790 --> 00:53:34,030 Would like to hear about strategies for optimizing the relationship between rules, analysts, and ML systems operating on the same domain. 499 00:53:34,030 --> 00:53:54,890 I'm going to take that and I'm going to spin that a little bit and say, like, maybe how do we... maybe that's like an internal communication problem, so less of a hype issue and more of a how do you go about getting analysts comfortable with thinking about sort of the statistical way that these machine learning tools operate? 500 00:53:54,890 --> 00:54:03,330 So how can we effectively skill up analysts in using these tools and understanding what they mean and not just sort of blindly trusting the result? 501 00:54:08,720 --> 00:54:13,500 So I think a lot of people are already intuitively doing that when they're doing their work. 502 00:54:13,500 --> 00:54:19,800 A lot of analysts are building up their own internal models of what's likely to be good and what isn't. 503 00:54:20,360 --> 00:54:39,680 I think if we can show how a model, you know, how an ML model kind of does the same thing and show, you know, that in a lot of cases the model's learning the same things that you would think that would be useful for an analyst, I think that goes a long way to bridging the two. 504 00:54:39,820 --> 00:54:42,520 I knew we'd get you on the explainability train somehow. 505 00:54:47,120 --> 00:54:48,300 Anyone else? 506 00:54:48,300 --> 00:54:51,520 I think, sorry John, go ahead. 507 00:54:51,880 --> 00:55:09,680 Oh yeah, I was just going to say a lot of the like basic building blocks for ML, so like things like historical data or like intermediate representations and transformations of data are things that analysts wish they had available to them but like can't because, 508 00:55:09,680 --> 00:55:13,520 you know, maybe like a spunk is too slow or something like that, right? 509 00:55:13,520 --> 00:55:36,740 And so I think sort of also allowing these intermediate representations and stuff to people so they can use them, whether it's in the form of like a querying engine or something like that, is something we've had a lot of good feedback it and help with and help bridge that gap. 510 00:55:37,840 --> 00:55:54,880 And I think it's important that data science doesn't have to be done only in the data science team because I think a lot of the analysts, like Rob was mentioning, have these mental models in their head already and the conversations are fun when we can say, 511 00:55:54,880 --> 00:56:02,560 you know, there's a little bit of terminology we have to add, these are called inputs, you have a prediction label, just articulate that very precisely and then boom, that's all it is. 512 00:56:02,560 --> 00:56:06,340 The model is just an equation that takes you from, you know, input to output. 513 00:56:07,540 --> 00:56:16,460 And I think it helps to say, you know, go off and try that on your own without the data science team in the middle at all. 514 00:56:16,840 --> 00:56:24,540 And that we're also here to advise and maybe help out with better inputs, data cleaning, that sort of stuff. 515 00:56:24,540 --> 00:56:29,820 But by no means does it have to be, you know, relegated to like the data science tower. 516 00:56:32,980 --> 00:56:34,360 Yeah, for sure. 517 00:56:34,360 --> 00:56:49,060 Integrating, like, getting everyone kind of back to what Rob was saying about the mass democratization of data science and machine learning, getting it integrated across the organization I think is a huge win for the organization. 518 00:56:49,060 --> 00:56:56,860 Sort of giving everyone a little, just like a little baseline of maybe statistical literacy and understanding how to use these tools. 519 00:56:57,080 --> 00:56:58,940 Ariel, any thoughts on? 520 00:57:01,100 --> 00:57:03,540 No, I don't have any thoughts on this. 521 00:57:03,540 --> 00:57:04,740 Sorry, not trying to put you on the spot. 522 00:57:04,740 --> 00:57:05,380 It's all good. 523 00:57:05,380 --> 00:57:06,200 It's all good. 524 00:57:06,200 --> 00:57:06,980 Okay. 525 00:57:08,840 --> 00:57:09,480 Cool. 526 00:57:09,480 --> 00:57:12,500 So, that looks like the bulk of the questions from Discord. 527 00:57:12,500 --> 00:57:16,940 I really am sorry if you had a question in Discord and I missed it. 528 00:57:16,940 --> 00:57:24,540 Please feel free to, like, do it again and maybe at me and I'll see if we can wrangle some responses in the Discord chat. 529 00:57:24,620 --> 00:57:28,880 But I want to thank all you guys for being here and doing this panel with me. 530 00:57:28,880 --> 00:57:33,420 I had a lot of fun once we got past the inevitable technical problems. 531 00:57:33,420 --> 00:57:34,500 I hope you did, too. 532 00:57:34,740 --> 00:57:36,380 And thanks so much for a great conversation. 533 00:57:36,380 --> 00:57:37,760 I really appreciate your time. 534 00:57:38,380 --> 00:57:38,840 Thanks, Rich. 535 00:57:38,840 --> 00:57:39,700 Thanks, everybody. 536 00:57:39,800 --> 00:57:40,080 Cool. 537 00:57:40,080 --> 00:57:40,380 Thanks. 538 00:57:40,380 --> 00:57:40,900 Thanks, Rich. 539 00:57:40,900 --> 00:57:41,880 Thanks, everyone. 540 00:57:42,000 --> 00:57:43,080 This was awesome. 541 00:57:43,940 --> 00:57:44,480 Ciao. 542 00:57:44,480 --> 00:57:45,120 Cheers.