WEBVTT 00:03.070 --> 00:04.010 Okay. 00:04.290 --> 00:05.490 Sorry. 00:12.280 --> 00:13.160 Sorry. 00:13.160 --> 00:14.880 I just figured it out. 00:15.560 --> 00:18.060 The audio should now be working. 00:18.920 --> 00:20.000 Seriously. 00:20.000 --> 00:20.620 Okay. 00:20.620 --> 00:20.920 Are we... 00:22.080 --> 00:24.540 So, on Twitch chat, I am so sorry. 00:24.540 --> 00:26.980 We should have this all sorted now. 00:26.980 --> 00:31.280 I found out what caused the feedback loop. 00:31.280 --> 00:32.100 Okay. 00:33.420 --> 00:40.520 So, on Twitch chat, can you please give us a thumbs up? 00:40.540 --> 00:41.320 Okay. 00:42.640 --> 00:45.000 Thumbs up if the audio is not working for everyone. 00:45.000 --> 00:48.460 So, you can hear Will, and it's not doing the weird Hall of Mirrors thing. 00:49.080 --> 00:50.040 Yay. 00:51.660 --> 00:54.140 I'm terrified to open Twitch at this point. 00:54.140 --> 00:55.100 Yeah. 00:56.040 --> 00:57.000 Cool. 00:57.940 --> 00:59.540 I'm muting myself now. 00:59.540 --> 01:00.900 The audio is now working. 01:00.900 --> 01:02.700 I am adding everyone's names. 01:02.700 --> 01:05.600 So, I am so sorry for everyone. 01:06.880 --> 01:10.660 This time, if it doesn't work, I'll off myself with a sword in the background. 01:11.760 --> 01:12.720 Okay. 01:13.000 --> 01:13.620 Okay. 01:13.620 --> 01:14.780 So, have at it, Will. 01:14.780 --> 01:16.300 It's what the people want. 01:17.300 --> 01:19.840 Rich, whoever I am this week. 01:20.060 --> 01:20.940 Okay. 01:21.780 --> 01:23.000 Well, last time. 01:23.280 --> 01:24.720 Last time through. 01:24.820 --> 01:32.980 So, again, welcome, everyone, to the Hype in AI panel, where I'm going to talk about does AI in InfoSec actually live up to the hype? 01:32.980 --> 01:36.440 I'm not going to pitch the topic again, because I'm tired of it. 01:36.440 --> 01:39.820 So , Ariel, please introduce yourself one last time. 01:39.820 --> 01:41.260 All right. 01:41.260 --> 01:41.860 Hello. 01:41.860 --> 01:43.160 My name is Ariel Hirt-Voss. 01:43.160 --> 01:48.620 I'm a research scientist at OpenAI, where I work on misuse and abuse of the AI technologies that we develop there. 01:48.620 --> 01:55.560 I'm also a PhD student at Harvard University, and I've been working on a book for No Starch Press on how to do red teaming for AI systems, specifically. 01:55.960 --> 01:58.080 And that's most of what I do. 01:59.120 --> 01:59.900 Cool. 01:59.960 --> 02:01.420 My name is Rich Huerang. 02:01.420 --> 02:05.140 I am a data scientist at Sophos. 02:05.240 --> 02:11.280 I have been working at the intersection of security, machine learning, and privacy for about 10 years. 02:11.280 --> 02:14.580 I have a PhD from the University of California at Santa Barbara. 02:14.980 --> 02:15.660 Will? 02:18.850 --> 02:20.010 My name is Will. 02:20.010 --> 02:29.950 I am a network operator, and I work on building tools, AI tools for red teaming and attacking machine learning systems. 02:30.810 --> 02:31.870 Very cool. 02:31.870 --> 02:32.370 Masuda? 02:33.470 --> 02:34.610 My name is Masuda. 02:34.610 --> 02:36.750 I'm a data scientist at Rapid7. 02:36.870 --> 02:43.730 I think a lot about incident detection, anomaly detection, and I'm a relative newcomer to security. 02:43.730 --> 02:51.290 I have a PhD in physics and spent some time in quantum computing research before formally switching over to data science and now security. 02:51.970 --> 02:53.010 Awesome. 02:53.150 --> 02:54.190 Rob? 02:55.270 --> 02:56.410 I'm Rob Brandon. 02:56.410 --> 03:00.550 Been working in the tech and security field for probably about 25 years now. 03:00.550 --> 03:04.210 Currently working for Booz Allen Hamilton as a researcher and threat hunter. 03:04.410 --> 03:06.850 And about eight years ago, I was doing some sock work. 03:06.850 --> 03:18.850 I got a little bit of a exposure to this machine learning stuff and went back to school and got a PhD from the University of Maryland, and been working in the intersection ever since. 03:19.870 --> 03:20.870 Awesome. 03:20.930 --> 03:23.390 And last, but certainly not least, John. 03:23.990 --> 03:24.970 Hey, everyone. 03:24.970 --> 03:27.150 I'm John Seymour, aka DeltaZero. 03:27.150 --> 03:32.270 I work as a data scientist at Salesforce, where I do intrusion detection on the DNR team. 03:32.570 --> 03:42.590 I also like to just kind of put my hands in everything, so I've touched a lot of, you know, offensive security and data science and all sorts of things. 03:43.030 --> 03:46.550 But yeah, mostly currently do intrusion detection. 03:48.230 --> 03:49.270 Awesome. 03:49.490 --> 03:53.650 So again, thanks to everyone for being here and taking part in this discussion. 03:53.710 --> 03:59.550 I kind of want to kick it off with a very sort of like high-level first question. 03:59.910 --> 04:04.070 So machine learning has actually been used in a wide range of in-physic problems now. 04:04.070 --> 04:09.130 So we've seen it in malware detection, where it's probably sort of the biggest success story. 04:09.130 --> 04:13.010 But then spam blocking is probably the oldest field it's been used in. 04:13.010 --> 04:18.690 And then other stuff like network traffic classification, analyst alert ranking, and triage. 04:18.770 --> 04:31.810 So what, if anything, do we get as sort of a value-add above and beyond sort of like the old-school tried-and-true signature-based or rule-based techniques for security? 04:31.810 --> 04:39.430 And so what I'm kind of hoping we can dig into here is like, what are the strengths and weaknesses of AI or ML or whatever you want to call it? 04:39.430 --> 04:49.250 And are there classes of problems that we should always be using ML for or maybe like should just completely give up and never waste time and money? 04:49.250 --> 04:57.990 And are there specific areas where if a vendor says, oh yes, we can solve this problem with AI, right, that immediately sets off a red flag in your head that they might not know what they're talking about? 04:58.490 --> 05:00.730 So open question, please dive in. 05:02.890 --> 05:04.690 Or I can call on somebody. 05:06.310 --> 05:17.290 So for intrusion detection, because this is what I spent the most time thinking about, I think the class of problems really depends strongly on what the data makes available. 05:17.690 --> 05:26.710 And just as an example for how I've approached the problem, when I first started, I thought, you know, we're awash in data, everything's digital, records are there for everything. 05:27.310 --> 05:35.450 And I viewed my role as analogous to the people that study volcanoes by looking at very small earthquake signatures, and have gotten really good at predicting these eruptions. 05:35.450 --> 05:37.350 And I thought, yeah, that's what I'm going to do. 05:37.350 --> 05:38.690 Can anything be more glamorous? 05:38.690 --> 05:43.970 I'm going to really understand these signatures and use them to predict, you know, incoming attacks. 05:43.970 --> 05:49.110 I very quickly realized it's hard to get a labeled data set of incidents. 05:49.310 --> 05:52.010 People don't want to share when they've been compromised. 05:53.070 --> 05:54.130 And so that's a problem. 05:54.130 --> 05:59.830 But even if I were to get a library of these attacks, attackers behave very differently. 05:59.830 --> 06:04.410 So I'd have to get a representative set of different attacker behavior. 06:04.410 --> 06:06.410 And also, they're opportunistic. 06:06.410 --> 06:13.310 So it really depends on the very specifics of network configurations and how different companies, you know, set up their assets. 06:13.530 --> 06:22.350 So really, now I'm talking about simulating different types of incidents over many different types of attackers over tons of different environments. 06:23.150 --> 06:27.910 So that very quickly put me off the classification use case, let's say. 06:28.710 --> 06:32.270 But there's still a lot of data. 06:32.270 --> 06:33.910 And so I think anomaly detection is great. 06:33.910 --> 06:35.970 We can baseline a lot of things. 06:36.830 --> 06:40.730 The volume-based baselining is maybe straightforward and not that interesting. 06:40.730 --> 06:48.490 But I think there's an art in defining what are the attributes that we expect to look really regular in reference to particular intrusions. 06:48.490 --> 06:51.750 So I think that's a really useful way to go. 06:51.750 --> 06:59.070 I'm very suspicious if someone were to tell me I've got a method to, you know, precisely detect a compromise for you. 06:59.070 --> 07:03.430 So you're mostly thinking about network traffic in that case? 07:03.990 --> 07:09.070 Network traffic, let's say endpoint data, you know, logs from endpoint data. 07:10.250 --> 07:11.370 Cool. 07:11.370 --> 07:12.690 Anyone else? 07:12.690 --> 07:20.730 I guess, Will, I'd really be interested to hear your input on this, since you work on the other side of the problem a lot of the time. 07:20.730 --> 07:21.470 Yeah, yeah. 07:21.470 --> 07:24.470 I don't come at it from a machine learning area. 07:25.410 --> 07:31.110 I'd say the nice bit is scale and building context around an event. 07:31.110 --> 07:32.730 I think that's really helpful. 07:34.010 --> 07:41.050 You know, I think the people who have been doing incident response for a long time, you know, they have the knowledge. 07:41.050 --> 07:49.510 They just don't necessarily have all the pieces to find, you know, a direction. 07:49.510 --> 08:00.190 So, you know, we always, I think as data scientists, or as the data is coming across, it's often incomplete, where you don't know what you need to solve the problem. 08:00.190 --> 08:06.930 So you only have, you know, half of the information that you actually need, and the rest is kind of fuzzy. 08:06.930 --> 08:10.250 But you know, maybe a direction that you're trying to go with it. 08:10.530 --> 08:13.290 So I think machine learning can help pull out those insights. 08:13.330 --> 08:19.650 But you know, I think it requires that still like a human, a human touch, a human direction. 08:19.650 --> 08:23.530 I think that is probably pretty limited as we go forward in time. 08:23.530 --> 08:31.230 But currently, where it's at it, you know, it's still pretty ineffective in networks for detections and things. 08:31.850 --> 08:39.630 And you potentially the other use cases. 08:40.130 --> 08:48.690 So proactive, secure configuration, it's like, out of this machine learning thing, the only thing we've got is just more alerts. 08:48.930 --> 08:53.590 I think that that may remain, you know, and people can't necessarily trust them. 08:53.870 --> 09:04.690 But I do think it's going forward in time, you know, it's going to be a very, very different use case, especially as you get data scientists with data sets that, you know, they start to understand. 09:05.910 --> 09:08.330 And, you know, we're all on the same playing field. 09:08.330 --> 09:15.350 So, you know, once we're on a Windows host, for example, everybody has access to the same information, more or less. 09:15.450 --> 09:21.570 And it's really just about how you're viewing that information inside of a given context, I think. 09:23.070 --> 09:34.070 Yeah, so one, one thing that I think kind of popped out from both those answers was, there's a lot of like human tuning and human feature extraction that's going on in there. 09:34.070 --> 09:43.010 So the, you do see a lot of marketing literature about, oh, just, you know, dump raw bytes in and detections, magical detections come out the other end. 09:43.010 --> 09:45.470 So any, any thoughts on claims like that? 09:45.470 --> 09:50.210 Do you really need a, like, is the human element, like mandatory still at this point? 09:50.210 --> 09:52.490 Or is there any path? 09:52.490 --> 09:53.950 No, I don't think it's mandatory. 09:54.370 --> 10:01.910 So for example, any given user is going to operate on a host, you know, within some range. 10:02.230 --> 10:04.530 So I think the anomaly detection is really nice. 10:04.530 --> 10:20.450 So let's say you have 1000 users, 998 of those users are going to come in, log in at 8am every day, they're going to visit the same things, you know, then you have that 2% that are, you know, doing something that they're not supposed to be doing or whatever it is. 10:20.570 --> 10:28.650 So I don't think you necessarily need to label data, but you, you need to baseline your network. 10:28.650 --> 10:37.470 And I think the issue for machine learning is when you try and scale it across, let's say 10 different companies, that's going to be really difficult. 10:37.530 --> 10:50.250 So but I think machine learning is such that you could, you could build a network for each individual network, it's just going to take time. 10:50.250 --> 10:54.550 But from a vendor's perspective, like that doesn't scale as well. 10:54.550 --> 10:58.510 And so I think it gets a little more difficult in that way. 11:00.770 --> 11:11.870 I kind of disagree with the anomaly detection approach, just because every network is just full of anomalies when you start going to look for anomalies. 11:11.870 --> 11:19.070 And while it may just be 2%, that 2% is still going to be the source of, you know, to, you know, millions and millions of alerts. 11:20.290 --> 11:21.250 Yeah, yeah, right. 11:21.250 --> 11:31.430 I should say, in specific cases, like, let's say log on, log on times, so, or if, you know, host where users are logging on. 11:31.430 --> 11:33.170 So yeah, you're absolutely right. 11:33.170 --> 11:35.250 I think anomalies are everywhere. 11:35.250 --> 11:44.210 But in terms of malicious events, or potentially malicious events, I think you could cut it down to a little bit where you can get some high fidelity. 11:44.230 --> 11:57.750 You can narrow it down, but there's still going to be a huge number of false positives, especially when you look at so many networks, the admins are doing things that look like malware, just because they have this tool that somebody put together 20 years ago, 11:57.750 --> 12:00.270 and nobody knows how the whole thing works anymore. 12:00.630 --> 12:10.050 And I think that's one of the interesting things about looking at network security in this space is when you get down to it, most networks aren't really planned and designed. 12:10.470 --> 12:15.890 They're more like a organism that's kind of grown and developed over the past 20 years. 12:16.690 --> 12:21.090 And I think that that's the type of thing that machine learning is really useful for. 12:21.090 --> 12:38.410 If you approach your environment as a space to be analyzed, rather than something that's designed, then machine learning techniques can be really useful for helping you weed out what's interesting and what's not. 12:39.230 --> 12:54.010 I totally agree that anomaly detection is rife with false positives, but I think that's where the human interaction becomes really important, because what the machine learning algorithm can do is find things that are mathematically anomalous. 12:54.130 --> 13:12.850 And then we have to go back to someone who has some knowledge of the network and say, oh, but these are network backups that happen, you know, at 10am every Tuesday, throw those out, and like all of these things that add contextual information, I think those can help denoise quite significantly. 13:13.090 --> 13:19.810 And there are ways to automate them as well, but those are add-ons to the actual algorithm. 13:20.310 --> 13:30.910 Okay, but I think that gets right into where one of the things I see as a big problem in this space, at least from the analyst perspective. 13:31.770 --> 13:44.210 I've seen countless cases where you'll have a researcher come in and say, yeah, I've got this great anomaly detection method, you know, let's go ahead and get some of the domain experts to look at the results. 13:44.430 --> 13:50.690 And the domain experts are already overworked, you know, nobody wants to spend their time tuning your anomaly detection system. 13:50.950 --> 13:59.190 And I think that's one of the key downsides and reasons that those systems just aren't practical from an applied perspective, you know. 14:00.050 --> 14:04.310 You don't have the people to give you the feedback on that. 14:04.510 --> 14:18.550 And I think that's, you know, and I hate to say it, but I think in a lot of cases that's why the only way to really do machine learning and AI successfully in this field is to spend the time acquiring both skill sets. 14:20.030 --> 14:27.270 I actually really strongly agree with you on that point of actually acquiring both skill sets being important. 14:27.410 --> 14:49.880 But one of the pieces that I do actually slightly disagree with you on is, um, well, I guess more I see just the ability to sort of identify costs between false positives and false negatives and be able to tweak that really easily is a huge powerful tool I see for at least statistical thinking about alerts and events, 14:49.880 --> 14:50.640 right. 14:50.660 --> 15:03.640 And, like, I do see, you know, like Suricata or whatever rule set you have, like, we see a ton of, you know, false positives, even in rule-based alerts, right, too. 15:03.660 --> 15:06.440 Because, as you say, like, networks are noisy. 15:07.760 --> 15:31.280 So I think one of the benefits that you can sort of think of is more data science is like a layer on top to sort of help control, sort of, you know, even filter down further, like, as you're saying, like, you know, filter out the benign alerts, the non-anomalous sort of alerts that you get for rule-based, 15:31.280 --> 15:32.940 at least as a starting point. 15:33.920 --> 15:48.340 Yeah, I mean, I think Brian Kovar had a really good point at a Canvas a couple years ago where he was saying a lot of the benefit you get out of machine learning isn't in detecting things, it's in throwing away the things that you don't want to look at. 15:51.770 --> 16:06.250 I'll also say, like, one of the things that we've generally had success with in our place is sort of, so, like, we have a lot of different rules for detecting, for example, C2 toolkits like Empire, things like that, right? 16:06.850 --> 16:16.610 And one of the things that, you know, when we went ahead and actually, like, tested out Empire was that it's very config-driven, right? 16:16.610 --> 16:23.650 You can change things like, you know, easily change things like how often you beacon out to the C2 host or things like that. 16:23.690 --> 16:28.090 And a lot of these things are sort of things that you would build into a rule, right? 16:28.090 --> 16:40.710 So, if, for example, you're taking a Let's Encrypt certificate as you're, like, you know, one of your main things that you build into your rules, like, you can easily change things like the certificate subject to evade, right? 16:41.310 --> 16:55.570 Some of the things that we've actually found a lot of use for is taking these rules and sort of trying to generalize them just a bit more to where they'll capture, like, a large number of variants of, you know, Empire something. 16:55.570 --> 17:08.550 So, not just specifically targeted to this particular certificate subject or structure, but, like, other variants as well for things that are easily modifiable by the attacker. 17:11.890 --> 17:12.490 Cool. 17:12.490 --> 17:18.850 I think a good point, kind of, about that is that you need to have a plan for how you're going to deal with the alerts. 17:18.850 --> 17:26.010 And that's kind of a key thing for anything, you know, you have to have a plan for what are you going to actually do with this model. 17:26.390 --> 17:28.350 Yeah, 100% agree with that. 17:29.530 --> 17:35.570 Yeah, I want to jump in really quick and give Ariel a chance to chime in if you've got anything. 17:35.790 --> 17:37.950 I don't have anything significantly different. 17:38.870 --> 17:55.370 I firmly agree with the necessity to have both skill sets of understanding security and also understanding the data science, especially when you, if you don't understand the underlying data of what it is you're working with, you're going to have a really hard time having any sort of alerts or whatever that you're trying to build that actually tells you anything useful. 17:55.690 --> 17:56.930 That's all I would add. 18:00.550 --> 18:01.630 Yeah, for sure. 18:02.490 --> 18:19.990 I think, so, we've kind of been maybe a little bit in the weeds of, like, different, yeah, like, you need to be really, like, focused on sort of the technical details of, how do I tune my anomaly detection system, or what do I know about sort of the security rules that I'm building behind this? 18:19.990 --> 18:32.230 I guess maybe part of what contributes to all of the hype is the fact that, like, this is, like, a really technical, really specialized discipline that not a lot of people understand. 18:32.450 --> 18:39.010 And so that sort of, A, makes it look like magic to people that maybe don't really understand it as well. 18:39.010 --> 18:47.510 And B, makes it really easy to oversell and to hype up because whoever's doing the marketing and the hyping can be pretty sure that the other person doesn't understand that. 18:47.630 --> 19:06.970 So, as people who are sort of, like, at the coalface, right, working, like, on these specific problems, what can we do to sort of, like, help communicate what the capabilities of these machine learning systems are and how can we sort of, like, demystify them and cut through the hype and make it give people, 19:06.970 --> 19:14.130 like, a good idea of, yes, they actually do do really good things and you can do really powerful, useful things with them, but no, they're not magic. 19:14.690 --> 19:17.910 Like, how do we sort of, like, bridge that communications gap? 19:20.390 --> 19:25.590 I'm going to say it's, I mean, InfoSec is ripe for it. 19:25.590 --> 19:30.570 Like, every year, like, the black hat vendor, you know, two years ago, it was all about application whitelisting. 19:30.750 --> 19:32.250 And now you don't hear about it anymore. 19:32.250 --> 19:33.950 All you hear about is machine learning. 19:34.750 --> 19:36.810 Maybe two years, it'll be something else. 19:37.070 --> 19:41.870 But I think it's, it's, and marketing is just, is just that. 19:41.870 --> 19:45.270 It's, you know, it's designed to make everything look better than it is. 19:45.470 --> 20:00.490 But I think in terms of practitioners, or, you know, if you're working with a company or at a company, it's, you know, saying, yeah, the hype is real, because this is, it can be really powerful, but it's, you're right, it's not going to do your dishwasher. 20:00.650 --> 20:07.110 It, you know, it can assist in the things you have, but it's the ultimate garbage in garbage out. 20:07.930 --> 20:17.170 So, you know, like, it's like the first, the first thing you find out when you install a SIM in your network is not anything security related, it's how much stuff is misconfigured. 20:17.690 --> 20:20.970 And I think the same is true for, for machine learning. 20:20.970 --> 20:23.610 It's like, once you put it in, it's going to take a little bit. 20:23.610 --> 20:43.330 But if you continue to work and feed and spend that time up front, like some of the best organizations, from the security perspective that we work with, are just ones that have just put the legwork in, you know, they've gone through and they've handpicked the DLLs that are allowed to run on every host, 20:43.330 --> 20:49.350 you know, and they haven't just plugged in a box, pat themselves on the back and, you know, gone got a beer. 20:49.690 --> 21:00.530 So I think it's like, doesn't matter what you're installing, I think the legwork needs to be done with it is obviously misconstrued that AI will solve your problem. 21:00.530 --> 21:01.790 Maybe we'll get there one day. 21:01.790 --> 21:09.070 But I think, you know, for now, it's still going to take some legwork, which is to be expected. 21:11.390 --> 21:22.550 I think some of the hype also comes from, I mean, we've interacted with the results of AI in different contexts, Netflix recommendations, Amazon recommendations. 21:22.890 --> 21:31.130 And as we think about how to do that for security, it's natural to say, you know what, just give me a dashboard with recommendations for what to do next. 21:31.130 --> 21:32.350 And it'll be AI powered. 21:32.350 --> 21:35.370 And it'll be just as disruptive as, you know, Netflix. 21:36.730 --> 21:49.470 And I think security professionals are much more skeptical of recommendations that they would get from that kind of dashboard than I as a consumer would be about what recommendations Amazon gives me for my clothing purchases. 21:50.310 --> 22:01.370 So I think one way I'm trying to think about this hype or talk about this hype is to just think about, like, our users are using the output of this AI very, very differently. 22:03.050 --> 22:06.810 I've heard it referred to as high stakes versus low stakes AI. 22:06.810 --> 22:11.110 In low stakes, you know, if my email is spam, I click it away. 22:11.110 --> 22:12.290 It's not a big deal. 22:12.290 --> 22:22.030 But in security, if our predictions are wrong, someone can be compromised or we could be taking someone's attention greatly away from the things they ought to be looking at instead. 22:22.030 --> 22:29.630 So they're intrinsically interacting with our work very differently than if we were, you know, correspondingly in e-commerce or something. 22:30.450 --> 22:40.990 And I feel like that's a conversation I'd like to have with folks in marketing or product management as they're trying to brainstorm how to talk about this publicly. 22:41.490 --> 22:49.410 I feel like the way that users use this is more specific to security, let's say. 22:50.910 --> 22:59.830 Actually, what's funny is there's two companies I can think of who their marketing departments talk about how they don't use ML in their security products. 23:00.070 --> 23:05.950 So in terms of, like, the skeptical and the skepticism that exists, these companies actually market that they don't use ML. 23:07.690 --> 23:09.410 How well does that work for them? 23:09.530 --> 23:10.190 Do you know? 23:10.850 --> 23:12.430 I guess they're still in business, so... 23:12.430 --> 23:13.230 Well, that's good. 23:13.230 --> 23:14.110 Okay. 23:14.270 --> 23:19.290 I do know one of them, and they are really, really solid. 23:20.210 --> 23:21.850 They are a solid company. 23:22.950 --> 23:23.590 That's cool. 23:23.590 --> 23:25.070 Or a solid product, I should say. 23:25.650 --> 23:30.390 Yeah, I do wonder when we're going to see companies going back to good old-fashioned AI. 23:31.210 --> 23:34.750 You could argue that signatures are just good old-fashioned AI, right? 23:34.750 --> 23:35.450 Rule-based. 23:39.630 --> 23:44.970 John, any thoughts on bridging the hype gap or bringing hype back to reality? 23:45.770 --> 23:48.750 Um, well, yeah, like... 23:51.050 --> 23:53.310 I don't really have much to add. 23:53.310 --> 23:59.030 I actually agree with both Will and Vasudha's comments that they've made. 23:59.030 --> 24:03.230 Like, it's... yeah. 24:03.630 --> 24:05.530 Marketing is marketing. 24:05.990 --> 24:11.550 It's really good to focus on sort of what we're actually able to do well. 24:11.550 --> 24:17.810 And I think one of the things that we data scientists can actually do a lot better on is communicating failures. 24:17.810 --> 24:29.390 And I have started to see, you know, conferences like Kamla's, as well as others, sort of welcome, sort of, hey, we tried this thing. 24:29.390 --> 24:30.090 It didn't work. 24:30.090 --> 24:31.370 Here's what we did. 24:31.370 --> 24:35.410 Don't do what we did, basically, sort of talks. 24:35.410 --> 24:46.010 And I think sort of focusing on, even as data scientists, being able to very clearly articulate what we were able to do, what we weren't able to do. 24:46.390 --> 24:50.250 I think that's actually very important for bridging the gap long term. 24:55.190 --> 24:57.030 I definitely agree with that. 24:57.830 --> 25:10.330 I think that given that a lot of the data science that we see in InfoSec kind of comes from academia, it sort of ends up with a lot of the same incentives that academia has for publishing stuff that you want it to look good. 25:10.330 --> 25:17.810 You don't really want to share your failures because you don't really get any sort of, you don't get anything out of that as an academic, unfortunately. 25:18.350 --> 25:25.650 And I think sort of taking a lot of this security data science out of that context and putting it more in, like, here's some practical stuff. 25:25.650 --> 25:31.030 Let's talk about what works and what doesn't work is exactly kind of the direction we should be taking things. 25:34.890 --> 25:35.690 Cool. 25:35.910 --> 25:44.630 Yeah, I want to loop back a little bit to something Vasudha said a second ago, where she referred to high stakes and low stakes decisions that are made. 25:45.210 --> 26:01.990 So yeah, I think it's obvious to me at least that a lot of the decisions that get made when you're talking about using ML or artificial intelligence or whatever you want to call it this week for InfoSec and security in general, those tend to be higher stakes decisions. 26:01.990 --> 26:18.830 And one of the things that I've seen people push a lot more for in recent past maybe two years or so is when you do have high stakes decisions, having some notion of explainability or interpretability that comes along with those models. 26:19.050 --> 26:29.930 So I kind of wonder, and this is partly driven by my own experience using ML to detect malware, those models tend to be almost impossible to explain. 26:29.930 --> 26:41.870 So how big of a, how important is this do you think to be able to explain the decisions that a model makes, be it in a detection, a classification, or an anomaly detection context? 26:41.950 --> 26:57.970 And how can we get from where we are now, where everything's about gradient boosted decision trees, or big complicated neural networks that are almost impenetrable, to something that's maybe more explicable and more directly, something you can directly interrogate? 26:59.920 --> 27:01.660 I think it's everything. 27:01.660 --> 27:23.100 I think it's worth sacrificing a little bit of accuracy even, to get a simpler model that pulls out future importances, or that you can easily explain what feature in your data led to this prediction, or led to the the presentation of this particular record for review. 27:23.900 --> 27:46.060 I also think, this is my opinion only, I think human-in-the-loop is super important, and that if it's a high stakes decision, it's much better received as an augmentation that an analyst can take into account, rather than being something that would be fully automated and trusted to make a decision on its own. 27:48.670 --> 27:51.150 Yeah, I 100% agree with that. 27:51.150 --> 28:04.850 Like a lot, so especially like if you're sort of starting a data science modeling, like a branch in detection or response or something, your analysts aren't going to trust your data science models at all, right? 28:05.090 --> 28:21.210 So really, like starting with something that's very clear and obvious, like, you know, this machine has, is supposed to be fired walled off from the rest of the network, and it's uploading hundreds of thousands of gigabytes, you know, to the outside world, 28:21.210 --> 28:27.050 like those sorts of things are going to start, you know, building the trust as you go. 28:27.890 --> 28:33.630 I'm also a really, really big proponent of the Google rules of ML that they do. 28:33.630 --> 28:44.610 And one of the things that they actually really, really recommend is starting with, you know, whenever you're building like a product or a machine learning based system, starting with something that's easily baselineable. 28:44.610 --> 28:51.110 And then that allows you to actually improve upon it and actually be able to measure like how much you've improved. 28:51.110 --> 28:58.170 When you have sort of these black box models, you lose a lot in sort of as you're trying to improve the model. 28:58.430 --> 29:06.550 You spend a lot of effort trying to actually figure out whether you've actually improved or how you can actually improve it further. 29:08.030 --> 29:11.230 I think organizationally, like buy-in is pretty huge. 29:11.230 --> 29:15.510 So as explainable as you can make it. 29:15.510 --> 29:20.510 And you know, like analysts, security analysts, you know, routiners are like, they're naturally curious. 29:20.510 --> 29:34.050 So if you're showing them something that is, how did you get this, like, they're gonna, they're gonna go and look at it and figure out and they, you know, even if you're right, if it's not as accurate, or it's not as effective, they'll figure out how to, 29:34.050 --> 29:48.750 to make it that way, or they'll at least figure out, you know, what inputs, what, what inputs require what outputs, and then you can start making it a little more complicated or sophisticated or whatever. 29:48.750 --> 29:56.850 I think we often, you know, it's like we often are misguided in the fact that we think complexity is sophistication. 29:57.430 --> 30:02.970 When in reality, I think a lot of times simplicity is, is actually sophistication. 30:02.990 --> 30:04.430 It's much cleaner. 30:05.630 --> 30:18.190 So I think we're kind of falling into the trap that people tend to fall into here, where we're thinking that there has to be one silver bullet system that generates everything you need to know to make a decision. 30:18.310 --> 30:31.630 In a lot of cases, I don't think explainability is necessarily that important, as long as you can go grab more information around whatever you're looking at to do a deeper analytic and provide more information to the analyst. 30:31.630 --> 30:39.110 Like if you've got a binary, there's no reason that you have to rely solely on the machine learning model to make the decision of whether it's good and bad. 30:39.270 --> 30:44.490 But you can use that as a tipper to do some more in-depth analytics on the binary. 30:44.490 --> 30:57.210 You've got, you know, start running, throwing it through some sandboxes, you know, do more expensive computation that you maybe couldn't, you can't do if you, you may not be able to do it for every single binary, but you know, you could do it for a few hundred an hour. 30:57.690 --> 31:12.450 You know, you know, in most other cases, there's additional context that you could automatically collect and provide to the analyst to make their decision on whether they should do something with the alert or not, rather than just relying on whatever the output of the model is. 31:12.490 --> 31:13.610 And that's just a good point. 31:13.610 --> 31:17.310 I think, like, I know Microsoft, the Windows Defender does that. 31:17.530 --> 31:23.130 So anything overtly malicious gets blocked on the host and anything that they're unsure about gets sent to a cloud. 31:26.900 --> 31:30.140 I think that sometimes explainability can be a little bit of red herring. 31:30.140 --> 31:31.760 In general, I think that it's good. 31:31.760 --> 31:48.920 I mean, for all the reasons stated above about building trust and, and having baselines for understanding how you can improve, but when you have, when you have like a human judge making a decision about whether or not someone's guilty, you're, there's no part of the process whereby you go back to the judge and you say, 31:48.920 --> 31:53.060 what are the pieces of evidence that convinced you that this person is guilty? 31:53.060 --> 32:00.600 And sometimes they can explain why and sometimes they can't, but humans are very good about trying to come up with stories to explain what the decisions that we make. 32:00.740 --> 32:14.640 So to some extent, I kind of view it as a little bit I don't know, naive to try to come up with like purely explainable reasons for why things are the way they are. 32:14.640 --> 32:19.280 Because oftentimes you can't really come up with like a true reason for why something is the way that it is. 32:23.220 --> 32:24.120 Cool. 32:24.260 --> 32:34.700 So I have kind of a twofer that I want to, I guess, point at specifically at Ariel and Will for initial responses. 32:34.700 --> 32:41.080 So the first bit is when we implement ML into these security systems, right? 32:41.080 --> 32:44.740 We bake it in, we make it a big, sometimes a critical part. 32:45.140 --> 32:55.220 And this is for Ariel, are we kind of opening up new vulnerabilities or new risks that we need to be worried about, right? 32:55.220 --> 33:01.600 Is it, anytime you add a new component, you at least change the attack surface. 33:01.640 --> 33:15.460 So do we really know quite what we're getting into as far as introducing new elements to an attack surface when we start putting ML into important decision-making points in these security systems? 33:15.620 --> 33:17.920 And then I'll come back to you, Will, for the second bit in a sec. 33:19.280 --> 33:20.960 My answer would be yes. 33:21.600 --> 33:27.400 But I think that a lot of that can be mitigated by making sure that you still have human in the loop. 33:27.700 --> 33:34.980 I kind of view a lot of these AI tools as sort of like centaur chess where you have an AI system that assists a human. 33:34.980 --> 33:42.360 I don't really anticipate there being a situation which you'd want a system to fully be automated to do everything because you're always going to run into issues. 33:42.360 --> 33:52.860 And like we discussed earlier about interpretability, if you don't have ways of understanding what's going on or like baselines or anything, you're going to be kind of for a lot of this stuff. 33:53.440 --> 34:05.820 That said, there has been a lot of good research around trying to understand what these potential vulnerabilities are, but they're all, at least for the most part, most of them have been developed inside laboratory conditions, which makes them very hard to translate it into the real world. 34:07.780 --> 34:09.000 Yeah, yeah. 34:09.080 --> 34:14.760 There's a lot of messed up stuff that's going to go down, but I don't think it's going to be as bad as maybe some people think it will be. 34:15.880 --> 34:16.500 I don't know. 34:16.500 --> 34:21.420 We are kind of clear on how poorly machine learning works for a lot of things and how well it works for other things. 34:21.420 --> 34:31.460 And I think there's a decent amount of healthy skepticism about machine learning at this point to sort of be some level of protection for us as we go forward adding it to more things. 34:34.180 --> 34:35.400 Okay, cool. 34:35.400 --> 34:36.140 Thanks. 34:36.720 --> 34:42.580 And then I'm going to kick the second part over to Will, and then I'd kind of like to hear from everyone else on the same topics. 34:43.040 --> 34:52.980 So do you see a risk when we make models interpretable or explainable and we say, okay, I think this thing was bad because of X. 34:54.020 --> 35:04.160 Do you think that that is making them necessarily easier to evade or bypass, or is that a controllable risk, or does it not really matter to you? 35:04.160 --> 35:05.300 That doesn't really matter to me. 35:05.300 --> 35:10.640 So I think when you have to sell something, making it explainable is nice. 35:11.200 --> 35:13.140 Like that's what you're trying to do. 35:13.140 --> 35:18.540 You need to make it sellable, but I don't care if I'm using a I don't care. 35:18.540 --> 35:20.480 I don't have to explain to anybody. 35:20.480 --> 35:23.120 I'm not trying to sell it to somebody, right? 35:23.120 --> 35:31.460 But if you want ML to take hold in an organization, you're trying to sell a product and no one's going to trust it, then you need to make it explainable. 35:31.460 --> 35:33.820 But for everybody else, it doesn't matter. 35:34.060 --> 35:37.160 Like it can be as black box as you want. 35:37.680 --> 35:39.780 And I think that's an important distinction. 35:40.200 --> 35:46.480 So it's like the... it needs to be explainable if you're trying to sell it. 35:47.740 --> 35:51.680 You know, but even internally, it can be useful. 35:51.940 --> 35:53.600 An explainable model is more useful. 35:53.620 --> 35:57.700 I don't necessarily think it's easier to attack or what. 35:57.700 --> 36:13.140 And it would be difficult to assume like from an attacker's perspective, like I generally always assume that it's going to be a black box and that you're not going to get 100% of the model, but you might be able to get some sort of, you know, some copycat, 36:13.140 --> 36:16.820 something that helps you explain the model that you're looking at. 36:17.980 --> 36:29.140 So yeah, explainability, my opinion is more about making it palatable to deploy into an organization. 36:30.740 --> 36:31.520 Okay. 36:31.520 --> 36:31.860 Yeah. 36:31.860 --> 36:37.140 So I find it... it's interesting because there's, I think, two different threads on the explainability that people are picking up on. 36:37.140 --> 36:42.480 First is building trust to be able to get people to adopt the models. 36:42.480 --> 36:52.080 And second is reducing risk by giving analysts who are seeing these alerts more context to kind of work with them. 36:52.080 --> 37:10.040 So, I mean, do those seem like those are... I guess what I'm saying is like pitching it as like we're building trust almost seems it's a little bit cynical, like a sales job, whereas we need to be able to explain it because these are sort of critical, 37:10.040 --> 37:14.940 maybe even life and death decisions, seems like a much higher ethical bar. 37:14.940 --> 37:17.400 So where does the balance between those two fall? 37:17.400 --> 37:21.960 Or are they necessarily even in conflict, right? 37:23.160 --> 37:25.580 I would add a third column to that. 37:25.580 --> 37:45.700 I would say putting aside the issue of having to convince someone else to trust these results for my own quality control, I look to explainability to convince myself that I haven't overfit really badly or that I'm not worried about data or that I haven't had data leakage happen or something. 37:45.700 --> 37:56.780 So just going back and evaluating why the model made the decision that it did gives me better confidence that it's picking up on real signal and not something spurious. 37:59.100 --> 38:07.480 Yeah, I also think explainability also depends on how much domain expertise the person that's consuming the results of the model has. 38:09.060 --> 38:10.380 Yeah, that's true. 38:12.800 --> 38:33.860 I'm always going to come back to the malware because that's what has the most research on it, but I mean, if you've got a malware analyst looking at a binary that he was delivered by a neural network said that, hey, this is malicious, that malware analyst is going to make a decision based on their own domain expertise, 38:33.860 --> 38:38.800 regardless of what the model says it made its decision on. 38:38.800 --> 38:41.900 I think that's the case for a lot of other things as well. 38:43.320 --> 38:58.120 But that leads us to the problem of the people that get the most value out of the explainability are also the ones that are least qualified to make the decision in the first place. 39:00.200 --> 39:14.860 Okay, so I'm going to reframe that a little bit more positively and say maybe it's a way to maybe help turn less experienced analysts and to help them perform like more experienced analysts maybe, hopefully. 39:15.100 --> 39:16.960 Or is that too optimistic? 39:17.460 --> 39:25.400 I don't think it's optimistic, but you still have the chicken and the egg problem of they have to understand the results enough. 39:25.580 --> 39:31.960 And if they have that understanding of the domain, they may not derive as much value from the explainability. 39:32.160 --> 39:36.600 Yeah, actually, I'm gonna there's a product called Artemis by Endgame. 39:36.600 --> 39:39.320 And it's like a talk to your sim product. 39:39.660 --> 39:43.960 And it's, you know, they were like, this is everything machine learning that you ever wanted. 39:43.960 --> 39:45.720 I don't know what their marketing is. 39:45.720 --> 39:57.120 But it's basically they haven't, they've only made information retrieval easier, but they it still requires an analyst that knows what they need to be looking for. 39:57.580 --> 40:05.780 So it actually, we already had a language to retrieve information from a sim, you know, whatever, OS query, whatever it might be, we didn't need machine learning to do that. 40:06.400 --> 40:17.800 What we needed machine learning to do is potentially add some context around a particular data point, not to say like, hey, you know, like a chatbot on top of your sim, we didn't need that. 40:19.960 --> 40:37.700 Um, I, pardon me if I'm wrong, or misunderstanding, Rob, but I thought your point was more that basically, the domain expert is going to know what features basically would be most helpful in determining which whether this binary was malicious or benign. 40:37.700 --> 40:51.360 And so it's kind of not the right thing to put that on the data science to try to, you know, to be the ones to actually explain why the model chose a particular, you know, feature or something in its decision. 40:51.380 --> 40:53.280 Am I correct on that or? 40:53.780 --> 41:02.440 Partly, what I'm saying is that the model, the models explainability isn't necessarily going to be helpful for the people that need it. 41:04.320 --> 41:09.240 You know, it can maybe say this, this particular segment of the binary is what I thought is bad. 41:09.620 --> 41:23.020 But then you need the expertise to know whether that means anything, you know, it could just be a packer, it could just be, you know, this particular thing is using get proc address to load things for non malicious reasons. 41:23.020 --> 41:34.080 And there's, there's a lot of context around things where, if you already know it, then you don't need the AI to tell you that this is why this is bad. 41:34.340 --> 41:40.540 And if you need the AI to tell you that this is bad, you may not understand it well enough to know whether the AI is wrong or not. 41:44.070 --> 41:51.370 Yeah, sensing a lot of very healthy skepticism of ML from all quarters here, which warms my dark little heart. 41:52.190 --> 41:57.050 So one more, one more question to kind of carry us out to the top of the hour. 41:57.510 --> 42:00.710 So we've touched on anomaly detection. 42:00.710 --> 42:04.050 I think that's people, I think there's a little bit of a network focus here. 42:04.050 --> 42:07.990 So that's what a lot of people are sort of talking about as a main tool. 42:07.990 --> 42:14.430 There's also binary classification for things like static file artifacts, be it malware detection or maldoct detection. 42:15.210 --> 42:22.850 So these are sort of maybe the, among the simpler forms of ML, right? 42:22.850 --> 42:43.310 And it doesn't get into things like, it hasn't like, we haven't seen, as far as I know, any serious, like, successes as far as using more complicated forms like reinforcement learning or, you know, like active, active learning with like really good human-in-the-loop integrations in the InfoSec space yet. 42:43.390 --> 42:56.770 So just sort of among the vast menu of new, of existing techniques that really haven't made a dent in InfoSec, where do you think, where do people think that the field is going next? 42:56.770 --> 43:02.850 Like what's going to be the next, the next big breakthrough as far as applying machine learning to security problems? 43:06.220 --> 43:10.040 I think that reinforcement learning is coming. 43:10.100 --> 43:16.800 I think it's going to have its moment at some point, but I think that that's going to depend very heavily on the kind of data that we can get into these kind of systems. 43:17.280 --> 43:29.720 I've seen a lot of people trying to use it for adversarial simulation, but haven't really had good data that really allows it to do a good job that's better than what we already have with Cobalt Strike and some of those other things. 43:30.400 --> 43:40.360 But I have seen some very convincing sort of research in the RL space around trying to use RL for like code generation kind of stuff. 43:40.360 --> 43:48.140 And that gives me some, I guess, I don't know if it's hope because there's also like a negative side to any sort of adversary simulation using RL. 43:48.620 --> 43:50.580 But I think it'll definitely be interesting. 43:51.160 --> 43:51.260 Yeah. 43:51.260 --> 43:55.920 If it is a good adversarial simulation, it's also probably a decent adversary. 43:55.920 --> 43:56.160 Yeah. 43:56.160 --> 44:04.540 I think reinforcement learning is the next, like there's a lot of attack simulation stuff that's being talked about. 44:04.540 --> 44:11.060 I know MITRE has some talks where they're discussing reinforcement learning for attack simulation. 44:11.060 --> 44:21.020 I think for actual attackers, the difficult part is you can't get on a network and learn the network and then attack it, right? 44:21.020 --> 44:25.520 You need to be learning from all of your general experience. 44:25.520 --> 44:36.000 But I think from a defensive standpoint, reinforcement learning could be, you know, because you can iterate through your network infinitely, but it's just matching what is useful. 44:36.000 --> 44:39.560 I think, I hate, I shouldn't say that. 44:39.560 --> 44:47.240 Adversary simulation is a bit of a, it's a bit of a buzzword. 44:47.540 --> 44:49.400 I think it's not a buzzword. 44:49.400 --> 44:56.960 How do I, it's like you, from the offensive side for the last three years, they've just been dropping techniques left and right. 44:56.960 --> 45:05.080 And so it's like, in some ways we call it adversary simulation because the techniques that we dropped are getting picked up by adversaries. 45:05.560 --> 45:09.500 And so it's like, we're emulating ourselves via a proxy. 45:10.300 --> 45:14.900 But it's because we wanted to do the research, but it hasn't been really helpful. 45:14.900 --> 45:22.460 But if I think of the value, you know, that the public research has generated in the defensive space, it's insane. 45:22.460 --> 45:27.940 You know, like there are entire products that are based off a GitHub project. 45:28.120 --> 45:36.180 So, you know, you can't say it's terrible, but I think the adversary simulation piece is a bit overdone. 45:36.180 --> 45:53.840 I wish organizations would spend more time in their own networks, just figuring out where their data is and doing some of the simple stuff like network segmentation and less time focusing on, you know, whatever APT is they think is going to come after them next. 45:53.880 --> 45:56.320 I think that's probably more useful. 45:56.500 --> 45:59.180 But reinforcement learning would be my answer. 46:00.560 --> 46:14.960 Yeah, I'm fairly skeptical of reinforcement learning, just because the problems in this space don't lend themselves to easily defining a target for the learning algorithm to test itself against. 46:15.240 --> 46:32.240 You know, from the defensive side, it's really hard to say, you know, we can't easily define what secure means in any meaningful sense to really be able to teach an agent, whether it's in a secure state or not, to determine, you know, what actions it should be taking. 46:32.380 --> 46:41.660 From the attack perspective, you know, like you said, the attacker can't just throw a agent on to go learn the network, because that's going to be super noisy. 46:41.660 --> 46:54.500 And, you know, hopefully the defenders will notice that that's going on as it's probing everything and everything so my answer to what's the next big thing is, I don't think there's going to be a next big thing. 46:54.960 --> 47:20.300 I think the real positive impact or the real kind of game changing impact is going to be smaller, you know, people applying machine learning to solve small problems tactically in a lot more places, you know, it's going to be the sock analyst who sits down and just makes a naive Bayes classifier to triage the AV alerts they've got, 47:20.300 --> 47:26.060 you know, it's going to be the small projects like that, widely distributed that are going to really make the difference. 47:27.600 --> 47:35.940 Okay, so we've got a couple votes for reinforcement learning, one vote for mass democratization of machine learning. 47:36.020 --> 47:38.540 Vasudha, you gotta pick for us. 47:38.540 --> 47:42.560 I'll pile on to the small projects done with ML. 47:42.720 --> 47:44.880 But I will add on to that. 47:45.440 --> 47:49.280 I'm really excited about graph theoretic approaches for that. 47:49.580 --> 47:58.920 Because I think as we think about very like, maybe making classifications on very specific endpoints, that that's a machine learning problem that I think we can do well. 47:58.940 --> 48:17.280 But to go from that to bridge from that to talking about an intrusion means understanding network effects, adding in other data sets to endpoint logs, it's not possible to do that using local, like single node properties. 48:17.280 --> 48:24.160 And so I'm, I'm kind of waiting to see what the what the network graph picture will do for us. 48:26.160 --> 48:27.360 Very cool. 48:27.360 --> 48:32.280 So I know we're kind of at the top of the hour, but we started a little bit late. 48:32.280 --> 48:35.240 Can people stay for another couple minutes? 48:35.240 --> 48:38.820 I want to try and take a couple of questions from the Discord, if that's okay. 48:39.720 --> 48:40.500 Awesome. 48:41.140 --> 48:41.600 Cool. 48:41.600 --> 48:45.200 I'm just gonna, so if I don't call your question out, please don't be offended. 48:45.200 --> 48:47.160 I'm just kind of scrolling through. 48:47.900 --> 49:00.020 So the first one that jumps out at me, FS Montenegro says, my perennial question is, what are the questions and practices that we should use to probe claims of a vendor's ML quality? 49:00.020 --> 49:02.460 I particularly like that one. 49:02.460 --> 49:03.760 Anyone want to take a stab? 49:03.760 --> 49:04.380 Yeah. 49:04.800 --> 49:06.780 How do you call BS? 49:12.250 --> 49:16.990 I'm going to put... I guess, I guess I can start right. 49:16.990 --> 49:25.170 So there's, there's a lot of different, you know, pitfalls in, you know, data science and, and performing machine learning. 49:25.170 --> 49:36.230 And one of the things I would like to see more of from, you know, skeptics or people trying to evaluate these models is just the simple questions like, how did you split your train and test? 49:36.430 --> 49:41.350 Like, did you, did you, you know, consider data leakage in sort of the time domain, right? 49:41.350 --> 49:44.850 Like, are you learning from future samples in your past samples? 49:45.010 --> 49:46.650 Those sorts of questions. 49:46.650 --> 49:50.530 I don't have much experience on the actual buying product side though. 49:50.530 --> 49:56.390 So I probably, that that's about as far as my, my opinion on the subject. 49:58.690 --> 49:59.830 I'm going to go with... 50:04.170 --> 50:05.830 Let's go with Will then Rob. 50:05.830 --> 50:06.670 Oh, sorry. 50:07.250 --> 50:13.130 I was just going to say, you know, from an attacker's perspective, these things are being deployed, but they haven't really been tested. 50:13.550 --> 50:22.950 So I think attackers are ultimately going to be the ones that decide whether or not ML is a viable solution for network defense. 50:23.510 --> 50:29.870 And we just, from an attack perspective, there's just not many people looking at it currently. 50:29.890 --> 50:33.870 So I would say that's probably a good way. 50:33.870 --> 50:35.570 Eventually it'll just happen, right? 50:35.570 --> 50:37.090 Attackers will have to adapt. 50:37.090 --> 50:42.390 And so I think that's going to be a good, a good litmus test for it. 50:45.450 --> 50:46.410 Rob? 50:47.390 --> 51:00.310 One of the first things I look at when I'm looking at a ML product is whether the data that they are purporting to operate on is even reasonably possible to make a decision on. 51:00.510 --> 51:13.450 And a lot of the products out there are saying, you know, we're going to look at an email, we're going to look at a phishing email and, you know, we'll predict what the person had for breakfast and, you know, all kinds of other claims that just don't really follow from the data that they have available. 51:14.490 --> 51:20.610 So I think, you know, to me, that's really one of the big things I look at when I'm evaluating an ML system. 51:24.370 --> 51:24.950 Cool. 51:24.950 --> 51:25.830 Good answers. 51:25.990 --> 51:26.510 Anyone else? 51:28.730 --> 51:30.850 Plus one for Rob's answer. 51:30.850 --> 51:32.690 I think the data tells all, really. 51:32.910 --> 51:33.530 Yeah. 51:35.490 --> 51:36.590 Okay, cool. 51:37.330 --> 51:39.130 From... I'm going to get your name wrong, I'm sorry. 51:39.250 --> 51:40.830 Scudzi, there's a question. 51:40.830 --> 51:46.410 Is there any focus in these vendor tools that provide good introspection into how the models are making inferences? 51:46.410 --> 51:46.670 Yeah. 51:46.670 --> 51:50.410 So do we have explainable ML for security out there? 51:50.410 --> 51:52.490 Does anyone know of anything offhand? 51:58.250 --> 52:03.210 I hesitate to... It kind of depends on what you mean by explainable. 52:05.230 --> 52:07.710 Okay, that's... Yes. 52:07.870 --> 52:10.230 If you look at the... 52:13.270 --> 52:17.210 I hate it when I forget the name of the paper I was an author on. 52:17.590 --> 52:17.590 But 52:20.870 --> 52:24.070 eating binary is one that I did with Ed Raff. 52:24.070 --> 52:35.850 You know, one of the big things we looked at for the explainability was we took a look at what parts of the binary was the model keying in on, you know, for making its decision. 52:36.070 --> 52:52.810 You know, I think there's been some work in things like that, but not necessarily explainability in terms that we can write an English sentence that will tell you why the model made its decisions and the things that the model has learned. 52:54.490 --> 52:59.930 Yeah, I mean, I know there are a number of companies that are beginning to push... 53:00.690 --> 53:06.910 I think explainability is probably a bridge too far, but at least they're working on enriching context, right? 53:06.910 --> 53:12.290 I contractually oblige to mention that Sophos has done a lot of research in that area. 53:13.190 --> 53:14.350 But yeah, I agree. 53:14.350 --> 53:19.230 I don't think there's anything that is going for specific explainability. 53:20.930 --> 53:22.590 I'm browsing through Discord. 53:22.590 --> 53:23.270 Okay. 53:23.270 --> 53:25.890 One dad pants revolution. 53:25.890 --> 53:26.730 Awesome. 53:26.790 --> 53:34.030 Would like to hear about strategies for optimizing the relationship between rules, analysts, and ML systems operating on the same domain. 53:34.030 --> 53:54.890 I'm going to take that and I'm going to spin that a little bit and say, like, maybe how do we... maybe that's like an internal communication problem, so less of a hype issue and more of a how do you go about getting analysts comfortable with thinking about sort of the statistical way that these machine learning tools operate? 53:54.890 --> 54:03.330 So how can we effectively skill up analysts in using these tools and understanding what they mean and not just sort of blindly trusting the result? 54:08.720 --> 54:13.500 So I think a lot of people are already intuitively doing that when they're doing their work. 54:13.500 --> 54:19.800 A lot of analysts are building up their own internal models of what's likely to be good and what isn't. 54:20.360 --> 54:39.680 I think if we can show how a model, you know, how an ML model kind of does the same thing and show, you know, that in a lot of cases the model's learning the same things that you would think that would be useful for an analyst, I think that goes a long way to bridging the two. 54:39.820 --> 54:42.520 I knew we'd get you on the explainability train somehow. 54:47.120 --> 54:48.300 Anyone else? 54:48.300 --> 54:51.520 I think, sorry John, go ahead. 54:51.880 --> 55:09.680 Oh yeah, I was just going to say a lot of the like basic building blocks for ML, so like things like historical data or like intermediate representations and transformations of data are things that analysts wish they had available to them but like can't because, 55:09.680 --> 55:13.520 you know, maybe like a spunk is too slow or something like that, right? 55:13.520 --> 55:36.740 And so I think sort of also allowing these intermediate representations and stuff to people so they can use them, whether it's in the form of like a querying engine or something like that, is something we've had a lot of good feedback it and help with and help bridge that gap. 55:37.840 --> 55:54.880 And I think it's important that data science doesn't have to be done only in the data science team because I think a lot of the analysts, like Rob was mentioning, have these mental models in their head already and the conversations are fun when we can say, 55:54.880 --> 56:02.560 you know, there's a little bit of terminology we have to add, these are called inputs, you have a prediction label, just articulate that very precisely and then boom, that's all it is. 56:02.560 --> 56:06.340 The model is just an equation that takes you from, you know, input to output. 56:07.540 --> 56:16.460 And I think it helps to say, you know, go off and try that on your own without the data science team in the middle at all. 56:16.840 --> 56:24.540 And that we're also here to advise and maybe help out with better inputs, data cleaning, that sort of stuff. 56:24.540 --> 56:29.820 But by no means does it have to be, you know, relegated to like the data science tower. 56:32.980 --> 56:34.360 Yeah, for sure. 56:34.360 --> 56:49.060 Integrating, like, getting everyone kind of back to what Rob was saying about the mass democratization of data science and machine learning, getting it integrated across the organization I think is a huge win for the organization. 56:49.060 --> 56:56.860 Sort of giving everyone a little, just like a little baseline of maybe statistical literacy and understanding how to use these tools. 56:57.080 --> 56:58.940 Ariel, any thoughts on? 57:01.100 --> 57:03.540 No, I don't have any thoughts on this. 57:03.540 --> 57:04.740 Sorry, not trying to put you on the spot. 57:04.740 --> 57:05.380 It's all good. 57:05.380 --> 57:06.200 It's all good. 57:06.200 --> 57:06.980 Okay. 57:08.840 --> 57:09.480 Cool. 57:09.480 --> 57:12.500 So, that looks like the bulk of the questions from Discord. 57:12.500 --> 57:16.940 I really am sorry if you had a question in Discord and I missed it. 57:16.940 --> 57:24.540 Please feel free to, like, do it again and maybe at me and I'll see if we can wrangle some responses in the Discord chat. 57:24.620 --> 57:28.880 But I want to thank all you guys for being here and doing this panel with me. 57:28.880 --> 57:33.420 I had a lot of fun once we got past the inevitable technical problems. 57:33.420 --> 57:34.500 I hope you did, too. 57:34.740 --> 57:36.380 And thanks so much for a great conversation. 57:36.380 --> 57:37.760 I really appreciate your time. 57:38.380 --> 57:38.840 Thanks, Rich. 57:38.840 --> 57:39.700 Thanks, everybody. 57:39.800 --> 57:40.080 Cool. 57:40.080 --> 57:40.380 Thanks. 57:40.380 --> 57:40.900 Thanks, Rich. 57:40.900 --> 57:41.880 Thanks, everyone. 57:42.000 --> 57:43.080 This was awesome. 57:43.940 --> 57:44.480 Ciao. 57:44.480 --> 57:45.120 Cheers.