[00:03.070 --> 00:18.060]  Okay. Sorry. Sorry. I just figured it out. The audio should now be working.
[00:18.920 --> 00:20.920]  Seriously. Okay. Are we...
[00:22.080 --> 00:26.980]  So, on Twitch chat, I am so sorry. We should have this all sorted now.
[00:26.980 --> 00:31.280]  I found out what caused the feedback loop.
[00:31.280 --> 00:40.520]  Okay. So, on Twitch chat, can you please give us a thumbs up?
[00:40.540 --> 00:45.920]  Okay. Thumbs up if the audio is not working for everyone. So, you can hear Will,
[00:45.920 --> 00:48.460]  and it's not doing the weird Hall of Mirrors thing.
[00:49.080 --> 00:50.040]  Yay.
[00:51.660 --> 00:54.140]  I'm terrified to open Twitch at this point.
[00:54.140 --> 01:02.700]  Yeah. Cool. I'm muting myself now. The audio is now working. I am adding everyone's names.
[01:02.700 --> 01:05.600]  So, I am so sorry for everyone.
[01:06.880 --> 01:10.660]  This time, if it doesn't work, I'll off myself with a sword in the background.
[01:11.760 --> 01:12.720]  Okay.
[01:13.000 --> 01:14.780]  Okay. So, have at it, Will.
[01:14.780 --> 01:20.940]  It's what the people want. Rich, whoever I am this week. Okay.
[01:21.780 --> 01:28.680]  Well, last time. Last time through. So, again, welcome, everyone, to the Hype in AI panel,
[01:28.680 --> 01:32.980]  where I'm going to talk about does AI in InfoSec actually live up to the hype?
[01:32.980 --> 01:36.440]  I'm not going to pitch the topic again, because I'm tired of it.
[01:36.440 --> 01:39.820]  So, Ariel, please introduce yourself one last time.
[01:39.820 --> 01:44.640]  All right. Hello. My name is Ariel Hirt-Voss. I'm a research scientist at OpenAI,
[01:44.640 --> 01:48.620]  where I work on misuse and abuse of the AI technologies that we develop there.
[01:48.620 --> 01:52.360]  I'm also a PhD student at Harvard University, and I've been working on a book for No Starch Press
[01:52.360 --> 01:58.080]  on how to do red teaming for AI systems, specifically. And that's most of what I do.
[01:59.120 --> 02:06.760]  Cool. My name is Rich Huerang. I am a data scientist at Sophos. I have been working
[02:06.760 --> 02:11.280]  at the intersection of security, machine learning, and privacy for about 10 years.
[02:11.280 --> 02:15.660]  I have a PhD from the University of California at Santa Barbara. Will?
[02:18.850 --> 02:24.710]  My name is Will. I am a network operator, and I work on building tools,
[02:25.390 --> 02:29.950]  AI tools for red teaming and attacking machine learning systems.
[02:30.810 --> 02:32.370]  Very cool. Masuda?
[02:33.470 --> 02:40.130]  My name is Masuda. I'm a data scientist at Rapid7. I think a lot about incident detection,
[02:40.130 --> 02:45.250]  anomaly detection, and I'm a relative newcomer to security. I have a PhD in physics and
[02:45.250 --> 02:48.690]  spent some time in quantum computing research before formally switching over
[02:48.690 --> 02:54.190]  to data science and now security. Awesome. Rob?
[02:55.270 --> 03:00.550]  I'm Rob Brandon. Been working in the tech and security field for probably about 25 years now.
[03:00.550 --> 03:04.210]  Currently working for Booz Allen Hamilton as a researcher and threat hunter.
[03:04.410 --> 03:09.830]  And about eight years ago, I was doing some sock work. I got a little bit of a exposure to this
[03:09.830 --> 03:14.450]  machine learning stuff and went back to school and got a PhD from the University of Maryland,
[03:14.990 --> 03:18.850]  and been working in the intersection ever since.
[03:19.870 --> 03:23.390]  Awesome. And last, but certainly not least, John.
[03:23.990 --> 03:29.530]  Hey, everyone. I'm John Seymour, aka DeltaZero. I work as a data scientist at Salesforce,
[03:29.530 --> 03:35.150]  where I do intrusion detection on the DNR team. I also like to just kind of put my hands in
[03:35.150 --> 03:41.710]  everything, so I've touched a lot of, you know, offensive security and data science and all sorts
[03:41.710 --> 03:46.550]  of things. But yeah, mostly currently do intrusion detection.
[03:48.230 --> 03:53.650]  Awesome. So again, thanks to everyone for being here and taking part in this discussion.
[03:53.710 --> 03:59.550]  I kind of want to kick it off with a very sort of like high-level first question.
[03:59.910 --> 04:04.210]  So machine learning has actually been used in a wide range of in-physic problems now. So
[04:04.210 --> 04:09.130]  we've seen it in malware detection, where it's probably sort of the biggest success story.
[04:09.130 --> 04:13.010]  But then spam blocking is probably the oldest field it's been used in.
[04:13.010 --> 04:18.690]  And then other stuff like network traffic classification, analyst alert ranking, and triage.
[04:18.770 --> 04:24.830]  So what, if anything, do we get as sort of a value-add above and beyond sort of like the
[04:24.830 --> 04:32.410]  old-school tried-and-true signature-based or rule-based techniques for security? And so what
[04:32.410 --> 04:37.910]  I'm kind of hoping we can dig into here is like, what are the strengths and weaknesses of AI or ML
[04:37.910 --> 04:44.050]  or whatever you want to call it? And are there classes of problems that we should always be
[04:44.050 --> 04:49.670]  using ML for or maybe like should just completely give up and never waste time and money? And are
[04:49.670 --> 04:53.970]  there specific areas where if a vendor says, oh yes, we can solve this problem with AI, right,
[04:53.970 --> 04:57.270]  that immediately sets off a red flag in your head that they might not know what they're talking
[04:57.270 --> 05:04.690]  about? So open question, please dive in. Or I can call on somebody.
[05:06.310 --> 05:10.890]  So for intrusion detection, because this is what I spent the most time thinking about,
[05:10.890 --> 05:17.290]  I think the class of problems really depends strongly on what the data makes available.
[05:17.690 --> 05:21.850]  And just as an example for how I've approached the problem, when I first started,
[05:21.850 --> 05:25.890]  I thought, you know, we're awash in data, everything's digital, records are there for
[05:25.890 --> 05:31.330]  everything. And I viewed my role as analogous to the people that study volcanoes by looking at
[05:31.330 --> 05:35.450]  very small earthquake signatures, and have gotten really good at predicting these eruptions.
[05:35.450 --> 05:39.050]  And I thought, yeah, that's what I'm going to do. Can anything be more glamorous? I'm going to
[05:39.050 --> 05:43.970]  really understand these signatures and use them to predict, you know, incoming attacks.
[05:43.970 --> 05:50.530]  I very quickly realized it's hard to get a labeled data set of incidents. People don't want to share
[05:50.530 --> 05:56.010]  when they've been compromised. And so that's a problem. But even if I were to get a library of
[05:56.010 --> 06:02.770]  these attacks, attackers behave very differently. So I'd have to get a representative set of
[06:02.770 --> 06:08.510]  different attacker behavior. And also, they're opportunistic. So it really depends on the very
[06:08.510 --> 06:12.710]  specifics of network configurations and how different companies, you know, set up their
[06:12.710 --> 06:19.110]  assets. So really, now I'm talking about simulating different types of incidents over many different
[06:19.110 --> 06:24.850]  types of attackers over tons of different environments. So that very quickly put me off
[06:24.850 --> 06:32.730]  the classification use case, let's say. But there's still a lot of data. And so I think
[06:32.730 --> 06:38.470]  anomaly detection is great. We can baseline a lot of things. The volume-based baselining is
[06:38.470 --> 06:43.030]  maybe straightforward and not that interesting. But I think there's an art in defining what are
[06:43.030 --> 06:48.490]  the attributes that we expect to look really regular in reference to particular intrusions.
[06:48.490 --> 06:53.790]  So I think that's a really useful way to go. I'm very suspicious if someone were to tell me
[06:53.790 --> 06:59.070]  I've got a method to, you know, precisely detect a compromise for you.
[06:59.070 --> 07:03.430]  So you're mostly thinking about network traffic in that case?
[07:03.990 --> 07:09.070]  Network traffic, let's say endpoint data, you know, logs from endpoint data.
[07:10.250 --> 07:17.010]  Cool. Anyone else? I guess, Will, I'd really be interested to hear your input on this,
[07:17.010 --> 07:20.730]  since you work on the other side of the problem a lot of the time.
[07:20.730 --> 07:27.510]  Yeah, yeah. I don't come at it from a machine learning area. I'd say the nice bit is scale
[07:27.510 --> 07:35.770]  and building context around an event. I think that's really helpful. You know, I think the
[07:35.770 --> 07:41.050]  people who have been doing incident response for a long time, you know, they have the knowledge.
[07:41.050 --> 07:49.810]  They just don't necessarily have all the pieces to find, you know, a direction. So, you know,
[07:49.810 --> 07:56.280]  we always, I think as data scientists, or as the data is coming across, it's often incomplete,
[07:56.990 --> 08:02.130]  where you don't know what you need to solve the problem. So you only have, you know,
[08:02.130 --> 08:07.710]  half of the information that you actually need, and the rest is kind of fuzzy. But you know,
[08:07.710 --> 08:12.170]  maybe a direction that you're trying to go with it. So I think machine learning can help pull out
[08:12.170 --> 08:18.710]  those insights. But you know, I think it requires that still like a human, a human touch, a human
[08:18.710 --> 08:25.290]  direction. I think that is probably pretty limited as we go forward in time. But currently, where
[08:25.290 --> 08:36.990]  it's at it, you know, it's still pretty ineffective in networks for detections and things. And you
[08:37.630 --> 08:45.470]  potentially the other use cases. So proactive, secure configuration, it's like, out of this
[08:45.470 --> 08:51.210]  machine learning thing, the only thing we've got is just more alerts. I think that that may remain,
[08:51.210 --> 08:56.370]  you know, and people can't necessarily trust them. But I do think it's going forward in time,
[08:56.370 --> 09:01.350]  you know, it's going to be a very, very different use case, especially as you get data scientists
[09:01.350 --> 09:07.450]  with data sets that, you know, they start to understand. And, you know, we're all on the same
[09:07.450 --> 09:13.390]  playing field. So, you know, once we're on a Windows host, for example, everybody has access
[09:13.390 --> 09:17.510]  to the same information, more or less. And it's really just about how you're viewing that
[09:17.510 --> 09:25.310]  information inside of a given context, I think. Yeah, so one, one thing that I think kind of
[09:25.310 --> 09:31.610]  popped out from both those answers was, there's a lot of like human tuning and human feature
[09:31.610 --> 09:38.030]  extraction that's going on in there. So the, you do see a lot of marketing literature about, oh,
[09:38.030 --> 09:43.190]  just, you know, dump raw bytes in and detections, magical detections come out the other end. So
[09:43.190 --> 09:48.350]  any, any thoughts on claims like that? Do you really need a, like, is the human element,
[09:48.350 --> 09:53.950]  like mandatory still at this point? Or is there any path? No, I don't think it's mandatory.
[09:54.370 --> 10:01.910]  So for example, any given user is going to operate on a host, you know, within some range.
[10:02.230 --> 10:06.650]  So I think the anomaly detection is really nice. So let's say you have 1000 users,
[10:08.410 --> 10:12.230]  998 of those users are going to come in, log in at 8am every day, they're going to
[10:12.230 --> 10:18.110]  visit the same things, you know, then you have that 2% that are, you know, doing something that
[10:18.110 --> 10:22.910]  they're not supposed to be doing or whatever it is. So I don't think you necessarily need
[10:24.530 --> 10:31.030]  to label data, but you, you need to baseline your network. And I think the issue for machine
[10:31.030 --> 10:36.070]  learning is when you try and scale it across, let's say 10 different companies, that's going
[10:36.070 --> 10:43.110]  to be really difficult. So but I think machine learning is such that you could, you could
[10:44.510 --> 10:51.670]  build a network for each individual network, it's just going to take time. But from a vendor's
[10:51.670 --> 10:56.590]  perspective, like that doesn't scale as well. And so I think it gets a little more difficult
[10:57.450 --> 10:58.510]  in that way.
[11:00.770 --> 11:08.910]  I kind of disagree with the anomaly detection approach, just because every network is just
[11:08.910 --> 11:13.650]  full of anomalies when you start going to look for anomalies. And while it may just be 2%,
[11:13.650 --> 11:18.230]  that 2% is still going to be the source of, you know, to, you know, millions and millions of
[11:18.230 --> 11:19.070]  alerts.
[11:20.290 --> 11:26.690]  Yeah, yeah, right. I should say, in specific cases, like, let's say log on, log on times,
[11:26.690 --> 11:33.450]  so, or if, you know, host where users are logging on. So yeah, you're absolutely right. I think
[11:33.450 --> 11:38.910]  anomalies are everywhere. But in terms of malicious events, or potentially malicious
[11:38.910 --> 11:44.210]  events, I think you could cut it down to a little bit where you can get some high fidelity.
[11:44.230 --> 11:48.010]  You can narrow it down, but there's still going to be a huge number of false positives,
[11:48.010 --> 11:52.770]  especially when you look at so many networks, the admins are doing things that look like
[11:52.770 --> 11:57.750]  malware, just because they have this tool that somebody put together 20 years ago,
[11:57.750 --> 12:02.090]  and nobody knows how the whole thing works anymore. And I think that's one of the interesting
[12:02.090 --> 12:06.270]  things about looking at network security in this space is when you get down to it,
[12:06.270 --> 12:13.610]  most networks aren't really planned and designed. They're more like a organism that's kind of grown
[12:13.610 --> 12:18.790]  and developed over the past 20 years. And I think that that's the type of thing that machine
[12:18.790 --> 12:28.890]  learning is really useful for. If you approach your environment as a space to be analyzed,
[12:28.890 --> 12:33.270]  rather than something that's designed, then machine learning techniques can be really
[12:33.270 --> 12:38.410]  useful for helping you weed out what's interesting and what's not.
[12:39.230 --> 12:44.930]  I totally agree that anomaly detection is rife with false positives, but I think that's
[12:44.930 --> 12:50.730]  where the human interaction becomes really important, because what the machine learning
[12:50.730 --> 12:54.810]  algorithm can do is find things that are mathematically anomalous. And then we have
[12:54.810 --> 13:00.410]  to go back to someone who has some knowledge of the network and say, oh, but these are network
[13:00.410 --> 13:05.190]  backups that happen, you know, at 10am every Tuesday, throw those out, and like all of these
[13:05.190 --> 13:12.850]  things that add contextual information, I think those can help denoise quite significantly.
[13:13.090 --> 13:19.810]  And there are ways to automate them as well, but those are add-ons to the actual algorithm.
[13:20.310 --> 13:27.270]  Okay, but I think that gets right into where one of the things I see as a big problem in this space,
[13:27.270 --> 13:34.510]  at least from the analyst perspective. I've seen countless cases where you'll have a researcher
[13:34.510 --> 13:38.750]  come in and say, yeah, I've got this great anomaly detection method, you know, let's go ahead and
[13:39.710 --> 13:45.990]  get some of the domain experts to look at the results. And the domain experts are already
[13:45.990 --> 13:50.690]  overworked, you know, nobody wants to spend their time tuning your anomaly detection system.
[13:50.950 --> 13:57.410]  And I think that's one of the key downsides and reasons that those systems just aren't practical
[13:57.410 --> 14:04.310]  from an applied perspective, you know. You don't have the people to give you the feedback on that.
[14:04.510 --> 14:09.350]  And I think that's, you know, and I hate to say it, but I think in a lot of cases that's why
[14:09.990 --> 14:15.270]  the only way to really do machine learning and AI successfully in this field is to
[14:15.950 --> 14:18.550]  spend the time acquiring both skill sets.
[14:20.030 --> 14:26.290]  I actually really strongly agree with you on that point of actually acquiring both skill sets being
[14:26.290 --> 14:32.440]  important. But one of the pieces that I do actually slightly disagree with you on is,
[14:32.440 --> 14:40.480]  um, well, I guess more I see just the ability to sort of identify costs between false positives
[14:40.480 --> 14:46.740]  and false negatives and be able to tweak that really easily is a huge powerful tool I see for
[14:46.740 --> 14:54.240]  at least statistical thinking about alerts and events, right. And, like, I do see, you know,
[14:54.240 --> 15:00.580]  like Suricata or whatever rule set you have, like, we see a ton of, you know, false positives,
[15:00.580 --> 15:06.440]  even in rule-based alerts, right, too. Because, as you say, like, networks are noisy.
[15:07.760 --> 15:14.400]  So I think one of the benefits that you can sort of think of is more data science is like a layer
[15:14.400 --> 15:21.680]  on top to sort of help control, sort of, you know, even filter down further, like, as you're saying,
[15:21.680 --> 15:30.500]  like, you know, filter out the benign alerts, the non-anomalous sort of alerts that you get
[15:30.500 --> 15:36.460]  for rule-based, at least as a starting point. Yeah, I mean, I think Brian Kovar had a really
[15:36.460 --> 15:42.020]  good point at a Canvas a couple years ago where he was saying a lot of the benefit you get out
[15:42.020 --> 15:47.240]  of machine learning isn't in detecting things, it's in throwing away the things that you don't
[15:47.240 --> 15:56.190]  want to look at. I'll also say, like, one of the things that we've generally had success with
[15:56.190 --> 16:03.210]  in our place is sort of, so, like, we have a lot of different rules for detecting, for example,
[16:03.210 --> 16:09.290]  C2 toolkits like Empire, things like that, right? And one of the things that, you know,
[16:09.290 --> 16:16.610]  when we went ahead and actually, like, tested out Empire was that it's very config-driven, right?
[16:16.610 --> 16:22.070]  You can change things like, you know, easily change things like how often you beacon out to the C2
[16:22.070 --> 16:26.890]  host or things like that. And a lot of these things are sort of things that you would build
[16:26.890 --> 16:32.730]  into a rule, right? So, if, for example, you're taking a Let's Encrypt certificate as you're,
[16:32.730 --> 16:38.070]  like, you know, one of your main things that you build into your rules, like, you can easily change
[16:38.070 --> 16:42.790]  things like the certificate subject to evade, right? Some of the things that we've actually
[16:42.790 --> 16:49.370]  found a lot of use for is taking these rules and sort of trying to generalize them just a bit more
[16:49.370 --> 16:55.790]  to where they'll capture, like, a large number of variants of, you know, Empire something. So,
[16:55.790 --> 17:02.370]  not just specifically targeted to this particular certificate subject or structure, but, like,
[17:02.890 --> 17:08.550]  other variants as well for things that are easily modifiable by the attacker.
[17:11.890 --> 17:17.630]  Cool. I think a good point, kind of, about that is that you need to have a plan for how you're
[17:17.630 --> 17:20.310]  going to deal with the alerts. And that's kind of a key thing for
[17:21.370 --> 17:26.010]  anything, you know, you have to have a plan for what are you going to actually do with this model.
[17:26.390 --> 17:28.350]  Yeah, 100% agree with that.
[17:29.530 --> 17:35.570]  Yeah, I want to jump in really quick and give Ariel a chance to chime in if you've got anything.
[17:35.790 --> 17:41.950]  I don't have anything significantly different. I firmly agree with the necessity to have both
[17:41.950 --> 17:45.890]  skill sets of understanding security and also understanding the data science,
[17:46.530 --> 17:50.510]  especially when you, if you don't understand the underlying data of what it is you're working with,
[17:50.510 --> 17:53.350]  you're going to have a really hard time having any sort of alerts or whatever that you're trying
[17:53.350 --> 17:56.930]  to build that actually tells you anything useful. That's all I would add.
[18:00.550 --> 18:07.390]  Yeah, for sure. I think, so, we've kind of been maybe a little bit in the weeds of, like,
[18:07.390 --> 18:13.590]  different, yeah, like, you need to be really, like, focused on sort of the technical details of,
[18:13.590 --> 18:18.030]  how do I tune my anomaly detection system, or what do I know about sort of the security rules
[18:18.030 --> 18:25.550]  that I'm building behind this? I guess maybe part of what contributes to all of the hype is the fact
[18:25.550 --> 18:30.670]  that, like, this is, like, a really technical, really specialized discipline that not a lot of
[18:30.670 --> 18:37.590]  people understand. And so that sort of, A, makes it look like magic to people that maybe don't
[18:37.590 --> 18:42.450]  really understand it as well. And B, makes it really easy to oversell and to hype up because
[18:42.870 --> 18:46.450]  whoever's doing the marketing and the hyping can be pretty sure that the other person doesn't
[18:46.450 --> 18:51.750]  understand that. So, as people who are sort of, like, at the coalface, right, working, like,
[18:51.750 --> 18:59.730]  on these specific problems, what can we do to sort of, like, help communicate what the capabilities
[18:59.730 --> 19:04.290]  of these machine learning systems are and how can we sort of, like, demystify them and cut through
[19:04.290 --> 19:09.790]  the hype and make it give people, like, a good idea of, yes, they actually do do really good
[19:09.790 --> 19:14.130]  things and you can do really powerful, useful things with them, but no, they're not magic.
[19:14.690 --> 19:17.910]  Like, how do we sort of, like, bridge that communications gap?
[19:20.390 --> 19:27.250]  I'm going to say it's, I mean, InfoSec is ripe for it. Like, every year, like, the black hat
[19:27.250 --> 19:31.510]  vendor, you know, two years ago, it was all about application whitelisting. And now you don't hear
[19:31.510 --> 19:36.050]  about it anymore. All you hear about is machine learning. Maybe two years, it'll be something
[19:36.050 --> 19:42.970]  else. But I think it's, it's, and marketing is just, is just that. It's, you know, it's designed
[19:42.970 --> 19:48.250]  to make everything look better than it is. But I think in terms of practitioners, or, you know,
[19:48.250 --> 19:54.250]  if you're working with a company or at a company, it's, you know, saying, yeah, the hype is real,
[19:54.250 --> 19:59.510]  because this is, it can be really powerful, but it's, you're right, it's not going to do
[19:59.510 --> 20:06.010]  your dishwasher. It, you know, it can assist in the things you have, but it's the ultimate garbage
[20:06.010 --> 20:12.430]  in garbage out. So, you know, like, it's like the first, the first thing you find out when you
[20:12.430 --> 20:16.130]  install a SIM in your network is not anything security related, it's how much stuff is
[20:16.130 --> 20:22.050]  misconfigured. And I think the same is true for, for machine learning. It's like, once you put it
[20:22.050 --> 20:28.630]  in, it's going to take a little bit. But if you continue to work and feed and spend that time
[20:28.630 --> 20:33.850]  up front, like some of the best organizations, from the security perspective that we work with,
[20:35.070 --> 20:38.970]  are just ones that have just put the legwork in, you know, they've gone through and they've
[20:40.050 --> 20:44.410]  handpicked the DLLs that are allowed to run on every host, you know, and they haven't just
[20:44.410 --> 20:50.950]  plugged in a box, pat themselves on the back and, you know, gone got a beer. So I think it's like,
[20:50.950 --> 20:56.550]  doesn't matter what you're installing, I think the legwork needs to be done with it is obviously
[20:58.370 --> 21:02.890]  misconstrued that AI will solve your problem. Maybe we'll get there one day. But I think,
[21:02.890 --> 21:09.070]  you know, for now, it's still going to take some legwork, which is to be expected.
[21:11.390 --> 21:17.510]  I think some of the hype also comes from, I mean, we've interacted with the results of AI
[21:17.510 --> 21:26.190]  in different contexts, Netflix recommendations, Amazon recommendations. And as we think about how
[21:26.190 --> 21:29.630]  to do that for security, it's natural to say, you know what, just give me a dashboard with
[21:29.630 --> 21:34.330]  recommendations for what to do next. And it'll be AI powered. And it'll be just as disruptive as,
[21:34.330 --> 21:42.150]  you know, Netflix. And I think security professionals are much more skeptical of
[21:42.150 --> 21:46.310]  recommendations that they would get from that kind of dashboard than I as a consumer would be about
[21:46.310 --> 21:51.670]  what recommendations Amazon gives me for my clothing purchases. So I think one way I'm
[21:51.670 --> 21:57.310]  trying to think about this hype or talk about this hype is to just think about, like, our users
[21:57.310 --> 22:04.770]  are using the output of this AI very, very differently. I've heard it referred to as
[22:04.770 --> 22:11.110]  high stakes versus low stakes AI. In low stakes, you know, if my email is spam, I click it away.
[22:11.110 --> 22:16.710]  It's not a big deal. But in security, if our predictions are wrong, someone can be compromised
[22:16.710 --> 22:21.190]  or we could be taking someone's attention greatly away from the things they ought to be looking at
[22:21.190 --> 22:27.330]  instead. So they're intrinsically interacting with our work very differently than if we were,
[22:27.330 --> 22:32.730]  you know, correspondingly in e-commerce or something. And I feel like that's a conversation
[22:32.730 --> 22:37.050]  I'd like to have with folks in marketing or product management as they're trying to brainstorm
[22:38.370 --> 22:45.290]  how to talk about this publicly. I feel like the way that users use this is
[22:47.090 --> 22:49.410]  more specific to security, let's say.
[22:50.910 --> 22:55.990]  Actually, what's funny is there's two companies I can think of who their marketing departments
[22:55.990 --> 23:00.750]  talk about how they don't use ML in their security products. So in terms of, like,
[23:00.750 --> 23:05.950]  the skeptical and the skepticism that exists, these companies actually market that they don't use ML.
[23:07.690 --> 23:10.190]  How well does that work for them? Do you know?
[23:10.850 --> 23:12.430]  I guess they're still in business, so...
[23:12.430 --> 23:13.230]  Well, that's good.
[23:13.230 --> 23:21.850]  Okay. I do know one of them, and they are really, really solid. They are a solid company.
[23:22.950 --> 23:23.590]  That's cool.
[23:23.590 --> 23:25.070]  Or a solid product, I should say.
[23:25.650 --> 23:29.170]  Yeah, I do wonder when we're going to see companies going back to good old-fashioned
[23:29.590 --> 23:35.450]  AI. You could argue that signatures are just good old-fashioned AI, right? Rule-based.
[23:39.630 --> 23:44.970]  John, any thoughts on bridging the hype gap or bringing hype back to reality?
[23:45.770 --> 23:48.750]  Um, well, yeah, like...
[23:51.050 --> 23:57.730]  I don't really have much to add. I actually agree with both Will and Vasudha's comments
[23:57.730 --> 24:08.750]  that they've made. Like, it's... yeah. Marketing is marketing. It's really good to focus on sort
[24:08.750 --> 24:14.090]  of what we're actually able to do well. And I think one of the things that we data scientists
[24:14.090 --> 24:21.210]  can actually do a lot better on is communicating failures. And I have started to see, you know,
[24:22.130 --> 24:28.450]  conferences like Kamla's, as well as others, sort of welcome, sort of, hey,
[24:28.450 --> 24:33.610]  we tried this thing. It didn't work. Here's what we did. Don't do what we did, basically,
[24:34.210 --> 24:40.010]  sort of talks. And I think sort of focusing on, even as data scientists, being able to very clearly
[24:41.810 --> 24:46.010]  articulate what we were able to do, what we weren't able to do.
[24:46.390 --> 24:50.250]  I think that's actually very important for bridging the gap long term.
[24:55.190 --> 25:02.430]  I definitely agree with that. I think that given that a lot of the data science that we see in
[25:02.430 --> 25:06.630]  InfoSec kind of comes from academia, it sort of ends up with a lot of the same incentives that
[25:06.630 --> 25:11.050]  academia has for publishing stuff that you want it to look good. You don't really want to share
[25:11.050 --> 25:16.470]  your failures because you don't really get any sort of, you don't get anything out of that as an
[25:16.470 --> 25:22.710]  academic, unfortunately. And I think sort of taking a lot of this security data science out of that
[25:22.710 --> 25:26.950]  context and putting it more in, like, here's some practical stuff. Let's talk about what works and
[25:26.950 --> 25:31.030]  what doesn't work is exactly kind of the direction we should be taking things.
[25:34.890 --> 25:39.630]  Cool. Yeah, I want to loop back a little bit to something Vasudha said a second ago,
[25:39.630 --> 25:47.330]  where she referred to high stakes and low stakes decisions that are made. So yeah, I think it's
[25:47.330 --> 25:52.090]  obvious to me at least that a lot of the decisions that get made when you're talking about using
[25:52.090 --> 25:58.990]  ML or artificial intelligence or whatever you want to call it this week for InfoSec and security in
[25:58.990 --> 26:05.190]  general, those tend to be higher stakes decisions. And one of the things that I've seen people push
[26:05.270 --> 26:12.470]  a lot more for in recent past maybe two years or so is when you do have high stakes decisions,
[26:12.470 --> 26:17.810]  having some notion of explainability or interpretability that comes along with those
[26:17.810 --> 26:25.790]  models. So I kind of wonder, and this is partly driven by my own experience using ML to detect
[26:25.790 --> 26:33.850]  malware, those models tend to be almost impossible to explain. So how big of a, how important is this
[26:33.850 --> 26:38.750]  do you think to be able to explain the decisions that a model makes, be it in a detection,
[26:38.850 --> 26:44.890]  a classification, or an anomaly detection context? And how can we get from where we are now,
[26:44.890 --> 26:49.510]  where everything's about gradient boosted decision trees, or big complicated neural networks
[26:49.510 --> 26:56.150]  that are almost impenetrable, to something that's maybe more explicable and more directly, something
[26:56.150 --> 27:04.140]  you can directly interrogate? I think it's everything. I think it's worth sacrificing a
[27:04.140 --> 27:11.860]  little bit of accuracy even, to get a simpler model that pulls out future importances, or that
[27:11.860 --> 27:18.620]  you can easily explain what feature in your data led to this prediction, or led to the
[27:18.620 --> 27:27.420]  the presentation of this particular record for review. I also think, this is my opinion only,
[27:27.420 --> 27:31.760]  I think human-in-the-loop is super important, and that if it's a high stakes decision,
[27:34.260 --> 27:41.220]  it's much better received as an augmentation that an analyst can take into account, rather than
[27:41.220 --> 27:46.060]  being something that would be fully automated and trusted to make a decision on its own.
[27:48.670 --> 27:55.170]  Yeah, I 100% agree with that. Like a lot, so especially like if you're sort of starting
[27:56.010 --> 28:00.350]  a data science modeling, like a branch in detection or response or something,
[28:00.350 --> 28:06.970]  your analysts aren't going to trust your data science models at all, right? So really,
[28:07.550 --> 28:14.330]  like starting with something that's very clear and obvious, like, you know, this machine has,
[28:14.330 --> 28:18.810]  is supposed to be fired walled off from the rest of the network, and it's uploading hundreds of
[28:18.810 --> 28:23.410]  thousands of gigabytes, you know, to the outside world, like those sorts of things are going to
[28:23.410 --> 28:30.730]  start, you know, building the trust as you go. I'm also a really, really big proponent of the
[28:30.730 --> 28:36.870]  Google rules of ML that they do. And one of the things that they actually really, really recommend
[28:36.870 --> 28:40.530]  is starting with, you know, whenever you're building like a product or a machine learning
[28:40.530 --> 28:46.010]  based system, starting with something that's easily baselineable. And then that allows you
[28:46.010 --> 28:51.770]  to actually improve upon it and actually be able to measure like how much you've improved. When you
[28:51.770 --> 28:57.290]  have sort of these black box models, you lose a lot in sort of as you're trying to improve the
[28:57.290 --> 29:02.410]  model. You spend a lot of effort trying to actually figure out whether you've actually
[29:02.410 --> 29:10.190]  improved or how you can actually improve it further. I think organizationally, like buy-in is
[29:10.190 --> 29:18.110]  pretty huge. So as explainable as you can make it. And you know, like analysts, security analysts,
[29:18.110 --> 29:22.110]  you know, routiners are like, they're naturally curious. So if you're showing them something that
[29:22.110 --> 29:27.450]  is, how did you get this, like, they're gonna, they're gonna go and look at it and figure out
[29:27.450 --> 29:32.490]  and they, you know, even if you're right, if it's not as accurate, or it's not as effective, they'll
[29:32.490 --> 29:39.690]  figure out how to, to make it that way, or they'll at least figure out, you know, what inputs, what,
[29:39.690 --> 29:44.890]  what inputs require what outputs, and then you can start making it a little more
[29:46.490 --> 29:51.730]  complicated or sophisticated or whatever. I think we often, you know, it's like we often
[29:52.330 --> 29:58.590]  are misguided in the fact that we think complexity is sophistication. When in reality,
[29:58.590 --> 30:04.430]  I think a lot of times simplicity is, is actually sophistication. It's much cleaner.
[30:05.630 --> 30:10.590]  So I think we're kind of falling into the trap that people tend to fall into here, where we're
[30:10.590 --> 30:16.990]  thinking that there has to be one silver bullet system that generates everything you need to know
[30:16.990 --> 30:22.350]  to make a decision. In a lot of cases, I don't think explainability is necessarily that important,
[30:23.570 --> 30:27.770]  as long as you can go grab more information around whatever you're looking at
[30:27.770 --> 30:32.050]  to do a deeper analytic and provide more information to the analyst. Like if you've
[30:32.050 --> 30:37.170]  got a binary, there's no reason that you have to rely solely on the machine learning model to make
[30:37.170 --> 30:42.470]  the decision of whether it's good and bad. But you can use that as a tipper to do some more
[30:42.470 --> 30:46.230]  in-depth analytics on the binary. You've got, you know, start running, throwing it through some
[30:46.830 --> 30:52.770]  sandboxes, you know, do more expensive computation that you maybe couldn't, you can't do if you,
[30:52.770 --> 30:56.270]  you may not be able to do it for every single binary, but you know, you could do it for a few
[30:56.270 --> 31:02.110]  hundred an hour. You know, you know, in most other cases, there's additional context that
[31:02.110 --> 31:07.470]  you could automatically collect and provide to the analyst to make their decision on whether
[31:07.470 --> 31:11.250]  they should do something with the alert or not, rather than just relying on whatever the output
[31:11.250 --> 31:16.170]  of the model is. And that's just a good point. I think, like, I know Microsoft, the Windows Defender
[31:16.170 --> 31:21.030]  does that. So anything overtly malicious gets blocked on the host and anything that they're
[31:21.030 --> 31:29.500]  unsure about gets sent to a cloud. I think that sometimes explainability can be a little bit of
[31:29.500 --> 31:33.740]  red herring. In general, I think that it's good. I mean, for all the reasons stated above about
[31:33.740 --> 31:40.480]  building trust and, and having baselines for understanding how you can improve, but when you
[31:40.480 --> 31:45.320]  have, when you have like a human judge making a decision about whether or not someone's guilty,
[31:45.320 --> 31:49.080]  you're, there's no part of the process whereby you go back to the judge and you say, what are
[31:49.080 --> 31:54.480]  the pieces of evidence that convinced you that this person is guilty? And sometimes they can
[31:54.480 --> 31:58.620]  explain why and sometimes they can't, but humans are very good about trying to come up with stories
[31:58.620 --> 32:03.620]  to explain what the decisions that we make. So to some extent, I kind of view it as a little bit
[32:03.740 --> 32:14.260]  I don't know, naive to try to come up with like purely explainable reasons for why things are the
[32:14.260 --> 32:17.820]  way they are. Because oftentimes you can't really come up with like a true reason for why something
[32:17.820 --> 32:31.420]  is the way that it is. Cool. So I have kind of a twofer that I want to, I guess, point at specifically
[32:31.420 --> 32:40.160]  at Ariel and Will for initial responses. So the first bit is when we implement ML into these
[32:40.160 --> 32:46.140]  security systems, right? We bake it in, we make it a big, sometimes a critical part. And this is
[32:46.140 --> 32:53.340]  for Ariel, are we kind of opening up new vulnerabilities or new risks that we need
[32:53.340 --> 33:00.840]  to be worried about, right? Is it, anytime you add a new component, you at least change the attack
[33:00.840 --> 33:07.720]  surface. So do we really know quite what we're getting into as far as introducing new elements
[33:07.720 --> 33:13.340]  to an attack surface when we start putting ML into important decision-making points
[33:13.340 --> 33:17.920]  in these security systems? And then I'll come back to you, Will, for the second bit in a sec.
[33:19.280 --> 33:25.540]  My answer would be yes. But I think that a lot of that can be mitigated by making sure that you
[33:25.540 --> 33:31.300]  still have human in the loop. I kind of view a lot of these AI tools as sort of like centaur
[33:31.300 --> 33:37.300]  chess where you have an AI system that assists a human. I don't really anticipate there being a
[33:37.300 --> 33:41.400]  situation which you'd want a system to fully be automated to do everything because you're always
[33:41.400 --> 33:47.220]  going to run into issues. And like we discussed earlier about interpretability, if you don't have
[33:47.220 --> 33:51.140]  ways of understanding what's going on or like baselines or anything, you're going to be kind of
[33:51.140 --> 33:56.060]  for a lot of this stuff. That said, there has been a lot of good research around trying to
[33:56.060 --> 34:00.680]  understand what these potential vulnerabilities are, but they're all, at least for the most part,
[34:00.680 --> 34:04.040]  most of them have been developed inside laboratory conditions, which makes them very hard to
[34:04.040 --> 34:10.860]  translate it into the real world. Yeah, yeah. There's a lot of messed up stuff that's going
[34:10.860 --> 34:14.760]  to go down, but I don't think it's going to be as bad as maybe some people think it will be.
[34:15.880 --> 34:20.080]  I don't know. We are kind of clear on how poorly machine learning works for a lot of things and
[34:20.080 --> 34:24.600]  how well it works for other things. And I think there's a decent amount of healthy skepticism
[34:24.600 --> 34:29.620]  about machine learning at this point to sort of be some level of protection for us as we go forward
[34:29.620 --> 34:39.380]  adding it to more things. Okay, cool. Thanks. And then I'm going to kick the second part over to
[34:39.380 --> 34:45.000]  Will, and then I'd kind of like to hear from everyone else on the same topics. So do you see
[34:45.260 --> 34:50.960]  a risk when we make models interpretable or explainable and we say, okay, I think this thing
[34:50.960 --> 35:00.960]  was bad because of X. Do you think that that is making them necessarily easier to evade or bypass,
[35:00.960 --> 35:04.960]  or is that a controllable risk, or does it not really matter to you? That doesn't really matter
[35:04.960 --> 35:10.640]  to me. So I think when you have to sell something, making it explainable is nice.
[35:11.200 --> 35:17.740]  Like that's what you're trying to do. You need to make it sellable, but I don't care if I'm using a
[35:17.740 --> 35:22.240]  I don't care. I don't have to explain to anybody. I'm not trying to sell it to somebody,
[35:22.680 --> 35:27.800]  right? But if you want ML to take hold in an organization, you're trying to sell a product
[35:28.260 --> 35:32.680]  and no one's going to trust it, then you need to make it explainable. But for everybody else,
[35:32.680 --> 35:38.840]  it doesn't matter. Like it can be as black box as you want. And I think that's an important
[35:38.840 --> 35:46.480]  distinction. So it's like the... it needs to be explainable if you're trying to sell it.
[35:47.740 --> 35:53.600]  You know, but even internally, it can be useful. An explainable model is more useful.
[35:53.620 --> 35:59.580]  I don't necessarily think it's easier to attack or what. And it would be difficult to
[36:00.500 --> 36:05.840]  assume like from an attacker's perspective, like I generally always assume that it's going to be
[36:05.940 --> 36:10.200]  a black box and that you're not going to get 100% of the model, but you might be able to get some
[36:10.200 --> 36:16.000]  sort of, you know, some copycat, something that helps you explain the model that you're looking
[36:16.000 --> 36:28.120]  at. So yeah, explainability, my opinion is more about making it palatable to deploy into an
[36:28.120 --> 36:34.240]  organization. Okay. Yeah. So I find it... it's interesting because there's, I think,
[36:34.240 --> 36:38.000]  two different threads on the explainability that people are picking up on. First is
[36:38.800 --> 36:45.440]  building trust to be able to get people to adopt the models. And second is reducing risk by giving
[36:45.440 --> 36:54.100]  analysts who are seeing these alerts more context to kind of work with them. So, I mean,
[36:54.100 --> 37:01.980]  do those seem like those are... I guess what I'm saying is like pitching it as like we're
[37:01.980 --> 37:07.760]  building trust almost seems it's a little bit cynical, like a sales job, whereas we need to
[37:07.760 --> 37:11.600]  be able to explain it because these are sort of critical, maybe even life and death decisions,
[37:11.600 --> 37:17.900]  seems like a much higher ethical bar. So where does the balance between those two fall? Or are
[37:17.900 --> 37:26.140]  they necessarily even in conflict, right? I would add a third column to that. I would say
[37:27.740 --> 37:32.860]  putting aside the issue of having to convince someone else to trust these results for my own
[37:32.860 --> 37:39.540]  quality control, I look to explainability to convince myself that I haven't overfit really
[37:39.540 --> 37:45.800]  badly or that I'm not worried about data or that I haven't had data leakage happen or something. So
[37:45.800 --> 37:51.940]  just going back and evaluating why the model made the decision that it did gives me better confidence
[37:51.940 --> 38:01.060]  that it's picking up on real signal and not something spurious. Yeah, I also think explainability
[38:01.060 --> 38:06.560]  also depends on how much domain expertise the person that's consuming the results of the model
[38:06.560 --> 38:15.260]  has. Yeah, that's true. I'm always going to come back to the malware because that's
[38:15.260 --> 38:19.160]  what has the most research on it, but I mean, if you've got a malware analyst
[38:19.680 --> 38:27.280]  looking at a binary that he was delivered by a neural network said that, hey, this is
[38:27.280 --> 38:32.340]  malicious, that malware analyst is going to make a decision based on their
[38:32.340 --> 38:38.800]  own domain expertise, regardless of what the model says it made its decision on.
[38:38.800 --> 38:41.900]  I think that's the case for a lot of other things as well.
[38:43.320 --> 38:50.760]  But that leads us to the problem of the people that get the most value out of the explainability
[38:50.760 --> 38:58.120]  are also the ones that are least qualified to make the decision in the first place.
[39:00.200 --> 39:05.260]  Okay, so I'm going to reframe that a little bit more positively and say maybe it's a way
[39:05.260 --> 39:12.200]  to maybe help turn less experienced analysts and to help them perform like more experienced
[39:12.200 --> 39:19.480]  analysts maybe, hopefully. Or is that too optimistic? I don't think it's optimistic,
[39:19.480 --> 39:24.460]  but you still have the chicken and the egg problem of they have to understand the results
[39:24.460 --> 39:30.560]  enough. And if they have that understanding of the domain, they may not derive as much value
[39:30.560 --> 39:36.600]  from the explainability. Yeah, actually, I'm gonna there's a product called Artemis by Endgame.
[39:36.600 --> 39:42.380]  And it's like a talk to your sim product. And it's, you know, they were like, this is everything
[39:42.380 --> 39:47.120]  machine learning that you ever wanted. I don't know what their marketing is. But it's basically
[39:47.120 --> 39:53.280]  they haven't, they've only made information retrieval easier, but they it still requires
[39:53.280 --> 39:59.380]  an analyst that knows what they need to be looking for. So it actually, we already had a
[39:59.380 --> 40:03.940]  language to retrieve information from a sim, you know, whatever, OS query, whatever it might be,
[40:03.940 --> 40:09.000]  we didn't need machine learning to do that. What we needed machine learning to do is potentially
[40:09.000 --> 40:15.360]  add some context around a particular data point, not to say like, hey, you know,
[40:15.460 --> 40:17.800]  like a chatbot on top of your sim, we didn't need that.
[40:19.960 --> 40:26.800]  Um, I, pardon me if I'm wrong, or misunderstanding, Rob, but I thought your point was more that
[40:26.800 --> 40:33.320]  basically, the domain expert is going to know what features basically would be most helpful
[40:33.320 --> 40:40.260]  in determining which whether this binary was malicious or benign. And so it's kind of not the
[40:40.260 --> 40:46.300]  right thing to put that on the data science to try to, you know, to be the ones to actually explain
[40:46.300 --> 40:52.220]  why the model chose a particular, you know, feature or something in its decision. Am I correct
[40:52.220 --> 40:59.380]  on that or? Partly, what I'm saying is that the model, the models explainability isn't necessarily
[40:59.380 --> 41:06.540]  going to be helpful for the people that need it. You know, it can maybe say this, this particular
[41:06.540 --> 41:12.600]  segment of the binary is what I thought is bad. But then you need the expertise to know whether
[41:12.600 --> 41:18.100]  that means anything, you know, it could just be a packer, it could just be, you know, this
[41:18.100 --> 41:23.760]  particular thing is using get proc address to load things for non malicious reasons. And there's,
[41:24.640 --> 41:31.800]  there's a lot of context around things where, if you already know it, then you don't need the AI
[41:31.800 --> 41:37.040]  to tell you that this is why this is bad. And if you need the AI to tell you that this is bad,
[41:37.040 --> 41:40.540]  you may not understand it well enough to know whether the AI is wrong or not.
[41:44.070 --> 41:50.150]  Yeah, sensing a lot of very healthy skepticism of ML from all quarters here, which warms my
[41:50.150 --> 41:57.050]  dark little heart. So one more, one more question to kind of carry us out to the top of the hour.
[41:57.510 --> 42:02.990]  So we've touched on anomaly detection. I think that's people, I think there's a little bit of
[42:03.090 --> 42:07.990]  a network focus here. So that's what a lot of people are sort of talking about as a main tool.
[42:07.990 --> 42:12.970]  There's also binary classification for things like static file artifacts, be it malware detection or
[42:12.970 --> 42:23.090]  maldoct detection. So these are sort of maybe the, among the simpler forms of ML, right? And it
[42:23.090 --> 42:29.250]  doesn't get into things like, it hasn't like, we haven't seen, as far as I know, any serious,
[42:29.250 --> 42:37.210]  like, successes as far as using more complicated forms like reinforcement learning or, you know,
[42:37.210 --> 42:41.810]  like active, active learning with like really good human-in-the-loop integrations in the
[42:41.810 --> 42:49.430]  InfoSec space yet. So just sort of among the vast menu of new, of existing techniques that
[42:49.430 --> 42:55.990]  really haven't made a dent in InfoSec, where do you think, where do people think that the field
[42:55.990 --> 43:00.270]  is going next? Like what's going to be the next, the next big breakthrough as far as applying
[43:00.270 --> 43:08.880]  machine learning to security problems? I think that reinforcement learning is
[43:08.880 --> 43:13.100]  coming. I think it's going to have its moment at some point, but I think that that's going to
[43:13.100 --> 43:16.800]  depend very heavily on the kind of data that we can get into these kind of systems.
[43:17.280 --> 43:21.620]  I've seen a lot of people trying to use it for adversarial simulation, but haven't really
[43:21.620 --> 43:27.400]  had good data that really allows it to do a good job that's better than what we already
[43:27.400 --> 43:32.380]  have with Cobalt Strike and some of those other things. But I have seen some very convincing
[43:32.380 --> 43:40.360]  sort of research in the RL space around trying to use RL for like code generation kind of stuff.
[43:40.360 --> 43:45.060]  And that gives me some, I guess, I don't know if it's hope because there's also like a negative
[43:45.060 --> 43:50.580]  side to any sort of adversary simulation using RL. But I think it'll definitely be interesting.
[43:51.160 --> 43:56.160]  Yeah. If it is a good adversarial simulation, it's also probably a decent adversary. Yeah.
[43:56.160 --> 44:01.480]  I think reinforcement learning is the next, like there's a lot of attack simulation
[44:02.200 --> 44:08.060]  stuff that's being talked about. I know MITRE has some talks where they're discussing
[44:09.600 --> 44:13.780]  reinforcement learning for attack simulation. I think for actual attackers, the difficult part
[44:15.460 --> 44:21.380]  is you can't get on a network and learn the network and then attack it, right? You need
[44:21.380 --> 44:26.260]  to be learning from all of your general experience. But I think from a defensive
[44:26.260 --> 44:29.740]  standpoint, reinforcement learning could be, you know, because you can iterate through your network
[44:29.740 --> 44:38.300]  infinitely, but it's just matching what is useful. I think, I hate,
[44:38.300 --> 44:47.240]  I shouldn't say that. Adversary simulation is a bit of a, it's a bit of a buzzword.
[44:47.540 --> 44:54.560]  I think it's not a buzzword. How do I, it's like you, from the offensive side for the last three
[44:54.560 --> 44:58.380]  years, they've just been dropping techniques left and right. And so it's like, in some ways
[44:58.380 --> 45:03.600]  we call it adversary simulation because the techniques that we dropped are getting picked up
[45:03.600 --> 45:11.400]  by adversaries. And so it's like, we're emulating ourselves via a proxy. But it's because we wanted
[45:11.400 --> 45:17.180]  to do the research, but it hasn't been really helpful. But if I think of the value, you know,
[45:17.180 --> 45:22.800]  that the public research has generated in the defensive space, it's insane. You know, like
[45:22.800 --> 45:30.240]  there are entire products that are based off a GitHub project. So, you know, you can't say it's
[45:32.680 --> 45:37.280]  terrible, but I think the adversary simulation piece is a bit overdone. I wish organizations
[45:37.280 --> 45:43.280]  would spend more time in their own networks, just figuring out where their data is and doing some
[45:43.280 --> 45:50.520]  of the simple stuff like network segmentation and less time focusing on, you know, whatever
[45:50.520 --> 45:56.320]  APT is they think is going to come after them next. I think that's probably more useful.
[45:56.500 --> 45:59.180]  But reinforcement learning would be my answer.
[46:00.560 --> 46:06.480]  Yeah, I'm fairly skeptical of reinforcement learning, just because the problems in this
[46:06.480 --> 46:12.160]  space don't lend themselves to easily defining a target for the learning algorithm to
[46:12.960 --> 46:17.960]  test itself against. You know, from the defensive side, it's really hard to say,
[46:17.960 --> 46:25.880]  you know, we can't easily define what secure means in any meaningful sense to really be able to
[46:25.880 --> 46:31.140]  teach an agent, whether it's in a secure state or not, to determine, you know, what actions it
[46:31.140 --> 46:35.680]  should be taking. From the attack perspective, you know, like you said, the attacker can't just
[46:35.680 --> 46:42.580]  throw a agent on to go learn the network, because that's going to be super noisy. And, you know,
[46:42.580 --> 46:47.580]  hopefully the defenders will notice that that's going on as it's probing everything and everything
[46:47.580 --> 46:54.500]  so my answer to what's the next big thing is, I don't think there's going to be a next big thing.
[46:54.960 --> 47:02.000]  I think the real positive impact or the real kind of game changing impact is going to be
[47:03.360 --> 47:09.180]  smaller, you know, people applying machine learning to solve small problems tactically
[47:11.220 --> 47:15.300]  in a lot more places, you know, it's going to be the sock analyst who sits down and just
[47:15.300 --> 47:20.880]  makes a naive Bayes classifier to triage the AV alerts they've got, you know, it's going to be
[47:20.880 --> 47:26.060]  the small projects like that, widely distributed that are going to really make the difference.
[47:27.600 --> 47:32.200]  Okay, so we've got a couple votes for reinforcement learning, one vote for
[47:32.200 --> 47:38.540]  mass democratization of machine learning. Vasudha, you gotta pick for us.
[47:38.540 --> 47:44.880]  I'll pile on to the small projects done with ML. But I will add on to that.
[47:45.440 --> 47:50.720]  I'm really excited about graph theoretic approaches for that. Because I think as we
[47:50.720 --> 47:56.760]  think about very like, maybe making classifications on very specific endpoints, that that's a machine
[47:56.760 --> 48:02.300]  learning problem that I think we can do well. But to go from that to bridge from that to
[48:02.300 --> 48:07.060]  talking about an intrusion means understanding network effects,
[48:07.060 --> 48:14.740]  adding in other data sets to endpoint logs, it's not possible to do that using local,
[48:14.740 --> 48:20.980]  like single node properties. And so I'm, I'm kind of waiting to see what the
[48:20.980 --> 48:24.160]  what the network graph picture will do for us.
[48:26.160 --> 48:32.280]  Very cool. So I know we're kind of at the top of the hour, but we started a little bit late.
[48:32.280 --> 48:37.000]  Can people stay for another couple minutes? I want to try and take a couple of questions
[48:37.000 --> 48:43.720]  from the Discord, if that's okay. Awesome. Cool. I'm just gonna, so if I don't call your
[48:43.720 --> 48:47.160]  question out, please don't be offended. I'm just kind of scrolling through.
[48:47.900 --> 48:54.080]  So the first one that jumps out at me, FS Montenegro says, my perennial question is,
[48:54.080 --> 49:00.020]  what are the questions and practices that we should use to probe claims of a vendor's ML quality?
[49:00.020 --> 49:06.780]  I particularly like that one. Anyone want to take a stab? Yeah. How do you call BS?
[49:12.250 --> 49:19.390]  I'm going to put... I guess, I guess I can start right. So there's, there's a lot of different,
[49:19.390 --> 49:26.390]  you know, pitfalls in, you know, data science and, and performing machine learning. And one
[49:26.390 --> 49:30.250]  of the things I would like to see more of from, you know, skeptics or people trying
[49:30.250 --> 49:35.230]  to evaluate these models is just the simple questions like, how did you split your train
[49:35.230 --> 49:40.770]  and test? Like, did you, did you, you know, consider data leakage in sort of the time
[49:40.770 --> 49:45.850]  domain, right? Like, are you learning from future samples in your past samples? Those sorts of
[49:45.850 --> 49:52.250]  questions. I don't have much experience on the actual buying product side though. So I probably,
[49:52.250 --> 49:56.390]  that that's about as far as my, my opinion on the subject.
[49:58.690 --> 49:59.830]  I'm going to go with...
[50:04.170 --> 50:05.830]  Let's go with Will then Rob.
[50:05.830 --> 50:10.250]  Oh, sorry. I was just going to say, you know, from an attacker's perspective,
[50:10.250 --> 50:14.510]  these things are being deployed, but they haven't really been tested. So I think
[50:15.950 --> 50:19.450]  attackers are ultimately going to be the ones that decide whether or not
[50:19.450 --> 50:25.970]  ML is a viable solution for network defense. And we just, from an attack perspective,
[50:25.970 --> 50:31.590]  there's just not many people looking at it currently. So I would say that's probably
[50:32.710 --> 50:37.150]  a good way. Eventually it'll just happen, right? Attackers will have to adapt. And
[50:37.150 --> 50:42.390]  so I think that's going to be a good, a good litmus test for it.
[50:45.450 --> 50:46.410]  Rob?
[50:47.390 --> 50:52.210]  One of the first things I look at when I'm looking at a ML product is whether the data
[50:52.210 --> 51:00.310]  that they are purporting to operate on is even reasonably possible to make a decision on.
[51:00.510 --> 51:03.770]  And a lot of the products out there are saying, you know, we're going to look at an email,
[51:03.770 --> 51:07.850]  we're going to look at a phishing email and, you know, we'll predict what the person had
[51:07.850 --> 51:11.510]  for breakfast and, you know, all kinds of other claims that just don't really follow
[51:11.510 --> 51:16.830]  from the data that they have available. So I think, you know, to me, that's really one
[51:16.830 --> 51:20.610]  of the big things I look at when I'm evaluating an ML system.
[51:24.370 --> 51:26.510]  Cool. Good answers. Anyone else?
[51:28.730 --> 51:32.690]  Plus one for Rob's answer. I think the data tells all, really.
[51:32.910 --> 51:39.790]  Yeah. Okay, cool. From... I'm going to get your name wrong, I'm sorry. Scudzi,
[51:39.790 --> 51:43.510]  there's a question. Is there any focus in these vendor tools that provide good
[51:44.250 --> 51:46.870]  introspection into how the models are making inferences? Yeah. So
[51:46.870 --> 51:52.490]  do we have explainable ML for security out there? Does anyone know of anything offhand?
[51:58.250 --> 52:03.210]  I hesitate to... It kind of depends on what you mean by explainable.
[52:05.230 --> 52:07.710]  Okay, that's... Yes.
[52:07.870 --> 52:10.230]  If you look at the...
[52:13.270 --> 52:17.590]  I hate it when I forget the name of the paper I was an author on. But
[52:20.870 --> 52:25.910]  eating binary is one that I did with Ed Raff. You know, one of the big things we looked at
[52:25.910 --> 52:33.390]  for the explainability was we took a look at what parts of the binary was the model keying in on,
[52:33.390 --> 52:37.310]  you know, for making its decision. You know, I think there's been
[52:37.310 --> 52:42.930]  some work in things like that, but not necessarily explainability in terms that
[52:44.250 --> 52:48.190]  we can write an English sentence that will tell you why the model
[52:48.190 --> 52:52.810]  made its decisions and the things that the model has learned.
[52:54.490 --> 52:59.930]  Yeah, I mean, I know there are a number of companies that are beginning to push...
[53:00.690 --> 53:05.850]  I think explainability is probably a bridge too far, but at least they're working on enriching
[53:05.850 --> 53:11.490]  context, right? I contractually oblige to mention that Sophos has done a lot of research in that
[53:11.490 --> 53:19.230]  area. But yeah, I agree. I don't think there's anything that is going for specific explainability.
[53:20.930 --> 53:27.730]  I'm browsing through Discord. Okay. One dad pants revolution. Awesome. Would like to hear
[53:27.730 --> 53:32.890]  about strategies for optimizing the relationship between rules, analysts, and ML systems operating
[53:32.890 --> 53:37.370]  on the same domain. I'm going to take that and I'm going to spin that a little bit and say,
[53:37.370 --> 53:41.690]  like, maybe how do we... maybe that's like an internal communication problem,
[53:41.690 --> 53:48.490]  so less of a hype issue and more of a how do you go about getting analysts comfortable with
[53:48.490 --> 53:55.330]  thinking about sort of the statistical way that these machine learning tools operate? So how can
[53:55.330 --> 54:00.930]  we effectively skill up analysts in using these tools and understanding what they mean and not
[54:00.930 --> 54:10.480]  just sort of blindly trusting the result? So I think a lot of people are already
[54:10.480 --> 54:14.720]  intuitively doing that when they're doing their work. A lot of analysts are building
[54:14.720 --> 54:21.660]  up their own internal models of what's likely to be good and what isn't. I think if we can show
[54:22.880 --> 54:26.420]  how a model, you know, how an ML model kind of does the same thing
[54:27.260 --> 54:32.160]  and show, you know, that in a lot of cases the model's learning the same things that you would
[54:32.160 --> 54:39.680]  think that would be useful for an analyst, I think that goes a long way to bridging the two.
[54:39.820 --> 54:42.520]  I knew we'd get you on the explainability train somehow.
[54:47.120 --> 54:48.300]  Anyone else?
[54:48.300 --> 54:51.520]  I think, sorry John, go ahead.
[54:51.880 --> 54:57.300]  Oh yeah, I was just going to say a lot of the like basic building blocks for ML, so like things like
[54:57.300 --> 55:05.400]  historical data or like intermediate representations and transformations of data are things that
[55:05.400 --> 55:11.340]  analysts wish they had available to them but like can't because, you know, maybe like a spunk is too
[55:11.340 --> 55:19.680]  slow or something like that, right? And so I think sort of also allowing these intermediate
[55:19.680 --> 55:26.340]  representations and stuff to people so they can use them, whether it's in the form of like a
[55:26.340 --> 55:32.880]  querying engine or something like that, is something we've had a lot of good feedback
[55:32.880 --> 55:40.680]  it and help with and help bridge that gap. And I think it's important that data science
[55:40.680 --> 55:45.600]  doesn't have to be done only in the data science team because I think a lot of the analysts,
[55:45.600 --> 55:52.640]  like Rob was mentioning, have these mental models in their head already and the conversations
[55:52.640 --> 55:57.000]  are fun when we can say, you know, there's a little bit of terminology we have to add,
[55:57.000 --> 56:01.400]  these are called inputs, you have a prediction label, just articulate that very precisely and
[56:01.400 --> 56:05.500]  then boom, that's all it is. The model is just an equation that takes you from, you know, input to
[56:05.500 --> 56:13.860]  output. And I think it helps to say, you know, go off and try that on your own without the data
[56:13.860 --> 56:21.720]  science team in the middle at all. And that we're also here to advise and maybe help out with better
[56:21.720 --> 56:27.960]  inputs, data cleaning, that sort of stuff. But by no means does it have to be, you know, relegated
[56:27.960 --> 56:36.340]  to like the data science tower. Yeah, for sure. Integrating, like,
[56:36.340 --> 56:42.960]  getting everyone kind of back to what Rob was saying about the mass democratization of data
[56:42.960 --> 56:47.220]  science and machine learning, getting it integrated across the organization I think is a huge win
[56:47.780 --> 56:52.420]  for the organization. Sort of giving everyone a little, just like a little baseline of maybe
[56:52.420 --> 56:58.940]  statistical literacy and understanding how to use these tools. Ariel, any thoughts on?
[57:01.100 --> 57:04.740]  No, I don't have any thoughts on this. Sorry, not trying to put you on the spot.
[57:04.740 --> 57:11.180]  It's all good. It's all good. Okay. Cool. So, that looks like the bulk
[57:11.180 --> 57:16.940]  of the questions from Discord. I really am sorry if you had a question in Discord and I missed it.
[57:16.940 --> 57:23.240]  Please feel free to, like, do it again and maybe at me and I'll see if we can wrangle some responses
[57:23.240 --> 57:28.880]  in the Discord chat. But I want to thank all you guys for being here and doing this panel with me.
[57:28.880 --> 57:34.500]  I had a lot of fun once we got past the inevitable technical problems. I hope you did, too.
[57:34.740 --> 57:37.760]  And thanks so much for a great conversation. I really appreciate your time.
[57:38.380 --> 57:40.380]  Thanks, Rich. Thanks, everybody. Cool. Thanks.
[57:40.380 --> 57:43.080]  Thanks, Rich. Thanks, everyone. This was awesome.
[57:43.940 --> 57:45.120]  Ciao. Cheers.