start	end	text
1470	10050	Hello everyone and welcome to our next talk which is mine. To introduce myself, which is wrong,
10050	16490	I did a postdoc in machine learning after getting my PhD in algebraic topology with some of the
16490	22450	people who did some of the math behind the concepts in this talk and I've just extended
22450	30810	it for this thing. So that's enough about me. Let me play my talk back for you.
32490	34530	So, here we go.
34530	39830	I welcome everyone to my talk on calculating concepts very quickly with this library called
39830	61730	Google which is based on... I hear no audio.
65790	66930	Oopsie daisy.
88690	91430	Um, there's no... do you hear audio from the talk?
98900	101260	Okay, well that's the problem I'm trying to...
122030	124050	Uh, sorry about this guys.
142810	147950	Uh, sorry, I don't know what's going on.
198250	203590	Um, I am... I'll apologize.
214220	219140	Key takeaway for this talk is this is a novel way of calculating drift.
219980	224440	Welcome everyone to my talk on calculating concept drift very quickly with this library
224440	229400	called Google which is based on cover trees. My name is Sven Kittel. I am a senior data
229400	233940	scientist at Elastic and my Twitter handle is co-mathematician.
234480	241660	So, the key outline... key takeaway for this talk is this is a novel way of calculating drift.
241660	245970	It's done using a new technique using cover trees.
246800	253600	And because it's so fast, it's log-in time to actually calculate drift for a new sample.
253600	259460	We can use it for on... in new ways that concept drift couldn't have been applied before now.
259660	262560	So, we're going to go over the objective, why we care about concept drift,
262560	268360	mathematically how it's done, then a couple results, and then finishing up with next steps.
268860	273780	So, why do we care about data set drifts? Well, our data drifts.
275300	281040	So, I work with malware and there are new malware families coming out every month.
281040	285300	There are new benign pieces, examples of software coming out every month.
285300	289580	Every time that Adobe patches Photoshop, it changes the pattern slightly and it changes
289580	294980	its location slightly in our data set. Every single time a new piece of malware comes up,
294980	300880	it's a new... it changes the way our... the malware data is distributed slightly.
301500	308040	Another problem... well, what those do, basically, which gets into the second problem,
308040	314720	they lower the efficacy of our models that we deploy. So, our models expect data in a sort of
314720	320340	the same sort of general location as our training data. And every time we deploy, we are kind of
320340	327640	fixing our model's view of the world at a certain day, and then we're training it and we're testing
327640	334800	it on essentially future data. So, it doesn't have a perspective for the... if it's trained in March,
334800	340800	it doesn't have... it might not work very well at the end... data coming in at the end of April.
341800	346620	So, one of the methods for dealing with this is you build a dashboard that basically tracks the
346620	353340	error. And when the error gets too high, you discard the model and go on for a new one.
353340	358460	But the problem is, I don't trust VirusTotalLabels, and I don't think anyone really should trust
359400	363000	VirusTotalLabels. And that's the best way we have because of the sheer volume of data
363000	370480	we have to kind of trust VirusTotalLabels. So, and for other things, I wouldn't trust
370480	376660	early labels on data because we have to move very quickly and sometimes with whitelisting and stuff.
378380	384520	And, you know, well, and sometimes we only get like a label when our customer complains and we'd rather
385160	389920	get ahead of that and figure out like, hey, where's the drift happening? What's going on?
389920	394900	And then additionally, aside from the fact that things are just changing all the time,
394900	399060	our models are also under attack. People are trying to bypass them by doing new and weird
399060	406580	things to their data, like packing a spam filter and seeing if that works, or like posting a
406580	411100	different pattern to see if that works, to see if they can get past like the Facebook spam filter.
411100	416020	They're constantly innovating with a specific goal in mind of bypassing our models.
416540	422140	This may not result be an adversarial example per se from the literature, but it sure
422140	428500	kind of acts like that sometimes. And additionally, we could be under attack via a poisoning system.
429420	434760	Maybe if the detector is fast enough, we can get ahead of those things beforehand.
435640	442020	So here's a trivial example. So we have these two data sets. One is our training data set. That's
442020	449500	we've got 1, 1, 2, 1, 1. And so our training data set has 70% 1s, 20% 2s, and 10% 3s.
449900	455420	And this is what we trained on. Realistically, there's no point in training a machine learning
455420	463140	model on a sequence of numbers, but suppose we do. But then we deploy this model, and we get
463140	468960	this other sequence of data. 2, 2, 1, 2, so on, so on. And when we actually deploy it, we see
468960	474420	data coming over the wire. We have 30% 1s, 60% 2s, and 10% 3s. So that's a little bit different
474420	481340	from our training set. So the real world is different from our training set. So in this case,
481340	486240	this is a... we can model this distribution very easily because it's a discrete space.
486240	492640	There are only three things that our data could be. There's only... it isn't continuous like a lot
492640	497600	of the stuff we have to deal with in data science. So we can actually compute the distribution fairly
497600	502980	easily. And this is a pretty good... we can feel confident in our estimate of the distribution.
503180	506760	We have to do some Bayesian statistics to do this properly,
508000	515100	which we're not getting into this talk. But this is basically how it works. And then once we have
515100	520460	like a concept of our training distribution, and our real world distribution, or our test
520460	528720	distribution, as I'll be calling it today, we can take the Kublai-Klepch divergence. So
529500	535930	this is a sort of a measure of the distance between this distribution and this distribution.
536200	543760	And it's very easy to calculate on this categorical discrete data because it's just
543760	550340	going to be, well, 0.7 times log of 0.7 over 0.3. So that's the category for 1s over the
551160	556120	1s. Take the log and then multiply it for the category for 1s. And that gives you the first
556120	561140	term. Second term is the same thing for 2s. And the third term is the same thing for 3s.
561240	566520	And then we add those all up and you get the KL divergence between these two distributions is
566520	574060	0.16. So that's all great. But the problem is our data looks like this. And I don't know whether
574060	582200	this has any difference. Are my blue points sampled from the same distribution as my orange
582200	588020	points? I don't know. It's really hard to tell by just looking at this picture.
588600	593840	And now, in this case, they are. So my orange points are sampled from a two-dimensional
594340	602040	Gaussian, two-dimensional normal distribution, where it's just, it's a nice sphere. And they're
603200	608180	both, the covariance matrix is 1, 1. And the covariance matrix with blues is 1, 1. And it
608180	612560	gives you this nice distribution. And it looks okay. Now, what I'm going to do is I'm going to
612560	619220	drift the blues slightly over. And can you tell, is this correct? Well, you know, every single time
619220	623580	that I put a blue, it's right next to some orange points. So this looks like it could be from the
623580	628500	distribution. I can't really tell with my eyes. And one of the techniques we might do is like,
628500	633040	oh, well, let me take the k-nearest neighbors and see if the distance from my k-nearest neighbors
633640	639140	over time goes up, then I'm out of distribution. But if you tell here, that's not going to tell
639140	646000	you much. If I go here, well, oh, I've got some outliers here now. So maybe that will work now.
646060	650200	If I go here, well, this is really distributed. So this center, the center of this distribution
650200	653960	should be around here. And the center for this distribution is around here. So that's really
653960	659480	drifted. But you've got some outliers over here. Now, what happens when there's like millions of
659480	663760	orange points and there's only a few hundred blue points? Well, you might not be able to tell the
663760	670060	difference with that in that case. So, and also like, there's all this, what I've described to
670060	674020	you is kind of a feeling. And to actually get at the math of this thing, I have to like model the
674020	679300	distributions that these came from. And that's quite a hard problem. And especially in high
679300	685320	dimensions. So things get kind of complicated. So the solution to this that I came up with is to
685320	691540	use a cover tree. Now, why do I want to use a cover tree? So the cover tree, basically,
692120	696420	the key takeaway for cover trees is it's a k-nearest neighbors data structure. There are
696420	706020	many like it. There's kd trees, there's b trees, there's kd trees, you know,
706020	714200	there's k-means trees, a whole bunch. And, well, the cover tree has this wonderful thing where
714940	722060	in 2016, Mario Maggiorni and Wenjing Lau proved that it can arbitrarily well approximate the
722060	727080	underlying distribution of the data. There's some caveats to that statement, and mathematically,
727080	731300	it will probably, to properly statement, will take it, take the whole page. But basically,
731300	736580	the whole concept of their proof is for a nice data set, and to know what a nice data set is,
736580	741600	you need to know what a low dimensional manifold is. And you have to have enough data from that.
742000	748940	But for a nice data set, which most of our data sets are pretty nice, sort of, kind of,
749120	753260	a cover tree will arbitrarily well approximate the underlying distribution of the data.
754460	759160	There's some caveats, and you can go read their paper if you want on exactly what that means.
759160	764140	But for us, what this means is, if I use a cover tree to build a model of my data for
764140	768680	k-nearest neighbors, I can take k-nearest neighbors, or I can just fiddle around with
768680	772500	the properties of the cover tree to, like, get information about my data set.
773420	777440	So, I can tell things like the local dimensionality, I can tell things like how
777440	781880	things glue together, and I can tell, like, sort of, the shape of my data fairly well. I can tell
781880	787580	clusters and things. I can infer what clusters are. Let's go build a very simple cover tree
787580	793060	in two-dimensional data. So, here's a, you know, infinity sign, a bow tie, whatever you want to
793060	797900	call it, and I start my cover tree by picking a point at random and then building a sphere.
798080	803360	You know, in this case, it's a circle that covers everything. So, this is the start. And now,
803360	807440	this doesn't help me that much. I haven't split up my data geometrically to, like,
807440	813420	divide and conquer the k-nearest neighbors structure, because the naive way would take
813960	818880	a linear time, big O of n, but we want to divide and conquer, so it takes log of n.
818880	823640	So, here's what I do. I shrink my sphere down, add another one, and I cover it again.
823760	829380	You can kind of tell that this object is longer than it is wide. It's kind of,
829380	833460	if you really squint, it's kind of a one-dimensional thing in this direction.
835100	839960	So, that's great. But now, if I go, okay, well, I'm going to split it up further, so really divide
839960	844920	and conquer, and I, so I've got these two nodes, and they're both children of the previous node on
844920	850220	the previous slide, and I'm going to build their children. So, here's their children.
850880	856760	So, I divide and conquer their stuff, and I can kind of tell that each of their children have
856760	863580	four, and if I, if you squint a little bit, it's kind of square-shaped in this dimension.
863740	869000	At the scale, the top scale, it was too blurry. I could only see a line here. At the scale,
869000	874900	I can kind of see that it's up in, it's got a nice shape, square shape, and so there's four
874900	880120	children, and the reason, you know, that that dimensionality is relevant to the number of
880120	885860	children you have. And then I can go further, and you know, it's got some more, it's kind of split
885860	890680	up nicely, and I've got like a one-dimensional object here again, because all these children
890680	895940	are kind of one-dimensional-ish, and I can split up some more, and you can see that it's kind of
895940	900980	conforming to your data nicely, and at every single step of the way, I can make some inferences about
900980	906380	what the shape of my data is. They're very cheap, because it's basically counting.
906980	913740	So how do we build a probability distribution? So on slide two, we built a very simple
913740	921980	probability distribution, where I had a discrete space, and now I've got a tree, and essentially,
921980	926540	when I have a tree, at every single node, I can either go left, or right, or down, or you know,
926540	933660	in this case, my number of children I have is variable, but I will be at a parent node,
933660	940700	and I have to go to one of its children. So I always have discrete choices, a discrete choice
940700	947460	of where to go. So how does that apply? So here is a very, like, simple cover tree,
947460	954840	and I've colored it by the total, like, the population of this node.
955900	962520	So this color here is the relative population of this node. So about 66% of the data
963400	970960	is underneath this node. About 33% of the data is underneath this node. And you can see that
970960	977920	colors are kind of changing. I get a kind of a picture of how dense my data is, but that doesn't
977920	982680	help me, because I need to make choices. It doesn't help me make the choices at all steps.
982800	990820	So if I go here, this helps me make my choices. So here, I've got 66% of my data over here, and I've
990820	998840	got 33% of my data over here. And then once I've taken... I have to take one of those two choices.
998840	1005040	If I take this choice, I'm now at this node, and I have two choices again. And the relative
1005040	1011340	probability of taking this choice versus this one... Well, this is basically a 0.1 probability
1011340	1018440	of taking this path, and this is a 0.9 probability of taking this path. And once I get to this node,
1018440	1023400	I have a probability of one of going straight down, because I have no other choices.
1023400	1029200	And every single point, I have a discrete distribution, like we had on page one, which
1029200	1035840	tells me where I'm going. And because my tree is geometrically motivated, these choices,
1035840	1042680	which were all discrete and easy to compute, and easy to count, and easy to keep track of,
1042680	1047260	now I can compute the KL divergence, because it's trivial. It's all... it's just counting,
1047260	1054520	taking logs, and summing things up. So this is my prior distribution. As I said,
1054520	1059500	we're going to use some Bayesian statistics. And so here is my prior. How... this is built
1059500	1066900	with my training set. How do I include my test set? So when I see a piece of data coming over
1066900	1074200	the wire, I can make an inference of where it should belong in this tree. So if it was already...
1074200	1078400	if it was in my training set and didn't get involved in building the tree,
1078400	1083460	where would have it ended up? So I have a point here. So this is my new point that came over the
1083460	1091020	wire. And I queried and said, oh well, this point is covered by this parent, and it's covered by
1091020	1097460	that parent, and that parent, and all the way up. So the path for this point goes up this way.
1097460	1101980	And now for each of those elements in those paths, I can increment my probabilistic distribution
1101980	1106860	and get a posterior one that involves... that includes the new information that I got,
1106860	1111320	that there's a new point over here. And now I can do that again with another point.
1111320	1116100	Oh yep, cool. I have a posterior... this point's been added, and you can see that the distribution
1116100	1121140	changes. And then I can add some more points, and I can add some more points. So these are
1121140	1126260	all my posterior. And I can update my original thing by taking the path of each of the points
1126260	1133540	and updating all the little discrete distributions that I have. So this enables me to
1134100	1140940	sort of quickly make a posterior distribution out of my prior distribution, and everything
1140940	1148100	is discrete. And then I can take the KL divergence between my posterior discrete distributions and
1148100	1154780	my prior discrete distributions. So in this case, this one has a probability of 0.33,
1154780	1161100	and here I have a probability of 0.66. And afterwards, after I've included all my training
1161100	1167060	set, this is a probability of like 0.4. And this, you know, this is a probability of 0.6,
1167060	1171980	and this is a probability of 0.4. And those are very different. And I can plug those into the
1171980	1176900	KL divergence thing and get a positive number for the KL divergence at this node. And then I can do
1176900	1182220	it for this node, and this node, and this node, and get all the KL divergences and add them up.
1182220	1186420	And then I can get the total KL divergence with respect to my original tree
1186940	1190860	of the posterior distribution with respect to the prior distribution,
1192560	1200520	which is really useful. And so this basically, yeah, does it. And the code for doing this
1201120	1204960	enables you to do it very quickly, because it's basically just counting
1206240	1215680	a couple logs and some other statistical equations. And the slide just covers how you do it.
1216260	1224500	So let's go back to that original test set. So I've got basically a blown up version of
1225020	1230880	those slides of the Gaussian distributions where I had them, where a large number of points from my
1230880	1236280	training distribution, which was a single Gaussian sampled at the center of the thing.
1236280	1242680	And then I have a small number of points from a test distribution. And in this case, I have
1242680	1248300	100,000 points from my training distribution and only 500 points from my test distribution.
1248300	1256620	And these both have a covariance matrix of all ones. So it's actually quite, and it's
1256620	1264160	20-dimensional. This is actually harder than most datasets. MNIST is about 10-dimensional,
1264160	1267140	if you really get down to it. If you want to know what that is, you can
1267600	1272260	contact me on Twitter and ask me why MNIST might be lower dimensional.
1275140	1280420	And between the training distribution, I build the tree, fix it, and then I make a posterior
1280420	1287140	distribution for a bunch of test distributions. And I take the KL divergence. And you can see
1287140	1293740	it's a bit noisy. The reason being is I generate a lot of the exact position of your points. It's
1293740	1297940	fairly sensitive to these. So this is one of the problems. And one of the things I'll talk about
1297940	1303880	at the end is like, there are ways to normalize this and resolve this, because I've designed it
1303880	1308740	for speed, not really for accuracy. And I'll get into why I did that afterwards, because honestly,
1308740	1314400	it came to the drift stuff backwards. But you can see, the further you get, the further you
1314400	1320680	separate the two distributions, the higher the number gets. So the sanity test, the unit test
1320680	1327160	of does this work is correct. And I'll have to compare this to other methods.
1327860	1332940	This takes big O of, sorry, Wasserstein takes big O of N to the fourth.
1333440	1338180	Wasserstein is the only other drift calculator that I really know
1338740	1345280	works. And is model independent and doesn't have some funny business with
1345280	1354800	building an estimated distribution initially. This takes K log N. Another big difference between
1354800	1359320	Wasserstein is the traditional way of doing Wasserstein requires a relatively equal size
1359320	1366160	test and training set. You can't have the test set be 0.5% of the training set.
1366720	1373340	That would just not work so great on the traditional method I know of. And this one,
1373340	1378040	honestly, it's online. You can do, the test set can be a fraction of the size of the training
1378040	1386140	set. And because it's online, it can do it in real time. And this is stupidly fast. You can track
1386140	1392620	for the Ember open source, Ember malware data set, you build a reasonable couple tree,
1392620	1401580	you can track 16,000 new samples per second on my laptop. So that's fast enough that inference
1401580	1406140	wouldn't be, this would not be the bottleneck. Inference would be the bottleneck if you would
1406900	1412640	build a cloud distribution, a cloud model, and put this as a guard in front of that cloud model.
1413820	1421460	So we built it so that it can go fast enough to defend. And now here is where the thing originally
1421460	1428720	came from, and then we backtracked onto Drift. So here's the test set attack. So this was
1428720	1437120	originally categorized by Ian Goodfellow, and some defenses have been proposed by Nicholas Carlini
1438140	1447120	and others. So, but here is the gist of the attack. Normal users just query. They don't
1447120	1450840	care about whether they got a false positive or a false negative, they just keep querying.
1451820	1458280	And for us, we think of a normal user as being like an enterprise user, trying to figure out
1458280	1464800	whether all the reports coming over the wire of new binaries are malicious or benign, or they'd
1464800	1471560	be submitting every single binary that they see on their network to a service that will tell them
1471560	1480840	whether it was malicious or benign for an ADV product. But a malicious user would be submitting
1480840	1487520	things with a specific goal in mind. They don't care about what they care deeply what the label
1487520	1493140	is, and they care deeply about defining a false negative. They want to find a malicious sample
1493140	1497380	we classify as benign. So what they'll do is they'll try a bunch of stuff until they find
1497380	1503600	something that is misclassified as benign and is actually malicious and bypasses our model,
1503600	1513840	and then they deploy it as far as much as possible. So if you do, if you apply this to,
1513840	1520420	you know, the fast drift calculators to that, you can attach a little drift calculator to each user.
1520560	1528500	And as samples come over the wire, you track it for that user. And if the user starts
1528500	1532860	exploiting something, they're going to get a very boring distribution where it's just
1532860	1541520	one point repeated over and over again. And you spike your distribution, your KL divergence
1541520	1545820	spikes up and then becomes very normal because it's essentially you're tracing the same path
1545820	1552240	over and over and over again. And you can see that there's an exploration phase where the
1552240	1557640	malicious users try to look for a benign misclassified sample, and when they find one,
1557640	1562240	there's an exploitation phase. And the KL divergence rises rapidly. You can see that
1562240	1570320	that this is in log of scale. So this is literally hundreds of times more KL divergence than the
1570320	1575280	benign samples. And here, it's the same thing. Hundreds of times more KL divergence than
1575280	1584220	the benign and the attackers were unsuccessful. Because of this, you can build, you can probably
1584220	1594460	build a very good online defense of your system against a set attack. And here's where basically
1594460	1601880	the system was originally built sort of with this mind and other attacks in mind, because
1601880	1608440	I'm more interested in defending my models than calculating drift. But because it's so
1608440	1613220	bloody fast, it does this really well. And because it's so bloody fast, you can retrofit it.
1613220	1621080	So here's where I'll talk about some next steps. So as I mentioned previously,
1621080	1627000	this thing is a bit noisy. There are ways to fix it. In this case,
1627000	1633700	window sizes and some tuning things that I didn't go into in these slides because I've
1633700	1639860	already got 20 minutes. There are some tuning systems you can use to really clean up and
1640100	1644900	de-noise. There's also some interesting waking functions you can do inside the tree because
1645500	1656900	there's some noise from an overactive leaf node might end up just building a very high KL
1656900	1661380	divergence for just that one node and it throws everything off. And there's a couple other little
1661380	1665360	things like that that still need to be cleaned up before this goes into production. But the
1665360	1671560	system itself is looking very promising and is honestly the fastest of its kind
1672320	1686220	that I can see in the world. And that's basically it for Gokul's ability to detect benign test set
1686220	1693480	and other drift things. The library is at elastic.com, oh sorry, github slash elastic
1694060	1701040	slash Gokul. It's named after my grandmother because the base algorithm for this thing is
1701040	1704820	called GRMA. And if you say that while drunk, you can kind of get grandma.
1705160	1710940	And grandma's a bit too on the nose, so I named it Gokul.
1712680	1715420	Anyway, well, looking forward to hearing you guys on Twitter
1715420	1720980	and Twitch and Slack or whatever. So thank you very much and good night.