start	end	text
3720	14660	Sorry about the no sound, people. I will figure this out. Every day,
14660	25960	it just decides to grab the wrong audio input. So let's go back to the beginning and see if
25960	31920	this works. Hi everyone. Today's talk is about machine learning privacy. The title of the talk
31920	39100	is secrets are lies, sharing is caring, privacy is theft. My name is Nahid Farhadi and I'm a
39100	45520	software developer in Capital One. I will be giving this talk with my colleague Vincent Pham.
45520	51160	The outline of our talk is as below. We will give an introduction on the importance of privacy
51160	55260	and we will talk about privacy attacks, we propose some defense techniques,
55260	60280	and finally we will show a demo of an attack as well as defense techniques.
61920	68300	Machine learning needs data. So what if this data has sensitive information like medical data
68300	76620	or financial data? Even if we trust the algorithms that are generating the model or model developers,
76620	82080	sometimes just using the statistics by looking at the output of a model, we can find out some
82080	88260	information about the input of the model. This type of threat to ML can be black box or white
88260	94260	box, meaning that the adversary doesn't necessarily need to have any access to the
94260	101760	model specifics. So what is really privacy? Privacy means that if we are using some data
101760	108560	from our users for training a model, but just by looking at the output of the model,
108560	112440	no one should be able to get information on the input of the model.
113760	120480	There is a trade-off between privacy and utility. If we have too much privacy,
120480	125920	then it will be very difficult to use the model. If we don't have much privacy, then
125920	134020	it will be useful, but then we will endanger our customers. PPML is there to define or find
134020	142360	the optimized point between utility and privacy. How can we sacrifice less by getting enough privacy?
143460	147220	My colleague will be talking about privacy attacks next.
148060	152920	In terms of privacy attack, there are two main categories. One is the classical, where the
152920	157280	private data is in the train, where it's in the raw set. And there are the ML-enabled one, which
157280	162020	I'll be focusing more on. In the next slide, you can see that there are several attack surfaces
162020	168980	that attackers can utilize to perform a privacy attack. The first is the physical domain,
168980	174400	in terms of network protection, it could be attack network traffic. Then there's the digital
174400	180140	representation, which is the TCP dump. There's also the machine learning model, which is just
180140	186840	the model itself, where you have an input and do a prediction on the output. In terms of this,
186840	190120	it's the attack probability. And then you have the physical domain, which is the shutdown
190120	195360	infrastructure. For this talk, we'll focus on the machine learning model. In the next slide,
195360	201460	we will see our first type of attack, which is the linkage attack. With this attack, in the early
201460	206780	90s, Tina Sweeney found that she was able to identify the identification of the governor of
206780	211520	Massachusetts by linking his health records to a voter information just by using three variables,
211520	217560	his gender, date of birth, and a zip code. In the next slide, she was able to identify that
218100	223360	using these three variables alone, she was able to uniquely identify 87% of the population in the
223360	231600	U.S. using the census, the 1990 census. In 2000, Philip Goh identified that using these three
231600	238740	records, he was able to identify only 64% of the population. This could be a result of people
238740	245840	urbanizing more, moving from the rural areas to the cities. But this provides an important point
245840	251400	where using just very simple features, attackers can identify back to the original source.
251930	258520	In the next slide, we look at another popular linkage attack. In 2006, Netflix released a
258520	263300	public data set, a competition where they're offering a prize to improve their recommendation.
263300	269560	And their FAQs denoted that there is no private information in the data set itself,
269560	275120	but researchers have found that if they were able to link the Netflix accounts back to the IMDb,
275120	280000	just by using the name of the person on the IMDb, their public dates, and public ratings,
280000	287820	they'll identify personable information back to Netflix. You might be wondering why this might be
287820	297120	damaging since the people tend to put their public information on IMDb. This could be because they
297120	304100	might be biasing or selecting a set of records to disclose on IMDb, but not on Netflix recommendation.
304100	309540	For example, they were able to identify that for a particular person, they were able to identify
309540	315960	just based on his Netflix record, that he might have a voting preference from his voting record on
315960	321020	Power and Terror, Norm Chomsky and Our Times, and Periodic 9-11. They might be able to infer on his political
321020	326080	preference based on the Jesus of Nazareth and Gospel of John ratings, and also his eating habit based
326080	333320	on his recommendations on supersizing. In the next slide, we look at a new type of attack called
333320	338140	the reconstruction attack. This is more prone to models that tend to keep the training information
338140	345560	in the structure itself, such as near-neighbor classifier and kernel-based SVM. But this is also
345560	351760	prone to neural networks as well, as we can see in the next slide, where in production environments,
351760	357820	researchers and even implementers tend to reuse neural network models just because of something
357820	365340	called style transfer, or style transfer where they're able to utilize one model and
367300	371900	create a new prediction from that same model using the inputs and architecture of it,
371900	377860	and reconstruct back to the original input. In the next slide, we also see a similar type of
377860	383700	attack called model inversion attack, where in this case, a feature such as an image is fed into
383860	388820	a neural network model and produces a set of probabilities, and then the researcher can use
388820	394120	these probabilities and name and reconstruct an image back to the real thing. For example,
394120	399380	in the right side of the slide, you can see that in the left image, there's this reconstructed image,
399380	403660	and in the right is the original image, and they're very similar to each other.
404180	408860	In the next slide, we also have a membership inference attack, where an adversary can
408860	413920	generate multiple shadow network models that resembles the production environment,
413920	419000	produce the probabilities from these models, create a new model called the attack model to
419000	425400	feed in these probabilities, and learn what observations are trained by the production model
425400	429540	and what are not trained by the production model, identify what are the members and what are not the
429540	438380	members. In the next slide, we also have a core attack. This is more prone to models such as LSTM,
438380	448480	or models that learn from a time series of data points. For this example, you might have a
448480	452020	model that learns from social security, and it does an autocomplete, and one of the autocomplete
452020	458100	could be some sort of sensitive information, such as the phone number or the social security of a
458100	466340	person. This is more prone to maybe like if you're over-memorizing from the neural network model
466340	472980	itself, where the records might appear at least five times. In the next slide, we have a summary
472980	478840	of different privacy attacks that I've talked about earlier on. You could take a pause here
478840	484360	and just look through the difference, compare the different approaches, and look at the examples,
484360	490950	but we can move on if you're ready to move on. Next slide.
491830	498690	All right, so now we're really talking about methods and techniques to preserve privacy.
499650	507890	To preserve privacy, we have four main points to protect. The training data and input and output
507890	513910	of the model shouldn't be visible to anyone who is not authorized to visit them. And finally,
513910	519930	the model privacy, which makes sure that the model cannot be stolen by any malicious party.
520170	527110	Two main categories to preserve privacy are secure computation and privacy preservation. In case of
527110	534090	secure computation, we transform or distribute our computations in a way that is not readable
534090	540330	by any unauthorized person. And in terms of privacy preservation techniques, we use injection,
540330	544950	noise injection, or masking and hiding techniques in order to preserve privacy.
545790	551910	The first method of privacy preservation is homomorphic encryption. Homomorphic encryption
551910	558610	is a type of encryption where you are doing the same computations and a third party can
558610	564130	do your computations for you, but not on the original form of the data, on an encrypted form
564130	575290	of data. This means that if you use a third party company for your ML computations, you can totally
575290	581010	give your data to them in a transformed form, which is using homomorphic encryption, and they
581010	585790	can do the computations for you and give you back the results. And then you can decrypt the data
585790	592470	and get the plain text or cipher text as you want. The second method is secure multi-party
592470	599730	computation. In case of this technique, we have multiple parties who all contribute into building
599730	607550	an ML model or a function or to compute a value for a function. But the thing is that the input
607550	615150	is distributed between all of these parties. So as you see, part of input A1 is here, A2 is here,
615150	623690	A3 is here, and so on. So this way, if an adversary wants to attack the outputs, they need to
623690	629850	be able to hack into all of the parties in order to make sense of the data. Otherwise, getting
629850	637110	information from one of the parties is meaningless. Another technique is federated machine learning.
637110	643470	Federated machine learning is very similar to the previous method. However, the difference is that
643470	652030	we have multiple sets of users, and these users all contribute into building a classifier.
652750	659250	The way that we handle the creation of the outputs is that we assign some sort of weights
659250	665950	and locally update the average of classifiers by assigning weights to each of the trained models.
667830	672550	The most important technique to preserve privacy is differential privacy.
672930	679050	Basically, differential privacy means that it's a noise injection technique so that
679050	685550	the output of your model shouldn't reveal any information about the input of the model.
685550	691690	Whether your model is trained using the data of this person or the data of this person
691690	695110	shouldn't be revealed in the outcome of your algorithm.
697230	704270	It can be non-interactive or interactive. For example, in case of non-interactive,
704270	710710	we pre-compute the stats, the amount of noise, the amount of relationship between the input
711350	717970	features, and then inject noise to that. In case of interactive, the amount of noise is
719270	723770	injected based on user's requests.
725350	731910	What does differential privacy mean? It means that if person i change their information
732510	741570	from x i to x prime i, then the probability of their output should not change by that much.
741570	750010	What does it mean again? It means that as a person, if one of my features is changing,
750010	756990	and as a person who is contributing into training a model, that change shouldn't cause a huge change
756990	762510	in the output. Because if that causes that huge change, it will reveal some information
762510	770770	about this instance or observance as an input. For example, if I change the color of my hair
770770	777810	from brown to green, and that has a huge effect on the output of the model, then just by observing
777810	784610	that change, an adversary will know that I changed the color of my hair. So using differential
784610	792730	privacy, we want to make that change as minimal as possible. And how do we do that? By computing
792730	799050	what is the effect of that change and making that as a noise input into our model.
799870	807370	So looking again at the differential privacy, you see here that we have two datasets. We call these
807370	814690	neighbor datasets, meaning that all of the features in both datasets are actually equal,
814690	822210	except that one feature. In here, it is x3 and x3 prime. We want to make sure that when we change
822210	830790	the input data from x3 to x3 prime, the output that is generated by the model is changing
830790	836290	minimally. So as you see, the probability of generating a specific output in the first case
836290	843930	to the second case is minimal, or epsilon. The next method is called PATE, private
843930	852090	aggregation of teacher ensembles. In this case, we have several classifiers that use some data
852090	859470	use some features in order to train a model. Then we have two sets of voters. It's like an
859470	865270	inversion programming type of technique. These voters choose the majority voting generated by
865270	873130	all of these classifiers. And if we have equal amount of votes, for example, in this case,
873130	880730	we have two cancers and two healthy results, then we will calculate a Gaussian noise to
880730	887270	inject to the voters data. So basically, we again go back into our training data,
887270	895530	and we take a look and see that how much noise does each classifier needs to be injected into.
895530	902770	Then again, we look at we inject that noise and we get the results off of this chart based on the
902770	908330	results. So based on the amount of noise injected, the model might generate the value cancer or
908330	915430	might generate the value healthy. The important thing is that whatever the output is, for example,
915430	921610	in this case, it is cancer, the adversary cannot correlate it back to the data that is trained
921610	928610	in order to get the value of cancer here. Because even if we don't have we have the value healthy
928610	933670	here by the noise injection, these might also generate the value cancer. So we don't really
933670	940070	know is James Smith the cause of generating the result cancer or if someone similar to
940070	946670	James Smith in this training data set is contributing into that result. Here is the
946670	952290	comparison of the defense methods in terms of what they're emphasizing on, what are they trying to
952290	959170	protect, whether it is the data owner or the model protection, and their use cases in different
959170	965390	applications. Most importantly, most of these methods are really effective when they're used
965390	970570	in combination with each other, one from each category. For example, homomorphic encryption
970570	976790	and differential privacy or SMP and differential privacy, PATE and federated lightning plus
976790	984870	homomorphic encryption. Here are the practical methods for privacy preservation. They are very
984870	992530	easy to use. There are already packages that are generated for that like TensorFlow or PyTorch
992530	999670	or IBM privacy package. We are going to show an example of how to use one of these packages in a
999670	1008270	real world sample. In this demo, we're going to show how you're going to implement a solution
1008270	1012610	from an adversary point of view, and then from a defense point of view, how you can easily apply
1012610	1019350	differentiable private method into your model. So here in the references, you'll see the data
1019350	1025050	that we use, which is a data source on Kaggle of the purchase dataset. This contains information
1025670	1031330	that you can use to create labels, such as a binary classifier. In some cases, it could be like
1031330	1036430	fraud, or if it's fraudulent or not, or it could be abusive or not. Then we can also create a set
1036430	1045730	of multi-class labels itself, such as the class for a customer, or it could be the category of a
1045730	1052690	merchant, or any other multi-class that you can think of relevant to your use case. And you can
1052690	1057850	see that we're also going to show you the TensorFlow privacy method as well. So first, we're going to
1057850	1064150	look at how a shadow network is created. It's really easy. First, you create the set of shadow
1064150	1070450	networks. Then you output the predictions from the shadow networks, feed that into a TAC network,
1070450	1075910	and then train a target network that is similar to what you think the production environment would
1075910	1082010	look like, and then see whether the prediction from that target network is predicted as an in-member
1082010	1087850	or out-member from the TAC network. If we look at the shadow network, all you have to do is create
1088130	1094010	a very simple for loop. So if you go down, you can see that all you have to do is create
1094150	1099870	n number of shadow networks. They could be similar to each other, or they could be of different
1099870	1106690	structure. But for our use case, we're going to create one type of TensorFlow model, a very simple
1106690	1115070	one with an initial density of 64 nodes, and then dropout, and then another dense layer of 24 nodes,
1115070	1121350	and then finally the softmax for a prediction of the multi-class or binary class. Each of these
1121350	1128410	n shadow networks will have the same networks, and it'll produce a prediction of how likely it is
1128410	1134370	to be in a certain class or not. If we look at the TAC network, this is where then you're feeding
1134370	1139930	the predictions of the shadow networks. We're then training a very fine model to distinguish
1139930	1145610	the threshold of whether a particular observation is within a network or not. So this is very
1145610	1151910	simple. You're training samples from the shadow network. You're going to identify if it came from
1151910	1158810	that particular shadow network or not, and that would be pretty much your label. If we go to the
1158810	1166370	target DP, this will reveal how easy it is to implement a differential private method for your
1166370	1171510	defense mechanism if you already have a deep learning TensorFlow model in production. So all
1171510	1176710	you really have to do is import the TensorFlow privacy library and then update your optimizer.
1176710	1181610	So for in cell 11, you can see that the model is the same, but in cell 12, you can see that
1181610	1186450	the only thing that changes is the optimizer, which is a DP gradient descent constant optimizer
1186450	1191810	with four inputs that you would have to fine-tune. Dial-to-norm clip, noise multiplier,
1191810	1196410	non-micro-batches, and learning rate. And then you have your differential private method.
1199170	1206630	And then if we go back to the first one, here we show results of the different number of shadow
1206630	1211550	networks with the different number of classes, 10 classes, one-inch classes, and the binary two
1211550	1215630	classes, and whether it was differential private or not. And you can see the different types of
1215630	1224230	accuracy, test accuracy, and then the recall and precision rate as well. This is just a summary.
1224230	1228350	If you're interested, you can pause here and look at it for further details.
1235590	1240190	Okay, so this is the end of our presentation. Thank you so much for your time.
1240190	1245350	If you have questions, please reach out to our emails included in the slide deck.