start	end	text
1240	7640	Okay, hi everyone. This is Zoltan and Hiram's...
9800	17000	in conjunction with Microsoft and Cujo AI. They're launching a very interesting security competition
17680	21180	which kind of relates back to some of the stuff we were talking about in opening remarks
21800	54640	and I'll let them take it away. Sorry about the audio.
59930	66650	I will get the audio working. I am sorry. Why is it not capturing the right thing?
82320	87920	I'm Balazs and this is a project together with Hiram Anderson.
89220	96380	We are going to talk about machine learning detection bypasses and in the past you probably
96380	106040	have seen some researches where people modified existing images and the goal was to bypass
106040	114480	machine learning classifiers. So for example for a human viewer the new image looks like the
114480	122240	original one but for a machine learning classifier this looks like something different. Like it will
122240	131440	believe that this looks like an ostrich. In this talk we are going to present machine learning
131440	138620	bypasses when it comes to malicious software and there have been some interesting researches in
138620	148040	the past regarding this topic as well. For example last year there was a new research published
148480	156580	where people extracted strings from a known game executable, appended this to a known malware
156580	165000	and it was able to bypass production machine learning model. I also did some research in
165000	171120	the past and like four years ago you were able to bypass some machine learning models
171120	175520	just by packing a sample with UPX.
177020	183800	In order to advance the field of offensive and defensive machine learning based malware
183800	192000	detection, last year we created a challenge where you had to download 50 working malware samples,
192000	196260	you had to download three machine learning models with eight weights,
196260	200520	modify the malware samples to evade detection by all models
200520	209460	and if you were lucky and you had the most points you were able to win this nice GPU card.
210180	216480	In total 70 people registered for this competition and at least 11 people were
216480	223860	able to bypass at least one machine learning model. Congratulations to the winner William
223860	232920	Fleschmann and I highly recommend you to check out his blog post on how he won this competition.
234100	241640	There were some other write-ups and papers as well. I do recommend you to check out those as
241640	248140	well. You can see one from Jakub and one from Fabrizio on the following links.
249100	256740	When it comes to win this competition, multiple approaches were used. Some people started
256740	266840	with a simple packer like the one I mentioned, but unfortunately some of the samples are already
267400	281080	packed in a way and this means that if you use UPX or something similar it will not work anymore.
283200	291320	Another great approach was to add new sections to the executable. For example, you can extract
291900	298820	the end user license agreement resource from Microsoft Files and add it to the malware
298820	307920	samples multiple times. This approach was really good at bypassing the detection for the ML models
307920	313740	but unfortunately again this broke some of the malware binaries.
314980	324040	Fun fact, if you just simply add sections to a malware sample, you might be able to bypass
324040	333400	some antivirus detection because for performance reasons some AV engines check the number of
333400	343720	sections before they evaluate the rest of the rules. At the end, the winning strategy was
343740	351690	to just append random data to the end of the executable. This is called as an overlay.
352280	358910	Even though this is a very simple strategy, it worked during last year's competition.
362750	372800	This is also an easy way to bypass if the sample has any kind of self-protection, for example.
373740	379240	Just by increasing the size of the sample, again, you might be able to bypass some
379240	388000	antivirus engines. Again, they can have a file size in their rules.
391220	401200	And one important thing, as you can see on the top right image, that's a visual representation
401200	411780	of a malware. And I just appended some random strings to the end of this sample. And if you
411780	422820	look at the green visual, you can clearly see how this changed the visual representation of the
422820	432180	sample. For us, there were some key takeaways from last year's competition. For example,
432180	437820	some of the machine learning models are way too academic, but not very effective in practice.
438740	447740	Turned out it's not just us, but everybody thinks that the DF tool is awesome. This is a Python
447740	456560	package you can use to modify binaries. And as it is the case with malware, it is always tricky
457050	461940	to deal with them. For example, some of the samples do not reproduce the same
462820	469360	indicators of compromise over time. This can be because, for example, the command and control
469360	475640	server is down, and dealing with packed and protected samples can be hard sometimes.
477740	486260	I also checked the SSDeepHashes of some of the samples, and it was interesting to see that
486780	495860	whenever people added repeating patterns to the sample, for example, the same section, or they
496820	503220	added the same overlay over and over again to the sample, then it created a repeating pattern
503220	513180	in the SSDeepHash as well, which can be used for detecting a sample which uses
513180	520980	machine learning evasion, for example. This year, we created the Defender and the
520980	526080	Attacker challenge. In the Defender challenge, you had to create your own machine learning model
526080	532500	and submit this to the competition in a Docker format, and in the Attacker challenge,
532500	539680	now the machine learning models are not available for you, so this is now a black box challenge.
540440	546120	And if you win this competition, either the Defender or the Attacker challenge,
546120	552160	you can win some Azure credits for your machine learning research plans.
552830	558860	The defensive track is already over. We received two submissions that passed the minimum
558860	567500	requirements, and the offensive track already started, so I highly recommend you to go to our
567500	578100	website, analsec.io, and check out what the competition is. In this year, we have used the
578400	586920	malware families, and if you go to our website, review the terms of service, and then you can
586920	595200	download the 50 provided malware samples, and after that, it is your time to modify the samples
595200	606340	in order to evade the detection. And new to this year, you can use an API to check your samples or
606340	613600	submit your samples. I also recommend you that you verify that the malware functionality remains
613600	622520	the same in your local Windows box. Then, when you upload the zip files, or you can just upload the
622520	631440	partial zip files, meaning that you only submit some of the samples and not all of them,
631440	639940	you can receive one point for each bypassed machine learning model, which means that
639940	648760	for every sample, you can get up to three points. And as usual, highest score wins.
648880	656940	The details about this will be provided by Hyrum, and in order to claim your prize, you have to
656940	666520	publish your solution. Please note that you have to keep the file names as it were in the
666520	676520	original zip file. This helps us to track which file you modified originally. We also provide some
676520	683460	additional tips and tricks you might use in this competition. Some of them may not make sense, but
684100	690440	you can modify an executable in a lot of different ways. For example, you can add or
690440	697660	remove signatures, change section names properties, modify the import or export tables, create TLS
697660	704620	callbacks, change the PA header, fix or change the checksums, add, modify, or remove the version
704620	711760	information, create new entry points, or just change some code or data in it.
713040	721340	Still, it's not allowed to create droppers or self-extracting archives, because this will kind
721340	728660	of defeat the purpose of the whole competition. And this year, keep in mind that multiple
728660	733770	registration is against the rules, and it will result in immediate disqualification.
734190	740550	Please do join our Slack channel, where you can discuss everything with us,
740550	749570	and you can also discuss your progress with the other participants of this competition.
751970	760250	Just a side note, the whole frontend was created in Python Flask admin,
760250	766550	we are using Cloudflare, Nginx, and GUnicorn for scalability and performance reasons.
766610	775390	There are some backend scripts running with Python, scheduled by Chrome, and as it was
775390	783490	the case last year, we still use the VMware Sandbox to evaluate the samples. As mentioned,
783490	792360	we already have an API, so if you want to check your sample against machine learning,
792690	797650	against all the machine learning models, or just against one machine learning model,
797650	805930	you can use the API just to do that. And also, you can use the API to get the results.
805930	812530	And if you are satisfied with bypassing the machine learning models, you can upload your
812530	820610	zip files and query the zip status and the sample statuses as well with the API keys.
820690	826570	This is all I wanted to share with you guys, but please welcome Hiram, who will
827490	834630	present you some other tips and tricks you can use to win this competition. Thank you.
836640	842940	I'm going to describe to you the example solution in the machine learning security evasion
842940	850500	competitions attacker challenge that has just begun. The models that you'll be attacking this
850500	856780	year have been submitted by participants of the previous round in the defender challenge.
856940	862960	Two of the models from the previous round have qualified to be included in this round.
862960	869740	In addition, we have hosted our own model for you to attack. That model is trained on the
869740	877200	Ember dataset and includes some basic capability to detect adversarial examples. The source code
877200	884220	and model weights for this defended Ember model are provided on the competition's GitHub site.
884880	892940	However, the remaining models are to you complete black boxes where you only get to observe the hard
892940	900040	label predictions. That is a zero or a one for an output that you provide to the machine learning
900040	907680	models. The final leaderboard ranking will be set by the following rank ordered criteria.
907980	914620	First, the total number of evasions with one point for each of the three ML models
914620	922060	times 50 malware samples, meaning that the maximum score is 150. Remember though that each
922060	928640	evasive sample must reproduce its original functionality in a sandbox in order to be
928640	937020	awarded a point. Functionality is verified only when you upload a zip file containing
937020	944760	your candidate malware samples. It will not be verified when you merely query the machine
944760	952840	learning models through the API. In the event of a tie for point number one, contestants will
952840	960820	be ranked by the number of model queries used through the API. And lastly, the timestamp of
960820	967180	your final zip upload would break any subsequent tie. More than likely though, we won't get to
967180	973800	point number three, so you should feel incentivized to continue competing right up until the competition
973800	979540	deadline. Even if you see a perfect score on the leaderboard, because you might achieve that same
979540	986200	perfect score but do it more efficiently. So as a contestant, you can choose any strategy you'd
986200	993740	like to compete. But to demonstrate one possible strategy, we have released some example code on
993740	999940	the competition's GitHub site. You can find more information about the nitty-gritty details of this
999940	1007000	approach on the website. It essentially is using a discrete optimization technique over a space of
1007000	1012880	functionality preserving file modifications. However, the general strategy might be more useful
1013580	1020400	for you to adopt. The strategy consists of, is really simple, consists of doing a bunch of bulk
1020400	1026760	work using an algorithm in part A and then kind of batting cleanup for manual manipulation of
1026760	1032880	malware samples in part B. And I'm going to be describing and demoing the code for part A today.
1033100	1039300	In it, because we'd like to be efficient in the number of queries against the hosted machine
1039300	1046400	learning models, we'll actually break this attack into two parts. An offline attack where we use the
1046400	1053020	defended Ember model for which we have code to kind of work out our strategy and generate initial
1053020	1061020	malware samples. We hope that those seeds might evade some of the online models that are hosted.
1061520	1066400	And then in the online attack, we'll take those initial seeds and the algorithm will further
1066400	1074760	optimize and discover additional file modifications required to evade the online hosted models.
1074760	1081060	Some tricks that we're using here include label smoothing, where we're converting the hard label
1081060	1089940	outputs into a soft score by averaging four things, the three hard label outputs from the
1089940	1098800	hosted machine learning models, as well as a local score from a local machine learning Ember model
1098800	1104480	that will be used as a heuristic to kind of guide the optimization process.
1105520	1111920	So as I demo this code, I want you to please be aware that this code writes malware to disk. So
1111920	1123280	please do run this code only using a Linux VM. To begin, we initialize the attack by analyzing a
1123280	1131260	connect collection of benign files. This init subcommand extracts elements of these benign files
1131260	1138420	that will be later injected into the malware. To launch our offline attack, we'll run a local
1138420	1146080	copy of the Ember model in the top window. Then in the bottom window, we'll use the run command,
1146080	1151800	passing in malware samples that we downloaded after registering on the website.
1151980	1159620	The tool will then write successful evasion attempts to pass one slash success that we've
1159620	1165840	specified in the command line, and failed attempts to pass two slash failure folder.
1166020	1174120	And also included in each output directory will be the history of file modifications
1174830	1182130	that will be useful if we'd like to pick up to resume a failed attempt.
1183470	1188140	So to demonstrate that, in a second pass of the offline attack,
1188140	1196300	we'll start with the pass one failures and iterate on the optimization approach,
1196300	1200830	again storing successes and failures to a pass two directory.
1204210	1209100	So after doing that a number of times and having collected a bunch of candidate samples offline
1209670	1217140	that evade the local defended Ember model, we'll then use those candidates as seeds for
1217140	1223820	an online attack, which now counts against our API query usage. To do an online attack,
1223820	1231820	simply use this tool with the dash dash online flag, and the optimization will then continue
1232720	1238900	trying to find file modifications that will bypass all three of the hosted models.
1241200	1246080	Of course, you want to do perhaps as many, you know, as many iterations as necessary,
1246080	1251560	and the online version of this attack. But after you've done so, in a final pass of the online
1251560	1260540	attack, you can now collect the successful samples into a zip file that you would then validate in a
1260540	1269000	Windows 10 virtual machine, and then upload to the website for validation and leaderboard scoring.
1269000	1273760	I want to point out that since there is a chance that by running this code,
1273760	1279000	file modifications might break some of the samples, you should always run these samples
1279000	1285620	in a Windows 10 sandbox before uploading to the competition website. Also note that zip file
1285620	1293520	uploads contribute against your API query count. So it is to your benefit to double check your work
1293520	1299120	and make sure that any files you upload are functional, so you don't have to redo that work
1299120	1306420	and upload again. As a final note, kind of tricky that since the hosted models might be actually
1306420	1313100	changing state and learning from the queries that you and others are giving them, there's a
1313100	1320700	possibility that an evasive variant sample that you discovered along the way may no longer evade
1320700	1325360	a model by the time that you upload your zip file. So be... I don't know that will be the case, but
1325360	1331300	please be aware that that is a possibility. So with that, good luck on the competition.
1332220	1337920	Visit the website at emailsecta.io. The competition will run for over six weeks.
1338540	1345720	And those who are ranked first and second on the leaderboard will win our
1345720	1352220	grand and first prizes respectively, so long as they publish their solution.
1353620	1356560	And with that, I'd like to thank our sponsors,
1356560	1362020	Microsoft and Cujo AI, with partners MRG Efitas and VMRay.