1
00:00:01,240 --> 00:00:03,020
Okay, hi everyone.

2
00:00:03,400 --> 00:00:07,640
This is Zoltan and Hiram's...

3
00:00:09,800 --> 00:00:13,640
in conjunction with Microsoft and Cujo AI.

4
00:00:14,020 --> 00:00:24,740
They're launching a very interesting security competition which kind of relates back to some of the stuff we were talking about in opening remarks and I'll let them take it away.

5
00:00:51,600 --> 00:00:54,640
Sorry about the audio.

6
00:00:59,930 --> 00:01:01,950
I will get the audio working.

7
00:01:01,950 --> 00:01:03,270
I am sorry.

8
00:01:04,010 --> 00:01:06,650
Why is it not capturing the right thing?

9
00:01:22,320 --> 00:01:27,920
I'm Balazs and this is a project together with Hiram Anderson.

10
00:01:29,220 --> 00:01:48,280
We are going to talk about machine learning detection bypasses and in the past you probably have seen some researches where people modified existing images and the goal was to bypass machine learning classifiers.

11
00:01:48,280 --> 00:02:00,640
So for example for a human viewer the new image looks like the original one but for a machine learning classifier this looks like something different.

12
00:02:00,640 --> 00:02:06,000
Like it will believe that this looks like an ostrich.

13
00:02:06,320 --> 00:02:22,480
In this talk we are going to present machine learning bypasses when it comes to malicious software and there have been some interesting researches in the past regarding this topic as well.

14
00:02:22,580 --> 00:02:42,260
For example last year there was a new research published where people extracted strings from a known game executable, appended this to a known malware and it was able to bypass production machine learning model.

15
00:02:42,480 --> 00:02:55,520
I also did some research in the past and like four years ago you were able to bypass some machine learning models just by packing a sample with UPX.

16
00:02:57,020 --> 00:03:16,260
In order to advance the field of offensive and defensive machine learning based malware detection, last year we created a challenge where you had to download 50 working malware samples, you had to download three machine learning models with eight weights,

17
00:03:16,260 --> 00:03:29,460
modify the malware samples to evade detection by all models and if you were lucky and you had the most points you were able to win this nice GPU card.

18
00:03:30,180 --> 00:03:40,720
In total 70 people registered for this competition and at least 11 people were able to bypass at least one machine learning model.

19
00:03:41,120 --> 00:03:52,920
Congratulations to the winner William Fleschmann and I highly recommend you to check out his blog post on how he won this competition.

20
00:03:54,100 --> 00:03:58,880
There were some other write-ups and papers as well.

21
00:03:58,980 --> 00:04:02,320
I do recommend you to check out those as well.

22
00:04:02,320 --> 00:04:08,140
You can see one from Jakub and one from Fabrizio on the following links.

23
00:04:09,100 --> 00:04:15,380
When it comes to win this competition, multiple approaches were used.

24
00:04:15,380 --> 00:04:41,080
Some people started with a simple packer like the one I mentioned, but unfortunately some of the samples are already packed in a way and this means that if you use UPX or something similar it will not work anymore.

25
00:04:43,200 --> 00:04:49,480
Another great approach was to add new sections to the executable.

26
00:04:49,480 --> 00:05:00,760
For example, you can extract the end user license agreement resource from Microsoft Files and add it to the malware samples multiple times.

27
00:05:01,000 --> 00:05:13,740
This approach was really good at bypassing the detection for the ML models but unfortunately again this broke some of the malware binaries.

28
00:05:14,980 --> 00:05:39,360
Fun fact, if you just simply add sections to a malware sample, you might be able to bypass some antivirus detection because for performance reasons some AV engines check the number of sections before they evaluate the rest of the rules.

29
00:05:40,620 --> 00:05:49,400
At the end, the winning strategy was to just append random data to the end of the executable.

30
00:05:49,400 --> 00:05:51,690
This is called as an overlay.

31
00:05:52,280 --> 00:05:58,910
Even though this is a very simple strategy, it worked during last year's competition.

32
00:06:02,750 --> 00:06:12,800
This is also an easy way to bypass if the sample has any kind of self-protection, for example.

33
00:06:13,740 --> 00:06:20,660
Just by increasing the size of the sample, again, you might be able to bypass some antivirus engines.

34
00:06:20,660 --> 00:06:28,000
Again, they can have a file size in their rules.

35
00:06:31,220 --> 00:06:42,640
And one important thing, as you can see on the top right image, that's a visual representation of a malware.

36
00:06:42,640 --> 00:06:50,740
And I just appended some random strings to the end of this sample.

37
00:06:50,740 --> 00:07:03,820
And if you look at the green visual, you can clearly see how this changed the visual representation of the sample.

38
00:07:05,800 --> 00:07:11,620
For us, there were some key takeaways from last year's competition.

39
00:07:11,620 --> 00:07:17,820
For example, some of the machine learning models are way too academic, but not very effective in practice.

40
00:07:18,740 --> 00:07:26,100
Turned out it's not just us, but everybody thinks that the DF tool is awesome.

41
00:07:26,120 --> 00:07:31,880
This is a Python package you can use to modify binaries.

42
00:07:32,760 --> 00:07:38,500
And as it is the case with malware, it is always tricky to deal with them.

43
00:07:38,500 --> 00:07:45,720
For example, some of the samples do not reproduce the same indicators of compromise over time.

44
00:07:46,360 --> 00:07:55,640
This can be because, for example, the command and control server is down, and dealing with packed and protected samples can be hard sometimes.

45
00:07:57,740 --> 00:08:27,460
I also checked the SSDeepHashes of some of the samples, and it was interesting to see that whenever people added repeating patterns to the sample, for example, the same section, or they added the same overlay over and over again to the sample, then it created a repeating pattern in the SSDeepHash as well,

46
00:08:27,460 --> 00:08:36,340
which can be used for detecting a sample which uses machine learning evasion, for example.

47
00:08:36,840 --> 00:08:42,280
This year, we created the Defender and the Attacker challenge.

48
00:08:42,280 --> 00:08:59,680
In the Defender challenge, you had to create your own machine learning model and submit this to the competition in a Docker format, and in the Attacker challenge, now the machine learning models are not available for you, so this is now a black box challenge.

49
00:09:00,440 --> 00:09:12,160
And if you win this competition, either the Defender or the Attacker challenge, you can win some Azure credits for your machine learning research plans.

50
00:09:12,830 --> 00:09:15,540
The defensive track is already over.

51
00:09:15,540 --> 00:09:35,800
We received two submissions that passed the minimum requirements, and the offensive track already started, so I highly recommend you to go to our website, analsec.io, and check out what the competition is.

52
00:09:35,820 --> 00:09:58,700
In this year, we have used the malware families, and if you go to our website, review the terms of service, and then you can download the 50 provided malware samples, and after that, it is your time to modify the samples in order to evade the detection.

53
00:09:58,860 --> 00:10:08,020
And new to this year, you can use an API to check your samples or submit your samples.

54
00:10:08,240 --> 00:10:17,340
I also recommend you that you verify that the malware functionality remains the same in your local Windows box.

55
00:10:18,020 --> 00:10:42,000
Then, when you upload the zip files, or you can just upload the partial zip files, meaning that you only submit some of the samples and not all of them, you can receive one point for each bypassed machine learning model, which means that for every sample,

56
00:10:42,000 --> 00:10:44,800
you can get up to three points.

57
00:10:44,840 --> 00:10:48,760
And as usual, highest score wins.

58
00:10:48,880 --> 00:10:59,160
The details about this will be provided by Hyrum, and in order to claim your prize, you have to publish your solution.

59
00:10:59,840 --> 00:11:07,980
Please note that you have to keep the file names as it were in the original zip file.

60
00:11:07,980 --> 00:11:13,200
This helps us to track which file you modified originally.

61
00:11:14,400 --> 00:11:20,720
We also provide some additional tips and tricks you might use in this competition.

62
00:11:20,720 --> 00:11:29,220
Some of them may not make sense, but you can modify an executable in a lot of different ways.

63
00:11:29,220 --> 00:11:47,860
For example, you can add or remove signatures, change section names properties, modify the import or export tables, create TLS callbacks, change the PA header, fix or change the checksums, add, modify, or remove the version information, create new entry points,

64
00:11:47,860 --> 00:11:51,760
or just change some code or data in it.

65
00:11:53,040 --> 00:12:04,620
Still, it's not allowed to create droppers or self-extracting archives, because this will kind of defeat the purpose of the whole competition.

66
00:12:04,880 --> 00:12:13,770
And this year, keep in mind that multiple registration is against the rules, and it will result in immediate disqualification.

67
00:12:14,190 --> 00:12:29,570
Please do join our Slack channel, where you can discuss everything with us, and you can also discuss your progress with the other participants of this competition.

68
00:12:31,970 --> 00:12:46,550
Just a side note, the whole frontend was created in Python Flask admin, we are using Cloudflare, Nginx, and GUnicorn for scalability and performance reasons.

69
00:12:46,610 --> 00:13:01,690
There are some backend scripts running with Python, scheduled by Chrome, and as it was the case last year, we still use the VMware Sandbox to evaluate the samples.

70
00:13:02,170 --> 00:13:21,070
As mentioned, we already have an API, so if you want to check your sample against machine learning, against all the machine learning models, or just against one machine learning model, you can use the API just to do that.

71
00:13:21,310 --> 00:13:25,930
And also, you can use the API to get the results.

72
00:13:25,930 --> 00:13:40,610
And if you are satisfied with bypassing the machine learning models, you can upload your zip files and query the zip status and the sample statuses as well with the API keys.

73
00:13:40,690 --> 00:13:53,250
This is all I wanted to share with you guys, but please welcome Hiram, who will present you some other tips and tricks you can use to win this competition.

74
00:13:53,330 --> 00:13:54,630
Thank you.

75
00:13:56,640 --> 00:14:07,180
I'm going to describe to you the example solution in the machine learning security evasion competitions attacker challenge that has just begun.

76
00:14:07,400 --> 00:14:16,780
The models that you'll be attacking this year have been submitted by participants of the previous round in the defender challenge.

77
00:14:16,940 --> 00:14:22,960
Two of the models from the previous round have qualified to be included in this round.

78
00:14:22,960 --> 00:14:27,600
In addition, we have hosted our own model for you to attack.

79
00:14:27,600 --> 00:14:36,360
That model is trained on the Ember dataset and includes some basic capability to detect adversarial examples.

80
00:14:36,360 --> 00:14:44,220
The source code and model weights for this defended Ember model are provided on the competition's GitHub site.

81
00:14:44,880 --> 00:14:54,720
However, the remaining models are to you complete black boxes where you only get to observe the hard label predictions.

82
00:14:54,720 --> 00:15:01,120
That is a zero or a one for an output that you provide to the machine learning models.

83
00:15:01,980 --> 00:15:07,680
The final leaderboard ranking will be set by the following rank ordered criteria.

84
00:15:07,980 --> 00:15:20,500
First, the total number of evasions with one point for each of the three ML models times 50 malware samples, meaning that the maximum score is 150.

85
00:15:20,700 --> 00:15:30,280
Remember though that each evasive sample must reproduce its original functionality in a sandbox in order to be awarded a point.

86
00:15:30,280 --> 00:15:39,980
Functionality is verified only when you upload a zip file containing your candidate malware samples.

87
00:15:39,980 --> 00:15:47,580
It will not be verified when you merely query the machine learning models through the API.

88
00:15:47,940 --> 00:15:58,800
In the event of a tie for point number one, contestants will be ranked by the number of model queries used through the API.

89
00:15:59,000 --> 00:16:04,820
And lastly, the timestamp of your final zip upload would break any subsequent tie.

90
00:16:04,820 --> 00:16:14,760
More than likely though, we won't get to point number three, so you should feel incentivized to continue competing right up until the competition deadline.

91
00:16:15,500 --> 00:16:22,060
Even if you see a perfect score on the leaderboard, because you might achieve that same perfect score but do it more efficiently.

92
00:16:23,400 --> 00:16:27,540
So as a contestant, you can choose any strategy you'd like to compete.

93
00:16:27,540 --> 00:16:35,660
But to demonstrate one possible strategy, we have released some example code on the competition's GitHub site.

94
00:16:35,660 --> 00:16:41,700
You can find more information about the nitty-gritty details of this approach on the website.

95
00:16:41,700 --> 00:16:49,820
It essentially is using a discrete optimization technique over a space of functionality preserving file modifications.

96
00:16:50,040 --> 00:16:55,300
However, the general strategy might be more useful for you to adopt.

97
00:16:55,380 --> 00:17:09,180
The strategy consists of, is really simple, consists of doing a bunch of bulk work using an algorithm in part A and then kind of batting cleanup for manual manipulation of malware samples in part B.

98
00:17:09,180 --> 00:17:12,880
And I'm going to be describing and demoing the code for part A today.

99
00:17:13,100 --> 00:17:23,320
In it, because we'd like to be efficient in the number of queries against the hosted machine learning models, we'll actually break this attack into two parts.

100
00:17:23,320 --> 00:17:34,860
An offline attack where we use the defended Ember model for which we have code to kind of work out our strategy and generate initial malware samples.

101
00:17:34,860 --> 00:17:41,020
We hope that those seeds might evade some of the online models that are hosted.

102
00:17:41,520 --> 00:17:54,760
And then in the online attack, we'll take those initial seeds and the algorithm will further optimize and discover additional file modifications required to evade the online hosted models.

103
00:17:54,760 --> 00:18:24,480
Some tricks that we're using here include label smoothing, where we're converting the hard label outputs into a soft score by averaging four things, the three hard label outputs from the hosted machine learning models, as well as a local score from a local machine learning Ember model that will be used as a heuristic to kind of guide the optimization process.

104
00:18:25,520 --> 00:18:31,820
So as I demo this code, I want you to please be aware that this code writes malware to disk.

105
00:18:31,820 --> 00:18:37,140
So please do run this code only using a Linux VM.

106
00:18:38,040 --> 00:18:45,720
To begin, we initialize the attack by analyzing a connect collection of benign files.

107
00:18:45,720 --> 00:18:54,500
This init subcommand extracts elements of these benign files that will be later injected into the malware.

108
00:18:55,660 --> 00:19:02,640
To launch our offline attack, we'll run a local copy of the Ember model in the top window.

109
00:19:02,640 --> 00:19:11,800
Then in the bottom window, we'll use the run command, passing in malware samples that we downloaded after registering on the website.

110
00:19:11,980 --> 00:19:25,840
The tool will then write successful evasion attempts to pass one slash success that we've specified in the command line, and failed attempts to pass two slash failure folder.

111
00:19:26,020 --> 00:19:42,130
And also included in each output directory will be the history of file modifications that will be useful if we'd like to pick up to resume a failed attempt.

112
00:19:43,470 --> 00:20:00,830
So to demonstrate that, in a second pass of the offline attack, we'll start with the pass one failures and iterate on the optimization approach, again storing successes and failures to a pass two directory.

113
00:20:04,210 --> 00:20:22,220
So after doing that a number of times and having collected a bunch of candidate samples offline that evade the local defended Ember model, we'll then use those candidates as seeds for an online attack, which now counts against our API query usage.

114
00:20:22,560 --> 00:20:38,900
To do an online attack, simply use this tool with the dash dash online flag, and the optimization will then continue trying to find file modifications that will bypass all three of the hosted models.

115
00:20:41,200 --> 00:20:48,520
Of course, you want to do perhaps as many, you know, as many iterations as necessary, and the online version of this attack.

116
00:20:48,520 --> 00:21:07,780
But after you've done so, in a final pass of the online attack, you can now collect the successful samples into a zip file that you would then validate in a Windows 10 virtual machine, and then upload to the website for validation and leaderboard scoring.

117
00:21:08,400 --> 00:21:23,580
I want to point out that since there is a chance that by running this code, file modifications might break some of the samples, you should always run these samples in a Windows 10 sandbox before uploading to the competition website.

118
00:21:23,700 --> 00:21:30,180
Also note that zip file uploads contribute against your API query count.

119
00:21:30,200 --> 00:21:40,380
So it is to your benefit to double check your work and make sure that any files you upload are functional, so you don't have to redo that work and upload again.

120
00:21:40,500 --> 00:22:03,180
As a final note, kind of tricky that since the hosted models might be actually changing state and learning from the queries that you and others are giving them, there's a possibility that an evasive variant sample that you discovered along the way may no longer evade a model by the time that you upload your zip file.

121
00:22:03,180 --> 00:22:08,580
So be... I don't know that will be the case, but please be aware that that is a possibility.

122
00:22:09,020 --> 00:22:11,300
So with that, good luck on the competition.

123
00:22:12,220 --> 00:22:14,800
Visit the website at emailsecta.io.

124
00:22:14,800 --> 00:22:17,920
The competition will run for over six weeks.

125
00:22:18,540 --> 00:22:32,220
And those who are ranked first and second on the leaderboard will win our grand and first prizes respectively, so long as they publish their solution.

126
00:22:33,620 --> 00:22:42,020
And with that, I'd like to thank our sponsors, Microsoft and Cujo AI, with partners MRG Efitas and VMRay.