1
00:00:41,340 --> 00:00:42,960
Thanks for tuning into our talk.

2
00:00:42,960 --> 00:00:45,640
I'm Spencer Cureton, and with me is Alan Sagenovic.

3
00:00:45,640 --> 00:00:52,720
We are on the abuse operations team at Heroku, where we keep users from doing bad things on, to, and from the platform.

4
00:00:52,920 --> 00:00:55,820
We're like mall cops for our corner of the internet.

5
00:00:56,360 --> 00:01:00,120
This talk focuses on abuse and how we hunt for it on Heroku.

6
00:01:00,580 --> 00:01:05,380
So Heroku is a platform as a service that was purchased by Salesforce about 10 years ago.

7
00:01:05,380 --> 00:01:12,900
It provides an easy interface to quickly go from idea to online application without having to manage and configure the underlying infrastructure.

8
00:01:13,400 --> 00:01:20,400
So to put it simply, we run anyone's code on the internet as directed from unknown sources millions of times a day.

9
00:01:21,140 --> 00:01:24,260
Okay, we are RCE as a service.

10
00:01:24,340 --> 00:01:31,980
The part of Heroku we're talking about is referred to as the free tier, or the common runtime, where all you need is an email address to create an account.

11
00:01:31,980 --> 00:01:36,160
Heroku has other offerings that are not like this, but we won't be talking about them today.

12
00:01:36,200 --> 00:01:41,420
Our focus will be on how we defend the common runtime from abusive bad actors.

13
00:01:42,540 --> 00:01:45,680
So just defining abuse is a challenge within itself.

14
00:01:46,000 --> 00:01:49,940
We define abuse as the unintended monetization of anything.

15
00:01:50,240 --> 00:01:58,300
So if you have a free service, a free trial, or any offering that has an angle to extract value, somebody is already trying to figure out a way to make money off of it.

16
00:01:58,300 --> 00:02:04,980
It's a very fine line between a business trying to get off the ground and an abuser trying to take advantage of your services.

17
00:02:05,140 --> 00:02:07,620
Sometimes it's obvious, like theft of resources.

18
00:02:07,860 --> 00:02:09,400
Other times it's not so clear.

19
00:02:09,600 --> 00:02:11,740
For example, think about a social media poll.

20
00:02:11,740 --> 00:02:17,220
It's intended for honest input, but it's gameable because it's not one person, one vote.

21
00:02:17,240 --> 00:02:18,920
It's one account, one vote.

22
00:02:18,920 --> 00:02:22,380
So the person in control of the most accounts wins.

23
00:02:22,380 --> 00:02:27,460
The problem isn't new, the scams aren't new, but we see them leveraging more and more of the cloud.

24
00:02:27,780 --> 00:02:31,480
And we feel like we have a unique perspective that we would like to share.

25
00:02:32,640 --> 00:02:41,500
So in a more practical layer, we see a portion of this bigger abuse picture realized through misuse, malice, and crime on our platform.

26
00:02:41,960 --> 00:02:48,160
Legal documents like the Acceptable Use Policy, the AUP, provide a list of prohibited content and prohibited actions.

27
00:02:48,160 --> 00:02:54,640
So it's the list of don'ts for users to follow, and also acts as a set of guidelines for our team to uphold.

28
00:02:55,180 --> 00:02:58,340
Obviously, this can't cover every type of abuse activity.

29
00:02:58,360 --> 00:03:04,840
And even if it did, it would act as a laundry list for an abuser to know what kind of badness could be performed from the platform.

30
00:03:05,440 --> 00:03:08,580
Let's look at how we group violations of the AUP.

31
00:03:09,060 --> 00:03:12,340
There are unintentional and intentional violations.

32
00:03:12,340 --> 00:03:20,580
Now, unintentional could be a customer that deploys a poorly written app that consumes more than its share of resources, and now they've become a noisy neighbor.

33
00:03:21,000 --> 00:03:23,020
So think about like a runaway web scraper.

34
00:03:23,020 --> 00:03:26,400
Not malicious, but kind of like they were accidentally jerks.

35
00:03:26,880 --> 00:03:31,140
Then we have the intentional abusers, and these are the folks that make this job really interesting.

36
00:03:31,480 --> 00:03:37,040
They have skills that range from script pity to rogue advanced developer, which we call our apex actors.

37
00:03:38,160 --> 00:03:43,600
Skids will be skids and deploy code found on GitHub, YouTube, or whatever their buddy gave them to run.

38
00:03:43,860 --> 00:03:50,520
At times, we'll get what feels like a swarm of abuse centered around a particular app, like a proxy kit or a peer-to-peer app.

39
00:03:50,780 --> 00:03:59,420
Then there are the quick and loud abusers that could care less about being detected and will attempt to stand up a DDoS farm and attack a target as quickly as possible.

40
00:03:59,880 --> 00:04:04,740
Now, the actors that are patient, skilled, and criminal are the more dangerous group.

41
00:04:04,740 --> 00:04:08,740
These actors attempt to iterate past our detection capabilities.

42
00:04:08,740 --> 00:04:17,960
They may try to social engineer support staff in order to gain insight into how they were detected after being suspended, and will try anything to maximize their use of the platform.

43
00:04:18,160 --> 00:04:22,660
Now, these users are not exploiting technical vulnerabilities within the platform.

44
00:04:22,740 --> 00:04:25,980
They take advantage of gaps in business logic.

45
00:04:25,980 --> 00:04:29,020
So technically, everything's working as designed.

46
00:04:29,200 --> 00:04:32,100
The majority are not too advanced, but are very persistent.

47
00:04:32,100 --> 00:04:37,700
So if you like to think about APT, small lowercase a, big capital P.

48
00:04:37,760 --> 00:04:39,700
And again, financially motivated.

49
00:04:39,860 --> 00:04:51,060
Now, persistent here means that after an account has been suspended, the same actor will try to create a new account with a different email address or a sub-inbox variant of the original email address.

50
00:04:51,100 --> 00:04:55,060
We'll cover a few campaigns during the show and tell section of this talk.

51
00:04:56,400 --> 00:05:05,440
Overall, we define our team, Abuse Operations, as a specialized DFIR team that requires a wide view across the business as shown in this slide.

52
00:05:05,580 --> 00:05:12,680
Our goal is to push bad actors from the platform and put a spotlight on abuse patterns for engineering teams and the business to address.

53
00:05:12,700 --> 00:05:18,620
In order to identify those bad actors and stay on top of them, we need to be proactive and hunt.

54
00:05:18,660 --> 00:05:22,000
Now I'll hand it over to Alan to discuss the art of the hunt.

55
00:05:22,220 --> 00:05:22,780
Alan?

56
00:05:23,180 --> 00:05:24,700
Hi, thank you, Spencer.

57
00:05:24,700 --> 00:05:25,540
I'm Alan.

58
00:05:25,540 --> 00:05:29,120
I'm Akam on the Discord, if you want to find me later.

59
00:05:29,640 --> 00:05:36,900
Now, we're quickly going to breeze through this, how we hunt, mostly just to give better context around the next section, which is really going to be the meat of it.

60
00:05:37,420 --> 00:05:38,760
Now, there's a lot on these slides.

61
00:05:38,760 --> 00:05:40,340
I'm not going to talk about every point.

62
00:05:40,340 --> 00:05:43,060
Just kind of the more interesting stuff, the more pertinent stuff.

63
00:05:43,060 --> 00:05:49,380
If you want to hear the philosophy behind this, go look at the talk that we did at that other conference that happened a little earlier this week.

64
00:05:50,360 --> 00:05:58,000
Besides, I think that you guys at the Blue Team Village probably do some semblance of this already.

65
00:05:58,000 --> 00:06:05,100
So we gather suspicious indicators from any of a number of different places, as you can see listed.

66
00:06:05,160 --> 00:06:14,540
For example, an app with an abnormally high CPU would generate some sort of event-based alarm that would give us a hint to go look at it.

67
00:06:14,540 --> 00:06:20,320
On the slide, as you can see, there's a short list of some of the indicators that we collect of the various places we collect them from.

68
00:06:20,320 --> 00:06:31,040
Now, because we're highly instrumented, and because Heroku runs on Heroku, this is how we can actually gain home-field advantage, by putting these internal and external indicators together.

69
00:06:31,040 --> 00:06:35,620
Now, for this moment, just understand that the indicators that we're getting are considered really low-value.

70
00:06:35,620 --> 00:06:40,180
By themselves, they don't have any kind of context, or rarely enough context to actually make a decision.

71
00:06:41,800 --> 00:06:48,200
Now, given a starting suspicious indicator or two, we form some sort of hypothesis about the intent of the actor.

72
00:06:48,200 --> 00:06:52,580
We start looking at related indicators and try to prove or disprove that hypothesis.

73
00:06:52,620 --> 00:07:03,080
Now, each of these indicators is not proof of intent of any kind individually, but as you can see by this list, you can see that they collectively start making a really strong case.

74
00:07:03,080 --> 00:07:08,320
Collection of low-value indicators does become a very good footprint of malicious activity.

75
00:07:10,380 --> 00:07:15,120
To increase the confidence of what we're looking at, we keep asking that question, what else you got?

76
00:07:15,120 --> 00:07:21,980
For instance, a face-rolled email address, like the one shown, is a pretty good indicator that the user doesn't really value the account.

77
00:07:22,020 --> 00:07:25,060
Same with app names, if they got a face-rolled app name, too.

78
00:07:25,060 --> 00:07:29,720
Rumor is a great indicator of badness, though, because it could be just for discarding it.

79
00:07:29,720 --> 00:07:30,940
Test, discard.

80
00:07:30,960 --> 00:07:40,600
However, a face-rolled email address with a collection of face-rolled app names, in addition to multiple attempts to verify a credit card, is worth a deeper look.

81
00:07:43,490 --> 00:07:45,110
We try to establish intent.

82
00:07:45,410 --> 00:07:46,870
There are two goals here.

83
00:07:46,870 --> 00:07:48,590
One is to determine what the actor is trying to do.

84
00:07:48,590 --> 00:07:52,010
And the other is to determine what the actor is trying to do before they do it.

85
00:07:52,030 --> 00:07:57,810
Now, the key piece here is that goal number two is something that we reserve for actors that we've actually seen before.

86
00:07:57,810 --> 00:08:00,030
Actors that have tried to do something bad before.

87
00:08:00,030 --> 00:08:03,630
This is how we avoid a situation of doing pre-crime.

88
00:08:03,630 --> 00:08:08,350
We don't want to necessarily kick people off the platform before they've done bad.

89
00:08:08,350 --> 00:08:15,030
But if we have indicators that they are a same returning actor that has already been declared persona non grata, then maybe we want to know as early as possible.

90
00:08:15,030 --> 00:08:16,490
Now, there is an exception to this.

91
00:08:16,550 --> 00:08:19,630
There is software that we don't let people deploy to the platform.

92
00:08:19,790 --> 00:08:25,710
But we're not talking about that in any great degree, although we'll mention it later in contrast to other things.

93
00:08:27,930 --> 00:08:31,690
Now, making the decision and taking the action is kind of the crux of the matter.

94
00:08:32,010 --> 00:08:36,510
We've collected a bunch of indicators that give us a good idea of what they're trying to do.

95
00:08:36,510 --> 00:08:37,470
So we take the action.

96
00:08:37,470 --> 00:08:42,670
If the intent is bad, then the action is suspension, evidence collection, so on and so forth.

97
00:08:42,670 --> 00:08:46,970
If it's not bad, watch lists, allow lists, or false positive checks.

98
00:08:47,750 --> 00:08:54,190
And sometimes it's not easy to determine what the intent is, in which case we'll throw it back out of the stack.

99
00:08:54,190 --> 00:08:57,870
We'll examine it again should it pop up again in a fresh alert.

100
00:08:59,090 --> 00:09:01,810
We investigate for more indicators.

101
00:09:02,470 --> 00:09:06,650
Now, really the question here is, have we found all the accounts that are associated with the badness?

102
00:09:06,810 --> 00:09:12,950
And have we found all the indicators associated with all the accounts that point to the badness?

103
00:09:12,950 --> 00:09:22,070
We cycle this a couple of different times to make certain that we've got as big a view as possible, even retroactively, so we can pick up early starts, early campaigns, so on and so forth.

104
00:09:22,070 --> 00:09:27,050
And this is all done to strengthen our ability to do automation.

105
00:09:30,150 --> 00:09:37,410
Automation is... we're doing it because, obviously, we never want to see the same abuse again, and this is how we reduce all the toil of all the things that we do.

106
00:09:37,590 --> 00:09:45,230
The automation of the footprint includes the alerting and the actioning on a subset of indicators, including what we call a partial match.

107
00:09:46,990 --> 00:09:49,570
And then we do it all over again.

108
00:09:49,810 --> 00:09:51,910
Now, some of that requires a little bit more.

109
00:09:52,110 --> 00:09:56,370
But in a nutshell, if you're a visual learner, here's what the loop looks like.

110
00:09:57,490 --> 00:09:58,530
Okay?

111
00:09:58,910 --> 00:10:00,370
Let's go next.

112
00:10:00,810 --> 00:10:05,390
Within our tooling, we have... is where our codified indicators live.

113
00:10:05,390 --> 00:10:12,730
So we take the indicators that we've identified, and it could be something as simple as a substring match on a registration of an email address.

114
00:10:12,730 --> 00:10:18,470
Or it could be something more complex, like a Levenstein distance to a particular word, or any of these kinds of things.

115
00:10:18,470 --> 00:10:23,350
And we codify them as, literally, as code into any of these tooling.

116
00:10:23,350 --> 00:10:28,230
Now, the column on the left, we have an event collection agent, similar to a classic SEM.

117
00:10:28,230 --> 00:10:32,630
We have database query tools that'll run automatically or manually when we need to check stuff.

118
00:10:32,630 --> 00:10:34,270
And we have malware analysis tools.

119
00:10:34,270 --> 00:10:41,310
So we'll do our queries within those tools so that we can actually get these indicators, these codified indicators out.

120
00:10:42,210 --> 00:10:46,110
Now, we also have, as everybody, we have a ticket system.

121
00:10:46,110 --> 00:10:49,930
Now, the ticket system is not normally considered part of the tooling in a lot of cases.

122
00:10:49,930 --> 00:11:01,550
We like to consider it part of the tooling because our ever-increasing goal to reduce Toro means that we also have to automate ticketing where it makes sense or where it's possible as well.

123
00:11:01,550 --> 00:11:03,150
And we have our suspension tools.

124
00:11:03,230 --> 00:11:13,070
Suspend at various levels, but the really important thing there is that they're set up in such a way that they have business rules and guardrails so that we don't accidentally suspend the entire world if we make a mistake.

125
00:11:14,190 --> 00:11:16,690
Now, we track two general types of activities.

126
00:11:16,690 --> 00:11:19,670
We track campaigns and prohibited softwares, as we previously said.

127
00:11:19,670 --> 00:11:24,370
Campaigns track actors, or sometimes they're victims, whatever makes more sense.

128
00:11:24,370 --> 00:11:29,550
And it also tracks the custom software that they might write to be able to do the things that they're trying to do.

129
00:11:30,230 --> 00:11:39,370
Whereas prohibited software is tracking malware, tracking C2, tracking known abusive software, including the stuff that has a deploy to Heroku button sitting in GitHub.

130
00:11:40,670 --> 00:11:43,410
Now, ultimately, we're looking to track actors.

131
00:11:43,610 --> 00:11:48,010
There doesn't seem to be a great way to do this, and we're building up towards more of that.

132
00:11:48,010 --> 00:11:57,730
But what we found is that actors really do end up being a collection of TTPs codified as a constellation of indicators, and we call that a footprint.

133
00:11:57,890 --> 00:12:00,850
Now, footprints can overlap, partially or fully.

134
00:12:01,230 --> 00:12:07,890
And how we implement footprint detection is how we detect actor behavior changes, and multiple actors doing the same thing.

135
00:12:07,910 --> 00:12:16,230
Tracking footprints is kind of difficult, though, and we're looking to be improving the process as we go on this year so we can speed up this whole loop.

136
00:12:16,230 --> 00:12:21,510
But even today, with what we have, we can detect pivots.

137
00:12:21,930 --> 00:12:25,710
A pivot is what we call a change to a TTP of a returning actor.

138
00:12:25,710 --> 00:12:27,850
Now, different from a pen tester pivot.

139
00:12:28,210 --> 00:12:36,550
If a footprint is a collection of indicators, then given enough of them, a partial match usually indicates a change in behavior of the same actor.

140
00:12:37,190 --> 00:12:38,390
Actors are people, too.

141
00:12:38,430 --> 00:12:47,270
When an actor gets suspended, they do have this tendency to test how by isolating for variables and making very small changes over time until they get it working again.

142
00:12:47,570 --> 00:12:55,610
So from a threshold detection, like shown, works really well as an accurate way to detect their return and still take some action.

143
00:12:56,010 --> 00:13:02,450
Then updating the footprint with the new indicators, reflecting the change, just keeps that footprint accurate.

144
00:13:02,450 --> 00:13:06,790
And this is designed to make it difficult for a persistent actor to regain that foothold.

145
00:13:06,890 --> 00:13:10,350
And it frustrates the heck out of them, and that's always a pleasure.

146
00:13:11,110 --> 00:13:13,350
We also have what we call a splash.

147
00:13:13,970 --> 00:13:26,510
When we block a particular actor under a footprint, sometimes what we'll see is that we'll see a whole bunch of different footprints or different partial footprint matches against that actor's footprint.

148
00:13:26,510 --> 00:13:27,850
And we call this a splash.

149
00:13:27,850 --> 00:13:29,850
It's like slapping a mud puddle.

150
00:13:30,070 --> 00:13:38,650
It's a whole bunch of different actors that are suddenly trying to get back onto the platform that have actually all been hiding under the same TTP umbrella, the same footprint.

151
00:13:39,150 --> 00:13:41,550
Sometimes they don't even know that they're doing it.

152
00:13:41,550 --> 00:13:52,170
Now, they're trying to... often it turns out that they're sharing TTPs, they're sharing techniques of doing things on the platform through everything from YouTube videos to forums to whatever.

153
00:13:52,830 --> 00:14:08,430
And sometimes when we find the source of those, if they're doing really particular badness, you know, phishing attacks and stuff like that, and watching them discuss how they're going to make changes, if we can keep on top of those changes and add them to the footprint before they're actually even deployed,

154
00:14:08,430 --> 00:14:11,250
then we can keep on top of them as well.

155
00:14:11,250 --> 00:14:13,490
Shout out to our fantastic Threat Intel team.

156
00:14:13,490 --> 00:14:17,530
They kind of help us find all that extra info if that is the need.

157
00:14:17,930 --> 00:14:29,910
Now, when we have all that tooling in place, when pivot detection is working and designed, and when we manage to stop the actor, sometimes even before they've begun, then they only have two choices.

158
00:14:29,910 --> 00:14:33,530
They can either go somewhere else, or they can entirely retool.

159
00:14:34,410 --> 00:14:40,770
Retooling requires them to change everything, because we've got footprints for nearly everything that they're doing.

160
00:14:40,770 --> 00:14:42,230
I don't want to say absolutely everything.

161
00:14:42,230 --> 00:14:42,930
We don't know.

162
00:14:42,930 --> 00:14:44,590
But pretty damn close.

163
00:14:44,990 --> 00:14:48,130
Now, this makes them indistinguishable from a new actor.

164
00:14:48,130 --> 00:14:51,690
But honestly, we don't really care at that point.

165
00:14:51,690 --> 00:14:54,930
If we end up being a show-up actor, so be it.

166
00:14:54,930 --> 00:14:56,470
We'll track them as a new actor.

167
00:14:56,470 --> 00:15:01,350
Sometimes they've actually retooled to the point where they decide to do something completely different than what they've been doing before.

168
00:15:02,890 --> 00:15:04,430
Now, we don't usually...

169
00:15:05,390 --> 00:15:14,970
Because we're pretty thorough in our collection, sometimes we still find evidence that a new actor is actually an old actor retooled, and that's kind of fascinating for different reasons.

170
00:15:14,970 --> 00:15:19,610
And when we find that evidence, we can still put it together and make great work of that.

171
00:15:19,610 --> 00:15:25,350
And sometimes that little tenuous link, as you probably know, is something that's really, really simple.

172
00:15:25,350 --> 00:15:34,470
Something as simple as putting a couple of numbers, the same 1, 2, 3, 4 numbers in there, constantly using four dots in their email address.

173
00:15:34,850 --> 00:15:40,290
Now, given that as our hunt loop, let's go back to Spencer for the show and tell.

174
00:15:42,190 --> 00:15:42,990
All right.

175
00:15:42,990 --> 00:15:43,790
Thanks, Alan.

176
00:15:44,190 --> 00:15:46,930
Let's look at a few campaigns that have touched our hearts.

177
00:15:46,970 --> 00:15:49,530
And, well, we'd like to think we've touched theirs, too.

178
00:15:50,230 --> 00:15:57,690
The first actor appeared on our radar by spinning up 100 apps per user account across multiple accounts, all from free email providers.

179
00:15:57,850 --> 00:16:03,830
So a user account is limited to running five apps until they verify their account by adding a payment method, like the credit card.

180
00:16:03,830 --> 00:16:06,090
After that, 100 apps is the new max.

181
00:16:06,190 --> 00:16:12,110
So here we have new accounts immediately adding a credit card and going full tilt to 100 apps.

182
00:16:12,570 --> 00:16:13,730
That's not normal.

183
00:16:13,810 --> 00:16:15,510
Normal users don't do this.

184
00:16:15,510 --> 00:16:19,150
That activity stands out and gets on our radar fast.

185
00:16:19,170 --> 00:16:22,890
And to top things off, fraudulent credit cards were used to verify each account.

186
00:16:22,890 --> 00:16:25,110
And all of the apps were identical.

187
00:16:26,210 --> 00:16:31,970
So we inspected one of the apps and saw that they were running a Tor hidden service accessible through SSH.

188
00:16:31,990 --> 00:16:35,730
Here's a screenshot of the script that would run once their app was deployed.

189
00:16:37,990 --> 00:16:42,930
We suspended the accounts and very soon the actor was back and we had their attention.

190
00:16:44,650 --> 00:16:49,010
Soon after, we saw their underlying intent, mining Monero.

191
00:16:49,010 --> 00:16:53,990
The apps were mining with LibXMR and sending all traffic through Tor.

192
00:16:54,630 --> 00:17:03,590
Now after seeing this, we believe that the initial wave of apps that were running Tor were really part of some capabilities testing, so some real live bad actor DevOps.

193
00:17:04,770 --> 00:17:11,390
We continued to suspend the accounts and the actor would pivot, each time trying to minimize their footprint to avoid detection.

194
00:17:11,410 --> 00:17:19,790
So the top image is from an app that packed the Tor and mining binaries in a zip file, which would then be unzipped with a unique passphrase at the time of deployment.

195
00:17:19,910 --> 00:17:27,470
The bottom image is from a later pivot, this time using GPG encrypted payloads with unique names and unique passphrases.

196
00:17:27,470 --> 00:17:31,670
So decrypt, unpack, and mine, sending everything through Tor.

197
00:17:32,070 --> 00:17:34,190
Now this user was really fast.

198
00:17:34,370 --> 00:17:40,270
They appeared to be retooling their apps and back online within just four hours of being suspended.

199
00:17:41,630 --> 00:17:46,990
And this type of activity at this scale threatened the performance of some of the shared resources.

200
00:17:46,990 --> 00:17:51,950
Other customers were impacted due to these noisy neighbor situations, so we had to stay on top of this actor.

201
00:17:51,950 --> 00:17:57,030
But they were automating their operation and getting faster, mostly during our off hours.

202
00:17:57,150 --> 00:18:02,030
So playing never-ending whack-a-mole was unsustainable and really a path toward team burnout.

203
00:18:02,710 --> 00:18:08,370
So we took a step back and we looked at all of the data provided from this actor's use of the platform.

204
00:18:08,370 --> 00:18:09,610
Home field advantage, right?

205
00:18:09,610 --> 00:18:14,490
This turned into a list of indicators which we could use to form a TTP footprint.

206
00:18:14,950 --> 00:18:18,030
The actual footprint is very close to what's on this slide.

207
00:18:18,250 --> 00:18:24,750
As you can imagine, to make the most from cryptocurrency mining, the actor needs to spread out across as many accounts as possible.

208
00:18:25,210 --> 00:18:29,710
So to make it worth their time, they try to automate any part of their operation, like account creation.

209
00:18:30,050 --> 00:18:34,250
That kind of automation generates patterns that we can use for detection.

210
00:18:34,250 --> 00:18:47,490
In this case, we look for groups of accounts that signed up from an IP address, that happened to be a Tor exit node, from a particular free email provider, and had an email address that matched a certain naming pattern.

211
00:18:47,570 --> 00:18:53,650
If a new account matched those three indicators, we already have high confidence that it's an account controlled by this actor.

212
00:18:53,970 --> 00:19:02,890
Next, the accounts would quickly create five apps, all with a positive size, and they would leave the names as the default Heroku haiku.

213
00:19:02,890 --> 00:19:11,330
Then sampling some of the apps, we could confirm if they had a GPG blob, and if they had the startup script, which decrypted and unpacked the payload.

214
00:19:11,510 --> 00:19:14,270
Again, normal users don't do this.

215
00:19:15,090 --> 00:19:18,870
So when we joined these indicators together, the false positive rate was zero.

216
00:19:18,870 --> 00:19:20,610
Very high confidence, right?

217
00:19:20,770 --> 00:19:26,990
So now matching these 10 indicators, low value by themselves, gave us a high value decision.

218
00:19:26,990 --> 00:19:32,830
We can now identify this actor without having to pick everything apart, every time, on every pivot.

219
00:19:33,030 --> 00:19:37,930
Just focus on what's different, which saved us a bunch of time, while wasting the actors.

220
00:19:38,270 --> 00:19:41,050
They soon gave up, and appeared to have left the platform.

221
00:19:42,350 --> 00:19:44,090
Let's look at another campaign.

222
00:19:44,330 --> 00:19:47,150
This actor came to our attention through a phishing report.

223
00:19:47,210 --> 00:19:54,230
We suspended their account, and very soon, we saw that the same phish was reported again, so we tracked them as a campaign.

224
00:19:54,230 --> 00:19:58,090
They were targeting customers of a specific insurance provider.

225
00:19:58,410 --> 00:20:06,370
When inspecting one of their apps, we see the following directory structure, which contains message-sending capabilities in the mailer.py file.

226
00:20:06,630 --> 00:20:13,010
Here, they would use third-party messaging add-ons to either distribute their messages or handle their loot.

227
00:20:13,730 --> 00:20:19,710
We suspended the accounts and hunted for more, and found an entirely different app used for the same campaign.

228
00:20:19,830 --> 00:20:24,210
This time, it was a JavaScript React app, which had some added functionality.

229
00:20:24,310 --> 00:20:28,550
Like, now they were going after credit card information, as well as credentials.

230
00:20:29,430 --> 00:20:35,170
This screenshot shows the collection of credit card number, expiration date, and CVV in French.

231
00:20:37,630 --> 00:20:43,310
Really quick, I want to share these screenshots from another pivot, where the actor repurposed somebody else's kit.

232
00:20:43,310 --> 00:20:47,470
You can probably Google for some of these keywords, and still find it out there somewhere on the internet.

233
00:20:48,770 --> 00:20:51,650
We developed a TTP footprint for this actor.

234
00:20:51,650 --> 00:20:58,710
This time, it was more app-focused, since they didn't need to spread out across multiple accounts to profit like the crypto-miner did.

235
00:20:58,950 --> 00:21:05,450
We also used these unique indicators to update rules in our detection and response tooling, just whenever possible, whenever it was appropriate.

236
00:21:05,710 --> 00:21:08,010
But this actor was still very persistent.

237
00:21:08,370 --> 00:21:09,910
We want to break their spirit.

238
00:21:09,910 --> 00:21:13,330
We don't want to help them improve their coding skills as we go back and forth.

239
00:21:13,330 --> 00:21:19,550
Typically, being in proximity to the sensitive information that they were gathering is like being in proximity to toxic waste.

240
00:21:19,550 --> 00:21:22,430
Unless you're the bad actor, then it has value.

241
00:21:22,430 --> 00:21:25,470
But it's only valuable if the information is valid.

242
00:21:25,750 --> 00:21:33,770
So considering this, we did our best to get that information into the appropriate hands, so that passwords, payment information could all get reset and nullified.

243
00:21:33,970 --> 00:21:36,790
Let's strip the value and really waste their time.

244
00:21:37,290 --> 00:21:43,470
Also, we found ways to notify the other service providers that were used in the campaign, which put even more pressure on this actor.

245
00:21:43,910 --> 00:21:48,910
These two things disrupted their operations enough that it appears to have pushed them from the platform.

246
00:21:49,330 --> 00:21:57,410
But some of the external indicators that we used to track them, like what you can find on EARLScan, suggest that they're still active, just not on our platform.

247
00:21:59,170 --> 00:22:01,570
The third campaign is from a couple of years ago.

248
00:22:01,570 --> 00:22:07,930
It started as a DMCA takedown request, claiming that a Heroku app was distributing a copyrighted e-book.

249
00:22:10,450 --> 00:22:18,450
When we checked out the link that was reported, the PDF was not the full e-book, but had the same title and same cover art as the real book.

250
00:22:18,670 --> 00:22:21,730
But it also had a link to download the book on another site.

251
00:22:22,150 --> 00:22:23,430
Nothing weird there, right?

252
00:22:23,670 --> 00:22:29,790
This screenshot is not from what was reported, but it was from one of the other apps that we found while researching this campaign.

253
00:22:31,290 --> 00:22:39,470
When we investigate an app, we're looking for just enough information to establish intent with high confidence, while hunting for unique indicators and general functionality.

254
00:22:39,970 --> 00:22:44,710
Now, with an app like this, we don't have enough time to become subject matter experts and pick everything apart.

255
00:22:44,710 --> 00:22:52,890
So we typically start with something like the index.php file, or look for some other unique files that really stand out, directories.

256
00:22:53,010 --> 00:22:56,930
So in this case, these were some of the files and directories that stood out.

257
00:22:56,990 --> 00:23:01,610
func.php, the document generator PHP files, and the keyw directory.

258
00:23:01,610 --> 00:23:05,150
Now, func is Indonesian for function, by the way.

259
00:23:05,150 --> 00:23:07,450
It's going to be important in a few seconds.

260
00:23:07,990 --> 00:23:11,110
Let's see what's in the keyw directory first.

261
00:23:11,810 --> 00:23:13,610
Lots of text files.

262
00:23:15,870 --> 00:23:18,090
Lots of text files.

263
00:23:19,490 --> 00:23:27,030
Each filled with lots of book titles, from New York Times bestsellers to obscure auto-wiring diagrams.

264
00:23:27,090 --> 00:23:29,670
Now, these screenshots are just from one of the files.

265
00:23:29,970 --> 00:23:33,270
Overall, this app had over 300,000 titles.

266
00:23:34,190 --> 00:23:40,290
To understand how these titles are being used, we went back to the root and searched for the keyw string in the PHP files.

267
00:23:40,290 --> 00:23:46,270
Now, in this screenshot, we see that they were creating sitemap XML files for each book index file.

268
00:23:46,610 --> 00:23:47,370
Huh.

269
00:23:48,370 --> 00:23:51,570
And they would make sitemaps for each book title.

270
00:23:52,010 --> 00:23:56,570
These sitemaps would be sent to search engines for...

271
00:23:57,290 --> 00:23:58,590
Black Hat SEO!

272
00:23:59,270 --> 00:24:05,090
We saw other tactics like cloaking, where search engines would return different results than a normal visitor.

273
00:24:05,450 --> 00:24:07,890
Some apps would perform multiple redirects.

274
00:24:07,890 --> 00:24:14,770
And then they would do link manipulation by taking these long book titles, slicing them up, creating a whole bunch of different links, and sending them back out to the search engines.

275
00:24:15,410 --> 00:24:18,490
And I'm sure there's more, but, I mean, this was more than enough to establish intent.

276
00:24:18,490 --> 00:24:20,870
We don't want this kind of activity on our platform.

277
00:24:21,490 --> 00:24:24,210
Now, we had to see how widespread it was.

278
00:24:25,210 --> 00:24:28,890
Turned out, there were thousands of accounts participating in this campaign.

279
00:24:29,470 --> 00:24:40,850
Many were found just by looking at patterns from the sign-up data, like source IP and user-agent string combinations, as well as the use of email sub-inboxing patterns to create multiple accounts.

280
00:24:40,850 --> 00:24:43,690
So, you know, like how you can move the dot around in a Gmail account?

281
00:24:44,210 --> 00:24:45,210
It's been fixed.

282
00:24:45,450 --> 00:24:50,190
The overall campaign included a variety of apps that were ultimately doing the same thing.

283
00:24:50,550 --> 00:24:56,150
Some were even sniping orphaned subdomains of reputable customers in order to boost their rankings with the search engines.

284
00:24:56,210 --> 00:24:57,550
It's been fixed as well.

285
00:24:57,670 --> 00:25:00,750
So, abusing our systems to abuse the search engine algorithms.

286
00:25:00,810 --> 00:25:02,870
I mean, it's just abuse all the way down.

287
00:25:03,450 --> 00:25:06,030
Now, here's a quick look at some of the other indicators.

288
00:25:06,030 --> 00:25:09,530
Now, we mentioned hunting within the sign-up data.

289
00:25:09,530 --> 00:25:17,270
These accounts were created within the typical working hours of the Indonesian time zone, and only on five days out of the week.

290
00:25:17,510 --> 00:25:23,250
They weren't non-stop sign-ups, but you see these bursts of accounts being created a few times a day within those working days.

291
00:25:23,390 --> 00:25:26,650
Other fun indicators were typos and Indonesian words.

292
00:25:26,990 --> 00:25:34,070
One that had me scratching my head at first was S-U-K-S-E-S, which translates to how it sounds.

293
00:25:34,070 --> 00:25:35,230
Success.

294
00:25:35,630 --> 00:25:38,150
So, we successfully booted them from the platform.

295
00:25:39,490 --> 00:25:46,790
Now, I wish we could dive deeper and share more, but we want to briefly mention some of the lessons that we've learned while fighting abuse on a platform.

296
00:25:46,870 --> 00:25:52,150
First, demotivating a bad actor can be very effective and satisfying.

297
00:25:52,610 --> 00:26:02,970
Know your environment and hunt for internal indicators, no matter how little value they appear to have, because collectively they can show intent, build confidence, and help make a high-value decision.

298
00:26:02,970 --> 00:26:09,110
Also, check if there are ways to include external indicators, like OSINT or threat intelligence, and enrich that data.

299
00:26:09,450 --> 00:26:17,310
For us, not focusing on things like suspension count, but rather focusing on patterns that the bad actors were using to take advantage of our services.

300
00:26:17,310 --> 00:26:25,410
Those patterns are worth reporting back to the business so they can be addressed and fixed, and ultimately create a platform that's less hospitable for abuse.

301
00:26:26,750 --> 00:26:28,290
Thanks again for attending our talk.

302
00:26:28,290 --> 00:26:33,710
If you're interested in more of this, but maybe for more of a philosophical or business approach, check out our Black Hat Talk.

303
00:26:33,710 --> 00:26:34,590
Thank you.

304
00:26:35,510 --> 00:26:36,890
Thank you.

305
00:26:36,910 --> 00:26:39,350
Any questions in text talks in Track 1?