1
00:00:08,650 --> 00:00:14,850
Today we're going to be chatting about experience that Jeff and I have had building bug bounty programs, managing programs.

2
00:00:14,870 --> 00:00:17,690
So we'll dive right on into it then.

3
00:00:17,690 --> 00:00:20,170
So a little bit about us.

4
00:00:20,170 --> 00:00:21,690
Perfect, we do have everything working here.

5
00:00:21,690 --> 00:00:22,610
So Jeff?

6
00:00:24,450 --> 00:00:25,490
Hello.

7
00:00:25,650 --> 00:00:27,650
Cool, so quick intro.

8
00:00:27,950 --> 00:00:31,450
Sorry I lost my voice, I blamed a DJ yesterday at O'Shea's.

9
00:00:31,710 --> 00:00:41,690
I enjoy driving fast cars, racing cars professionally, building cars, security research and building bug bounty programs.

10
00:00:41,690 --> 00:00:45,750
It's where I started my career many years ago and just very passionate about it.

11
00:00:45,750 --> 00:00:49,230
And I'm a senior security engineer at GitHub at the moment.

12
00:00:49,850 --> 00:00:51,470
And I am Logan McLaren.

13
00:00:51,470 --> 00:00:53,270
I'm based out of Ottawa, Canada.

14
00:00:53,270 --> 00:00:55,110
So a bit of a trip to get here.

15
00:00:55,110 --> 00:00:59,910
I've been an enthusiast in the security world for longer than I want to admit.

16
00:00:59,910 --> 00:01:02,270
Also a senior security engineer at GitHub.

17
00:01:02,270 --> 00:01:06,870
And following along from a great tip from the last talk, we are not representing GitHub here today.

18
00:01:06,870 --> 00:01:10,890
Just sharing context on that before we dive on into it.

19
00:01:12,050 --> 00:01:19,770
So to chat a little bit about what we're going to cover here today, we are going to be as sort of platform and product agnostic as we can be going through this.

20
00:01:19,770 --> 00:01:24,750
The idea is really to get you set up through different stages of building a bug bounty program.

21
00:01:24,750 --> 00:01:35,890
So whether you're coming in from a position where you have nothing that's public facing right now, or you are already set up with maybe a private program or vulnerability disclosure program and you want to see how that's going to grow.

22
00:01:35,950 --> 00:01:37,190
We've both lived through this.

23
00:01:37,190 --> 00:01:39,350
We've had a lot of pain and suffering with it.

24
00:01:39,350 --> 00:01:43,650
So we're hoping to share some of that info so that you don't have to have that same pain and suffering.

25
00:01:44,130 --> 00:01:47,850
So starting off, we are going to make some assumptions that you've got some fundamentals in place.

26
00:01:47,850 --> 00:01:49,370
We'll sort of talk about what those are.

27
00:01:49,370 --> 00:01:56,370
But this is going to be things like having vulnerability lifecycle tracking, understanding what your asset inventory is, so you can actually start building something on top of it.

28
00:01:56,970 --> 00:02:04,870
Next up is going to be the groundwork to actually get a program started, building a plan to grow from there, sort of getting to an MVP so you can figure out how you want to scale.

29
00:02:05,510 --> 00:02:07,210
Operations for it as well here.

30
00:02:07,210 --> 00:02:09,850
Sorry, we're on a new platform for this.

31
00:02:09,890 --> 00:02:17,990
So the operations, looking at your communications plans, your program policies, how you want to handle transparency, disputes, all that, all the fun that goes with it.

32
00:02:18,790 --> 00:02:22,270
Last up for the main stuff here is also focusing on relationships.

33
00:02:22,330 --> 00:02:25,830
This is something that Jeff and I certainly feel is overlooked in a lot of places.

34
00:02:25,970 --> 00:02:36,990
If you are building a bounty program, effectively you're building something that's user-facing, customer-facing, so you want to be sure that you are putting sort of the best image forward possible for it.

35
00:02:37,210 --> 00:02:40,450
But yeah, just to clarify again, we are sort of expecting these fundamentals to be in place.

36
00:02:40,450 --> 00:02:45,250
Our goal here is to name drop the talk, getting you scoped for success.

37
00:02:46,890 --> 00:02:49,470
But yeah, so no fundamentals for this.

38
00:02:49,470 --> 00:02:50,870
So Jeff, over to you.

39
00:02:50,870 --> 00:02:51,710
Cool.

40
00:02:51,770 --> 00:02:52,310
All right.

41
00:02:52,310 --> 00:03:00,870
Yep, like you said, we're going to walk through the expectations and necessities for programs, policies already in place before starting or launching your program.

42
00:03:00,870 --> 00:03:04,950
However, if you don't have them, you can still follow along and implement what we're recommending.

43
00:03:04,950 --> 00:03:07,290
So we're taking kind of like a broad approach on this one.

44
00:03:08,470 --> 00:03:13,710
So to start with, without understanding your attack surface, you can't really scope responsibly.

45
00:03:13,830 --> 00:03:16,790
You should ask yourself, what are you exposing publicly?

46
00:03:16,790 --> 00:03:21,950
What types of threats might you face, right, based on the industry or assets that you have?

47
00:03:22,790 --> 00:03:26,750
Know what's publicly accessible and also what you want your researchers to focus on.

48
00:03:26,810 --> 00:03:29,670
What's important for you as a company?

49
00:03:29,670 --> 00:03:33,090
Why are you kind of wanting to host the program?

50
00:03:35,370 --> 00:03:38,570
And yep, build an asset inventory if you don't have one.

51
00:03:38,730 --> 00:03:43,670
A great colleague of ours said, own your assets before someone else owns them, right?

52
00:03:43,670 --> 00:03:54,830
And so building one, even if you have, you know, a very immature asset inventory, ask yourself, how can we work on that before we actually launch the program?

53
00:03:55,250 --> 00:03:58,330
And so you put in scope what's important to you.

54
00:03:59,150 --> 00:04:11,670
We don't specifically endorse any, like we said earlier, but some examples, some tools are Detectify, HackerOne Assets, and many more to kind of get you started if you need a starting point.

55
00:04:12,130 --> 00:04:19,170
To understand your asset inventory, you should categorize that you should understand relative risk.

56
00:04:19,170 --> 00:04:26,350
Some examples of this is like using a scoring or criticality system like Carver to determine risk.

57
00:04:27,030 --> 00:04:32,830
And then various platforms have tools for this, as well as purpose-built products for enumeration.

58
00:04:32,830 --> 00:04:38,690
We can discuss how we're handling it at GitHub, but that's another talk for another day.

59
00:04:40,910 --> 00:04:47,090
Finally, like you mentioned earlier, establishing a vulnerability remediation practice, it's huge.

60
00:04:47,090 --> 00:04:53,810
You should have at least something basic internally before you start receiving a ton of reports, right?

61
00:04:53,810 --> 00:04:58,290
And the way you can start is kind of understanding what qualifies as an issue.

62
00:04:58,290 --> 00:05:01,830
What is a security vulnerability for your org and for your company?

63
00:05:01,830 --> 00:05:02,950
Have clear ownership.

64
00:05:02,950 --> 00:05:04,570
This is a tough one.

65
00:05:04,590 --> 00:05:14,690
You really want to ensure that when you do receive something on an asset, you know exactly who owns it, what engineering team will be fixing it.

66
00:05:14,690 --> 00:05:28,330
And that sounds simple, but I've seen organizations have issues with, at the end of the day, they've had the vulnerability for a few months now, a few weeks, and they're having issues with clear ownership on who should fix it.

67
00:05:28,390 --> 00:05:32,310
And so having that in the beginning will save you a lot of time.

68
00:05:33,010 --> 00:05:36,970
And ultimately agreeing upon timelines for remediation.

69
00:05:37,490 --> 00:05:43,030
And if you can, just to add one more thing there, is having a good exceptions process.

70
00:05:43,030 --> 00:05:55,430
Because if a vulnerability comes in and you don't really know, it's the first of its class, but in the end it's working as expected, or you just can't fix it at the moment, you should have a defined exceptions process.

71
00:05:55,430 --> 00:06:01,370
Because that bug will be sitting there for a while, so you should know how to deal with it before you keep getting them from other folks.

72
00:06:02,230 --> 00:06:04,890
So now Logan's going to talk about some groundwork.

73
00:06:05,210 --> 00:06:17,750
Yeah, so now that we sort of are established on what we're hoping is in place, let's start getting into how we can turn that into at least the start of a bounty program, or if you're already there again, how to grow it from that point.

74
00:06:18,090 --> 00:06:24,690
So first thing we'll say is, understand what the different program options are, and how these deal with different levels of maturity.

75
00:06:24,690 --> 00:06:31,410
So one of the things that we'll generally recommend is if you're starting from scratch, you can look at something like a VDP, a Vulnerability Disclosure Program.

76
00:06:31,670 --> 00:06:34,730
This tends to be a lot lower lift to get started.

77
00:06:34,730 --> 00:06:36,870
You don't necessarily need specific tooling.

78
00:06:36,870 --> 00:06:44,190
This could be a mailing list, it could be something like Private Vulnerability Reporting on GitHub, or Confidential Issues on GitLab, anything like that.

79
00:06:44,190 --> 00:06:49,790
I mean, there are some huge projects out there that effectively do this, like the Git Security Mailing List, which is a Google group at this point.

80
00:06:49,790 --> 00:06:54,250
So there's many ways to go through and build repeatable processes out of this.

81
00:06:54,250 --> 00:07:03,990
As Jeff mentioned, understanding which teams are going to own stuff when they come in, or understanding how you're handling severities, how you're doing issue assignment, SOAs, all of this.

82
00:07:03,990 --> 00:07:12,330
This is all things that you can build with a Vulnerability Disclosure Program before you really have to worry about the money side of it, or platform, or tooling, or really any of that.

83
00:07:12,570 --> 00:07:22,390
But broadly, the sort of progression we'd suggest, if you're looking at this, is to start out with a Vulnerability Disclosure Program to get those reps in, and then from there maybe look at moving into a private program.

84
00:07:22,390 --> 00:07:29,270
You get a bit more control around the number of reports you're going to get, potentially who's actually working on your program as well, depending on how you build that out.

85
00:07:29,270 --> 00:07:35,990
And then look at going public once you have those processes in place, because otherwise you're just going to get buried the second you open things up.

86
00:07:37,250 --> 00:07:52,870
And when you do look at moving into a paid program, whether that's a private program, public program, whatever, as Jeff sort of alluded to earlier, we want you to scope with intention for this, but also, we both think, importantly, pair up with a platform that can support you through it.

87
00:07:52,870 --> 00:07:58,650
And again, we're not going to specifically endorse anything, whether it's HackerOne, Integrity, BugCrowd, whatever, you get the same idea for it.

88
00:07:58,650 --> 00:08:07,970
But the key things for this is you get a lot of stuff that you may not think is super important out of the gate, and then as soon as you don't have it, it bites you in the ass immediately.

89
00:08:08,290 --> 00:08:18,850
So this is going to be things like having educational resources, a code of conduct, collaboration on triage and process, mediation process, if shit really hits the fan, and you can sort of go from there.

90
00:08:18,850 --> 00:08:22,750
And I guarantee you all of those will happen as soon as you get money involved for it.

91
00:08:22,810 --> 00:08:29,230
But on a more positive note, a lot of this sets you up for a situation where it's much easier for folks to submit stuff to you.

92
00:08:29,230 --> 00:08:37,270
Certainly sending stuff in through email to a mailing list, even if you're using PGP-signed emails or something, it's not the greatest feeling for a lot of it.

93
00:08:37,270 --> 00:08:49,450
And for a lot of folks that are, again, at least in the world we work in for this, a lot of people who are doing bug hunting for this are already familiar with these platforms, they're ready to send stuff in through it, they know what it looks like, they have a reputation on there already,

94
00:08:49,450 --> 00:08:50,750
they know the ins and outs.

95
00:08:50,890 --> 00:08:53,090
So you're sort of good to go from that.

96
00:08:53,430 --> 00:08:58,450
But again, speaking of scoping with intention, be very careful about scope creep for this.

97
00:08:58,450 --> 00:09:04,670
Again, with the Vulnerability Disclosure Program, you get a little more latitude because you don't necessarily have to fix things.

98
00:09:04,850 --> 00:09:06,570
Take that with a grain of salt.

99
00:09:06,890 --> 00:09:20,270
But when you do start moving into broader stuff, if you are getting reports and you have money on the line for those and you don't actually have ownership of those or a way to remediate stuff, or an exceptions process if you want to accept risks for it,

100
00:09:20,270 --> 00:09:27,610
you're going to be in for a lot of frustration on both sides of that relationship, both for your company, your organization, and for the hackers that are working on it.

101
00:09:30,430 --> 00:09:31,470
There we go.

102
00:09:32,250 --> 00:09:40,030
And another area that a lot of folks will overlook, again, until it bites them, is make sure you are bringing your legal and PR teams into this.

103
00:09:40,030 --> 00:09:42,070
This is all customer-facing stuff.

104
00:09:42,070 --> 00:09:47,350
As soon as you have money on the line as well, you have a lot of reputational impact that's going to come from that.

105
00:09:47,350 --> 00:09:49,110
You have a lot of legal obligations.

106
00:09:49,230 --> 00:09:54,310
And this is somewhere else that having a platform involved rather than doing a homegrown thing can be very, very helpful.

107
00:09:54,390 --> 00:10:05,270
As soon as you have to worry about trade sanctions for where you want to send money to, currency conversion, all of those moving parts, if you have a partner that can take care of a lot of that for you, has a good reputation for it, has all the processes,

108
00:10:05,270 --> 00:10:09,750
you're going to be saving yourself, again, a lot of frustration, a lot of pain, a lot of suffering.

109
00:10:10,090 --> 00:10:16,270
And also for many of these platforms, if not all of them at this point, it gives you standardized tooling for disclosure.

110
00:10:16,270 --> 00:10:18,910
They have their own terms for how you want to handle disclosure.

111
00:10:18,910 --> 00:10:22,690
You don't necessarily need to reinvent the wheel or have something that's bespoke to you.

112
00:10:22,870 --> 00:10:26,210
Same deal for safe harbor terms, things like that.

113
00:10:26,210 --> 00:10:29,030
So HackerOne has the gold standard safe harbor, I think they call it.

114
00:10:29,030 --> 00:10:31,110
Many other platforms have something similar.

115
00:10:31,470 --> 00:10:35,970
In our experience, the companies we've worked with, we have some bespoke ones, but it's all sort of the same idea.

116
00:10:35,970 --> 00:10:38,190
If you send stuff in, you're safe.

117
00:10:38,190 --> 00:10:43,490
But again, you're getting a lot of this for free, out of the box when you're partnering up with somebody.

118
00:10:46,450 --> 00:10:47,710
Forgot about those.

119
00:10:48,450 --> 00:10:57,230
And then as you're going through as well, as you progress through each of these stages, you're getting a lot of information around what that next stage is going to look like.

120
00:10:57,330 --> 00:11:04,450
So when you have a vulnerability disclosure program to start out with, again, you're getting the reps in for how you want to build out your processes for it.

121
00:11:04,450 --> 00:11:09,610
When you move into a private program, you're getting a better idea of what some of the budgeting is going to look like, your cost per bug.

122
00:11:09,610 --> 00:11:15,450
If you need to fine-tune your severity models, you can do that in a little bit of a more controlled environment than if you're already public.

123
00:11:15,790 --> 00:11:20,190
And this also gives you a really good opportunity to start building out some tooling for this.

124
00:11:21,150 --> 00:11:32,710
And if you're interested specifically in tooling, we're not going to talk about it here, but Jeff did a great talk at B-Sides SF earlier this year called Life of a Bug, which goes through how GitHub does all their bug remediation, bug tracking, all that.

125
00:11:32,850 --> 00:11:33,690
Very helpful.

126
00:11:33,690 --> 00:11:36,130
It saves us a colossal amount of time.

127
00:11:36,130 --> 00:11:40,850
It also makes auditing really easy, which is another thing that having a good platform can really help with.

128
00:11:40,850 --> 00:11:46,470
So if we need to go through and say, hey, are we consistently paying out the same way for these types of vulnerabilities as they come through?

129
00:11:46,470 --> 00:11:48,090
Are we consistent in our messaging?

130
00:11:48,150 --> 00:11:49,530
All of that is already there.

131
00:11:49,530 --> 00:11:50,830
We know exactly what it looks like.

132
00:11:50,830 --> 00:11:53,530
It's the same thing every single time for it.

133
00:11:54,670 --> 00:11:57,350
And yeah, also looking at budgeting and staffing for this.

134
00:11:57,430 --> 00:12:01,970
When you're going through, again, VDP, you're getting an idea of the amount of time it's going to take you to remediate something.

135
00:12:01,970 --> 00:12:04,930
You move into a private program, you're getting an idea of the costs for it.

136
00:12:04,930 --> 00:12:09,570
But that's also going to have an uptick in interest, generally, for the number of reports you're going to get.

137
00:12:09,670 --> 00:12:18,270
So you can start figuring out, okay, if we're getting 100 reports per year, 100 reports per month, maybe, what does that look like in terms of time and effort for our company?

138
00:12:18,270 --> 00:12:21,310
And then when you go public, yeah, all hell's going to break loose.

139
00:12:21,330 --> 00:12:28,730
Especially if you manage to get featured as a new program on any of these platforms, you're going to get a big uptick in interest almost immediately.

140
00:12:28,730 --> 00:12:34,950
You need to be prepared for that because nothing is worse than just having things that will linger for months while people are expecting it to get fixed.

141
00:12:37,090 --> 00:12:40,190
Oh, and I will give a quick shout-out for...

142
00:12:40,190 --> 00:12:47,130
there's a fantastic resource, and we do have a slide that has links to all these resources at the end, as well as these slides in there.

143
00:12:47,410 --> 00:12:53,190
There is a forum called Bug Bounty Community of Interest, which has fantastic articles around all of these.

144
00:12:53,190 --> 00:12:54,370
Definitely worth checking out.

145
00:12:54,370 --> 00:12:57,590
I do not remember the URL, but it is linked in the resources here.

146
00:12:58,470 --> 00:13:02,990
So moving forward, let's chat about what things look like once you're actually operational.

147
00:13:02,990 --> 00:13:06,850
You've got the baseline stuff you need in place for this.

148
00:13:07,030 --> 00:13:15,430
And again, our goal here is to talk about some of the less easily documentable things for it when you're going through and building a program.

149
00:13:15,490 --> 00:13:18,870
So the first one is going to be focusing on transparency for this.

150
00:13:18,910 --> 00:13:25,050
The more you can document and share with the community of hackers that's going to be working on your program, the better.

151
00:13:25,050 --> 00:13:29,490
And that's everything from process that you're going through, your timelines, your payout, your scope.

152
00:13:29,590 --> 00:13:32,490
There's nothing that's worse than ambiguity in any of this.

153
00:13:32,530 --> 00:13:36,950
Somebody's going to send in a report to you, you're not going to be sure if it's in your scope.

154
00:13:36,950 --> 00:13:38,390
That's going to suck for everyone.

155
00:13:38,390 --> 00:13:46,590
If they know specifically that it is in scope, and you wind up having to go through a mediation process, you've got clarity for that, again, on both sides for it.

156
00:13:46,630 --> 00:13:50,590
And it gives you a lot of opportunities as well when you start working with disclosure for it.

157
00:13:50,710 --> 00:14:04,030
The more transparent you can be around that, whether it's limited disclosure, full disclosure, whatever, if it's unilateral, both sides doing a coordinated disclosure, whatever, it's giving really good insight into how your program operates.

158
00:14:04,030 --> 00:14:14,270
Whether that's from a timeline perspective, or it's from a operation perspective, what your communications look like, what all of the individual moving bits and pieces look like, even payouts.

159
00:14:14,270 --> 00:14:22,930
You can start looking at some of these programs out there and say, hey, if I get an IDOR for this product, I have a pretty good idea of what the payout's going to be, so I can manage my own expectations for it.

160
00:14:23,070 --> 00:14:27,050
So that can be vulnerability classes, it can be specific assets in there as well.

161
00:14:27,110 --> 00:14:29,250
There's lots to look at for it.

162
00:14:31,810 --> 00:14:32,750
There we go.

163
00:14:32,790 --> 00:14:40,370
And building on that as well is making sure that you've got good clarity on what you want hackers to focus on when they are looking at your program.

164
00:14:40,370 --> 00:14:46,070
So I'll talk about GitHub very briefly for this, because I think we've done this all right.

165
00:14:46,070 --> 00:14:50,470
But we have specific lists of stuff that we do care about, things that we don't care about.

166
00:14:50,470 --> 00:14:54,350
And that's not to say that they're not necessarily security issues, but they're things we're aware of.

167
00:14:54,350 --> 00:14:58,730
It's accepted risk, so it's pretty easy for us to filter stuff out when it comes in.

168
00:14:58,730 --> 00:15:00,770
We've seen that with other programs we've worked on as well.

169
00:15:00,770 --> 00:15:03,230
It's a significant time saver on both sides.

170
00:15:03,230 --> 00:15:19,150
If you, as a researcher in the community, are looking at a program, and you immediately know, hey, I don't want to touch this asset, it's not going to get me any money, it's not going to be of interest, you can save yourself a lot of time and focus on something that instead will give you that level of payout that you're looking for.

171
00:15:19,510 --> 00:15:28,130
And with this as well, in addition to understanding what you do want to focus on, what you don't want to focus on, this is also an area to add a lot of clarity around your payout structures.

172
00:15:29,150 --> 00:15:33,770
For many programs out there, it's going to be a set thing, like, hey, if this is a low severity, it's $100.

173
00:15:33,790 --> 00:15:36,310
If it's medium, it's $500, so on and so forth.

174
00:15:36,310 --> 00:15:43,250
For others, it's a direct tie to CVSS, or VISS, or one of five million standards that people are coming up with now.

175
00:15:43,250 --> 00:15:49,590
But having clarity around it, having examples for it as well, if you're doing disclosure through your program, is incredibly helpful for it.

176
00:15:49,970 --> 00:15:54,510
And with that as well, having a communication plan for a lot of this is really, really helpful.

177
00:15:54,510 --> 00:16:06,870
Whether it is something that you want to handle specifically internally, whether you are staffing this program through a platform to do your triage work, or you're doing it yourselves, having a list of resources for that, having things like canned responses for it,

178
00:16:06,870 --> 00:16:08,290
can be very, very helpful.

179
00:16:08,410 --> 00:16:13,830
You're ensuring consistency for it, you're ensuring that when people work on your program, they know exactly what they're going to get for it.

180
00:16:13,870 --> 00:16:21,070
And it really does help with a lot of the auditing stuff as well, as we've alluded to, because you understand exactly what you're in for every single time for it.

181
00:16:22,090 --> 00:16:34,150
And I think one of the other things to put on here, and I say this coming from a customer-facing background before moving into the security world, with a lot of this, you have to remember that you are, quote-unquote, customer-facing, user-facing, whatever.

182
00:16:34,150 --> 00:16:37,410
So, whatever you say, you're going to be representing the group you work for.

183
00:16:37,410 --> 00:16:44,590
So, if I get a bounty report, and I am a giant ass to somebody, it's not that I was an ass, it's that GitHub was an ass .

184
00:16:44,590 --> 00:16:47,310
You don't want to necessarily be on the hook for that.

185
00:16:49,070 --> 00:16:53,830
And, speaking of being an ass, understanding how to handle disputes and duplicates.

186
00:16:53,830 --> 00:16:56,410
These will happen, they happen very frequently.

187
00:16:56,770 --> 00:17:05,730
More often than not, a lot of disputes are of dubious nature, but duplicates are going to happen very, very frequently.

188
00:17:05,730 --> 00:17:11,090
And this could be duplicates for stuff you're already tracking in your company as known issues, but maybe aren't accepted risks.

189
00:17:11,090 --> 00:17:16,450
It could be things where you have two researchers who submit the same report within minutes of each other even.

190
00:17:16,450 --> 00:17:20,070
So, having a clear, documented policy on that is very helpful as well.

191
00:17:20,070 --> 00:17:26,610
And that can be something as simple as, hey, if we get duplicate reports, if somebody sent it in first based on timestamp, they're going to get the award.

192
00:17:26,650 --> 00:17:33,210
Understanding whether or not you want to combine people onto single reports is also something that sounds a bit more simple than it actually is.

193
00:17:33,210 --> 00:17:42,250
That is a great way to lead yourself down a path of pain with mediation requests and other frustrations and arguments more frequently than not when that happens.

194
00:17:42,410 --> 00:17:45,350
So be conscious of that when it goes through.

195
00:17:45,350 --> 00:17:56,590
One of the other things that comes up and causes a lot of pain and suffering for programs and researchers is making sure you have something along the lines of a one -fix, one-bug, one-payout type of policy.

196
00:17:56,690 --> 00:17:59,150
There was a great example that popped up the other day.

197
00:17:59,150 --> 00:18:06,730
I don't remember what the asset was, what the company was, but somebody had found what they had considered to be 40 or 50 different crits that were coming in.

198
00:18:06,730 --> 00:18:10,110
It was leaking PII, a bunch of other problems.

199
00:18:10,110 --> 00:18:11,590
This was not for us, I'll say.

200
00:18:11,850 --> 00:18:18,830
But with that, those all got closed out as a single low-severity vulnerability because it was something that was being tracked.

201
00:18:18,830 --> 00:18:24,130
So this person had spent about a month compiling stuff, sent all of it in, and got completely hosed on it.

202
00:18:24,130 --> 00:18:36,190
So making sure you've got as much structure there as possible, you've got the policies in place to avoid that kind of frustration, is really helpful because otherwise you're going to wind up on the bug bounty forum slack, people are going to be flaming your program,

203
00:18:36,190 --> 00:18:37,510
and it's not fun for anyone.

204
00:18:37,510 --> 00:18:42,030
You don't want to be on the other side of that because it's not going to lead to a positive outcome for anyone.

205
00:18:43,670 --> 00:18:54,950
And last but not least for this bit, just chatting about doing your own triage versus working with a platform for it, both of these are incredibly feasible ways to do things.

206
00:18:54,970 --> 00:19:03,210
We've both worked with groups that are using hosted platform-managed triage services, whether it's, again, HackerOne, Integrity, whatever.

207
00:19:03,570 --> 00:19:08,230
These can be incredibly helpful because you don't need to change your own staffing for it.

208
00:19:08,230 --> 00:19:17,370
The key thing I'll note with that is on most of these platforms, that's going to come with an additional cost, but more importantly you also need to be set up to still do that triage when it comes through.

209
00:19:17,370 --> 00:19:29,030
A lot of the time, this is going to be lacking the business context for it, so they may not necessarily understand severity that's going along with it, especially as you're starting out as a new program that will build over time, but it's not going to be there out of the gate,

210
00:19:29,030 --> 00:19:33,530
so you need to be able to filter through whatever is coming in after that.

211
00:19:33,530 --> 00:19:47,470
So we've spoken with various other companies where they'll say, yeah, we're using a managed triage service, we don't actually have anybody full-time, it just goes into our AppSec team, but we're still spending 50, 60, 70 hours a week doing triage on this stuff because yeah,

212
00:19:47,470 --> 00:19:52,950
it's valid, but it's stuff that they don't have visibility into our internal bug base, so we don't need to do anything with it.

213
00:19:52,950 --> 00:19:55,290
So that's one of the things to consider for it.

214
00:19:55,550 --> 00:20:01,890
One of the other areas that comes with that as a bit of a selling point is looking at things like spam filtering, especially as you're going to a public program.

215
00:20:02,330 --> 00:20:06,110
I would suggest that that's not as important as it may sound like.

216
00:20:06,110 --> 00:20:17,290
You think you have a public program, you're going to get a lot of garbage, and you are, but if you have that context around your program already, your products, you have that expertise on it, a lot of that goes away pretty quickly.

217
00:20:17,490 --> 00:20:25,710
So for reference, we're one of the larger programs, I think, out there at GitHub, and we get about 20 or 30 spam reports a week, and those take under a minute to remediate.

218
00:20:25,710 --> 00:20:29,070
So for our entire team, it's a pretty negligible cost to do that.

219
00:20:29,910 --> 00:20:43,950
And yeah, going for in-house, the other side of this as well is you're looking at a situation where you can make the appropriate decision, whether that's from a payout standpoint, a triage process , whatever, you can make that very quickly because you have the full context around it,

220
00:20:43,950 --> 00:20:46,030
and you've got the expertise on it.

221
00:20:46,130 --> 00:20:59,710
Again, we've heard a lot of challenges, and that's not to say this solves the problem by any stretch, but there are many other cases where something without that context may show up as being a lower severity issue, when in fact it's something you might actually consider to be a crit and need to work on right away.

222
00:20:59,850 --> 00:21:04,690
So being able to make that judgment call very, very quickly instead of having to route it through another party can be helpful.

223
00:21:04,690 --> 00:21:05,890
So there's a lot of trade-offs here.

224
00:21:05,890 --> 00:21:11,470
It's not to say one way is any better than the other, but something to be conscious of that otherwise does not always show up that way.

225
00:21:12,210 --> 00:21:17,490
And with that, we'll talk about what I think both of us consider the most important intangible part of this.

226
00:21:17,490 --> 00:21:18,330
So over to you, Jeff.

227
00:21:18,330 --> 00:21:19,330
Cool, yeah.

228
00:21:19,770 --> 00:21:23,850
There's not a lot of documentation around how to build relationships.

229
00:21:24,030 --> 00:21:25,930
It's a rather new thing, right?

230
00:21:25,930 --> 00:21:29,010
And yeah, the program's up and running, right?

231
00:21:29,010 --> 00:21:30,070
You're getting value out of it.

232
00:21:30,070 --> 00:21:30,910
How do you grow from there?

233
00:21:30,910 --> 00:21:31,990
How do you create relationships?

234
00:21:31,990 --> 00:21:33,470
How do you celebrate the wins?

235
00:21:33,470 --> 00:21:35,550
How do you keep your researchers coming back?

236
00:21:38,510 --> 00:21:46,170
And so one thing is basically create relationships and celebrate... oh, I already said that, sorry.

237
00:21:46,170 --> 00:21:51,870
An empathetic approach helps researchers continue to invest time in your program and represent your org, right?

238
00:21:51,870 --> 00:22:01,970
So when you get a report and it's kind of like big bounty and they're like, oh, I really need like 50 bucks or I spent 20 hours on this.

239
00:22:02,150 --> 00:22:05,610
You can explore ways where you don't just turn that into a negative .

240
00:22:05,610 --> 00:22:08,750
You can just close it out as spam and send them on their way.

241
00:22:09,050 --> 00:22:16,470
One way you can do is like exploring defense in debt is one way of handling reports where you're like, okay, you know what?

242
00:22:16,470 --> 00:22:21,110
This might have not met the low severity, but it's a good idea.

243
00:22:21,110 --> 00:22:23,790
It's a good maybe security feature request or something.

244
00:22:23,950 --> 00:22:27,070
And so it's kind of, you know, you throw them $200, $500.

245
00:22:27,110 --> 00:22:28,970
We have something I get where we do that.

246
00:22:28,970 --> 00:22:33,610
And we tell them like, this didn't really meet the low severity, but this is what we're going to do with it.

247
00:22:33,610 --> 00:22:39,530
And so we're going to award you a lower than low severity bounty.

248
00:22:39,530 --> 00:22:40,210
And they like that.

249
00:22:40,210 --> 00:22:46,250
It encourages researchers to continue to try to find bones on your platform and to keep coming back.

250
00:22:47,890 --> 00:22:51,130
Yeah, so I was alluding to be conscious how you close the reports.

251
00:22:51,250 --> 00:22:53,890
Don't close them as spam immediately all the time.

252
00:22:53,890 --> 00:23:02,890
If they're actually trying or they were close, you can explore ways of using informative or duplicate.

253
00:23:02,890 --> 00:23:06,130
But like, those don't hurt someone's red points.

254
00:23:06,130 --> 00:23:12,470
And so those are one way that that's a way that maybe researchers can...

255
00:23:12,470 --> 00:23:15,170
they'll want to come back, you know, and keep working with you.

256
00:23:15,170 --> 00:23:20,010
And to jump in quickly while Jeff gets the slide sorted out here, because I think we had some technical issues with that.

257
00:23:20,010 --> 00:23:26,130
The scenario I mentioned earlier with those 50 reports that came in for that program, all of those got closed out as not applicable.

258
00:23:26,130 --> 00:23:32,410
That person's reputation went from being very well respected, scored on HackerOne, to being tanked in a single day.

259
00:23:32,430 --> 00:23:36,170
So it led to a lot of other discussions that, again, are not fun for anybody involved.

260
00:23:37,250 --> 00:23:37,910
Right, right.

261
00:23:37,910 --> 00:23:39,870
And so helping researchers grow, right?

262
00:23:39,870 --> 00:23:43,410
This is the education part that we spend a lot of time on.

263
00:23:43,710 --> 00:23:51,270
We rather educate rather than, like, you know, chastise if there are off-base submissions that aren't spam, right?

264
00:23:51,270 --> 00:23:54,770
What we do is we provide resources and direct feedback, right?

265
00:23:54,770 --> 00:23:59,750
It takes five minutes to let them know, hey, maybe you should check out this podcast that might help you out.

266
00:23:59,750 --> 00:24:01,270
Or check out Portswigger Academy.

267
00:24:01,270 --> 00:24:06,010
That might help you get a little closer to this, you know, LFI that you thought you found.

268
00:24:06,090 --> 00:24:11,230
Or this SSRF, you know, that's clearly not an SSRF, but you were looking for it, you know?

269
00:24:11,230 --> 00:24:18,390
So we have a lot of internal lists that we use to try to educate folks to...

270
00:24:18,390 --> 00:24:21,070
And it's cool, because eventually they do come back.

271
00:24:21,070 --> 00:24:25,050
We've seen a few folks where we helped them out, we educated them a little bit.

272
00:24:25,050 --> 00:24:28,410
Then they came back maybe a year later and they found like a medium or something.

273
00:24:28,410 --> 00:24:30,990
And they let us know, like, hey, this is cool, you know?

274
00:24:30,990 --> 00:24:34,170
You sent me on my way and I came back a year later.

275
00:24:34,170 --> 00:24:35,850
Those are special.

276
00:24:38,810 --> 00:24:39,870
Showcasing submissions.

277
00:24:39,890 --> 00:24:42,430
That's something that we love to do.

278
00:24:42,570 --> 00:24:48,410
You should explore opportunities for joint articles, blog posts, and the creation of content.

279
00:24:48,710 --> 00:24:54,070
Basically, you know, some examples are like walkthroughs of disclosed bugs, their life cycle.

280
00:24:54,070 --> 00:24:57,230
If you're not ready for disclosure, that's okay.

281
00:24:57,230 --> 00:24:58,210
It takes time.

282
00:24:58,210 --> 00:25:04,130
One way that you can do that is if you see a researcher who's just focusing... You can tell, they're just focusing on your program.

283
00:25:04,130 --> 00:25:08,350
They're sending in... Every two weeks you're getting one or two good bugs from them.

284
00:25:08,910 --> 00:25:09,670
Reach out .

285
00:25:09,670 --> 00:25:11,450
Form a relationship with them.

286
00:25:11,450 --> 00:25:16,450
Just ask them, hey, can I give you a researcher spotlight?

287
00:25:16,450 --> 00:25:23,870
I just want to write a blog and tell everybody, and maybe inspire other researchers to check out our program.

288
00:25:23,870 --> 00:25:37,310
And we've done that a few times where we'll just interview folks and we'll write a blog post with the answers and we'll give them something, you know, a platform where they can kind of talk about their own research and really plug their own stuff, you know,

289
00:25:37,310 --> 00:25:40,170
and why they like to come and hack on us.

290
00:25:40,270 --> 00:25:42,090
And that's pretty cool.

291
00:25:46,050 --> 00:25:52,050
Lastly, researchers and industry peers enjoy interacting and reading what programs are up to.

292
00:25:52,050 --> 00:25:53,950
You can do this in a lot of different ways.

293
00:25:53,950 --> 00:25:57,930
You can join, you know, Slack workspaces like Bug Bounty Forum.

294
00:25:57,930 --> 00:26:02,830
You can, you know, create a Twitter account and just tweet little things here and there.

295
00:26:02,890 --> 00:26:05,970
Like, we tweet our monthly stats and people like to engage with them.

296
00:26:05,970 --> 00:26:06,470
That's pretty cool.

297
00:26:06,470 --> 00:26:09,510
I like to see, like, how much did GitHub pay last month, you know?

298
00:26:09,510 --> 00:26:13,750
And some programs are pretty cool at promoting promotions.

299
00:26:13,750 --> 00:26:17,670
Like, oh, we are paying a 30% bonus on cross-site scriptings.

300
00:26:17,670 --> 00:26:26,570
And that, you know, if you have researchers new or, you know, returning researchers and they check that out they will most likely want to hack on you, you know?

301
00:26:26,570 --> 00:26:30,670
They'll be like, oh, 30%, 40% bonus sounds nice this month.

302
00:26:30,950 --> 00:26:37,470
And so, yeah, using these community discussions it's pretty cool for industry peers and then also researchers.

303
00:26:40,950 --> 00:26:46,870
So you've got your program scoped, operations are underway, relationship, you're building relationships.

304
00:26:46,870 --> 00:26:48,770
You know, what's next?

305
00:26:52,900 --> 00:27:03,840
So, now that you're operational and have your goals changed you should create if you haven't created one you should create one or update a roadmap of where you want to be.

306
00:27:03,840 --> 00:27:06,100
Where do you want to be in two quarters?

307
00:27:06,100 --> 00:27:07,380
Where do you want to be in a year?

308
00:27:07,840 --> 00:27:11,740
How do you really get, like, the return of investment for your bugs?

309
00:27:12,760 --> 00:27:15,300
Let's say you're getting all your bugs and you're fixing them correctly.

310
00:27:15,300 --> 00:27:15,920
What's next?

311
00:27:15,920 --> 00:27:18,780
Like, do you have the bandwidth to explore varying analysis?

312
00:27:18,780 --> 00:27:24,400
Can you enable something like CodeQL to really try to find more bugs with the bugs that you're getting already?

313
00:27:24,420 --> 00:27:26,060
You have access to the code base, right?

314
00:27:26,060 --> 00:27:34,880
So, using tools, tooling, staffing and all that to try to get more bugs of the bugs you get are so impactful.

315
00:27:35,340 --> 00:27:37,680
What needs to change from what you're doing now?

316
00:27:38,500 --> 00:27:40,200
Maybe you need more automation.

317
00:27:40,580 --> 00:27:41,840
How can you do that?

318
00:27:41,840 --> 00:27:44,080
What is working well that you want to double down on?

319
00:27:44,080 --> 00:27:48,320
What do you want to focus on for the next two months?

320
00:27:48,320 --> 00:27:50,920
What do you really want to focus on and then move on to the next thing?

321
00:27:50,920 --> 00:27:52,480
And what problems are you having?

322
00:27:52,720 --> 00:27:55,020
Is it time to focus on disclosure?

323
00:27:55,020 --> 00:27:57,280
Is it time to focus on communication?

324
00:27:57,280 --> 00:27:59,480
Maybe you can train your...

325
00:27:59,480 --> 00:28:09,240
If you're using platform triage, is it time to maybe spend a few weeks and months just really getting to know them so that that layer, that filtering layer gets a little better?

326
00:28:09,880 --> 00:28:12,220
So, what problems are you having?

327
00:28:12,220 --> 00:28:15,060
That's what you should focus on.

328
00:28:15,060 --> 00:28:19,480
And internally promote and celebrate findings from your program.

329
00:28:19,520 --> 00:28:23,200
Focus on blameless handling of bugs.

330
00:28:23,200 --> 00:28:29,420
A blameless culture really goes a long way for engineering and security teams.

331
00:28:29,780 --> 00:28:33,980
I've been at an organization where it was tougher to get things fixed.

332
00:28:33,980 --> 00:28:37,180
It was always, ah, I don't think that's an SSRF.

333
00:28:37,180 --> 00:28:39,680
I didn't mean to introduce an SSRF.

334
00:28:39,680 --> 00:28:43,420
But like, truly what it should be is we did that.

335
00:28:43,420 --> 00:28:45,120
We introduced the SSRF.

336
00:28:45,120 --> 00:28:47,660
How do we not introduce it going forward, right?

337
00:28:47,660 --> 00:28:49,020
It's not so simple.

338
00:28:49,020 --> 00:28:50,880
But like, lessons learned from these bugs, right?

339
00:28:50,880 --> 00:28:58,040
Instead of just paying them out, you know, having a hard time fixing them and then forgetting about them, you're just going to keep getting them.

340
00:28:58,040 --> 00:28:59,680
You're going to keep seeing these patterns.

341
00:28:59,700 --> 00:29:13,780
So, if you try, and it's a cultural thing and it's tough, but if you really are able to build a blameless culture, you'll be able to be way more successful with this bug bounty program that you're spinning up or you're revamping.

342
00:29:14,100 --> 00:29:16,020
Another thing is software has bugs, right?

343
00:29:16,020 --> 00:29:21,120
This is another feedback mechanism in addition to other tooling and processes.

344
00:29:21,980 --> 00:29:25,360
So, yeah, just understand that you're in it together.

345
00:29:30,260 --> 00:29:31,060
Cool.

346
00:29:31,060 --> 00:29:38,800
So, most public programs are very clear about their scopes, their reward structures, what they do, right?

347
00:29:38,800 --> 00:29:44,100
You can check out activity, everything's on there, based on the platform you use.

348
00:29:44,560 --> 00:29:51,480
So, you should compare yourself to industry peers, but you should understand you don't need to compete to be competitive, right?

349
00:29:51,480 --> 00:30:03,880
You're in different industries, you're in different maturity levels of your program, and just chat with industry peers, just see what they're doing, what can you implement to your program?

350
00:30:03,880 --> 00:30:06,720
Are you ready to implement some of the things that others are doing?

351
00:30:06,720 --> 00:30:08,080
Can you start simple?

352
00:30:08,080 --> 00:30:14,840
I like how this company has automation in place, and as a team of three, that's what I really need, and that's what you're going to focus on.

353
00:30:14,840 --> 00:30:20,600
So, talking and collaborating with industry peers will be fundamental, really.

354
00:30:21,180 --> 00:30:24,300
One more thing is, few people hack on a single program.

355
00:30:24,340 --> 00:30:27,160
Like Logan said earlier, these researchers, they bounce around, right?

356
00:30:27,840 --> 00:30:41,380
Most of them, they have a lot of experience working with a lot of companies, and just for some metric, GitHub has had a little more than 850 unique reporters last year, in 2022, right?

357
00:30:41,380 --> 00:30:43,480
And some of them were first-timers.

358
00:30:43,480 --> 00:30:45,680
I don't have the exact number, but that's a lot, right?

359
00:30:45,680 --> 00:30:49,040
More than 850 people coming to your program for the first time.

360
00:30:49,040 --> 00:31:01,160
That can sound scary, really, but like I said, chatting with industry peers kind of helps build and mature your program so you can handle that many reporters, right?

361
00:31:01,160 --> 00:31:10,120
Because a lot of these unique reporters, it's their first time engaging with your program, so it can be rough if they don't maybe understand how your program runs compared to others, right?

362
00:31:10,120 --> 00:31:19,540
So if you have a lot of the communication, automation, all that, it'll have a really smooth process for these new researchers, and they'll want to come back.

363
00:31:19,540 --> 00:31:20,260
And that's the goal, right?

364
00:31:20,260 --> 00:31:24,380
You don't want them to submit one report and then never again.

365
00:31:25,240 --> 00:31:38,000
And so, yeah, chatting with industry peers, engage with programs, understand what they're seeing, what are problems that they're not seeing and you're seeing, what are some common issues that are being submitted, you know, throughout the industry.

366
00:31:40,500 --> 00:31:46,780
Lastly, yeah, so you want to talk to bug hunters and review feedback.

367
00:31:46,880 --> 00:31:53,480
What we do is we go to conferences, we meet up with all the researchers that we can, and we just ask them, like, how's it going?

368
00:31:53,540 --> 00:31:55,640
What programs are you hunting on, right?

369
00:31:55,640 --> 00:32:01,740
And if we see a trend and we're seeing that one program is what a lot of people are focusing on, we'll check it out.

370
00:32:01,740 --> 00:32:03,480
Like, what are they doing that's pretty cool?

371
00:32:03,480 --> 00:32:04,660
What are we not doing?

372
00:32:04,660 --> 00:32:13,060
Like, at DEF CON last year, we had a few researchers that we chatted with for about five hours in one day, and they said, I really want swag.

373
00:32:13,060 --> 00:32:14,480
And so we said, okay, cool.

374
00:32:14,480 --> 00:32:15,280
We'll work on it.

375
00:32:15,280 --> 00:32:22,000
And so we launched a cool bug bounty swag store that only people who submit a valid report to us can get that kind of swag.

376
00:32:22,000 --> 00:32:23,760
So we gamified it a little bit, you know?

377
00:32:23,760 --> 00:32:31,640
And so what that causes is that these people, these researchers, they're like, I kind of want that swag that you can only... it's a challenge, you know?

378
00:32:31,640 --> 00:32:32,580
It's kind of like a CTF.

379
00:32:32,580 --> 00:32:36,800
You get some money, and you get cool swag that only you can get if you submit a bug.

380
00:32:36,840 --> 00:32:39,580
So finding ways to gamify it, you know?

381
00:32:39,700 --> 00:32:41,860
Launching a swag store is not that expensive.

382
00:32:42,740 --> 00:32:48,860
So yeah, basically talking to bug hunters and seeing what is something they would like, you know?

383
00:32:48,860 --> 00:32:51,920
Not all the requests are going to be things you can do, right?

384
00:32:51,920 --> 00:32:53,040
Some things take time.

385
00:32:53,040 --> 00:32:56,160
There's some things that we got asked for that'll probably take like a year or two.

386
00:32:56,160 --> 00:32:56,880
And that's okay.

387
00:32:56,880 --> 00:32:58,400
But at least you're working on them, you know?

388
00:32:58,440 --> 00:33:04,040
You're surveying the community, and you're interacting with folks on what is out there right now.

389
00:33:04,040 --> 00:33:04,940
Like, what is trending?

390
00:33:04,940 --> 00:33:07,800
What is something that maybe we should do that others are doing?

391
00:33:09,200 --> 00:33:11,720
I will say swag is a very common request.

392
00:33:11,720 --> 00:33:13,080
There are some programs out there.

393
00:33:13,080 --> 00:33:15,220
I think it's the Dutch government as well.

394
00:33:15,380 --> 00:33:17,080
Their approach is they can't pay for anything.

395
00:33:17,080 --> 00:33:21,120
They have a vulnerability disclosure program, but you get an awesome shirt that says, hey, I hacked the Dutch government.

396
00:33:21,120 --> 00:33:22,520
All I got was a shitty t-shirt.

397
00:33:22,520 --> 00:33:24,240
That kind of stuff is fantastic.

398
00:33:24,240 --> 00:33:25,120
It's very unique.

399
00:33:25,120 --> 00:33:27,220
There's no other place you can get that.

400
00:33:27,440 --> 00:33:28,840
So it's a shameless plug for us, I guess.

401
00:33:28,840 --> 00:33:31,720
If you want one of these wonderful GitHub hoodies, you can come hack on our program.

402
00:33:32,260 --> 00:33:32,900
Exactly.

403
00:33:33,680 --> 00:33:35,480
And I think that's it.

404
00:33:36,100 --> 00:33:38,960
So I promise this is not a rickroll .

405
00:33:38,960 --> 00:33:41,800
You can go to this, but the link is there if you don't believe me anyway.

406
00:33:42,020 --> 00:33:47,320
This is going to be a lot of stuff that we've talked about through this, resources to back it up, a lot of documentation.

407
00:33:47,600 --> 00:34:01,160
There were too many links to bother putting on here, so if you want a good weekend, week, month, year of reading, this is something we're continually updating with all sorts of bug bounty stuff, whether it's program building, operations, building out the fundamentals for it,

408
00:34:01,160 --> 00:34:02,700
sort of a dumping ground for that.

409
00:34:02,700 --> 00:34:05,880
We're trying to structure it in a lot of the same framework we have for this.

410
00:34:05,940 --> 00:34:12,240
We'll also have a copy of these slides with the speaker notes, once I edit them down a little bit, to be up there as well, if you want to loop back to that.

411
00:34:12,240 --> 00:34:14,200
It's going to have contact info for both of us, too.

412
00:34:14,200 --> 00:34:20,860
Feel free to reach out, any questions you have, whether here or later on, if anything comes to mind, we're always happy to chat bounty.

413
00:34:21,480 --> 00:34:24,040
So, thank you very much for sticking with us, hope you had a great DEF CON!