start	end	text
8650	11790	Today we're going to be chatting about experience that Jeff and I have had
11790	14850	building bug bounty programs, managing programs.
14870	17690	So we'll dive right on into it then.
17690	20170	So a little bit about us.
20170	22610	Perfect, we do have everything working here. So Jeff?
24450	27650	Hello. Cool, so quick intro.
27950	31450	Sorry I lost my voice, I blamed a DJ yesterday at O'Shea's.
31710	34750	I enjoy driving fast cars,
35530	38290	racing cars professionally, building cars,
38290	41690	security research and building bug bounty programs.
41690	45750	It's where I started my career many years ago and just very passionate about it.
45750	49230	And I'm a senior security engineer at GitHub at the moment.
49850	53270	And I am Logan McLaren. I'm based out of Ottawa, Canada.
53270	55110	So a bit of a trip to get here.
55110	59910	I've been an enthusiast in the security world for longer than I want to admit.
59910	62270	Also a senior security engineer at GitHub.
62270	64770	And following along from a great tip from the last talk,
64770	66870	we are not representing GitHub here today.
66870	70890	Just sharing context on that before we dive on into it.
72050	75410	So to chat a little bit about what we're going to cover here today,
75410	78310	we are going to be as sort of platform and product agnostic
78310	79770	as we can be going through this.
79770	82470	The idea is really to get you set up through different stages
82470	84750	of building a bug bounty program.
84750	88010	So whether you're coming in from a position where you have nothing
88010	91250	that's public facing right now, or you are already set up
91250	94110	with maybe a private program or vulnerability disclosure program
94110	95890	and you want to see how that's going to grow.
95950	97190	We've both lived through this.
97190	99350	We've had a lot of pain and suffering with it.
99350	101170	So we're hoping to share some of that info
101170	103650	so that you don't have to have that same pain and suffering.
104130	106190	So starting off, we are going to make some assumptions
106190	107850	that you've got some fundamentals in place.
107850	109370	We'll sort of talk about what those are.
109370	112130	But this is going to be things like having vulnerability lifecycle tracking,
112130	113950	understanding what your asset inventory is,
113950	116370	so you can actually start building something on top of it.
116970	120050	Next up is going to be the groundwork to actually get a program started,
120050	122750	building a plan to grow from there, sort of getting to an MVP
122750	124870	so you can figure out how you want to scale.
125510	127210	Operations for it as well here.
127210	129850	Sorry, we're on a new platform for this.
129890	132610	So the operations, looking at your communications plans,
132610	135310	your program policies, how you want to handle transparency,
135310	137990	disputes, all that, all the fun that goes with it.
138790	142270	Last up for the main stuff here is also focusing on relationships.
142330	145830	This is something that Jeff and I certainly feel is overlooked in a lot of places.
145970	148750	If you are building a bounty program,
148750	152130	effectively you're building something that's user-facing, customer-facing,
152130	153590	so you want to be sure that you are putting
153590	156990	sort of the best image forward possible for it.
157210	158310	But yeah, just to clarify again,
158310	160450	we are sort of expecting these fundamentals to be in place.
160450	163390	Our goal here is to name drop the talk,
163390	165250	getting you scoped for success.
166890	169470	But yeah, so no fundamentals for this.
169470	170870	So Jeff, over to you.
170870	173170	Cool. All right. Yep, like you said,
173170	175750	we're going to walk through the expectations and necessities
175750	178270	for programs, policies already in place
178270	180870	before starting or launching your program.
180870	182790	However, if you don't have them,
182790	184950	you can still follow along and implement what we're recommending.
184950	187290	So we're taking kind of like a broad approach on this one.
188470	191910	So to start with, without understanding your attack surface,
191910	193710	you can't really scope responsibly.
193830	196790	You should ask yourself, what are you exposing publicly?
196790	199390	What types of threats might you face, right,
199390	201950	based on the industry or assets that you have?
202790	204130	Know what's publicly accessible
204130	206750	and also what you want your researchers to focus on.
206810	209670	What's important for you as a company?
209670	213090	Why are you kind of wanting to host the program?
215370	218570	And yep, build an asset inventory if you don't have one.
218730	220890	A great colleague of ours said,
220890	223670	own your assets before someone else owns them, right?
223670	227370	And so building one, even if you have, you know,
227890	230390	a very immature asset inventory,
230390	232590	ask yourself, how can we work on that
232590	234830	before we actually launch the program?
235250	238330	And so you put in scope what's important to you.
239150	242710	We don't specifically endorse any, like we said earlier,
242710	246570	but some examples, some tools are Detectify, HackerOne Assets,
246570	249070	and many more to kind of get you started
249070	251670	if you need a starting point.
252130	255990	To understand your asset inventory,
255990	259170	you should categorize that you should understand relative risk.
259170	262590	Some examples of this is like using a scoring
262590	266350	or criticality system like Carver to determine risk.
267030	270050	And then various platforms have tools for this,
270050	272830	as well as purpose-built products for enumeration.
272830	275390	We can discuss how we're handling it at GitHub,
275390	278690	but that's another talk for another day.
280910	283890	Finally, like you mentioned earlier,
283890	287090	establishing a vulnerability remediation practice, it's huge.
287090	289610	You should have at least something basic internally
289610	293810	before you start receiving a ton of reports, right?
293810	296230	And the way you can start is kind of understanding
296230	298290	what qualifies as an issue.
298290	301830	What is a security vulnerability for your org and for your company?
301830	304570	Have clear ownership. This is a tough one.
304590	309350	You really want to ensure that when you do receive something on an asset,
309350	314690	you know exactly who owns it, what engineering team will be fixing it.
314690	317550	And that sounds simple, but I've seen organizations
317550	320690	have issues with, at the end of the day,
320690	324550	they've had the vulnerability for a few months now, a few weeks,
324550	328330	and they're having issues with clear ownership on who should fix it.
328390	332310	And so having that in the beginning will save you a lot of time.
333010	336970	And ultimately agreeing upon timelines for remediation.
337490	340470	And if you can, just to add one more thing there,
340470	343030	is having a good exceptions process.
343030	347030	Because if a vulnerability comes in and you don't really know,
347610	351310	it's the first of its class, but in the end it's working as expected,
351310	352990	or you just can't fix it at the moment,
352990	355430	you should have a defined exceptions process.
355430	357550	Because that bug will be sitting there for a while,
357550	358950	so you should know how to deal with it
358950	361370	before you keep getting them from other folks.
362230	364890	So now Logan's going to talk about some groundwork.
365210	370470	Yeah, so now that we sort of are established on what we're hoping is in place,
370470	372310	let's start getting into how we can turn that into
372310	374390	at least the start of a bounty program,
374390	377750	or if you're already there again, how to grow it from that point.
378090	380050	So first thing we'll say is,
380050	382170	understand what the different program options are,
382170	384690	and how these deal with different levels of maturity.
384690	386730	So one of the things that we'll generally recommend
386730	387970	is if you're starting from scratch,
387970	391410	you can look at something like a VDP, a Vulnerability Disclosure Program.
391670	394730	This tends to be a lot lower lift to get started.
394730	396870	You don't necessarily need specific tooling.
396870	398010	This could be a mailing list,
398010	401270	it could be something like Private Vulnerability Reporting on GitHub,
401270	404190	or Confidential Issues on GitLab, anything like that.
404190	406750	I mean, there are some huge projects out there that effectively do this,
406750	409790	like the Git Security Mailing List, which is a Google group at this point.
409790	414250	So there's many ways to go through and build repeatable processes out of this.
414250	418790	As Jeff mentioned, understanding which teams are going to own stuff when they come in,
418790	421250	or understanding how you're handling severities,
421250	423990	how you're doing issue assignment, SOAs, all of this.
423990	426990	This is all things that you can build with a Vulnerability Disclosure Program
426990	429450	before you really have to worry about the money side of it,
429450	432330	or platform, or tooling, or really any of that.
432570	436530	But broadly, the sort of progression we'd suggest, if you're looking at this,
436530	439830	is to start out with a Vulnerability Disclosure Program to get those reps in,
439830	442390	and then from there maybe look at moving into a private program.
442390	445450	You get a bit more control around the number of reports you're going to get,
445450	447730	potentially who's actually working on your program as well,
447730	449270	depending on how you build that out.
449270	452470	And then look at going public once you have those processes in place,
452470	455990	because otherwise you're just going to get buried the second you open things up.
457250	459470	And when you do look at moving into a paid program,
459470	462910	whether that's a private program, public program, whatever,
463810	465490	as Jeff sort of alluded to earlier,
465490	467350	we want you to scope with intention for this,
467350	470490	but also, we both think, importantly,
470490	472870	pair up with a platform that can support you through it.
472870	474750	And again, we're not going to specifically endorse anything,
474750	476890	whether it's HackerOne, Integrity, BugCrowd, whatever,
476890	478650	you get the same idea for it.
478650	480690	But the key things for this is you get a lot of stuff
480690	484130	that you may not think is super important out of the gate,
484130	486250	and then as soon as you don't have it,
486250	487970	it bites you in the ass immediately.
488290	490850	So this is going to be things like having educational resources,
490990	494590	a code of conduct, collaboration on triage and process,
494590	497610	mediation process, if shit really hits the fan,
497610	498850	and you can sort of go from there.
498850	500570	And I guarantee you all of those will happen
500570	502750	as soon as you get money involved for it.
502810	505190	But on a more positive note, a lot of this sets you up
505190	509230	for a situation where it's much easier for folks to submit stuff to you.
509230	511690	Certainly sending stuff in through email to a mailing list,
511690	514450	even if you're using PGP-signed emails or something,
514450	517270	it's not the greatest feeling for a lot of it.
517270	519390	And for a lot of folks that are, again,
519390	521290	at least in the world we work in for this,
521450	523130	a lot of people who are doing bug hunting for this
523130	524910	are already familiar with these platforms,
524910	526310	they're ready to send stuff in through it,
526310	527550	they know what it looks like,
527550	529450	they have a reputation on there already,
529450	530750	they know the ins and outs.
530890	533090	So you're sort of good to go from that.
533430	535790	But again, speaking of scoping with intention,
535790	538450	be very careful about scope creep for this.
538450	540430	Again, with the Vulnerability Disclosure Program,
540430	541410	you get a little more latitude
541410	544670	because you don't necessarily have to fix things.
544850	546570	Take that with a grain of salt.
546890	549950	But when you do start moving into broader stuff,
549950	552990	if you are getting reports and you have money on the line for those
552990	555490	and you don't actually have ownership of those
555490	557110	or a way to remediate stuff,
557110	560270	or an exceptions process if you want to accept risks for it,
560270	562390	you're going to be in for a lot of frustration
562390	564030	on both sides of that relationship,
564030	565810	both for your company, your organization,
565810	567610	and for the hackers that are working on it.
570430	571470	There we go.
572250	575090	And another area that a lot of folks will overlook,
575090	576350	again, until it bites them,
576350	580030	is make sure you are bringing your legal and PR teams into this.
580030	582070	This is all customer-facing stuff.
582070	584050	As soon as you have money on the line as well,
584050	587350	you have a lot of reputational impact that's going to come from that.
587350	589110	You have a lot of legal obligations.
589230	591050	And this is somewhere else that having a platform involved
591050	594310	rather than doing a homegrown thing can be very, very helpful.
594390	596330	As soon as you have to worry about trade sanctions
596330	597950	for where you want to send money to,
597950	600390	currency conversion, all of those moving parts,
600390	602490	if you have a partner that can take care of a lot of that for you,
602490	605270	has a good reputation for it, has all the processes,
605270	606990	you're going to be saving yourself, again,
607070	609750	a lot of frustration, a lot of pain, a lot of suffering.
610090	612390	And also for many of these platforms,
612390	614070	if not all of them at this point,
614070	616270	it gives you standardized tooling for disclosure.
616270	618910	They have their own terms for how you want to handle disclosure.
618910	620790	You don't necessarily need to reinvent the wheel
620790	622690	or have something that's bespoke to you.
622870	626210	Same deal for safe harbor terms, things like that.
626210	629030	So HackerOne has the gold standard safe harbor, I think they call it.
629030	631110	Many other platforms have something similar.
631470	633410	In our experience, the companies we've worked with,
633410	635970	we have some bespoke ones, but it's all sort of the same idea.
635970	638190	If you send stuff in, you're safe.
638190	640250	But again, you're getting a lot of this for free,
640250	643490	out of the box when you're partnering up with somebody.
646450	647710	Forgot about those.
648450	650730	And then as you're going through as well,
650730	653430	as you progress through each of these stages,
653430	654950	you're getting a lot of information around
654950	657230	what that next stage is going to look like.
657330	660330	So when you have a vulnerability disclosure program to start out with,
660330	662090	again, you're getting the reps in for how you want
662090	664450	to build out your processes for it.
664450	665950	When you move into a private program,
665950	667670	you're getting a better idea of what some of the budgeting
667670	669610	is going to look like, your cost per bug.
669610	671530	If you need to fine-tune your severity models,
671530	673770	you can do that in a little bit of a more controlled environment
673770	675450	than if you're already public.
675790	677650	And this also gives you a really good opportunity
677650	680190	to start building out some tooling for this.
681150	683630	And if you're interested specifically in tooling,
683630	684830	we're not going to talk about it here,
684830	687490	but Jeff did a great talk at B-Sides SF earlier this year
687490	689470	called Life of a Bug, which goes through how GitHub
689470	692710	does all their bug remediation, bug tracking, all that.
692850	696130	Very helpful. It saves us a colossal amount of time.
696130	697830	It also makes auditing really easy,
697830	699590	which is another thing that having a good platform
699590	700850	can really help with.
700850	702150	So if we need to go through and say,
702150	704550	hey, are we consistently paying out the same way
704550	706470	for these types of vulnerabilities as they come through?
706470	708090	Are we consistent in our messaging?
708150	710830	All of that is already there. We know exactly what it looks like.
710830	713530	It's the same thing every single time for it.
714670	717350	And yeah, also looking at budgeting and staffing for this.
717430	718950	When you're going through, again, VDP,
718950	720850	you're getting an idea of the amount of time it's going to take
720850	723270	you to remediate something. You move into a private program,
723270	724930	you're getting an idea of the costs for it.
724930	727410	But that's also going to have an uptick in interest,
727410	729570	generally, for the number of reports you're going to get.
729670	730970	So you can start figuring out,
730970	733470	okay, if we're getting 100 reports per year,
733470	735010	100 reports per month, maybe,
735010	737290	what does that look like in terms of time and effort
737290	739810	for our company? And then when you go public,
739810	741310	yeah, all hell's going to break loose.
741330	743810	Especially if you manage to get featured as a new program
744530	745910	on any of these platforms,
745910	747730	you're going to get a big uptick in interest
747730	750010	almost immediately. You need to be prepared for that
750010	751890	because nothing is worse than just having things that will
751890	754950	linger for months while people are expecting it to get fixed.
757090	760190	Oh, and I will give a quick shout-out for...
760190	761850	there's a fantastic resource,
761850	763530	and we do have a slide that has links to all these resources
763530	767130	at the end, as well as these slides in there.
767410	768690	There is a forum called
768690	770410	Bug Bounty Community of Interest,
770410	773190	which has fantastic articles around all of these.
773190	774370	Definitely worth checking out.
774370	776210	I do not remember the URL, but it is linked
776210	777590	in the resources here.
778470	780550	So moving forward, let's chat about
780550	782990	what things look like once you're actually operational.
782990	785050	You've got the baseline stuff
785050	786850	you need in place for this.
787030	789050	And again, our goal here is to talk about
789050	790310	some of the less
791450	792890	easily documentable things
792890	795430	for it when you're going through and building a program.
795490	796890	So the first one is going to be
796890	798870	focusing on transparency for this.
798910	800630	The more you can document
800630	802570	and share with the community of hackers
802570	805050	that's going to be working on your program, the better.
805050	807370	And that's everything from process that you're going through,
807370	809490	your timelines, your payout, your scope.
809590	810770	There's nothing that's worse than
810770	812490	ambiguity in any of this.
812530	814150	Somebody's going to send in a report to you,
814150	816950	you're not going to be sure if it's in your scope.
816950	818390	That's going to suck for everyone.
818390	820910	If they know specifically that it is in scope,
820910	823370	and you wind up having to go through a mediation process,
823370	824950	you've got clarity for that, again,
824950	826590	on both sides for it.
826630	828970	And it gives you a lot of opportunities as well when you start working
828970	830590	with disclosure for it.
830710	833070	The more transparent you can be around that,
833070	835210	whether it's limited disclosure, full disclosure,
835210	837510	whatever, if it's unilateral,
837510	838950	both sides doing a coordinated
838950	840250	disclosure, whatever,
840730	842890	it's giving really good insight into how
842890	845010	your program operates. Whether that's from a timeline
845010	847790	perspective, or it's from a
847790	849290	operation perspective, what your
849290	851170	communications look like, what all of the
851670	853110	individual moving bits and pieces look like,
853110	855230	even payouts. You can start looking at some
855230	856930	of these programs out there and say,
856930	858970	hey, if I get an IDOR for this product,
858970	860790	I have a pretty good idea of what the payout's going to be,
860790	862930	so I can manage my own expectations for it.
863070	864690	So that can be vulnerability classes,
864690	867050	it can be specific assets in there as well.
867110	869250	There's lots to look at for it.
871810	872750	There we go.
872790	874990	And building on that as well is making sure that you've got
874990	876970	good clarity on what you want hackers
876970	879070	to focus on when they are
879070	880370	looking at your program.
880370	882350	So I'll talk about GitHub
882350	884310	very briefly for this, because I think
884310	886070	we've done this all right.
886070	888490	But we have specific lists of stuff that we do care
888490	890470	about, things that we don't care about.
890470	892550	And that's not to say that they're not necessarily security
892550	894350	issues, but they're things we're aware of.
894350	896370	It's accepted risk, so it's pretty
896370	898730	easy for us to filter stuff out when it comes in.
898730	900350	We've seen that with other programs we've worked on
900350	902430	as well. It's a significant time saver on both
902430	904910	sides. If you, as a
904910	906650	researcher in the community, are looking
906750	908470	at a program, and you immediately know,
908470	910450	hey, I don't want to touch this asset,
910450	912310	it's not going to get me any money, it's not going to be
912310	914450	of interest, you can save yourself a lot of time
914450	916410	and focus on something that instead will give you
916410	918530	that level of payout that you're looking
918530	920810	for. And with this as well,
920810	922570	in addition to understanding
922570	924610	what you do want to focus on, what you don't want to focus
924610	926410	on, this is also an area to add a lot of
926410	928130	clarity around your payout structures.
929150	930670	For many programs out there,
930670	932390	it's going to be a set thing, like, hey, if this is
932470	934390	a low severity, it's $100. If it's medium,
934390	936310	it's $500, so on and so forth.
936310	938170	For others, it's a direct tie to
938170	940190	CVSS, or VISS, or
940190	942310	one of five million standards that people are coming
942310	944290	up with now. But having clarity around
944290	946250	it, having examples for it as well, if you're doing
946250	948390	disclosure through your program, is incredibly
948390	950290	helpful for it. And
950290	952010	with that as well, having a communication
952010	954510	plan for a lot of this is really, really helpful.
954510	956330	Whether it is something that you want to handle
956330	958170	specifically internally, whether
958170	960770	you are staffing this program through a
960770	962450	platform to do your triage work, or you're doing
962450	964410	it yourselves, having a list of resources
964410	966370	for that, having things like canned responses
966370	968290	for it, can be very, very helpful.
968410	970350	You're ensuring consistency for it, you're ensuring
970350	972270	that when people work on your program, they know exactly
972270	974350	what they're going to get for it. And it
974350	976290	really does help with a lot of the auditing stuff
976290	978170	as well, as we've alluded to, because you understand
978170	980270	exactly what you're in for every single time
980270	981070	for it.
982090	984030	And I think one of the other things to put on here,
984030	986470	and I say this coming from a customer-facing
986470	988210	background before moving into the security
988210	990330	world, with a lot of this, you have
990330	992050	to remember that you are, quote-unquote,
992050	994150	customer-facing, user-facing, whatever.
994150	996190	So, whatever you say, you're going to be representing
996190	998110	the group you work for. So, if I get
998230	1000830	a bounty report, and I am a
1000830	1002330	giant ass to somebody,
1002330	1004430	it's not that I was an ass, it's that GitHub was
1004430	1006330	an ass. You don't want to necessarily be
1006330	1009690	on the hook for that.
1009690	1010550	And,
1010550	1012330	speaking of being an ass, understanding
1012330	1014390	how to handle disputes and duplicates.
1014390	1017530	These will happen, they happen very frequently.
1017530	1018490	More often than not,
1018490	1019590	a lot of disputes are
1020890	1021690	of dubious
1021690	1023410	nature, but duplicates
1023410	1025730	are going to happen very, very frequently.
1025730	1027630	And this could be duplicates for stuff you're already
1027630	1029730	tracking in your company as known issues, but maybe
1029730	1031850	aren't accepted risks. It could be things
1031850	1033730	where you have two researchers
1033730	1035850	who submit the same report within minutes of each other
1035850	1037650	even. So, having a clear, documented
1037650	1040070	policy on that is very helpful as well.
1040070	1041690	And that can be something as simple as, hey,
1041690	1043650	if we get duplicate reports, if somebody sent it
1043650	1045730	in first based on timestamp, they're going to get
1045730	1047690	the award. Understanding whether or not you want
1047690	1049490	to combine people onto single reports
1049490	1051710	is also something that sounds a bit
1051710	1053710	more simple than it actually is. That is
1053870	1055670	a great way to lead yourself down a path
1055670	1057850	of pain with mediation requests
1057850	1059510	and other frustrations and
1059510	1062250	arguments more frequently than not when that happens.
1062410	1063590	So be conscious of
1063590	1065750	that when it goes through. One of the
1065750	1067830	other things that comes up and causes a lot
1067830	1069630	of pain and suffering for programs and
1069630	1071810	researchers is making
1071810	1073790	sure you have something along the lines of a
1073790	1075610	one-fix, one-bug, one-payout type
1075610	1077790	of policy. There was a great example
1077790	1079750	that popped up the other day. I don't remember
1079750	1081710	what the asset was, what the company was,
1081710	1083370	but somebody had found what they
1083370	1085530	had considered to be 40 or 50 different
1085530	1087890	crits that were coming in. It was leaking PII,
1087990	1090110	a bunch of other problems.
1090110	1091590	This was not for us, I'll say.
1091850	1093530	But with that,
1093530	1095710	those all got closed out as a single
1095710	1097510	low-severity vulnerability because it was
1097510	1099610	something that was being tracked. So this person had spent
1099610	1101510	about a month compiling stuff, sent
1101510	1103630	all of it in, and got completely hosed
1103630	1105710	on it. So making sure you've got
1105710	1107730	as much structure there as possible,
1107730	1109690	you've got the policies in place to avoid that kind
1109690	1111630	of frustration, is really helpful
1111630	1113550	because otherwise you're going to wind up on the
1113550	1115610	bug bounty forum slack, people are going to be flaming your
1115610	1117510	program, and it's not fun for anyone.
1117510	1119630	You don't want to be on the other side of that because it's
1119630	1122030	not going to lead to a positive outcome for anyone.
1123670	1125750	And last but not least for this bit,
1126170	1127450	just chatting about doing
1127450	1129930	your own triage versus working with a
1129930	1131430	platform for it, both of these are
1131430	1133650	incredibly feasible
1133650	1135570	ways to do things. We've both
1135570	1137630	worked with groups that are using
1137630	1139710	hosted platform-managed
1139710	1141390	triage services, whether it's, again,
1141390	1143210	HackerOne, Integrity, whatever.
1143570	1145410	These can be incredibly helpful because you don't
1145410	1147250	need to change your own staffing
1147250	1149530	for it. The key thing I'll note with that
1149530	1151370	is on most of these platforms, that's going to come with
1151370	1153370	an additional cost, but more importantly
1153370	1155450	you also need to be set up to still
1155450	1157370	do that triage when it comes through.
1157370	1159270	A lot of the time, this is going to be lacking
1159270	1161250	the business context for it, so they may not
1161250	1163390	necessarily understand severity that's going along with it,
1163390	1165330	especially as you're starting out as a new program
1165330	1167210	that will build over time, but
1167210	1169250	it's not going to be there out of the gate, so you
1169250	1171410	need to be able to filter through
1171410	1173530	whatever is coming in after that.
1173530	1175170	So we've spoken with various other
1175170	1177230	companies where they'll say, yeah, we're using
1178230	1179290	a managed triage service,
1179290	1181210	we don't actually have anybody full-time,
1181210	1183210	it just goes into our AppSec team, but we're
1183210	1185150	still spending 50, 60, 70 hours a week
1185150	1187030	doing triage on this stuff because
1187030	1188830	yeah, it's valid, but it's stuff that
1188830	1190950	they don't have visibility into our internal bug
1190950	1192950	base, so we don't need to do anything with it.
1192950	1195290	So that's one of the things to consider for it.
1195550	1197090	One of the other areas that comes with
1197090	1198930	that as a bit of a selling point is looking at things
1198930	1201110	like spam filtering, especially as you're going to a public
1201110	1203010	program. I would
1203630	1205010	suggest that that's not as important
1205010	1207050	as it may sound like. You think you have a public
1207050	1209050	program, you're going to get a lot of garbage, and you
1209050	1210970	are, but if you have that
1210970	1212990	context around your program already,
1212990	1215250	your products, you have that expertise on it,
1215370	1217290	a lot of that goes away pretty quickly.
1217490	1219050	So for reference, we're one of the larger
1219050	1221310	programs, I think, out there at GitHub,
1221310	1223230	and we get about 20 or 30 spam reports
1223410	1225030	a week, and those take under a minute to
1225030	1227270	remediate. So for our entire team, it's a
1227270	1230070	pretty negligible cost to do that.
1230070	1231390	And yeah, going for in-house,
1231390	1233530	the other side of this as well is you're looking at a
1233530	1235390	situation where you can make the
1235390	1237370	appropriate decision, whether that's
1237830	1239670	from a payout standpoint, a triage
1239670	1241430	process, whatever, you can make that very
1241430	1243350	quickly because you have the full context around
1243350	1245170	it, and you've got the expertise
1245170	1247750	on it. Again, we've heard a lot of challenges,
1247750	1249290	and that's not to say this solves the
1249290	1251210	problem by any stretch, but there are many
1251210	1253310	other cases where something without that context
1253310	1255230	may show up as being a lower
1255230	1257330	severity issue, when in fact it's something you might actually
1257330	1259710	consider to be a crit and need to work on right away.
1259850	1261210	So being able to make that judgment call
1261210	1263350	very, very quickly instead of having to route it through another
1263350	1265510	party can be helpful. So there's a lot of trade-offs
1265510	1267250	here. It's not to say one way is any better
1267250	1269530	than the other, but something to be conscious of that otherwise
1269530	1271470	does not always show up that way.
1272210	1273630	And with that, we'll talk about
1273630	1275510	what I think both of us consider the most
1275510	1277490	important intangible part of this.
1277490	1279330	So over to you, Jeff. Cool, yeah.
1279770	1280810	There's not a lot of
1281690	1283850	documentation around how to build relationships.
1284030	1285510	It's a rather new thing,
1285510	1287410	right? And yeah, the
1287410	1289670	program's up and running, right? You're getting value
1289670	1291430	out of it. How do you grow from there? How do you create
1291430	1293470	relationships? How do you celebrate the wins?
1293470	1295550	How do you keep your researchers coming back?
1298510	1299170	And so
1299170	1300730	one thing is
1302410	1303270	basically create
1303270	1305390	relationships and celebrate... oh, I already
1305390	1307230	said that, sorry. An empathetic approach
1307230	1309230	helps researchers continue to invest time in
1309230	1311170	your program and represent your
1311170	1311970	org, right? So
1313210	1315010	when you get a report and
1315010	1317350	it's kind of like big bounty and they're like, oh,
1317350	1319010	I really need like 50 bucks or
1319010	1321070	I spent 20 hours
1321070	1323130	on this. You can explore
1323130	1325430	ways where you don't just turn that into a
1325430	1327430	negative. You can just close it out as spam and
1327430	1329290	send them on their way. One
1329290	1330930	way you can do is like exploring
1331850	1333050	defense in debt is
1333990	1335430	one way of handling reports where
1335430	1337390	you're like, okay, you know what? This might have not met
1337390	1338730	the low severity, but
1340250	1341370	it's a good idea. It's a good
1341370	1343790	maybe security feature request or something.
1343950	1345330	And so it's kind of, you know, you throw
1345330	1347530	them $200, $500. We have something
1347530	1349250	I get where we do that. And we
1349250	1351530	tell them like, this didn't really meet the low severity,
1351530	1353450	but this is what we're going to do with
1353450	1354850	it. And so we're going to award you a
1354850	1356750	lower than low severity
1358610	1359530	bounty.
1359530	1360990	And they like that. It encourages
1360990	1363130	researchers to continue to try to find bones
1363130	1365010	on your platform and to
1365010	1366250	keep coming back.
1367890	1368990	Yeah, so I
1368990	1371130	was alluding to be conscious how you close the reports.
1371250	1372970	Don't close them as spam immediately
1372970	1375010	all the time. If they're actually
1375010	1377490	trying or they were close,
1377490	1379030	you can explore ways of using
1379030	1380850	informative or
1382050	1382890	duplicate.
1382890	1384690	But like, those don't hurt
1384690	1386490	someone's red points. And so
1386490	1388310	those are one way that
1389890	1390770	that's a way that
1390770	1392470	maybe researchers can...
1392470	1395170	they'll want to come back, you know, and keep working with you.
1395170	1396590	And to jump in quickly while Jeff
1396590	1398630	gets the slide sorted out here, because I think we had
1398630	1400770	some technical issues with that. The scenario
1400770	1402690	I mentioned earlier with those 50 reports that
1402690	1404710	came in for that program, all of those got
1404710	1406650	closed out as not applicable. That person's
1406650	1408970	reputation went from being very well respected,
1408970	1410530	scored on HackerOne, to being
1410530	1412530	tanked in a single day. So
1412530	1414470	it led to a lot of other discussions that, again,
1414470	1416170	are not fun for anybody involved.
1417250	1418290	Right, right. And so
1418290	1420290	helping researchers grow, right? This is
1420290	1422530	the education part that we spend a lot of
1422530	1424410	time on. We rather
1424410	1426450	educate rather than, like,
1426450	1428070	you know, chastise if
1428070	1430710	there are off-base submissions that aren't spam,
1430710	1432390	right? What we do is we
1432390	1434610	provide resources and direct feedback,
1434610	1436410	right? It takes five minutes to let
1436410	1438370	them know, hey, maybe you should check out this
1438370	1440270	podcast that might help you out. Or check out
1440270	1442210	Portswigger Academy. That might help you get a little
1442210	1444390	closer to this, you know,
1444390	1446350	LFI that you thought you found. Or this
1446350	1448410	SSRF, you know, that's clearly not
1448410	1450270	an SSRF, but you were looking
1450270	1452130	for it, you know? So we
1452570	1454130	have a lot of internal
1455670	1456350	lists that we
1456350	1458390	use to try to educate folks to...
1458390	1460290	And it's cool, because eventually they do
1460290	1462230	come back. We've seen a few folks where
1462910	1464410	we helped them out, we educated
1464410	1466410	them a little bit. Then they came back maybe a year later
1466410	1468410	and they found like a medium or something.
1468410	1470310	And they let us know, like, hey, this is
1470310	1472410	cool, you know? You sent me
1472410	1474170	on my way and I came back a year later.
1474170	1475850	Those are special.
1478810	1480330	Showcasing submissions. That's
1480330	1482430	something that we love to do.
1482570	1484450	You should explore opportunities for
1484450	1486710	joint articles, blog posts,
1486710	1488410	and the creation of content.
1488710	1490170	Basically, you know, some
1490170	1492930	examples are like walkthroughs of disclosed bugs,
1492930	1494350	their life cycle. If you're
1494350	1496530	not ready for disclosure,
1496530	1498210	that's okay. It takes time.
1498210	1500610	One way that you can do that is if you see a
1500610	1502310	researcher who's just focusing...
1502310	1504370	You can tell, they're just focusing on your program.
1504370	1506350	They're sending in... Every two weeks
1506350	1509450	you're getting one or two good bugs from them.
1509450	1510930	Reach out. Form a
1510930	1512390	relationship with them. Just ask
1512390	1515430	them, hey, can I give you
1515430	1517090	a researcher spotlight?
1517090	1518510	I just want to write a blog
1518510	1521010	and tell everybody, and maybe inspire
1521010	1521710	other researchers to
1522210	1523870	check out our program.
1523870	1525850	And we've done that a few times where we'll just interview
1525850	1527790	folks and we'll write a blog
1527790	1529810	post with the answers and we'll give
1529810	1531770	them something, you know, a platform
1531770	1533670	where they can kind of talk about their own
1533670	1535590	research and really plug
1535590	1537610	their own stuff, you know, and
1537610	1540170	why they like to come and hack on us.
1540270	1542090	And that's pretty cool.
1546050	1547650	Lastly, researchers
1547650	1549670	and industry peers enjoy interacting
1549670	1552050	and reading what programs are up to.
1552050	1553950	You can do this in a lot of different ways.
1553950	1555720	You can join, you know, Slack workspaces
1556630	1557930	like Bug Bounty Forum.
1557930	1559830	You can, you know, create a Twitter
1559830	1561890	account and just tweet little things
1561890	1563770	here and there. Like, we tweet our
1563770	1565970	monthly stats and people like to engage with them.
1565970	1567550	That's pretty cool. I like to see, like, how much
1567550	1569510	did GitHub pay last month, you know?
1569510	1571630	And some programs are pretty cool
1571630	1573750	at promoting promotions.
1573750	1575950	Like, oh, we are paying a 30%
1575950	1577670	bonus on cross-site scriptings.
1577670	1579430	And that, you know, if you have researchers
1579430	1581810	new or, you know, returning
1581810	1583750	researchers and they check that out
1583750	1585750	they will most likely want
1585750	1587310	to hack on you, you know? They'll be like, oh,
1587310	1589750	30%, 40% bonus sounds nice
1589750	1591870	this month. And so,
1591870	1593810	yeah, using these community discussions
1593810	1595730	it's pretty cool for industry
1595730	1597470	peers and then also researchers.
1600950	1603110	So you've got your program scoped, operations
1603110	1605110	are underway, relationship, you're
1605110	1606870	building relationships.
1606870	1608770	You know, what's next?
1612900	1613620	So,
1613620	1615760	now that you're operational and have your goals changed
1616520	1617700	you should create
1617700	1619140	if you haven't created one
1619560	1621680	you should create one or
1621680	1623840	update a roadmap of where you want to be.
1623840	1625640	Where do you want to be in two
1625640	1627380	quarters? Where do you want to be in a year?
1627840	1629780	How do you really get, like, the return
1629780	1631740	of investment for your bugs?
1632760	1633700	Let's say you're getting all
1633700	1635920	your bugs and you're fixing them correctly. What's next?
1635920	1637740	Like, do you have the bandwidth to explore
1637740	1639620	varying analysis? Can you enable
1639620	1641780	something like CodeQL to really try to
1641780	1644400	find more bugs with the bugs that you're getting already?
1644420	1645840	You have access to the code base,
1645840	1647280	right? So, using
1648620	1649900	tools, tooling,
1649900	1651840	staffing and all that to try to get more bugs
1651840	1653680	of the bugs you get are
1653680	1654880	so impactful.
1655340	1657680	What needs to change from what you're doing now?
1658500	1660200	Maybe you need more automation.
1660580	1661840	How can you do that?
1661840	1663820	What is working well that you want to double down
1663820	1665440	on? What do you want to focus on
1665440	1666320	for the next
1667580	1669500	two months? What do you really want to focus
1669500	1671540	on and then move on to the next thing? And what problems
1671540	1673480	are you having? Is it time
1673480	1675480	to focus on disclosure? Is it time
1675480	1677460	to focus on communication? Maybe
1677460	1679480	you can train your...
1679480	1681820	If you're using platform triage,
1681820	1683560	is it time to maybe spend a few
1683560	1685400	weeks and months just really getting
1685400	1687380	to know them so that that layer, that
1687380	1689240	filtering layer gets a little better?
1689880	1691540	So, what problems are you
1691540	1692220	having?
1692220	1694200	That's what you should focus
1694200	1696700	on. And internally
1696700	1698260	promote and celebrate findings
1698260	1700160	from your program. Focus
1700160	1702260	on blameless handling
1702260	1704140	of bugs. A blameless culture
1704900	1706180	really goes a long
1706180	1708300	way for engineering and security
1708300	1709420	teams.
1709780	1712300	I've been at an organization where it
1712300	1714160	was tougher to get things fixed. It was
1714160	1716240	always, ah, I don't think
1716240	1718260	that's an SSRF. I didn't mean to
1718260	1720260	introduce an SSRF. But like,
1720260	1722080	truly what it should be is
1722080	1724100	we did that. We introduced
1724100	1726140	the SSRF. How do we not
1726140	1728040	introduce it going forward, right? It's not
1728040	1730080	so simple. But like, lessons learned from
1730080	1731460	these bugs, right? Instead of just
1732100	1734140	paying them out, you
1734140	1736060	know, having a hard time fixing them and then forgetting
1736060	1738040	about them, you're just going to keep getting them.
1738040	1739960	You're going to keep seeing these patterns. So, if
1739960	1742300	you try, and it's a
1742300	1744440	cultural thing and it's tough, but if you really
1744440	1747180	are able to build a blameless culture,
1747180	1748200	you'll be able to be
1748200	1749980	way more successful with this bug bounty
1749980	1752120	program that you're spinning up or
1752120	1753780	you're revamping.
1754100	1756020	Another thing is software has bugs, right?
1756020	1758060	This is another feedback mechanism
1758060	1760100	in addition to other tooling and
1760100	1762040	processes. So,
1762040	1764240	yeah, just understand that you're
1764240	1765360	in it together.
1770260	1771920	Cool. So,
1773380	1774300	most public
1774300	1776840	programs are very clear about their scopes,
1776840	1778600	their reward structures, what they do,
1778600	1780540	right? You can check out activity, everything's
1780540	1782360	on there, based on the platform
1782940	1784100	you use.
1784560	1786460	So, you should compare yourself to industry
1786460	1788500	peers, but you should understand
1788500	1790180	you don't need to compete
1790180	1792800	to be competitive, right? You're in different industries,
1792800	1794320	you're in different maturity
1794320	1796440	levels of your program,
1796440	1798580	and just chat
1798580	1800400	with industry peers, just see
1800400	1802440	what they're doing, what can
1802440	1804400	you implement to your program? Are you ready
1804400	1806720	to implement some of the things that others are doing?
1806720	1808380	Can you start simple? I
1808380	1810660	like how this company has automation in place,
1810660	1812700	and as a team of three, that's what
1812700	1814840	I really need, and that's what you're going to focus on.
1814840	1816400	So, talking and
1816400	1818560	collaborating with industry peers will
1818560	1820600	be fundamental, really.
1821180	1822640	One more thing is, few people
1822640	1824460	hack on a single program. Like
1824460	1826640	Logan said earlier, these researchers, they bounce around,
1826640	1828640	right? Most of them,
1828640	1830380	they have a lot of experience
1830380	1832420	working with a lot of companies,
1832420	1834720	and just for some metric,
1834720	1836760	GitHub has had a little more
1836760	1838580	than 850 unique reporters
1838580	1840780	last year, in 2022,
1840780	1842480	right? And some of them were
1842480	1844720	first-timers. I don't have the exact number,
1844720	1846520	but that's a lot, right? More than 850
1846520	1849040	people coming to your program for the first time.
1849040	1850560	That can sound scary,
1850560	1851560	really, but
1854460	1856580	like I said, chatting with industry peers kind of
1856580	1858520	helps build and mature your program so you can
1858520	1860940	handle that many reporters,
1860940	1862600	right? Because a lot of these unique
1862600	1864660	reporters, it's their first time engaging with your program,
1864660	1866720	so it can be rough if they don't
1866720	1868760	maybe understand how your program
1868760	1870340	runs compared to others, right? So
1870340	1872660	if you have a lot of the communication, automation,
1872660	1874200	all that, it'll
1874200	1876200	have a really smooth
1876640	1878640	process for these new researchers, and they'll
1878640	1880580	want to come back. And that's the goal, right? You don't want
1880580	1882580	them to submit one
1882580	1884380	report and then never again.
1885240	1886520	And so, yeah, chatting with
1886520	1888480	industry peers, engage with programs, understand
1888480	1890520	what they're seeing, what are problems
1890520	1892780	that they're not seeing and you're seeing,
1892780	1894500	what are some common issues that are being
1894500	1896560	submitted, you know,
1896560	1898000	throughout the industry.
1900500	1901300	Lastly,
1903080	1904540	yeah, so you
1904540	1906780	want to talk to bug hunters and review feedback.
1906880	1908500	What we do is we go to
1908500	1910600	conferences, we meet up with all the researchers
1910600	1912540	that we can, and we just ask them,
1912540	1914480	like, how's it going? What programs are
1914480	1916600	you hunting on, right? And if we see a trend
1916600	1918660	and we're seeing that one program
1918660	1920800	is what a lot of people are focusing on,
1920800	1922600	we'll check it out. Like, what are they doing
1922600	1924660	that's pretty cool? What are we not doing?
1924660	1926740	Like, at DEF CON
1926740	1928620	last year, we had a few researchers that we
1928620	1930340	chatted with for about five hours
1930340	1932560	in one day, and they said, I really want
1932560	1934480	swag. And so we said, okay, cool.
1934480	1936360	We'll work on it. And so we launched a cool
1936360	1938300	bug bounty swag store that only
1938300	1940420	people who submit a valid report to us
1940420	1942660	can get that kind of swag. So we gamified
1942660	1944580	it a little bit, you know? And so what
1944580	1946360	that causes is that these people, these
1946360	1948300	researchers, they're like, I kind of want
1948300	1950840	that swag that you can only... it's
1950940	1952580	a challenge, you know? It's kind of like a CTF.
1952580	1954360	You get some money, and you get cool swag
1954360	1956800	that only you can get if you submit a bug.
1956840	1958460	So finding ways to
1958460	1960480	gamify it, you know? Launching a swag
1960480	1961860	store is not that expensive.
1962740	1964200	So yeah, basically
1964200	1966180	talking to bug hunters and seeing what
1966180	1968380	is something they would like,
1968380	1970260	you know? Not all the requests are going to be
1970260	1972300	things you can do, right? Some things
1972300	1974340	take time. There's some things that we got asked
1974340	1976160	for that'll probably take like a year or two.
1976160	1978400	And that's okay. But at least you're working on them, you know?
1978440	1980220	You're surveying the community,
1980220	1982400	and you're interacting with folks on what is
1983120	1984300	out there right now. Like, what
1984300	1986280	is trending? What is something that maybe we should
1986280	1987800	do that others are doing?
1989200	1990300	I will say swag
1990300	1992160	is a very common request. There are some
1992160	1994400	programs out there. I think it's the Dutch government
1994400	1996220	as well. Their approach is they
1996220	1998260	can't pay for anything. They have a vulnerability disclosure
1998260	2000200	program, but you get an awesome shirt that says, hey, I
2000200	2002520	hacked the Dutch government. All I got was a shitty t-shirt.
2002520	2004580	That kind of stuff is fantastic. It's very
2004580	2006500	unique. There's no other place you can
2006500	2008540	get that. So it's a shameless plug for us,
2008540	2010400	I guess. If you want one of these wonderful GitHub hoodies,
2010400	2011720	you can come hack on our program.
2012260	2014580	Exactly. And I think
2014580	2015480	that's it.
2016100	2018700	So I promise this is not a
2018700	2020720	rickroll. You can go to this, but the link is there
2020720	2022640	if you don't believe me anyway.
2022640	2025180	This is going to be a lot of stuff that we've talked about
2025180	2028040	through this, resources to back it up, a lot of
2028040	2029320	documentation. There were too many
2029320	2030280	links to bother putting on here, so if
2030280	2032480	you want a good weekend, week, month,
2032480	2034240	year of reading, this is something we're
2034240	2036360	continually updating with all sorts of bug bounty
2036360	2039140	stuff, whether it's program building, operations,
2039520	2040600	building out the fundamentals
2040600	2042700	for it, sort of a dumping ground for that.
2042700	2044140	We're trying to structure it in a lot of the
2044140	2046320	same framework we have for this. We'll also
2046320	2048400	have a copy of these slides with the speaker notes, once I
2048400	2050420	edit them down a little bit, to
2050420	2052240	be up there as well, if you want to loop back to that.
2052240	2054200	It's going to have contact info for both of us, too.
2054200	2056600	Feel free to reach out, any questions you have,
2056600	2058320	whether here or later on, if anything
2058320	2060140	comes to mind, we're always happy to chat
2060140	2062440	bounty. So, thank you very much for
2062440	2064040	sticking with us, hope you had a great DEF CON!