[00:12.740 --> 00:18.140]  Hello and welcome back. I hope you're looking forward to another fantastic talk here at the
[00:18.140 --> 00:26.240]  AppSec Village for DEF CON 29 in safe mode. It's important to remember you're 3-2-1.
[00:26.240 --> 00:32.260]  Even though we are separate from each other and socially distanced, your cohabitants will really
[00:32.260 --> 00:38.680]  appreciate that one in there. And just take care of yourselves. Get some good sleep, get some good
[00:38.680 --> 00:45.180]  food, make sure you take a shower. For our next talk, we've got David Waldrop talking about the
[00:45.180 --> 00:51.980]  DevOps and Agile Security Toolkit. After 25 years in the application development, David moved to
[00:51.980 --> 00:57.300]  information security just over seven years ago. His passion is helping developers write secure
[00:57.300 --> 01:03.060]  code. He's currently an information security advisor supporting the application development
[01:03.060 --> 01:08.700]  community at his employer. Let's put our hands together and welcome David Waldrop.
[01:10.000 --> 01:15.400]  Hi and welcome to the AppSec Village. I hope you're having a great DEF CON 2020,
[01:15.400 --> 01:21.780]  albeit in safe mode. My name is Dave and for the next 30-45 minutes, we're going to talk about the
[01:21.780 --> 01:26.420]  DevOps and Agile Security Toolkit. Now before I get started, I have to make the standard
[01:27.100 --> 01:32.100]  disclaimers. First, this presentation and any comments that I make are my own and do not
[01:32.100 --> 01:37.800]  necessarily represent those of my employer. Secondly, I will mention a few example products.
[01:37.800 --> 01:43.100]  Some are commercial, some are open source. These are not necessarily meant as recommendations.
[01:43.260 --> 01:47.500]  They are mentioned to serve as a starting point for your own investigation and review.
[01:47.500 --> 01:52.980]  These are things that I've run across in my journey, so take that as it is. Let's start
[01:52.980 --> 02:00.180]  with the two terms that are on everybody's buzzword bingo sheet these days, DevOps and Agile.
[02:00.180 --> 02:05.220]  We need to get definitions in place for the duration of this talk, so we have an idea that
[02:05.220 --> 02:11.780]  we're all dealing with the same playing field. Agile. Agile focuses on continuous iterative
[02:11.780 --> 02:17.180]  development and testing. Software is developed in smaller increments and then is integrated
[02:17.640 --> 02:24.320]  for final testing and delivery. Scrum, Kanban, and XP Extreme Programming are examples of Agile
[02:24.320 --> 02:31.000]  approaches. On the other hand, DevOps focuses on the collaboration between app dev and operations.
[02:31.000 --> 02:37.340]  The focus is on deploying code to production in a fast manner, usually automated.
[02:38.920 --> 02:44.100]  So you can see from the definitions that these are two separate concepts. However,
[02:44.100 --> 02:49.860]  many organizations tend to combine them into one initiative. Very, very rarely do you find
[02:50.600 --> 02:56.680]  a shop doing DevOps and not doing Agile, or vice versa. So no matter whether they start with DevOps
[02:56.680 --> 03:03.140]  or they start with Agile, the one piece that's usually late to the party is security. But no
[03:03.140 --> 03:08.060]  matter how late security is invited to the party, or whether we have to crash the party ourselves,
[03:08.060 --> 03:14.360]  we usually can still bring valuable resources without slowing things down, or at least without
[03:14.360 --> 03:18.420]  slowing them down a lot. And that's what we're going to talk about through the rest of this
[03:18.420 --> 03:25.740]  discussion. So for the past few years, I've worked with several different Agile and DevOps teams.
[03:25.980 --> 03:32.920]  My integration with this started as a pilot program where I was the information security
[03:32.920 --> 03:39.400]  representative that was assigned to help introduce Agile to our organization. More recently, I've
[03:39.940 --> 03:45.120]  implemented the information security integration with the Agile and DevOps teams across our
[03:45.120 --> 03:49.520]  organization. During this session, I'm going to share with you some things that really worked well
[03:49.940 --> 03:53.940]  and some things that just didn't seem to work at all. So hopefully you can take something
[03:54.340 --> 03:57.860]  away from this and apply it to your own organization.
[03:59.760 --> 04:03.720]  This particular slide will give you the map of what we're going to talk about for the rest of
[04:03.720 --> 04:08.100]  this discussion. We're going to talk about Agile staffing. We're going to talk about the eight
[04:08.100 --> 04:13.440]  simple questions. We're going to talk about the security champions program, and then go into
[04:13.440 --> 04:19.320]  developer training, which is my favorite part of my job. Security in the software development life
[04:19.320 --> 04:24.540]  cycle. And then we're going to hit the hat trick of securing your code. Static code analysis,
[04:24.540 --> 04:29.480]  dynamic code analysis, and open source analysis. And then finally, we're going to wrap things up
[04:29.480 --> 04:34.680]  by automating the heck out of it. So let's start with Agile staffing.
[04:35.660 --> 04:40.900]  Sherman set the way back machine from when I started with Agile. One information security
[04:40.900 --> 04:48.620]  person was assigned to one Agile team. That security person would be integrated into that team
[04:48.620 --> 04:57.520]  and attend every ceremony, every meeting, everything, every day. That didn't work.
[04:57.520 --> 05:01.600]  There are a couple of reasons I think that that didn't work. First of all, Agile teams tend to
[05:01.600 --> 05:09.560]  multiply. You can go from a proof of concept with one team to having close to 30 teams in less than
[05:09.560 --> 05:17.540]  two years. Most organizations do not have the information security bandwidth to support that
[05:17.540 --> 05:24.160]  type of staffing model. In addition, the scrum masters tend to treat every member on the team
[05:24.160 --> 05:28.320]  as though they can perform all tasks. And when you're dealing with a team of all developers,
[05:28.320 --> 05:34.420]  that makes perfect sense. This model tends to fall apart though when you put information security,
[05:34.420 --> 05:43.540]  QA, middleware, any other infrastructure or shared services team as a part of the Agile team.
[05:43.560 --> 05:52.300]  What happens is in the Agile process, their hours count against your available development hours.
[05:52.540 --> 06:00.160]  But the infrastructure people, the information security people do not have the skills or the
[06:00.160 --> 06:05.620]  capabilities to do some of the development. So you kind of see that there's a mismatch. You're
[06:05.620 --> 06:09.880]  given hours, but you're not allowed to use them. So it caused a lot of problems for some of our
[06:09.880 --> 06:15.260]  scrum masters early on. Again, we realized this was not the way to do it. So I ended up coming up
[06:15.260 --> 06:20.620]  with a two-tiered approach for trying to integrate information security into the Agile team.
[06:21.180 --> 06:27.740]  First, at a high level, at the enterprise level, we will have an architect, an information security
[06:27.740 --> 06:34.140]  architect, and they will inject into those high-level projects and basically set up guardrails
[06:34.140 --> 06:42.840]  and guidelines. So as projects are being devised at the visionary level, they can jump in and try
[06:42.840 --> 06:49.080]  to help guide them into making more secure decisions at that high level. Now when you get
[06:49.080 --> 06:53.860]  down into the weeds though, each Agile team does need to have somebody they can rely upon
[06:53.860 --> 07:01.680]  for information security. And so what we ended up doing is we created named resources within
[07:01.680 --> 07:08.260]  the information security team that are assigned to Agile teams. Now it's not like the previous
[07:08.260 --> 07:12.960]  model. You're not assigned, you're not sitting with them, you're not assigned to go to every meeting.
[07:12.960 --> 07:19.720]  We're going to talk about how we scaled that back, but we ended up finding a way where based on
[07:19.720 --> 07:26.020]  where information security can add the most value, we attend those ceremonies and only those
[07:26.020 --> 07:32.340]  ceremonies. So at the end of the day, each information security representative can support
[07:32.340 --> 07:37.680]  between six and eight different Agile teams. That is a model that is certainly sustainable.
[07:38.460 --> 07:45.120]  So what is a little different? Well first of all, we make sure that every Agile team has a named
[07:45.120 --> 07:50.200]  resource that is responsible for information security for them. So this gives them a named
[07:50.200 --> 07:55.360]  contact. If they ever need something, they know exactly who to call and where to start that
[07:56.600 --> 08:04.760]  quest. Secondly, we attend, or the Agile reps attend, the backlog refinement sessions. These
[08:04.760 --> 08:10.080]  are meetings that happen once every two weeks before the next sprint, and it defines what
[08:10.080 --> 08:15.100]  they're going to work on in that next sprint. It's a perfect opportunity to jump in and say,
[08:15.760 --> 08:22.300]  you're planning this work, we need to think about blah blah blah from an information security
[08:22.300 --> 08:29.460]  standpoint. We can get in just ahead of when development starts. It's a perfect time to bring
[08:29.460 --> 08:34.920]  up security issues, make recommendations, and answer questions. And then finally, we attend
[08:34.920 --> 08:40.740]  the quarterly planning increment meetings where the vision is cast for the next quarter, and we
[08:40.740 --> 08:46.320]  can make sure that we're on their radar for things that we think we need to be a part of.
[08:47.540 --> 08:53.340]  So this is kind of a high-level diagram. The purple arrow at the top is the information security
[08:53.340 --> 09:00.040]  architect. That particular person, as you can see from the diagram, sets guardrails for enterprise
[09:00.040 --> 09:05.880]  planning projects. All of the red arrows are the different places where our individual security
[09:05.880 --> 09:12.220]  representatives can or may inject into the Agile teams. We don't inject in all those places,
[09:12.220 --> 09:16.360]  but there are places where we can, and we give them that option to call us if they're needed.
[09:18.280 --> 09:22.480]  So let's move on to what I like to call the eight simple questions.
[09:22.600 --> 09:32.200]  This particular concept came up as a result of assigning our Agile reps to their teams.
[09:32.800 --> 09:38.920]  We found that the leadership within the Agile teams really wasn't sure when to call us. And
[09:38.920 --> 09:44.740]  since we weren't sitting there in every meeting, there was a possibility that something could come
[09:44.740 --> 09:51.560]  up that might fall off the radar, that we might not get contacted for, that they may not even think
[09:51.560 --> 09:57.420]  is important. So we decided that we need to give clear guidelines to the scrum masters and product
[09:57.420 --> 10:04.200]  owners as to when they should include information security. That's how I created the eight simple
[10:04.200 --> 10:10.920]  questions. So basically these eight questions are a guideline for that leadership team, and if they
[10:10.920 --> 10:16.620]  answer yes to any of them, they need to make sure to include their information security representative
[10:16.620 --> 10:23.720]  in those discussions. These are the questions as I currently have them. We're going to talk about
[10:23.720 --> 10:28.920]  that in just a bit because they change over time. These are not one and done kind of things.
[10:29.200 --> 10:34.060]  Am I dealing with any sensitive information? Now information that is sensitive, it's going to mean
[10:34.060 --> 10:37.860]  something different for each one of our organizations. If you're working in healthcare,
[10:37.860 --> 10:42.740]  if you're working with banking information, if you're working with retail with PCI information,
[10:43.080 --> 10:47.340]  you will be able to define what sensitive information means for your organization.
[10:47.560 --> 10:53.200]  Am I sending or receiving data outside of the company? Is there a new vendor involved?
[10:53.200 --> 10:58.820]  Am I going to introduce new software systems or libraries, even open source? And remember open
[10:58.820 --> 11:04.040]  source because we're going to talk about that. It's a unique challenge. Do I need access to
[11:04.040 --> 11:09.900]  secrets such as passwords, keys, or certificates? Will I need the ability to use single sign-on?
[11:09.900 --> 11:16.540]  Am I going to leverage the cloud? Will I need updated access roles? So these eight questions
[11:16.540 --> 11:22.380]  are what we are currently using as a guideline for the interaction between the Agile team
[11:22.380 --> 11:27.740]  and their information security representative. As I mentioned, the exact questions are going to
[11:27.740 --> 11:33.380]  differ on your organization. The questions themselves will actually change over time.
[11:33.380 --> 11:38.520]  I started I think with seven questions and then we had several issues come up with access roles
[11:38.520 --> 11:44.540]  and who to ask and blah blah blah. So we added a question for that too. So don't be afraid to
[11:44.540 --> 11:51.260]  take a stab at it and massage it and change it. That's the thing about Agile and DevOps,
[11:51.260 --> 11:55.500]  it's changing all the time and we as information security people need to be able to change our
[11:55.500 --> 12:03.600]  stuff as well. So the basic idea is this. Create a list of questions, usually 10 or less,
[12:03.600 --> 12:07.100]  something that's going to fit easily on a PowerPoint slide or a single piece of paper
[12:07.100 --> 12:12.580]  so they can take it to meetings with them. Give that to the Agile team leadership and they have
[12:12.720 --> 12:17.660]  a guideline. They now have a concrete way of knowing when they need to work with information
[12:17.660 --> 12:26.180]  security. So we found another advantage to this approach as we onboarded additional information
[12:26.180 --> 12:31.440]  security people to become representatives for the Agile teams. A lot of these folks were pretty new
[12:31.440 --> 12:35.740]  to the organization and they didn't really know when they needed to inject themselves into their
[12:35.740 --> 12:40.400]  teams. They didn't know what they needed to listen for in the backlog refinement sessions.
[12:40.400 --> 12:47.960]  So we used these 8 simple questions on our own people. It gave them guidelines and hints of what
[12:47.960 --> 12:53.500]  to listen for when they're working with their Agile teams. So this kind of became a win-win
[12:53.500 --> 13:01.420]  on both sides of the coin. So I think this was a great tool that we've been able to use.
[13:02.840 --> 13:07.920]  Another tool that I honestly have to thank Jim Manico for. Jim, if you're out there,
[13:07.920 --> 13:13.700]  thank you for this. This was a recommendation he made to me after one of our training sessions.
[13:13.760 --> 13:19.220]  And that is the Security Champions Program. A Security Champions Program is a team of
[13:19.220 --> 13:26.440]  application developers. We get one from each Agile team. And these developers have expressed
[13:26.440 --> 13:31.460]  an interest in information security, or secure coding, or just security in general.
[13:31.460 --> 13:40.560]  This is a voluntary program. The members of the team volunteer to come to the Security
[13:40.560 --> 13:46.740]  Champions Program. And we typically meet quarterly. I provide them lunch when we're in the office,
[13:46.740 --> 13:51.400]  thanks 2020. I look forward to getting back to the office and having lunch with these guys and
[13:51.400 --> 13:57.560]  gals. They're a great team. So the team has this following charter. We assist the Agile teams in
[13:57.560 --> 14:04.660]  terms of application security. We serve as a communication conduit between the Agile teams
[14:04.660 --> 14:10.440]  and information security. And then we meet quarterly to share experiences, we ask questions,
[14:10.440 --> 14:17.040]  we share knowledge. It has had some incredible benefits. First of all, security has a
[14:17.040 --> 14:22.480]  relationship with at least one person on every Agile team. I'm not saying we have them all.
[14:22.480 --> 14:28.460]  What I am saying is, sometimes we hear about things long before it percolates through the
[14:28.460 --> 14:33.320]  management chain. I might get a phone call about, hey, we're thinking about using this open source
[14:33.320 --> 14:38.740]  library. Is that cool? We take a quick look at it. We approve it. By the time the formal approver
[14:38.740 --> 14:44.700]  gets into my hands, we've already done it and they're coding. So it gives us a great communication
[14:44.700 --> 14:52.980]  conduit. And that helps me make things faster. I may have not mentioned this before. Hopefully,
[14:52.980 --> 14:58.220]  I have. Relationship is key. You're going to hear about relationship throughout this
[14:58.220 --> 15:05.560]  entire discussion. We build relationships with those Agile teams. It starts with having an
[15:05.560 --> 15:11.640]  assigned rep. It moves into a different relationship level when we have a security
[15:12.320 --> 15:19.440]  security champions team. Secure coding training takes it to yet another level. And you'll see why
[15:19.920 --> 15:24.060]  in just a few minutes as we talk about the developer training. But it's about building
[15:24.060 --> 15:29.500]  relationships. It's about having fun while we're doing the right thing for our organizations.
[15:30.000 --> 15:34.940]  So, so far, the team has done a lot of really cool things. We've reviewed vendors and products,
[15:34.940 --> 15:39.380]  both on the development side and on the information security side. We've selected
[15:39.880 --> 15:46.340]  our developer training programs for the last two years based on the input from the security
[15:46.340 --> 15:52.980]  champions team members. We've done a bunch of POCs together. We've cross consulted. Now,
[15:52.980 --> 15:59.960]  this was kind of an unexpected benefit is where if I had a team that was very well versed in one
[15:59.960 --> 16:06.620]  technology and they mentioned that in a security champions lunch, another team might reach out
[16:06.620 --> 16:12.900]  and say, hey, we're about to do that. Can we sit down and talk? So, it actually starts to
[16:12.900 --> 16:18.080]  cross pollinate some security knowledge and some technical knowledge between different application
[16:18.080 --> 16:23.880]  teams. And then finally, it served as a candidate group for application security openings within the
[16:23.880 --> 16:32.060]  InfoSec team. The software development life cycle. So, as we started to roll out security, a lot of
[16:32.060 --> 16:40.620]  people have asked us, how does this fit into our SDLC? So, I ended up creating a plan that maps
[16:40.620 --> 16:46.060]  information security phases to the software development life cycle as it currently exists.
[16:46.060 --> 16:52.640]  It is not a perfect fit, but I think it is a good representation of how information security can
[16:52.640 --> 17:00.620]  bring value to each step of the SDLC. So, this is what I like to call my dark side of the moon
[17:00.620 --> 17:06.700]  diagram. Those of you old enough to remember, probably recognize the album cover. So, if you
[17:06.700 --> 17:12.200]  take a look on the right, we have the software development life cycle from concept into design,
[17:12.200 --> 17:19.360]  up into coding testing and QA, QA and deployment, and then post-deployment at the very top.
[17:19.360 --> 17:25.240]  Now, if you look on the left side, you'll see that it matches up pretty closely with security
[17:25.240 --> 17:31.660]  knowledge, security design, secure coding techniques, security testing, and then security
[17:31.660 --> 17:39.000]  response. The each individual layer of the pyramid, I think, are deliverables that security
[17:39.000 --> 17:48.960]  can bring that help take that concept on the left and apply it to the SDLC phase on the right. And
[17:48.960 --> 17:53.920]  we're going to walk through those real quick. Training is the base of the pyramid. I think
[17:53.920 --> 17:59.860]  training is foundational to everything above it. If you don't have a solid foundation of
[17:59.860 --> 18:07.860]  training for your application developers, everything else is at risk. So, this particular
[18:07.860 --> 18:12.920]  approach, we're going to talk about in depth, again, because it's my favorite part of what I do.
[18:12.920 --> 18:18.020]  It includes things like lunch and learns, outside speakers, conferences like you're attending right
[18:18.020 --> 18:25.100]  now, video training, and things like that. The next level up is threat modeling. Threat modeling,
[18:25.100 --> 18:29.980]  the application team, and InfoSec, we sit together, we work together to examine the application
[18:29.980 --> 18:35.900]  design, the system flows, whatever they can bring to us that they have available in that design
[18:35.900 --> 18:43.360]  phase. We look, we try to work together and identify potential vulnerable vectors and try
[18:43.360 --> 18:49.980]  to figure out a way that we can mitigate them. Now, threat modeling, there are books upon books
[18:49.980 --> 18:54.580]  written about threat modeling. And there are people in my organization on my InfoSec team
[18:54.580 --> 19:00.720]  that are much better at it than I am. And they can actually create incredible documents and spend
[19:00.720 --> 19:05.300]  hours and days and days and days doing this. I think you need to start out small. I think it's
[19:05.300 --> 19:10.800]  about baby steps at first. Start with a discussion. This can be a one or two hour discussion where you
[19:10.800 --> 19:15.740]  have those deliverables from the design phase, and you whiteboard them. And you come out of there
[19:15.740 --> 19:23.600]  with to-dos, or questions, or things to research. It doesn't have to be this big, heavyweight,
[19:24.640 --> 19:30.400]  overbearing approach to threat modeling at first. It needs to be a conversation. That gets back to
[19:30.400 --> 19:36.020]  relationships. You need to have those relationships in place so you can have those conversations.
[19:36.560 --> 19:41.060]  The third level up is static code analysis. We're going to talk about this
[19:41.780 --> 19:47.800]  in much more detail later. But most tools are designed to identify potential vulnerabilities
[19:47.800 --> 19:52.860]  within the code. Now, static code analysis actually looks at the code and looks for
[19:52.860 --> 19:57.220]  patterns within the code without running it. Now, we're going to see a little bit later
[19:57.220 --> 20:05.300]  why that's important. I think I went the wrong way. There we go. Dynamic code analysis is the
[20:05.300 --> 20:10.300]  next layer up. And that is very similar to the previous layer, which was static code analysis.
[20:10.300 --> 20:15.320]  But instead of just looking at the code, the code is actually executed and tested for
[20:15.320 --> 20:24.360]  vulnerabilities. So, like I said, these two pieces are very, very similar. One is looking
[20:24.360 --> 20:30.000]  at patterns with the code. The other is actually exercising the code and looking for issues.
[20:30.000 --> 20:34.260]  The next is pin testing. Pin testing can be performed by your internal team if you have
[20:34.260 --> 20:40.560]  the skills and resources, or you can use an external firm. This is usually a greater
[20:41.240 --> 20:49.100]  scope than dynamic testing. It will actually look at things not just in your application,
[20:49.100 --> 20:55.320]  but system-wide for vulnerabilities and attack vectors. And usually there's a person involved.
[20:55.320 --> 20:59.020]  It's not just an automated exercise. We're going to see a little bit more when we talk about
[20:59.020 --> 21:03.800]  dynamic testing, the difference between that and pin testing. And then finally,
[21:03.800 --> 21:09.180]  when the code is in deployment, you need to have a vulnerability management process
[21:09.180 --> 21:18.440]  that you can classify, assign, and track identified issues. Right now, InfoSec meets
[21:18.440 --> 21:24.600]  regularly with a representative from each application team to review these and to make
[21:24.600 --> 21:29.540]  sure that they're working and they're on track. Again, those relationships are absolutely key
[21:29.540 --> 21:37.780]  to making this work. So now we are getting to what I consider my absolute favorite part of my job,
[21:37.780 --> 21:43.300]  and that is developer training. If you remember back to that pyramid we just looked at,
[21:43.300 --> 21:51.940]  this is the foundation for everything that InfoSec does with AppDev. When I came in
[21:52.780 --> 21:59.260]  to Information Security about seven years ago, I started a developer training program because I
[21:59.260 --> 22:05.520]  realized that the developers really are your last best hope for defense when it comes to
[22:05.520 --> 22:11.560]  application security. If your application is written correctly, then everything you throw
[22:11.560 --> 22:17.960]  in front of it, application firewalls, everything like that, that's gravy. That's nice, but you've
[22:17.960 --> 22:24.720]  got a really tight application, and that's where real security takes place. The problem is,
[22:24.720 --> 22:29.800]  I was a developer for roughly 25 years before coming to Information Security,
[22:30.440 --> 22:35.540]  and nobody taught this stuff. This is stuff you learned on your own if you had an interest in it,
[22:35.540 --> 22:43.980]  and it wasn't universally available to all developers. So I started small. I started with
[22:43.980 --> 22:51.860]  Lunch and Learns. It was really something that, hey, I think this is important, let's try it,
[22:51.860 --> 22:56.460]  let's see if it works. And it really did. We would end up having Lunch and Learns about every other
[22:56.460 --> 23:01.840]  month. I would create a class on a specific topic. My first one was cross-site scripting,
[23:01.980 --> 23:08.320]  a lot of fun. It's neat when you can show people cross-site scripting examples, even if you just
[23:08.320 --> 23:13.800]  run WebGoat or something like that. They get excited. They kind of dig it, and they start
[23:13.800 --> 23:19.100]  trying it on their own stuff. The content usually lasts about 40 or 45 minutes, a lot like this
[23:19.100 --> 23:23.680]  session, and then we leave 10 to 15 minutes for open discussion and questions. Now the key here
[23:23.680 --> 23:31.160]  is, make it fun. It has to be fun. I do things like, I'll go out and I'll buy cookies or some
[23:31.160 --> 23:36.480]  kind of treat, so we can all sit around, eat something together. There's usually some kind
[23:36.480 --> 23:40.520]  of door prize or door prizes, and they don't have to be expensive. You can just go down to
[23:40.520 --> 23:46.340]  five below and get a couple of wireless Bluetooth speakers. And people dig it. It's a lot of fun.
[23:46.700 --> 23:52.600]  Again, making it fun is absolutely key to a successful Lunch and Learn program.
[23:53.180 --> 23:58.440]  So once we got that up and running, I decided we needed to take this up a notch.
[23:58.920 --> 24:05.680]  There is a level of training that I couldn't provide that I think our developers really,
[24:06.280 --> 24:11.860]  and in order to do that, I knew that I had to reach out to industry experts.
[24:12.940 --> 24:16.860]  And what we ended up doing is, we were able to get commitment from management, both on
[24:16.860 --> 24:24.040]  the IT development side and on InfoSec side, that we could do one or two days where most,
[24:24.040 --> 24:28.480]  if not all, of the developers will attend a training event. Now here's a hint. If you are
[24:28.480 --> 24:35.280]  going to try this and you're doing Agile, put it in your Innovation Sprint. If you put it in the
[24:35.280 --> 24:40.960]  Innovation Sprint, which is usually the last sprint of the quarter, and the developers have
[24:41.100 --> 24:46.380]  a little more leeway of what they work on during the Innovation Sprint, you'll very likely get
[24:46.380 --> 24:52.380]  more participation than if you step on a middle of a sprint where there's actual deliverables due.
[24:52.380 --> 24:59.020]  So keep that in mind. I was really, really lucky. My very first interaction with this program,
[24:59.020 --> 25:05.100]  I was able to secure Jim Manico, and he came in and made an incredible impression on our
[25:05.100 --> 25:09.600]  development teams. Jim's been back, I think, three times now. Always a pleasure.
[25:10.420 --> 25:17.760]  Rich Mogul's been by twice. We've had a recent one with Ron Paris. Dr. Philippe Des Riques,
[25:17.760 --> 25:29.000]  phenomenal instructor. I just took his OAuth class, so if you're looking for OAuth,
[25:29.000 --> 25:34.520]  looking forward to that. The reason I actually want to call these gentlemen out by name is I
[25:34.520 --> 25:38.520]  think it's important to be grateful. I am very grateful for everything they've done for me
[25:39.080 --> 25:45.580]  and my development teams. And it's important to be grateful to those who help you along the way
[25:45.580 --> 25:49.940]  on this journey, because none of us make it alone. So gentlemen, thank you very much. If you
[25:49.940 --> 25:57.440]  see this or hear it later, your training, your mentorship has meant a lot to me and my
[25:57.440 --> 26:04.080]  development teams. So one cool aside is, I mentioned this earlier, the Security Champions
[26:04.080 --> 26:08.920]  team has been involved in the selection of the instructor for the last two years.
[26:08.920 --> 26:15.540]  This gives your application development teams a voice in the training. And we've seen that
[26:15.540 --> 26:24.100]  participation has actually increased. We had great participation before then, but when the developers
[26:24.100 --> 26:30.700]  actually had a voice in picking who's coming in and what they're going to talk about,
[26:31.300 --> 26:35.980]  we're finding that it's much more applicable to what's on their roadmap. It's much more
[26:35.980 --> 26:42.500]  applicable to the skills they're hoping to get. And participation goes beyond what I could have
[26:42.500 --> 26:47.920]  ever hoped for. So again, we're starting to see some of these initiatives are starting to
[26:48.480 --> 26:53.940]  weave together and starting to build kind of a program. It's kind of cool.
[26:54.420 --> 27:01.360]  So this next one, I really wish I was doing this right now. Take a developer to DEF CON.
[27:01.360 --> 27:07.480]  We had management agree to provide scholarships to developers that were selected by the information
[27:07.480 --> 27:12.860]  security team. So we would give them the money to buy their badges so they wouldn't have to go and
[27:12.860 --> 27:21.100]  get money out of their own bank account. So each year I've been able to bring up to three developers
[27:21.100 --> 27:30.380]  to DEF CON. That's been awesome. It has been just a phenomenal experience. They've gained a ton of
[27:30.380 --> 27:37.780]  knowledge and a ton of awareness by DEF CON. One of the things that I found really interesting was
[27:37.780 --> 27:42.700]  I really advised them not to restrict their sessions to things that they think will apply
[27:42.700 --> 27:48.920]  to their job. I did that my very first DEF CON and I wasn't really sure I was going to come back,
[27:48.920 --> 27:55.400]  to be honest with you. The next year, one of my co-workers strongly encouraged me to go. Thank you,
[27:55.400 --> 28:03.400]  Barry. And encouraged me to go and find a few fun things that just had nothing to do with my
[28:03.400 --> 28:09.840]  immediate job. The interesting thing is they ended up having more to do with the things that were
[28:09.840 --> 28:17.280]  coming down the road in my job than I ever could have imagined. So I fully encourage each one of
[28:17.280 --> 28:22.380]  these developers that I bring to DEF CON to experience the full DEF CON experience. Go to
[28:22.380 --> 28:29.620]  all the sessions. Try some of the capture the flags. Try whatever you want. Have a good time
[28:29.620 --> 28:37.640]  because it's really important for them to be immersed in the DEF CON experience.
[28:37.720 --> 28:42.920]  So like I said before, they come back with a changed perspective. It's a lot like that scene
[28:42.920 --> 28:48.800]  in The Matrix where that thing crawls out of Neo and he goes, my god, that thing is real?
[28:49.100 --> 28:55.520]  They understand now that the security is not just something that we're teaching and preaching about
[28:55.520 --> 29:03.660]  within InfoSec, that this stuff has real world application. And it's that realization that makes
[29:03.660 --> 29:11.260]  them a better developer, a more secure developer going forward. We've seen a trend where a lot of
[29:11.260 --> 29:16.620]  the folks that we've taken, actually, I think all but one of the developers that we've taken
[29:16.620 --> 29:23.300]  to DEF CON have eventually become security champions members. And then more recently,
[29:23.300 --> 29:29.360]  in the last couple of months, one of those developers, I think he was the first one I
[29:29.360 --> 29:36.160]  took to DEF CON with us, ended up joining the Information Security team on a full-time manner.
[29:36.200 --> 29:41.540]  And Shane, if you're listening, welcome. I'm really glad that you're part of the team.
[29:41.960 --> 29:46.220]  So there's a couple of other fun activities that we do in developer training. Again,
[29:46.220 --> 29:51.920]  got to keep it fun. Everything has to be fun. You learn better when you're having a good time. I
[29:51.920 --> 29:59.300]  look at DEF CON. We're all having a great time and we're learning stuff. So a couple of DEF CONs
[29:59.300 --> 30:04.640]  ago, for reasons of work, I couldn't make it. It was the only DEF CON since coming over to
[30:04.640 --> 30:11.820]  Information Security that I was not able to go to. And I must admit I was bummed. So what I decided
[30:11.820 --> 30:20.440]  to do was I was going to have my own little DEF CON. And we decided that, okay, let's just take
[30:20.440 --> 30:26.520]  people along for the ride. So we identified five videos from previous DEF CONs and we had a lunch
[30:26.520 --> 30:32.800]  and learn for five days that week where we watched it. We talked about it. We had drawings for DEF
[30:32.800 --> 30:39.440]  CON related swag or items or giveaways. We just kind of had a good time. It was kind of built up
[30:39.440 --> 30:46.000]  as, you know, hey, take a break. Come bring your lunch. Let's hang out. Watch something from DEF
[30:46.000 --> 30:52.700]  CON. Talk about it. You might win a t-shirt. You might win some stickers, whatever. And people
[30:52.700 --> 30:57.720]  really enjoyed it. It was a lot of fun. As a matter of fact, I'm probably going to do one this fall
[30:57.720 --> 31:03.140]  for the folks that didn't get to enjoy what you're doing right now, what you're enjoying right now.
[31:03.400 --> 31:08.260]  And then finally, I have an annual thank you lunch and learn. This was a tradition I started
[31:08.260 --> 31:13.660]  the first year we did lunch and learns. It's always the last lunch and learn of the year.
[31:13.660 --> 31:18.240]  And I typically will go out and I'll purchase quite a few giveaways. They don't have to be
[31:18.240 --> 31:25.080]  expensive. We do things for, you know, like say the Bluetooth speakers, boxes of candy. We've done
[31:25.080 --> 31:33.420]  some Christmas mug sets, you know, just weird things, but things that are kind of fun.
[31:33.560 --> 31:40.420]  And we do a shorter lunch and learn followed by a session where we just eat some good food
[31:40.420 --> 31:46.140]  and we have some drawings. And I have an opportunity to thank my development community
[31:46.140 --> 31:51.020]  for the support of the lunch and learn program and for the support they've given to information
[31:51.020 --> 31:58.240]  security. I think it's important to say thank you. And this has been just an incredibly popular
[31:58.240 --> 32:05.120]  lunch and learn every year. It's a lot of fun and I highly recommend it. So here are the keys to a
[32:05.120 --> 32:14.220]  successful training program. Make it applicable and give the developers a voice in the training.
[32:14.300 --> 32:19.360]  And usually if you give them a voice in the training, it will be applicable just because
[32:19.360 --> 32:24.780]  they're going to tell you what they need. I think I've mentioned this 1427 times, but
[32:24.780 --> 32:33.160]  make it fun. It's got to be fun. Share how much you care about this stuff. If you're engaged,
[32:33.160 --> 32:37.740]  you're passionate about it, people are going to feel that and they're going to gravitate to you
[32:37.740 --> 32:43.840]  and gravitate to that topic. And then finally build relationships. Listen, reward, and always
[32:43.840 --> 32:52.200]  express gratitude along the way in this journey. So we're getting into the next phase of the
[32:52.200 --> 32:57.640]  toolkit and this is what I like to call the analysis hat trick. Three types of analysis
[32:57.640 --> 33:03.980]  that if we implement all three, we're going to have a much more secure code base in production.
[33:03.980 --> 33:09.400]  And the first is static code analysis. You might remember that from our SDLC pyramid.
[33:10.220 --> 33:17.320]  So static code analysis scans the code without running it. It looks for known patterns of
[33:17.320 --> 33:25.180]  vulnerabilities. You can usually select what you scan for. Some will allow you to scan for PCI,
[33:25.180 --> 33:32.260]  I almost always scan for OWASP top 10. If you're not familiar with OWASP or OWASP top 10, I
[33:32.260 --> 33:37.380]  strongly recommend that you check it out. Some of the deliverables from that organization are
[33:37.380 --> 33:44.060]  absolutely stellar. I've been a member for I think seven years now, just after joining the InfoSec
[33:44.060 --> 33:51.940]  team. And I've leveraged a ton of their stuff. So great organization, hats off to you if you're out
[33:51.940 --> 33:57.220]  there. In an ideal world, the static code analysis tools would only report the confirmed
[33:57.220 --> 34:03.400]  vulnerabilities. But even if you tune the daylights out of this thing, you're still going to get false
[34:03.400 --> 34:08.580]  positives. And that becomes a problem. Because originally, when we first set up this process,
[34:08.580 --> 34:14.260]  I was the only one that could review the false positives and whitelist them or waive them,
[34:14.260 --> 34:20.140]  as it were. That slowed things down immensely, because that was not my full-time job. That was
[34:20.140 --> 34:26.180]  something else on top of my full-time job, and possibly another full-time job some days,
[34:26.180 --> 34:33.260]  it felt like. So that being said, I decided to leverage the security champions. These were folks
[34:33.260 --> 34:39.280]  that had some basic knowledge of security. They knew their applications better than I ever would.
[34:39.280 --> 34:47.080]  And they had a passion for making secure code. Seemed like a perfect fit. So what I ended up
[34:47.080 --> 34:54.840]  giving several members of the security champions team the ability to review and waive false
[34:54.840 --> 35:00.920]  positives. And that has worked out great. The end result was we've had a lot more
[35:01.780 --> 35:08.460]  throughput. We've been able to address false positives much more quickly. And the adoption
[35:08.460 --> 35:13.720]  of the static code analysis tools have actually increased as a result of not having that delay.
[35:14.260 --> 35:20.580]  So I also see static code analysis as an extension of developer training. A lot of these
[35:20.580 --> 35:29.660]  tools have an IDE component. So for example, I use the SpringToolSweep Eclipse. And you can
[35:30.540 --> 35:38.980]  use the plugin and execute your scan from within your developer tool. And it will take you straight
[35:38.980 --> 35:45.340]  to the line that's got the problem. And in some of these tools, you can even have recommendations
[35:45.340 --> 35:51.320]  pop up in the bottom of one of the windows that says, hey, you might try blank, blank, and blank
[35:51.320 --> 35:57.360]  to address this vulnerability. If developers are doing this while they're developing,
[35:57.360 --> 36:05.280]  not waiting for the build process, it becomes a teaching tool. Because as a developer uses this,
[36:05.280 --> 36:11.000]  he or she is going to identify and learn that, oh, I need to check my input fields to make sure
[36:11.000 --> 36:15.720]  they're the right type and right size. Okay. You do that enough times, it becomes second nature.
[36:15.720 --> 36:23.800]  So this becomes not only a way to secure your code, but a teaching opportunity as well.
[36:24.280 --> 36:31.800]  So studies have shown that static code analysis can identify up to 80% of vulnerabilities.
[36:31.800 --> 36:36.600]  That's pretty awesome. If I could tell you that you can do one thing and remove 80% of
[36:36.600 --> 36:40.840]  vulnerabilities, I think that'd be worth doing. So this might be something you might want to
[36:40.840 --> 36:46.580]  take a look at. Here's some example products. There's three products along the top that are
[36:46.580 --> 36:54.500]  commercial. OWASP has an entire list of code analysis tools at that link right there. Again,
[36:54.500 --> 36:59.900]  not meant as recommendations, but these were tools that I've come across in my journeys,
[36:59.900 --> 37:06.140]  and they might be worth at least acting as a starting point for your own investigation.
[37:07.200 --> 37:13.760]  So the second goal in our hat trick, the second piece to the tripod, is dynamic code analysis.
[37:13.760 --> 37:20.500]  This was also in the SDLC pyramid. So unlike static code analysis, dynamic code analysis
[37:20.500 --> 37:26.860]  actually executes the code. Now that has a prerequisite that the code has to be compiled
[37:26.860 --> 37:32.540]  and free of errors and can run before you can actually perform the dynamic code analysis.
[37:32.720 --> 37:36.460]  Dynamic code analysis, I mentioned this before, is different than pen testing,
[37:36.460 --> 37:41.180]  whereas dynamic code analysis looks for the flaws in the code that can be exploited.
[37:41.920 --> 37:48.000]  Pen testing looks for ways to exploit the system in general, often at a greater detail,
[37:48.000 --> 37:55.100]  and almost always involving a human attacker. So again, two different things. They're kind of
[37:55.100 --> 38:02.540]  similar, but I really think that dynamic code analysis is really, at best, a subset of a good
[38:02.540 --> 38:11.580]  penetration test. So dynamic code analysis tools can identify up to 85%, but if you combine them
[38:11.580 --> 38:19.300]  with the static code analysis tools on the market, up to 95% of your application's vulnerabilities
[38:19.820 --> 38:27.720]  can be identified early on. 95%, that's pretty darn impressive. So again, I'm going to throw a
[38:27.720 --> 38:34.340]  few products up here for you to take a look at. I've used several of these. Again, not
[38:34.340 --> 38:39.900]  recommendations, but a place for you to start. Give you a word of warning. I think I told you
[38:39.900 --> 38:44.140]  earlier on, I would tell you some things that work and some things that just didn't seem to
[38:44.140 --> 38:51.220]  work quite so well. Some of these tools can be weapons. I'm looking at you, Burp Suite.
[38:52.340 --> 38:58.440]  You need to do things like make sure you set the scope of your testing tool so it doesn't
[38:58.440 --> 39:04.280]  see that, oh, I've got a link to Facebook here. I will jump and follow that link in my crawl and I
[39:04.280 --> 39:12.220]  will start attacking Facebook. Set your scope absolutely to your application only. Run this
[39:12.220 --> 39:19.400]  stuff in test. Don't run it in production. If you find a SQL injection exploit and you start
[39:19.400 --> 39:26.000]  to execute it, it's very, very likely that you're going to corrupt or lose data on the database on
[39:26.000 --> 39:33.240]  the backside of that injection vulnerability. Also, watch out for forms. So let's say you're
[39:33.240 --> 39:39.880]  doing some dynamic testing and it's automated and you're getting ready to essentially attack a web
[39:39.880 --> 39:47.380]  form. Make sure you find out where that web form is going before you start your test. You can easily
[39:47.380 --> 39:55.000]  overrun a service desk by submitting thousands upon thousands of emails from a web mail form.
[39:55.000 --> 40:00.480]  Not saying that I've ever done that, but I know that it has happened. So the third goal in our
[40:00.480 --> 40:09.140]  hat trick is open source analysis. Open source is awesome. We all use open source. Some estimates
[40:09.140 --> 40:17.940]  put the percentage of open source in internally developed applications at 70%. That's 70%.
[40:18.600 --> 40:25.800]  Now the problem with that is, as a result, about 50% of our applications contain one or more open
[40:25.800 --> 40:36.740]  source pieces that has a high security vulnerability. And keeping that out of production is a huge,
[40:36.740 --> 40:42.700]  There are two different approaches I've taken when I've tried to address this problem. The first was
[40:42.980 --> 40:47.440]  a firewall slash proxy solution. There are products out there that you can actually set up that will
[40:47.440 --> 40:56.420]  proxy the main repos out in the wild. And they'll check as you pull down code libraries, they'll
[40:56.420 --> 41:02.680]  check them for license compliance. So you can say, hey, I'm okay with this license, this license,
[41:02.680 --> 41:08.200]  but I don't want that license. So you can set that up ahead of time and it'll check each time
[41:09.020 --> 41:13.540]  a module is pulled down that it is in compliance with your licensing. But more importantly,
[41:13.540 --> 41:19.520]  you can also set a vulnerability threshold. So you could, for example, say, hey, I don't want
[41:19.520 --> 41:26.200]  any critical or high translation. I don't want any CVE 7 or higher coming into my organization's
[41:26.200 --> 41:33.640]  local repository. So the problem with that is a lot of the tools that developers use,
[41:33.640 --> 41:39.440]  particularly in DevOps for automating building of systems, automating testing,
[41:40.220 --> 41:45.140]  have vulnerabilities because they really weren't meant to go live anywhere. They were basically
[41:45.140 --> 41:54.360]  developer aids, developer tools, and your policy is going to quarantine or prevent them from coming
[41:54.360 --> 42:01.340]  into your local repository. The result is you get a lot of phone calls. Hey, this module, I need it,
[42:01.340 --> 42:07.920]  I need it now, and it's quarantined. Well, that requires getting out on the firewall or proxy
[42:08.500 --> 42:14.340]  and looking at what's quarantined, determining how it's going to be used, where it's going to be used,
[42:14.340 --> 42:19.820]  and does that CVE apply in this use case, and then deciding whether you can waive it or not.
[42:19.820 --> 42:30.220]  It is a labor-intensive process. I didn't really like doing it this way at first,
[42:30.220 --> 42:36.920]  so I ended up moving to a build-based solution. Build-based solutions are very similar to what
[42:36.920 --> 42:41.380]  we're going to see in just a few minutes, where you basically add an open source review step to
[42:41.380 --> 42:48.060]  your automated build process. If you find that same vulnerability as it's going to production,
[42:48.060 --> 42:57.180]  let's say it's a CVE of 8, and it's going into production, you fail the build. At that point,
[42:58.600 --> 43:04.880]  the developers have the opportunity to go, okay, I need to get a security waiver, or I need to find
[43:05.140 --> 43:12.820]  a more recent or more updated library that doesn't have that CVE as an outstanding issue.
[43:12.820 --> 43:20.180]  So this ended up reducing the manual intervention. The only problem is you could end up having
[43:20.180 --> 43:27.240]  something on your internal network with a CVE of 7 or higher that's a developer tool. You've got
[43:27.240 --> 43:32.800]  to be very careful when you're whitelisting or removing items from quarantine to understand how
[43:32.800 --> 43:39.700]  it's going to be used, and if the CVE itself can be exploited in that manner. So here's some
[43:39.700 --> 43:46.860]  products once again. OWASP, I've mentioned them several times. Great organization. Again,
[43:46.860 --> 43:50.600]  these are not recommendations. These are simply places for you to start if you haven't
[43:50.600 --> 43:57.980]  looked at these already. And finally, let's talk about automation. So those last three steps,
[43:57.980 --> 44:05.280]  they seem like a lot. And originally we were doing them manually, and it became unwieldy
[44:05.280 --> 44:14.880]  very quickly. It just took a lot of time. It took a lot of effort. So what we were able to
[44:14.880 --> 44:20.420]  do was automate most of this into the build pipeline. So if you have a continuous integration
[44:20.420 --> 44:26.080]  tool, something like Jenkins, you can add steps that execute static code analysis, dynamic code
[44:26.080 --> 44:32.520]  analysis, and open source analysis with every build. So this gives you all of that protection
[44:32.520 --> 44:39.780]  over 95% from the studies with every single build that you do. Now, there's some things that are
[44:39.780 --> 44:46.900]  just too stinking big. So if you're still dealing with legacy applications, big old monolithic jars
[44:46.900 --> 44:54.660]  and ears, if you've got something that's somewhere in the hundreds of thousands of lines of code,
[44:54.660 --> 45:00.760]  sometimes close to a million lines of code, this is not going to go well for you because
[45:00.760 --> 45:07.200]  the dynamic scan, the static scan, those pieces are going to take a long, long time.
[45:07.200 --> 45:15.900]  I tested one application for a friend of mine just to try it. It was a legacy application,
[45:15.900 --> 45:23.380]  and the scan for just the static code analysis piece took over 12 hours. You can't stop a build
[45:23.380 --> 45:29.500]  for 12 hours and expect the developers not to be concerned. So I don't recommend this
[45:29.500 --> 45:38.840]  for large applications. If you do, you might do it as an asynchronous build step. So basically,
[45:38.840 --> 45:43.240]  it says, hey, I've got to go do static code analysis. I'm going to launch it and I'm going
[45:43.240 --> 45:49.940]  to continue with my build. You lose the advantage of being able to block at that point, but you
[45:49.940 --> 45:56.180]  still have an automated scan going along. You'll need to have a process in place to alert the
[45:56.180 --> 46:01.520]  development team as well as information security of the results of that scan, and then decide how
[46:01.520 --> 46:07.820]  you want to react. But you lose the ability to block. But fortunately, the industry is really
[46:07.820 --> 46:15.380]  moving towards smaller builds, APIs, microservices, things like that. And inline scans are very,
[46:15.380 --> 46:20.340]  very possible. They're very fast. These tools are tuned for this kind of work,
[46:20.340 --> 46:28.280]  and they run incredibly, incredibly fast. Just aside, some of the tools
[46:29.740 --> 46:36.080]  are still working on their automated pipeline plugins. So if you are looking at any of the
[46:36.080 --> 46:41.300]  tools, whether it's the ones I've mentioned or something else, make sure that you ask about
[46:41.300 --> 46:46.700]  the maturity of their continuous integration pipeline plugins.
[46:48.340 --> 46:53.400]  Otherwise, you might find yourself stuck manually doing this for a while until you can find a way
[46:53.400 --> 47:02.060]  to do that. I want to end with a couple of final thoughts. So the keys to success, hopefully you
[47:02.060 --> 47:08.800]  can read this, they are start small. Do one thing. For me, it was starting with a training program
[47:08.800 --> 47:15.320]  that literally led to everything else that you saw in this presentation. And as you saw in the
[47:15.320 --> 47:23.400]  presentation, this stuff tended to feed upon itself. The developer training led to more
[47:23.400 --> 47:29.260]  developer training, which led to a security program. So start small, and it will all come
[47:29.260 --> 47:35.800]  together. Get management buy-in by both Information Security and by AppSec Management.
[47:36.460 --> 47:42.060]  The centerpiece right there, that has to be the key for me. It has to be fun. When this stuff is
[47:42.060 --> 47:47.240]  no longer fun, I'm not going to do it anymore. We've got to keep making it fun, particularly for
[47:47.240 --> 47:52.780]  our developers. Developers are doing a lot of stuff. Adding security on top of everything else
[47:52.780 --> 47:59.980]  they're doing is a pain in the butt. And for the most part, it's not welcome, unless you can make
[47:59.980 --> 48:06.120]  it fun. If you can make it fun, it's going to happen. So just like we just mentioned, automate
[48:06.120 --> 48:10.940]  everything you possibly can. And remember, it's all about those relationships. The relationships
[48:10.940 --> 48:18.320]  with your scrum teams. It's the relationships with your developers. It's the relationships
[48:18.320 --> 48:25.400]  with your vendors and the trainers that you can get to come in. So building a good relationship
[48:26.080 --> 48:34.080]  is key. Don't try to reinvent the wheel. The best innovation is usually stolen from somewhere,
[48:34.080 --> 48:39.640]  or given from somewhere. So if you can take one piece from this discussion and implement it in
[48:39.640 --> 48:47.760]  your organization, I will feel blessed, and I'd love to hear about it. Take it slow. This stuff
[48:47.760 --> 48:55.340]  is new for all of us, and it's always changing. And finally, learn what works best in your
[48:55.340 --> 49:02.540]  environment. So I want to wrap up with thank yous. I want to thank you for spending the last 40,
[49:02.540 --> 49:08.060]  45 minutes, whatever it ended up being, with me and with AppSec Village. I want to thank AppSec
[49:08.060 --> 49:13.260]  Village. I got to volunteer there last year. I ended up spending the entire DEF CON there.
[49:13.880 --> 49:18.860]  After, I think that was my sixth year in information security, I finally found my home,
[49:18.860 --> 49:24.480]  and that is the AppSec Village. So thank you to the leadership of AppSec Village for doing this,
[49:24.480 --> 49:31.600]  and for giving me this opportunity. I want to reiterate my thanks to Jim Manico, and he's kind
[49:31.600 --> 49:38.240]  of been my spirit leader on this journey, and I appreciate all you've done. My colleagues Barry
[49:38.240 --> 49:44.340]  and Shane, my partners in crime, you guys are awesome. And finally, my best friend, my wife,
[49:44.340 --> 49:51.520]  who's put up with me doing this and several other things in information security. She's absolutely
[49:51.520 --> 49:57.880]  the best, and I can't thank her enough. So thank you guys very much. I appreciate you spending this
[49:57.880 --> 50:05.580]  time with me, and I hope you have a really, really excellent rest of your DEF CON. Thanks.