WEBVTT 00:00.520 --> 00:21.300 Our agenda will be the first introduction, we will introduce ourselves and what we do, and after that we will look at the Red Team side of the things, and the DevOps side of the things, and lastly we will meet them together in an example, and after that we will look at how can we maintain our automation codebase. 00:21.740 --> 00:27.780 So we're both Security Engineer Center and member of Blackbox Security and ITCAT. 00:27.780 --> 00:35.300 We're frequently blogging at Trento Tech and we're performing Red Team and Zero-Day Research at our leisure time. 00:35.740 --> 01:02.460 So what we're doing is we're planning a Red Team engagement and with that plan we are designing our infrastructure and iterate over it with our team, and once it's stable we're automating our infrastructure design using Terraform, and after that if the operation will take longer than usual or we will reiterate one of our infrastructure automation scripts, 01:02.460 --> 01:08.200 we will create a pipeline for change management so that we remove human error altogether. 01:09.020 --> 01:13.020 So the Red Team life cycle looks like this. 01:13.020 --> 01:40.940 Usually you're doing external reconnaissance and once you compromise a machine you do the same on internally until you get domain admin and domain dominance, and after that usually the company has some target machine and target file to exfiltrate, and until you exfiltrate that file you continue your loop of reconnaissance and remote code execution. 01:41.900 --> 01:46.220 The operational models for Red Team are like this. 01:46.220 --> 01:48.900 The first one is full scope annotation test. 01:48.900 --> 02:01.440 Although it's a controversial one, you can count it as a Red Team because you have black box scope and you're attempting to gain a full tilt into the target and steal the data. 02:01.660 --> 02:14.100 So it's really kind of targeted attack and although it provides a data point about the state of a security program, it needs a lot of time and resource. 02:14.380 --> 02:18.380 The similar one but the difference is the long-term Red Team operation. 02:18.380 --> 02:40.680 It really needs you to work towards the company like you're an actual attacker for a year or two, and it has advantages for the company that you will map their own network and the key divisions so that they can harden their security posture throughout the time. 02:40.680 --> 02:47.220 And the hardest part for a Red Team is you have to maintain your access over a long period of time. 02:47.780 --> 02:52.640 The other is the classic Red versus Blue wargames. 02:52.640 --> 03:02.640 It will strengthen the Blue Team's muscles against the Red Team activities, techniques, and procedures. 03:02.800 --> 03:16.380 And also Red Team's muscles against the Blue Team's in terms of they will see how the Blue Team handle their attack etc. 03:16.380 --> 03:29.880 So the wargames are usually in a simulated or some scenario-based environment and they are really good to have time to time within the company itself. 03:30.140 --> 03:32.520 And the last one is Adversary Simulation. 03:32.520 --> 03:41.900 In this Adversary Simulation scenarios, they are based on some APT or mock-up scenario based on a realistic timeline. 03:41.900 --> 03:49.600 So you're basically imitating some APT attack or some kind of scenario that the company would like to assess. 03:49.600 --> 03:59.400 So the goal is to exercise that attack without getting identified or discovered by Blue Teams. 03:59.400 --> 04:12.040 So these events are really good for the company because the states and the scenarios are step by step so they can see where they fail or where they are succeeding at every step. 04:12.580 --> 04:18.080 So in the light of our operational models, there is also Operation Security. 04:18.080 --> 04:26.380 It's a term from the military and it means that denying adversarial information from the opposite side. 04:26.380 --> 04:36.380 So as a Red Teamer, you should be able to deny any information that you may have from the Blue Teamers on the other side. 04:37.300 --> 04:44.880 In the wild, some threat actors have operational security mistakes throughout time. 04:44.880 --> 04:48.120 And let's look at the recent errors. 04:48.480 --> 04:51.700 And the first example is just botnets. 04:51.700 --> 05:01.240 They were discovered and doxxed because of not encrypting their C2 servers and the chat sessions. 05:01.320 --> 05:03.120 The other example is Forceful. 05:03.220 --> 05:13.680 He's a Russian botnet developer and he gave up his C2 server used to carry out the data stacks. 05:13.780 --> 05:15.820 So he also got exposed. 05:16.100 --> 05:26.960 And the last example is the recent one is IBM researchers found out a huge repository that got exposed due to a security setting misconfiguration. 05:27.160 --> 05:47.200 So if you look at these examples, you can clearly see that the security settings misconfiguration and also some error due to the C2 or the other elements that might be have human error is compromising operations. 05:47.200 --> 05:50.000 So it happens all the time. 05:50.100 --> 05:53.200 So that's why actually we need an automation. 05:54.060 --> 05:58.920 Infrastructure design for the standard penetration setup is like this. 05:58.920 --> 06:05.100 You usually have some team server and you're directly accessing organization throughout this team server. 06:05.100 --> 06:17.900 And usually the team server's assets, IP addresses or domains are white listed by the organization for penetration testing to test more clearly. 06:17.900 --> 06:31.240 But in the red teaming setup, you usually have different type of servers behind some kind of redirector, whether it would be traffic or Nginx or HAProxy, etc. 06:31.240 --> 06:35.580 So you have multiple servers behind the redirector. 06:35.580 --> 06:46.860 The reason for that is you can easily build defense and recover your infrastructure against exposures from the blue team. 06:46.860 --> 07:01.040 That's why the redirector approach is more blending into the organization's traffic when it comes to the operational side of things. 07:01.420 --> 07:07.380 So when we design our infrastructure, we should be able to consider a few things. 07:07.380 --> 07:10.600 The first one is the leanness of our infrastructure. 07:11.280 --> 07:21.940 Because we want smooth operation and complicated infrastructure means more maintenance time or more time to... more things to look at. 07:21.940 --> 07:25.420 So it's the last thing that we don't want. 07:25.420 --> 07:27.800 The second is the segmentation. 07:27.800 --> 07:48.360 Your tools of choice must be segmented according to the functionality and they must be accessed from the redirector, not to be directed directly because of the exposure risk from the blue teamers. 07:48.360 --> 07:55.340 So these three gain us interdependence for our infrastructure. 07:55.340 --> 08:05.900 Every part... since every part is interdependent from each other, if some of them... some of it got exposed, we can quickly destroy and set up again a new one. 08:05.900 --> 08:14.900 And the network footprint from the redirector is a thing to be considered so that we can imitate some real traffic from our redirector to the company. 08:14.900 --> 08:19.520 So blue teams are... don't find us that easily. 08:20.040 --> 08:24.200 And also the engagement domain is also important. 08:24.200 --> 08:32.280 Whenever we plan an operation to a company, we always choose a domain from that company's domain category. 08:32.400 --> 08:38.400 We... at least you should try to select it that way. 08:38.520 --> 08:54.360 And the payload and C2 specifications are important for the... for the opposite side because you should... you can... it's... is your operation more truly if you will use phishing. 08:54.440 --> 09:08.500 You should be able to consider the... the three elements when you design your infrastructure because you might need then some SFTP mail server or maybe third-party service if you will use. 09:08.560 --> 09:12.040 Also domain fronting if you will use it as well. 09:12.040 --> 09:16.820 You should consider it once you... you're starting your design. 09:16.820 --> 09:20.260 And last is access control and request processing. 09:20.260 --> 09:48.240 The... you should always exercise access control for your infrastructure and more importantly the request processing because if you have some kind of request processing mechanism on the redirectors, you can relay the blue team that if... if your redirector IP or domain got discovered and blue team were to visit that domain or IP address or were to mmap or some discovery... 09:48.240 --> 10:02.580 run some discovery, you can redirect them to or relay them to the actual domain or the website so that you can avoid their work, discovery work. 10:02.940 --> 10:09.380 So this... the last... the last thing is the one of the most important things when you consider your design. 10:09.980 --> 10:12.940 So let's look at the DevOps side of things. 10:12.940 --> 10:37.160 DevOps is mainly... is creating self-service infrastructure for teams and since it's self-service and contains automation, it removes human element and which also removes manual and slow procedures so that you can continue work continuously and more lean... lean and fast. 10:37.480 --> 10:39.780 So why we are using automation? 10:39.780 --> 10:56.620 For every red team engagement, since they... all engagements are most... most of the time are unique, you have to plan and design an infrastructure and you need to set up it from UI or some best script or some kind of means. 10:56.620 --> 11:04.300 And if you do that by hand for every engagement, it becomes more error-prone and slow procedures and it's boring. 11:04.300 --> 11:06.460 So we're using automation. 11:06.980 --> 11:15.660 So for automation, using infrastructure as code, it's... it helps you to define provision and manage your infrastructure. 11:15.660 --> 11:28.300 So if you codify all of your infrastructure, you can track version changes or also validate changes and remove human elements from the infrastructure deployment procedure because... 11:28.300 --> 11:32.460 because it will be done from the code that you're writing. 11:32.460 --> 11:37.720 So the provisioning and deployment process automation is key in here. 11:38.440 --> 11:42.480 For infrastructure as code, there are a couple of tools. 11:42.520 --> 11:49.660 Chef, Puppet, Ansible, Sustack, CloudFormation and Terraform are common tools for DevOps practices. 11:50.400 --> 11:52.700 But why we are using Terraform? 11:52.700 --> 12:01.500 Terraform and... except Terraform and CloudFormation, the other tools are mainly configuration management tools and designed to manage existing software. 12:01.500 --> 12:04.780 But CloudFormation and Terraform are provisioning tools. 12:04.780 --> 12:13.560 Although they have some little degree of configuration management capabilities, they are mainly... are for provisioning the infrastructure itself. 12:14.020 --> 12:20.400 And as we said, as the other slide states, CloudFormation is mainly for AWS. 12:20.400 --> 12:26.380 So Terraform has multiple cloud support, so we are using Terraform for it. 12:26.680 --> 12:38.780 And if you couple Docker with Terraform for the configuration management, all of your needs will resolve itself anyway. 12:39.340 --> 12:45.120 And also why we are using Terraform is also have another reason. 12:45.120 --> 12:48.260 In DevOps, there is a term called configuration drift. 12:48.260 --> 12:55.100 It happens when you maintain an infrastructure over a long period of time. 12:55.100 --> 13:01.000 They differentiate one another in terms of the software version, etc. 13:01.000 --> 13:11.820 So if you're using Terraform with Docker, you reduce the likelihood of these differences, because the every change is actually mean a new deployment. 13:12.400 --> 13:16.920 And another thing is Terraform have declarative style of coding. 13:17.300 --> 13:27.880 We can give an example for that is if you have 10 instances and you have to automate it in Ansible, you're automating it like this and in Terraform like this. 13:27.880 --> 13:39.280 But if you will have one more instance on top of that 10, you have to rewrite your Ansible to count is as 1. 13:39.340 --> 13:47.380 And on the other end, in Terraform, it's 11, and Terraform will take care of the mathematics itself. 13:47.380 --> 14:01.790 So you don't have to write new Ansible, you don't need to write useless scripts anymore, because with Terraform, you can update your existing one and it will keep working. 14:02.400 --> 14:08.760 So the last part is the samples require master and agent to operate. 14:09.020 --> 14:22.460 But Terraform is masterless because it directly talks with APIs, so that it removes the need of master, server, and agent, which will make our infrastructure more lean. 14:22.980 --> 14:31.560 And one of the design considerations that we have is the leanness, so Terraform also suits with it. 14:32.280 --> 14:39.220 And in the light of this, we have three common combinations when it comes to build and automate our infrastructure. 14:39.220 --> 14:45.060 The first one is Terraform and Ansible for provisioning and configuration management. 14:45.060 --> 14:59.200 But as I said, for Red Team operation, we need as lean as possible, so Ansible might not work in that scenario, because it requires master, server, and also some agents to operate. 14:59.420 --> 15:10.880 The other is, if you're using some virtual machines, you can use Packer to template your virtual machines and deploy it with Terraform. 15:10.920 --> 15:26.740 And the last and our recommended approach is using Docker and Kubernetes to orchestrate your infrastructure assets and deploy them with Terraform, so that everything is to be taken care of with Kubernetes. 15:26.880 --> 15:41.860 And also, nearly all of the cloud providers have managed Kubernetes service, so you can also take advantage of that and leave the management site for Kubernetes to them. 15:42.060 --> 15:54.240 And you can just deploy your Docker with the configurations that you want and leave the management site to them, so that you can operate more easily. 15:54.880 --> 15:57.730 And let's get the demo. 15:58.580 --> 16:00.860 Hi everyone, this is Caglar. 16:00.860 --> 16:05.220 Today we are going to demonstrate building a Red Team operation infrastructure in AWS. 16:05.340 --> 16:13.740 We choose AWS, but you are free to use other platforms as like Alibaba or Google Cloud Platform or your own private cloud. 16:13.860 --> 16:16.060 Our demo has four phases. 16:16.460 --> 16:21.880 Inspecting scripts, run scripts and build infrastructure, payload execution, and destroy. 16:21.920 --> 16:28.700 In our demo, we just create a simple network topology, but you can extend it with your needs or requirements. 16:30.800 --> 16:34.960 Let's take a quick look to our Terraform and Helm scripts. 16:34.960 --> 16:38.540 In our repo, we have four important Terraform scripts. 16:40.900 --> 16:52.640 The first one, vpc.tf file, lets you provision a logically isolated section of the AWS cloud, where you can launch AWS resources in a virtual network that you define. 16:52.640 --> 17:05.760 You have complete control over your virtual networking environment, including selection of your own IP address range, creation of subnets, and configuration of root tables and network gateways. 17:06.440 --> 17:12.660 Aks cluster file lets you create a Kubernetes cluster with one worker. 17:12.660 --> 17:20.640 In our demo, we deploy Nginx, Ingress, and Metasploit managed with Aks. 17:21.660 --> 17:27.900 Helm file is used to deploy Nginx and Metasploit to worker. 17:28.980 --> 17:31.860 The last one is security group. 17:31.860 --> 17:38.080 This file is used for access control of communication server and services. 17:38.840 --> 17:45.500 And also, we have a file to see outputs of our scripts for debugging. 17:48.340 --> 17:52.380 Let's start building our infrastructure with Terraform. 17:52.480 --> 17:59.600 First of all, if you are not using a root account, you have to create a user and attach a policy to access AWS services. 18:16.370 --> 18:20.710 After user creation, you should add credentials to your profile. 18:21.190 --> 18:23.790 Let's initialize our script and validate. 18:45.500 --> 18:47.140 It seems everything is fine. 18:57.310 --> 18:58.690 Just deploy it. 19:29.860 --> 19:32.100 It seems everything is ready now. 20:04.280 --> 20:10.380 Let's check status of Kubernetes classes, nodes, and ports. 20:23.390 --> 20:25.510 Let's execute our payload. 20:41.310 --> 20:44.650 Now, we are ready to destroy all infrastructure. 20:58.060 --> 20:59.720 Everything is clear now. 20:59.720 --> 21:01.020 Thanks for listening. 21:03.180 --> 21:05.760 Now, let's get the maintenance part. 21:06.040 --> 21:13.620 So, as you see, we are to validate, deploy, and destroy at every stage of our testing. 21:13.620 --> 21:16.720 So, we can automate it with GitLab CI. 21:17.280 --> 21:24.760 And for it, we use init and validate first to test our scripts head check. 21:24.760 --> 21:28.420 And after that, we are using apply to deploy it. 21:28.500 --> 21:36.680 And if we were to deploy, we should be able to get the state so that we can destroy it afterwards. 21:36.820 --> 21:55.620 And after deployment, the last part is the destroy, but it should always work in the pipeline because if you were having some problems with deployment, some of your assets would get deployed, but some of them are not. 21:55.620 --> 22:02.300 And if the destroy stage does not work, these deployed assets will be online all of the time. 22:02.560 --> 22:13.160 So, we should use destroy part always in our CI, always in our CI with the destroy command. 22:13.160 --> 22:18.880 And it will destroy our infrastructure after testing so that we can validate our codes. 22:18.880 --> 22:27.800 And we can also see if our infrastructure got deployed correct or not. 22:28.140 --> 22:39.840 So, main takeaways from our research and the work is choosing correct tools whether it would be some engagement tools or automation tools. 22:39.840 --> 22:47.160 You should always choose whichever you are comfortable with. 22:47.200 --> 23:01.540 And for every red team engagement, whether you're doing some long-term penetration testing or red teaming or adversary simulation, do not make your infrastructure complex. 23:01.620 --> 23:05.920 Try a lean and interdependent infrastructure. 23:05.920 --> 23:12.160 Try to build and the domain choice of the things as we mentioned before. 23:12.160 --> 23:16.700 Do not make your infrastructure complex so that you can manage it easily. 23:16.940 --> 23:28.560 And at last, use automation and CI so that you can test and validate all of your infrastructure so that you don't have any surprises during the engagement itself. 23:29.500 --> 23:35.380 If you have any more, if you have any questions, we will be on Discord throughout the day. 23:35.380 --> 23:36.620 You can ask it. 23:36.680 --> 23:44.600 And as I've said, we will share the link of the slides in the Discord as well so that you can also reach GitHub page.