Top of the hour, it's 5 p.m. in beautiful Las Vegas, Nevada, and at DEF CON 25. How is everyone? Tired? Yeah, I'm just going to take 10 minutes to ramble. Just going to put on open mic, I guess an open mic for the next 10 minutes. So, thank you so much for being with us here at the Packet Hacking Village. All day. Let me tell you, you really, really made this first day, actually, I'm feeling really amped up. I mean, seeing the crowd grow, you know, here throughout the whole day has been very, very welcoming. It's been awesome. It's been just awesome to see. Thank you so much for all your support. So, I just want to give you a, before, you know, before Peter comes on stage, I just want to give everyone a little preview for, I just want to give everyone a couple of notes. We have one more talk left for tonight. That starts at 6.10. It will be Cheryl Bezwas. She's with one of the big four, and she'll talk about Threat Intel for all. There's more than your data that meets the eye. So, let's talk about tomorrow, how that works. So, tomorrow we have a full slate of talk. In fact, we have, yeah, we have a full slate of talk tomorrow starting at 10.10. In the morning. And so, the first talk we'll have is Making Your Own Gadget. Making Your Own 802.11ac Monitoring Hacker Gadget by, I think, SecurityTube and the actual author of Aircrack. At 11, there's going to be a talk on the black art of wireless post-exploitation by Gabriel Ryan. 12.10 will be very interesting. It will be a talk on, you know. War Stories on Fortune 100 InfoStack on a State Budget by Eric Capuano. And he's going to talk about what it's like to work for, like, a big, you know, work doing InfoStack for the state of Texas. At 1 o'clock, Yaldoc. Gita Ziavari will talk about large-scale data mining for Threat Intelligence. And after that, we have another talk by CloudFlare on the past, present, and future of high-speed packet filtering by Gil Biotel. We have a talk on Visual Network and File Forensics. Modern Day Covert's TCP. We have one on, you know, deceiving domain admin hunters. And we have another talk then followed by another one on hunting down domain admins. The late talks get very interesting. So, at 17.40, we have Megan Rohde who will talk about strengthening. A SecOps team by leveraging neurodiversity. That is going to be pretty cool. And last, we have Sam Bone who will talk about passwords on a phone. I think that they messed it up. They messed up the typos here. Okay. So, we have a full slate of talks tomorrow. Just want to remind you for everyone here. All the talks for the speaker workshop at the Packet Hacking Village. Is that all? All the talks are recorded. They will be made available on video along with the, at the same time when the DEF CON videos are released on YouTube. So, that would be like two to three months. So, what about the slides? That actually comes a lot quicker. When I go home, within the next two weeks, a good chunk of the slides will be posted on the Wall of Sheep website at wallofsheep.com. So, the video, the estimated time, two to three months. Two to three months. And then the presentation slide, two to three, two weeks. Within the two weeks that we go home from Las Vegas. Okay. So, with that said, the next thing is, I'm going to take a few minutes. Okay. So, it's 5.04. I've got a few more minutes to ramble. So, it is 5 o'clock. No. Stupid phone. You know. I do want to take this time and opportunity right now, because if you take a look around, and I hope you, I hope for all of you here that the Packet Hacking Village have been a very pleasant experience for all of you here. I hope that you've really, really thoroughly taken advantage of what we've offered in terms of learning opportunities here. If you take a look around, if you take a look around, I mean, none of this stuff comes cheap. I mean, the posters, the banners, and all that stuff, the t-shirts. I want to give a very, very... You know, thank you to our sponsors at DEF CON 25 this year. Wallace Sheep Title Sponsor, Splunk. Packet Detective Sponsor, Fidelis Cyber Security. Capture the Packet Title Sponsor, Packet Sled. Capture the Packet Platinum Sponsor, Talos. Sheep City Title Sponsor, Dark Matter. Sheep City Sponsor, 360.cn. Again, we're also looking for people who are no Mandarin, because we have a whole bunch of nice devices donated from 360.cn. That is available for you to break, but we can't rip it. I can't read Mandarin. I can't read Mandarin. And HoneyTouch Title Sponsor, 802 Secure. Okay, we want to thank you, and most importantly, thank you, each and every one of you, for spending your today here at the Packet Hacking Village. One more thing, and I guess this is another good time to do so. Another friendly reminder is a public service announcement before I introduce Peter. Can you all hear? Me in the back. Is this good? Is this good? Okay. So I want to give a public service. Whoa. Oh, those are yours. Oh. You don't know I'm there? Cool. So before I introduce Peter, I want to give a public service announcement here at the Packet Hacking Village. And that's on an issue that is a serious problem in cybersecurity and also in tech. And that's... It's a no-no. Don't go there. Don't do anything stupid. In fact, sadly, we also had a few volunteers that were a victim of it last year. So, of course, this is something that's been near and dear to us, that we take very seriously. It's not tolerated. Absolute no-no. Don't go there. And as I tell every of my students on the second day of my class, you know, you know, don't... Give the community and the industry and also the field another more black eyes and a more bad rap. Okay? It's a no-no. It's an absolute no-no. So, with that said, now it is my pleasure to introduce to you Peter Wane. Hello. Hello. I see a lot of faces that I know that I wish weren't here. I'm going to warn you right now, this is not going to be great. But anyway, so to give you a brief primer of the way I present things normally, I tell a lot of bad jokes. They're an attempt to hide the lack of quality that I have in my slides. And then hopefully you will laugh at them. But in general, we're going to... Today I'm going to walk you through different post-compromise techniques that attackers use in AWS. In order to hide themselves and such. And hopefully that sounds good to you. Thank you. So, I'm Peter. And I'm a security researcher at AlienVol. I like cocktails and I like really like weird funky wine where you can't tell if it's bad or is it like really, really good. I'm from Texas and if you would like to follow me on Twitter where I talk very little about funky wine and even less about security, that's my handle down at the bottom. So, we're going to give you a brief intro to AWS. Can I get a brief show of hands of anyone who doesn't know what AWS is? Who knows what AWS is? Thank God. We're going to talk about a few common infection vectors, hiding techniques, the persistence techniques, and then some hardening tips that hopefully you can take back to your company and make sure that you don't get hacked like this. Someone just followed. Help me. Y'all are fast. So, what is AWS? So, the first question is who is Amazon and what are their web services? For the articles I've been seeing in Forbes, a couple years ago, Amazon only sold books. But nowadays, they seem to be like one of the forefront leaders in cloud computing. And everyone from like two, three person mom and pops to like major. And that's been right there. So, the first question is what's AWS and basically the way that AWS is, right now, most of the companies are storing everything inside AWS. They have the servers inside AWS and it becomes very lucrative for attackers to leverage this deployment in order to make money, get credentials, what have you. And the question is why should you care? How many people here host things inside AWS for the company? Thank you. I thought it was only one hand. It was going to be very poor talk. but we really should care because these days inside AWS as you lock down platforms more and more and more like for example I can only access your application on port 80 on 443 and most attackers aren't going to spend like 6 months fuzzing your platform to see what race conditions that you have inside to get in there. But if I happen to leverage some credentials I find at AWS I have the ability to back door everything from the application itself to your AMIs to whatever to what we're going to see in a little bit. So these are very common compromised vectors inside AWS. Number one is infected machines. As more and more people go to work remote and BYOD infected machines are always a problem even if they don't get infected on your corporate network. People take their laptops home, they bring them back in, they're infected and then from there they can infect the corporate network. And then second is phishing. As much as we try to train users phishing has always been and will always be a problem. We constantly read articles about someone pretending to be a CEO that you know has like one A off or one capital or what have you saying wire me $500,000 and someone in finance is like absolutely. You're down the hallway from me. I'm not going to ask you about this. I'm just going to wire it immediately. And then credential leakage. Credential leakage is a huge one. So there's popular applications such as Git rob that scrape through GitHub. There's other applications that scrape through big bucket or paste bin or what have you because too many people hard code credentials and then push them to paste bin or GitHub and ask questions about them. So from here you can find anything from SSH keys to secret tokens to what have you that you need to log into AWS. And I'm going to go ahead and say a lot of companies are cheap and people share credentials. So if you own that one set of credentials you own the entire corporate infrastructure. So this is a great way to do that. I'm going to assume none of you in here share credentials at your company because that's bad security, right? Y'all can say no. Alright and then the last one is social engineering. So I'm sure some of you in here are red teamers and you know you can go through this whole thing about like you know trying to get people's passwords, trying to use mini cats like scrape things out of memory or you can just call the front desk and say that you're a new user and you would like your password reset and generally they'll reset your password. This can also be another common infection vector into AWS. So besides user-based infection vectors there's also service-based. So service-based I want to say is because it's built in tools in AWS that can just be abused and leveraged in a way that they didn't originally intend. A major one is third-party monitoring services such as let's say a data dog or what have you for monitoring the uptime inside AWS. You can make your application and your corporate network as secure as you want but if one of your of these third parties get compromised and they have a token to log into your environment, whether it be the OAuth, what have you, then you effectively lost the battle. The second one is metadata leakage. The EC2 has a fair amount of metadata, so if you go to EC2 and select an object here. I seem to have put someone to sleep already. I like how everyone turned and saw him immediately. You can look at the AMI, you'll see the AMI version, you'll see the kernel, you'll see the region, and all this metadata is available through a web server that's only normally available through that instance itself, but if you have weak VPC separation or segmentation through your network, it's possible to query from one foothold that I had over here to query over here. And while this metadata may not seem important, it can, number one, tell me which regions you have. Number two, it can tell you which regions you have. Number three, it can tell you which hostings, and it can tell me common naming schemes you use. It can tell me different things where I can use that phishing that we talked about before and that social engineering to get people to leverage to get more data. There's also AMI poisoning. Once I have those set of credentials, it may be very difficult for me to hack your application in particular. Let's say you're just one of the best secure coders that's ever existed ever. But I back to your AMI and just add a nice account that has a cron job that sends me all the credentials I've logged through back to my web server. It doesn't matter how secure your web application is, does it? You can say no. Thank you. There you go. Like I said, we have to be interactive. We have to feel each other. It's going to make things go much more smoothly. And then there's instance profiles. Instance profiles, has anyone used them before? Let's see how in depth they go. All right. Instance profiles are defined by the AWS, usually by the architect who is like making everything. It determines which permissions are going to be available through the EC2 instances in the profile. For example, you'll be able to create an instance with a profile that has SQS for hitting whatever databases. Those permissions will usually just allow people to hit the API, but once created, that instance profile is associated with EC2 instance or a launch configuration, and when the instances are started, AWS creates a unique set that allows people to hit the API. It's a really simple process. The of access keys, a secret key and a security token, which makes them available through the instance, through that metadata that I talked about earlier. So that metadata that you'll query, it lists, let's say, for example, for layman's terms, usernames and passwords that are able to log in. And one of the lowest pieces of low-hanging fruit is this public EBS snapshots. So these snapshots are just like a snapshot of a virtual machine. But for some reason, some companies choose to make these snapshots public. Whether they tell themselves, I'm only going to make it public for five minutes so I can download it onto my local machine or I'm going to send it to someone, what have you, and then they forget to revoke it. So going through these snapshots, you can find anything from SSH keys to hard coded credentials to those secret keys that we talked about earlier to any sort of private data. And if you don't believe this, you can just go on to EC2 honestly and go to EBS and just select public, go through, download some images and see what you find. And that leads us to different ways of tapping. Attackers can hide in AWS. So number one is going to be CloudTrail. CloudTrail, if made properly, audits everything that happens inside Amazon. Everything from adding users, everything from monitoring their keys to everything. So one of the first loud things an attacker would do when they go through is delete all the logs. But more than likely, if someone like Eddie Lee in the back deletes all the logs, we'll notice. Bless you. Which leads us to the S3 trail. This is something that would be slightly more sneaky. It's just stopping logging. So instead of deleting all the logs, you still have your logs. I just stop. And everything I go, going forward, will be on logs. So you're going to have to audit every single profile, every single AMI to see if I've modified anything. And then that brings us to the S3 trail. So a lot of people push their logs into the S3 buckets because, you know, it's very simple to set up inside Amazon. So if I don't want to delete your logs and I don't want to stop logging because I assume you have correlation rules set up for that, I can modify the log rotation policy. So normally you would, normally you would rotate the logs about once a week, tar them, push them to an off-site storage, and I can modify that to every one minute. But then I can also change the location of where those logs write. And something you'll find out is there's also public S3 buckets. So I can change that from your corporate network to push to someone else's. So you don't notice anything about your logs. So I can modify that to every one minute. But then I can also change the location of where those logs stop logging because you no longer have access to those logs. And one of my favorite sneaky things to do that I've seen is the key management service. So inside this key management service, your logs will keep writing. They will keep going to your own S3 bucket or wherever drop box that you had to put it to. But I can instruct CloudTrail to use this particular private key to encrypt everything and then destroy it. So you won't have access to your logs. I will. But as the attacker, I don't really care. . . . . . . . . . . . . . . . . . . . . Well, leaves us over to persistence. One easy way of maintaining persistence inside AWS is just creating a new user. It's cloud. You'll see it more than likely. If you're a small company, you'll definitely notice someone adding 1 or 2 users but if you're a large conglomeration where users are added and deleted every single day, that might just be more noise. Particularly sneaky thing to do when you're creating a new user. Instead ofуб representing that new user creating let's say Jonathan, you'll create Jonathan with two A's or Jonathan with two N's so on first glance it looks very similar to everything that's in there. Another way is creating a temporary user. An acceptable duration when you're creating an IAM user, so identity management user, these sessions range from 900 seconds, about 15 minutes give or take, to roughly 1,000, sorry, 129,000 seconds which is about 36 hours as the default. These sessions, these sessions for the AWS owners are restricted to a maximum of one hour so you can create these, delete them, but those keys to log in will still be valid even though the user is gone. So they're not keeping too close of notice of like who's logging in and who's logging out. This is a very easy way to keep persistence. Another way is to create more user access keys. So by default, I believe you can have two to three access keys per user in there. So instead of making a new user or trying to typosquat a user, I'll just add an additional SSH key that I can use to log into various devices. And it brings us to roles. So let's say you're in an environment in which they have roughly 100 users. Yes, you could backdoor every single account there but you're going to have many, many, many, many, many keys. And you can keys to manage. Which brings us to roles. Roles are like let's say a superseding level above it to where you say role Bob has access to all these users X, Y, and Z. So it just makes it a little cleaner way to manage this and that goes back to the same thing which earlier we were talking about when a user deleted their keys are still valid. Well the one thing you will have to notice with roles is if one user loses that access in the role it could potentially affect the way that the role works as far as logging into devices. And backdooring AMIs. As I alluded earlier I don't have to necessarily hack your web application or your database or what have you. If I make myself a nice account or a cron job or a nice little script to scrape out whatever sensitive data you have and send it back to myself especially in ways where everyone loves auto scaling. So I only have to backdoor one AMI and then wait as auto scaling creates these new servers and destroys the old ones so eventually my infected AMI will eventually propagate throughout your entire network. And then another simple one is default security groups. The security groups as the VPCs you know the Amazon firewall. I can add very small things for myself or normally only allow port 80 and 443 but I'll allow port 1111 or 4444 to allow my backdoor that I put in the AMI earlier to connect back whenever these instances are set up. And here's where I have attempts to be more secure. Make sure you segregate users in the least privilege model where if you only need to have access to X, Y, and Z you should only have access to this and not everything. You should utilize that least privilege model and not use the root account just like on your computer. You shouldn't run things as root. I'm sure there's some people in here who run everything in Kali Linux as root. Don't do that. Use instance profiles. While you're using your own browser you should always have access to your own browser. So, the last one is segmentation. So, if you have some metadata leakage that we talked about before, if you use proper segmentation then your risk is mitigated. So, while they may own one part of your network, they're only segmented to that one particular piece. And the last part is audit absolutely everything. While it may seem noisy to have everything in CloudTrail when people log in, when new keys are created, it can be very, very useful when you're going through and attempting to search through attacks and you're trying to do something that the attackers came through. AWS config is also very useful in this. It's an integrated service in AWS that enables automatic enforcement and verification of AWS resource modifications. For example, if you have an AMI that's changed, AWS config will tell you that it's changed and you should log this. Along with CloudWatch. CloudWatch is another integrated monitoring service for AWS that enables organizations to collect, monitor, and set alarms for anything that massive changes have happened. AWS. The questions? Anyone? Sir? I cannot hear you. Yeah. I'm familiar with some of them, yes. Are you asking about some of the potential risks from using them? Yes. So what I talked about earlier, so when you give these people access to your AWS network, usually you're going to set up a user for them or you're going to give them like an access token to log in. So you are shifting some of the risk onto them to keep their network secure. So if they lose control of their network and these access tokens are taken, people potentially have access to everything inside your network. Well, hopefully you've set aside proper user segmentation to where they only have access to the one or two or three things that they need. But just like when people are doing vulnerability scans, they often give domain admin to the user that's scanning and they do something that's very similar inside AWS. Sir? Do you have any thoughts on using Docker or containers? So containerized systems, well, I'm going to go ahead and say that in my opinion, they have some of the same vulnerabilities as autoscaling inside AWS. For example, you have your particular Docker image that you launch and destroy and launch and destroy. I can still backdoor that same image in the same way I would backdoor an AMI. Anyone else? Sir? What do you think about the S3 problem that they have? Excuse me? What do you think about the S3 problem that Verizon has? That's going to be more of an uptime issue and I won't really comment on that. Anyone else? Sir? There's a logo in the bottom right corner. But honestly, any SIM that can ingest these logs and that you can create correlation on. The most important thing I'll say is creating a baseline. So what is normal inside your business? If you normally create one user every single Monday when you have new hires, that would be normal. So you can set a correlation role. If a user is created on a Tuesday or Wednesday, that would be abnormal to alert you. But like I said, any sort of SIM would be able to do it. But use the bottom right-hand corner logo. I saw one more hand in the back. Sir? Can you talk to the use of multiple accounts to increase your security profile in AWS? Yes. So the segmentation that we're talking about right now, we're talking about the security profile that you're speaking of earlier. So when you first create your account in AWS, you're generally given the root account, so to speak. From there, you want to create users that only have, for example, access to the SQL database and they only have access to launch these AMIs and they only have access to this. The permissions inside AWS are very rich and very full. If you've ever read the documentation, they go down very, very deep. I was talking more about actually using separate AWS accounts. Oh, I see. Not like user accounts, not IAM accounts, but actual user accounts. I see. So you're talking about AWS accounts, like numbered accounts. Yes. That's another way of segmentation. If you can segment from one root account up here, from, let's say, as you said, this AWS account that only has access, let's say, EC2 East, as opposed to this account that's a separate one that only has access to West 1 and they're completely different units, it's going to have about the same effect as user segmentation. Instead, it's going to be like zone segmentation, so to speak. Did I answer your question? Okay. I think we have room for one or two more. I was trying to escape. We have room for quite a lot of questions. I was trying to escape. It did not work. Anyone? Please. So I walked in a few minutes late. Forgive me if I missed something you already said. Finally someone uses the mic. Yeah, I walked in a few minutes late, so I apologize if I'm asking something that you've already mentioned. So a lot of the access that you're mentioning, the backdooring of the AMIs and stuff like that, is that assuming that those policies and whatnot have been applied to the roles or the groups that those users already possess, or are you talking some sort of privilege escalation? Is there a privilege escalation path that would be exploited in order to gain that access? So this is definitely like abusing credentials that they already have, whether they've scraped them via Bitbucket or what have you. Okay. Perfect. All right. You have a question, Eddie? Lambda functions. Lambda functions. Lambda functions are a nice black box inside AWS where you can define certain functions that are going to trigger off a certain set of functions that are going to trigger off a certain action happening. And honestly, that's as deep as I want to go into Lambda functions. Thank you, Eddie. Sir? Like Lambda or anything in general? Yes. Because no one understands Lambda. All right. Can you talk a little bit more about the initial scraping of the credentials to be able to get into an . Sure. So there's tools such as GitRob that can automatically scrape GitHub. And then so go on to Bitbucket and look at public repos. And then let's say download everything and grep for username, grep for password, grep for et cetera. You'll more than likely find many different user credentials. And then people often compromise different databases or what have you and just dump it on Pastebin and scrape those Pastebins, scrape those same things. So you can go from any amount of like AudraMagic or something like that. To yourself scraping. And then also since credential reuse from all like let's say the MySpace dump from a couple years ago. People are still using their passwords that they use in MySpace in high school. So you can take those and like go and try to brute force through to get into these same things. Or change small things from like fall 2017 to change it to fall 2015 or what have you. So it's all examples of those same credentials abuse. Sir? . I've seen them lock out one of my own instances when I leaked a key on Pastebin. So they seem to be fairly active. I can't say the response time but they blocked me. Amazon seems to roll out services with code names that don't really describe what they do. All the time. So I'm going to say yes. . He asked if there are any services sort of like Lambda that seem to be a black box that could be like leveraged by attackers. And I answered yes. And honestly we kind of don't know. They tend to just push out features all the time. And it takes some time to go through and see the security implications or risks that are introduced by using these services. . Sir. . . . . . . . . . . . . for like known CNCs to like known such techniques that are used. Okay, I see what you're asking. So, with the techniques that we presented right here, these are all tools that are built into Amazon that are just being abused in a way they weren't necessarily supposed to be used. But yeah, these are all known. These are all tools that we didn't design. These are all by AWS. All your defenses here have been about protecting your own account. Have you heard any rumors about being able to attack your account through another account on Amazon? That is weaknesses in their APIs or being able to jump from one account to another? As far as accounts that are in the same hierarchy. Like, for example, if I had an account that's like level three from here trying to work my way up to the root account, if the same, the passwords were similar, then yeah, you could jump up. But as far as pivoting to a completely different person, like, you know, Ming's AWS account, for example, I personally have no example, I have no ability to do that. Excuse me? Do you have any other questions? Well, you can, or you could do that kind of technically if they cross the account the icon minced so if you want to . Yes. . Okay. Yeah, I was more interested in if somebody from a random other account can use that to access my information in any way. Yeah. Okay. Yeah. Yeah. Um. Yeah. Okay. Yeah. Okay. Okay. Okay. Okay. Okay. Okay. Okay. So if we Bye let's just talk about Unterstützung. But let's just focus as much as we can question was answered. Anyone else? All right, I'm going to go ahead and say we're done.