start	end	text
1370	5750	So, welcome to my talk on Kubernetes called Attacking the Helmsman.
5910	9890	So, as Joan just said, I'm Mohit. I'm a security consultant at F-Secure.
10430	13410	I'm primarily a pen tester, but I also help our clients with, you know,
13410	17970	architectural diagrams and stuff from a security point of view.
18330	20470	So, a little bit about what we're going to be talking about today.
20470	23470	So, I'll start on a bit of background of Kubernetes,
24010	26830	and then moving to the management plane, you know, the different components
26830	29530	within there, how that gets, you know, affected by it being
29530	34790	within the cloud, and then a bunch of different security features
34790	36930	within Kubernetes and resources.
38450	40550	So, a little bit about Kubernetes.
40610	43730	So, Kubernetes, at least I've been told, is Greek for helmsman.
44410	47930	When I heard about this, I asked one of my Greek colleagues about this,
47930	50870	and he wasn't so sure about that. At least it didn't click in his head.
51410	55610	Later on, he came across Istio, which also happens to follow
55610	58390	that same sort of Greek, you know, naming convention.
58390	61110	And at that point, something clicked, and he was like, no, okay,
61110	63810	so it may be a thing, however, it's spelled a little bit differently,
63810	65430	and he pronounced it Kee-bee-nee-tees.
65910	67970	I hope I haven't butchered that.
68550	71190	For those of you who don't know, Kubernetes is a container
71650	75350	orchestration tool. It essentially helps you manage a large number
75350	77890	of containers over potentially a large number of hosts,
78730	81830	and it provides a lot of features to aid in that kind of management,
81830	86010	and you providing a desired state, and Kubernetes tries to maintain it.
86010	89470	And recently, it's also started moving into multiple clouds.
89570	92470	I remember, I think, a couple of coupons ago,
92470	94390	it was talked quite heavily there.
95370	99250	And Kubernetes is, I think, one of the more popular
99250	102170	container orchestration tools I've seen.
102170	104610	At least I haven't seen something that's more popular.
104830	105750	So, yeah.
106350	108490	So a little bit about the management plane.
108490	110630	Now, the top box here is the management plane,
110630	112850	and it is essentially what helps manage the cluster
112850	115050	from an automation perspective.
115050	118090	So at the beginning, you have the API server.
118090	122070	The API server is effectively a stateless REST-based API
122590	126490	that is used for users to interact with,
126490	130130	and it's kind of like the front end of the management plane,
130130	132490	which stores its data within ETCD.
132990	135290	You also have the controller manager and the scheduler.
135290	139630	The scheduler is the resource that kind of wants new pods,
139630	142450	which are the smallest item within Kubernetes.
142530	145030	It's created, and it doesn't have a node assigned.
145030	146990	It'll assign it to a node to deploy to,
146990	149390	and the controller manager is kind of like the brains
149390	152170	in the background, just as things are happening,
152170	154730	monitors their states and deploys things or updates things
154730	158590	as needed to make sure it's at where we,
158590	161490	or where the users desire the state of the cluster to be.
161770	164090	And then outside the management plane,
164090	165390	you have the various nodes.
165390	167670	These are the ones that actually do most of the work,
167670	170970	running all the containers for the various workloads.
171310	173210	Typically, each node contains a kubelet,
173210	176230	which kind of manages the pods on that node.
176250	178630	There's usually some sort of networking with kubeproxy,
178630	182710	which does like network-based routing and configuration.
183070	184830	And of course, you're going to have your container runtime
184830	186430	and your base OS.
188910	191170	So digging a little bit more into the API server.
191170	194110	So as I said earlier, this is kind of like the front end
194110	197690	of the cluster for people to communicate with.
197690	202050	And it performs some additional features on top of just storing that data.
202050	205130	It says, you know, applying any RBAC policies that may apply.
205130	207350	So, you know, if a user wishes to do something,
207350	209990	it would first validate that they are allowed to actually do that.
210230	213330	And also does additional data validation to make sure what it's been given
213330	214990	does make sense.
214990	218310	So it's not being asked to deploy, you know, a pod with incorrect specifications
218310	219110	or anything.
220370	222230	And once it has this data and is happy with it,
222230	225570	it gets stored in the ECD, which is effectively the golden grail
225570	228290	of information within the cluster, because it stores pretty much everything.
228570	231610	All the resources that are deployed in the cluster
231610	235930	are defined in YAML, which are stored in ETCD.
236110	238390	And therefore, if you have write access or read access to this,
238390	241730	you have pretty elevated access to the data within the cluster.
241770	245070	With write access, you know, you could write yourself new roles
245070	248150	or policies that would let you have cluster admin access.
248150	251810	And if you have read access, you can just read the secrets for like...
251810	254790	Sorry, not read the secrets, just read various data.
254850	258430	Secrets are a little bit more interesting as they tend to be encrypted in ETCD.
260050	264390	And the way ETCD is built, there's not really an application-level authentication.
264390	272420	So you kind of focus more on whitelisting the source addresses
272420	274160	that are permitted to talk to it.
274160	277360	Typically, it's recommended to only whitelist this to the Kube API server,
277360	280500	as that's really the thing that needs to talk to it.
282360	287840	As I said earlier, the Kubelet is the component that runs on each node.
287840	292080	And this tends to register itself with the API server to say, you know,
292080	298680	I'm a node here, please give me work and have permissions within the cluster
298680	301440	so I can access the data I need.
301440	303600	So for the port specifications, it needs to deploy pods
303600	306780	and any other data that may need to be deployed alongside it,
306780	309000	for example, the secrets for those pods.
309000	312840	And to have this access, it has a server's account to the API server.
312840	319120	And in itself, it exposes an API, typically on port 10.2.50.
319220	321900	And there have been attacks against this in the past as well.
321900	326900	So one of my colleagues, Alexander Keskis, wrote a blog post on this a while back,
326900	329800	where he was leveraging anonymous authentication on the Kubelet API
329800	333620	then to talk to the API server and get into various pods.
335700	342100	Now, the management plane within a cloud-managed Kubernetes cluster such as EKS or GKE
342840	346920	the management plane is simply managed by the cloud provider.
346980	350980	As such, when we're doing pentest engagements, these are typically considered out of scope.
352140	355820	And we kind of trust the cloud provider to manage that appropriately.
355860	359720	However, being in the cloud also adds cloud-specific interactions.
360060	365340	For example, instance metadata raises its ugly head again, where the VMs,
365340	371120	if deployed on VMs within the cloud, it would have access to the instance metadata
371120	378080	and theoretically pods could communicate with that to get AWS credentials
378080	382940	to talk to the rest of the environment if the nodes have it or anything else.
383180	387840	Now, there are various protections for this that are possible, but they're not always in play.
388580	390640	And the fact that it is also in the cloud environment,
390640	393840	typically IAM is also integrated with access to the cluster.
394080	397640	So when authenticating to a Kubernetes cluster, which is managed,
397640	402200	typically you're using your cloud provider's IAM or CLI to kind of get access
402200	405180	and generate your kube-config files so you can authenticate.
408970	413770	So a while back, we were doing an assessment where a couple of our guys noticed
413770	417250	that they could talk to the instance metadata within a cluster within GKE.
417630	423730	And within the instance metadata, they found tokens for talking to the kube-api server.
424030	429350	They enumerated the permissions for this and noticed that they were allowed to create CSRs.
429350	434030	And going back to the fact that kubelet self-registered, these CSRs did that.
434170	437750	You could essentially create a CSR to send it to the API server,
437750	443110	which when it gets signed is your authentication token as a kubelet to the API server.
443110	447530	So they could use this to start enumerating the different nodes and get the pods,
447530	451630	the secrets within and get to cluster admin in that manner.
451770	455110	So effectively from a pod that they can talk to the instance metadata,
455110	458930	they escalated themselves quite quickly to cluster admin.
459350	465290	Now back then they just did this manually, but recently I got told of a script that kind of did it.
465770	469910	I can't remember the person's name. Just a second.
472250	477490	A script by Brad Giesemann that's a quick bash script that kind of does this for you
477490	483250	and enumerates that data and, you know, gets you the secrets and stuff within the cluster.
484770	490050	Now Kubernetes is very much API based. So the API server is pretty much a whole bunch of APIs.
490110	493170	And by design, Kubernetes is modular.
493230	497010	And we're going to see this throughout the presentation where the way that it's set up
497010	501710	allows for the easy adding of new resources or configurations
501710	506510	so it can work better in people's individuals, you know, configurations.
507670	513030	So the ability to add additional APIs is present and it's also useful for identifying technology sites
513030	515870	when treating it in an offensive perspective.
516110	520250	As when new technologies, you know, tend to use creating custom APIs,
520250	524230	you can enumerate these APIs and figure out what kind of technologies are in use.
524450	530970	So this is the example output of this for a pretty bog standard cluster with some additional things.
530970	534970	And from this we can quickly enumerate that, you know, it's using Calico for networking,
534970	537370	it's using LinkedIn as a service mesh and so on.
537370	540170	So it's a quite useful way to enumerate this information.
543180	547060	So Kubernetes uses RBAC for permission control.
547380	551540	It's generally found to use RBAC. There are other ways it can be done, such as ABAC.
551800	554860	But most of the time it's RBAC.
555000	559040	And within RBAC there are three main high-level concepts within Kubernetes.
559180	561160	So the first one is the targets.
561280	563080	So these are what the permissions apply to.
563080	565360	So your users, your groups, your service accounts.
565580	567360	And then these get assigned roles.
567360	572480	And the roles grant you those permissions in, you know, the various areas of the cluster.
572800	576940	This can be limited by the API groups, the resource, the verb.
576940	580320	So, you know, whether you can create it or just get it or whatever.
580320	582520	And potentially also the resource name.
582820	589320	And now these are bound to those targets using role bindings, which basically are map roles onto your targets.
590300	596860	Now these RBAC permissions can either apply cluster-wide or within a single namespace.
596860	602920	And the way that kind of gets differentiated is with, you know, the role slash cluster role or role binding slash cluster role binding.
602920	607800	So if it were a cluster role and a cluster role binding, the permissions applied would be cluster-wide.
607840	612940	And all resources that, you know, are allowed by that role would be allowed to that target.
612940	617980	However, if it's, you know, using the role binding, then it's only within a specific namespace.
617980	619660	Those permissions are granted.
621020	627960	Now, of those three, you know, kind of targets I kind of mentioned, the only one really managed by Kubernetes is the service account.
628040	630600	And these are authenticated using Bero tokens.
630600	638480	And they are, you know, fully managed within Kubernetes, typically, with service accounts being created as a resource.
638480	641900	The authentication credentials are mounted to pods.
641900	645980	So they have access to those service accounts, which they can use to talk to the cluster.
645980	648820	Users and groups are not managed by Kubernetes.
648820	655040	And it's typically left to some sort of external service with authentication modules within Kubernetes,
655040	660860	providing that kind of access to those authentication mechanisms so it can kind of, you know, authenticate users.
661240	666580	And it kind of trusts those third-party systems to be correct and just follows it.
666580	673160	So, for example, using X.509 certificates, the common name provides the user and the organization, you know,
673160	676980	within a certificate, tells Kubernetes what groups it's in.
676980	682420	And it just validates that it's been signed by the appropriate certificate authority and says, yep, that's cool with me.
682420	693300	Or, for example, if you're running a Windows-based environment, it can also use the Active Directory as a kind of source of truth for what users are allowed or not.
695420	703460	So service accounts being, you know, in Kubernetes, when you create a new namespace, which is the way people tend to do segregation,
703540	707240	a new default service account is created within that namespace.
707240	712900	And this default service account is automatically mounted into every pod within that namespace by default.
713500	717520	Now, this service account does not have any permissions directly applied to it.
717520	721760	However, if it's in any groups, then it would have those permissions as well.
721760	725960	But then this is typically something you should avoid.
726080	732360	Because if you do assign any permissions to this default service account for one pod, and another pod is also using it,
732360	737100	then that other pod would also have those permissions, you know, kind of breaking the principles of least privilege.
738200	746180	So it's typically recommended to disable the auto-mounting of these default service accounts and creating bespoke service accounts for each time you'd need it.
747700	755120	Now, the groups that, you know, the default service account may be in, there are three groups it's in by default.
755440	760080	So that's system authenticated, which is for pretty much every authenticated entity within the cluster,
760080	763100	system service accounts, which is the group for all service accounts,
763100	769200	and then system service accounts and the name of the namespace is for all service accounts within that particular namespace.
769260	774360	So if any of those three groups have any permissions assigned to them, that default service account will also have it.
774360	779500	There's also a system unauthenticated group. So if, you know, the cluster allows anonymous authentication,
780140	786680	the user, well, not really a user, but the person who has communicating with the cluster will have the same permissions,
786680	789520	as would be granted by system unauthenticated.
790700	793780	So there was a time not too long ago where we were on assessment,
793780	800500	and the groups for system service accounts are actually over-provisioned, granting cluster admin access.
800500	804740	And therefore, you know, as soon as we compromise a single pod in the cluster,
805240	809440	we were able to use that service account, which was mounted into every pod by default.
809780	813880	And we very quickly got cluster admin into the cluster in that manner.
816380	821620	Now, when enumerating permissions, there are a bunch of tools I tend to use when enumerating permissions.
821660	826360	I think the one I use the most is a tool called Rakesh, get hub link on the slides.
826360	832240	And essentially, this takes, you know, your token that you used for authentication or whatever,
832240	836460	and it kind of enumerates all the permissions that are available and then gives you a nice tick or a cross
837040	842180	in a tabular format showing, you know, what you are and aren't allowed to do,
842180	846820	which, you know, if you've just compromised a single JWT token to authenticate for a service account,
846820	849320	it's great to kind of figure out what you can and cannot do.
850540	855740	But then, you know, when doing more of a review of RBAC, there's a tool called RBAC Lookup,
855740	860980	which kind of lists through all the subjects, all the targets, the roles and role bindings
860980	865360	that are present in the cluster. So, you know, if we've identified a role that is, you know,
865360	869900	considered risky in a way, so for example, it grants you excessive permissions,
869900	875700	we can quickly identify what targets have that role assigned to them.
876920	884920	The final tool I'd like to quickly mention is QBScan, which kind of takes the approach a bit more automated,
884920	890620	where it automatically tries to find those risky accounts, whether these are just our cluster admin by default
890620	898140	or they have permissions that could easily help them get to a cluster admin level of access.
898140	904180	So moving on to networking, there are a variety of things you need to consider with networking.
904180	911880	Now, Kubernetes, as I said earlier, being very modular, the actual networking interface you use is actually kind of up to you.
911880	917080	Kubernetes say that, you know, it should follow the CNI container networking interface kind of specification.
917220	923520	But apart from that, we're Gucci. So within that, you know, there are a bunch of things we need to kind of look into,
923520	928080	like pod-to-service, ingress, pod-to-pod, and the CNI itself.
928200	933000	Now, because CNI is pretty much freedom, there are a lot of CNIs out there,
933000	938480	and each of them provide different functionality and features.
939220	946640	So there are, you know, the cloud network-based CNIs, which kind of integrate with your cloud providers.
946640	951480	Networking interfaces like AWS VPC would, you know, kind of put you within the VPCs
951480	955100	as opposed to a weird overlay network within the cluster itself.
955400	960820	And then there are others that, you know, kind of provide in-transit encryption or other features.
961380	963020	And there's a whole bunch.
964580	969240	For traffic to get into the cluster, typically, you would find an ingress controller.
969240	974320	Some alternative methods are like a node port, which is just, you know, a high-numbered port,
974320	977540	usually above 30,000, where a service is exposed.
977640	981700	But more often than not, you're going to find some sort of ingress controller,
981700	985920	where it's typically a pod, such as, you know, Trafic or Nginx or something,
985920	991640	that has permissions within the cluster to list the services, which are the way you expose pods in a cluster,
991640	996400	and then gets traffic coming in and then tends to route them around internally in the cluster
996400	1001060	to the correct location or the correct service.
1003160	1005880	DNS for resolving within the cluster.
1006480	1011580	Now, if it's just a pod, it's automatically got a DNS name with the pod's IP address,
1011580	1015420	the name of the namespace, .pod.cluster.local.
1015460	1020780	However, it's usually not common for people to use the pod's DNS name for resolution.
1020780	1025080	It's more common not to use a headless service on top of that,
1025080	1028200	where after you've created a pod with the labels appropriate to it,
1028200	1033100	you create a headless service pointing to those pods, and it kind of simplifies that entire process,
1033100	1036780	as it can also do a load balancing between the various pods.
1037580	1042180	And typically, this is called, you know, the service name.namespace.service.cluster.local.
1042180	1049340	And because the search space for the DNS queries include namespace.service.cluster.local and service.cluster.local,
1049340	1052820	you can just do a query for service if you're in the same namespace,
1052820	1055600	or if you're in a separate namespace, service.namespace.
1056140	1060520	There's an additional DNS name that's usually also added, and that's hostname.subdomain.
1060520	1064000	And the hostname and subdomain is kind of extracted from the pod.
1064000	1071160	So the pod can specify the hostname and subdomain at which point it will be available using that kind of DNS name,
1071160	1074240	which kind of brings the idea of spoofing into play.
1074260	1078540	So, for example, if I were to set the subdomain to, say, com and the hostname to google,
1078540	1088500	suddenly I have a DNS queryable name, google.com, which would complete and resolve to an internal IP address.
1088960	1094260	So if you could, you know, somehow get yourself a google.com certificate, which I doubt you're going to get an external CA to do it,
1094260	1099100	but if you're using some sort of internal CA, it could let you do some interesting attacks in that manner,
1099100	1101320	or manhandling other connections.
1103260	1108680	Now, the Intel network is typically flat by default, because Kubernetes has this thing of, you know,
1108680	1113340	things should be able to talk to each other to facilitate, you know, microservices-based architecture,
1114040	1118340	which is great from, you know, a developer's point of view, where they can deploy their, you know,
1118340	1123020	series of pods and have them communicate in a way they need to, and it works.
1123060	1126020	However, from a security point of view, it adds a lot of issues,
1126020	1129960	because the pods in the cluster aren't only going to be the application pods,
1129960	1132540	they're also going to be, you know, a variety of management pods, etc.,
1132540	1136080	which if you can talk to, you can potentially do some dangerous things.
1137000	1141680	So, to kind of support locking down what kind of access pods have in the network,
1141680	1142900	there are network policies.
1143540	1146400	Now, network policies, effectively, are kind of like firewall rules,
1146400	1151000	where you can say, you know, these pods are allowed to talk to these pods or these CIDR ranges and nothing else,
1154440	1159940	and they're only allowed to, you know, get incoming traffic from these locations that are,
1159940	1162240	you know, permitted by the administrators.
1163460	1164860	So, it's kind of great.
1164880	1169440	Now, by default, there is no, like, deny rule or anything within network policies.
1169440	1173920	So, all the network policies are kind of like allow this connection.
1174240	1178180	So, the way it tends to work is if there are no network policies within the namespace,
1178620	1180060	everything's a flat network by default.
1180060	1184120	But as soon as you start adding the first network policy, it starts becoming,
1184120	1188180	you need to have a network policy allow that connection before it's permitted.
1188180	1191440	So, your default deny, instead of being a rule that says deny all connections,
1191440	1194400	is a network policy with zero rules in it.
1194440	1197080	And then you can kind of add the rules in as you need it.
1197880	1204280	Now, the Intel network, depending on the CNI you use, but most of them don't allow encryption in transit,
1204280	1205640	or they don't support it.
1206000	1208800	So, there are this concept of service meshes.
1209480	1213480	Now, service meshes, they typically provide encryption in transit,
1213480	1217640	but a whole bunch of other stuff like monitoring and other metrics that, you know,
1217640	1221400	your administrators can use to kind of help debug issues in the cluster
1221400	1224440	or just have an oversight of what's actually going on.
1224620	1229160	And the way this is typically done is through a sidecar container of a pod.
1229160	1231540	So, a pod is one or more containers.
1231580	1234220	So, this just would be another container in that pod.
1235340	1239300	And typically, these are automatically injected into the pod at runtime.
1239300	1246020	So, when that pod gets created, admission controllers are kind of used to inject those pods.
1246540	1251600	Now, admission controllers are functionalities within the API server that basically allow,
1251600	1257880	you know, modular-based additional features or checks to be added to that validation process
1257880	1259500	or request as they come in.
1259540	1264800	So, they can, you know, be used to write custom policies, for example, using OPA,
1264800	1271040	which is a policy as code tool, to provide custom checks that the API server can then use
1271040	1273320	to validate the data that's coming in.
1273980	1276140	Now, there are two types of admission controls.
1276140	1278080	There's mutating and validating.
1278440	1282600	Validating just validates the data and says, you know, this looks all right.
1282600	1283960	Yeah, I'm happy with this.
1284200	1287580	Mutating takes it a little bit further, where if it's not happy,
1287580	1291100	it has the option to change the request as it comes in.
1291100	1293500	So, for example, you have a pod coming in.
1293500	1295760	It feels like changing something.
1295760	1298260	It's well within its rights to do so.
1299360	1305220	So, the overall flow looks something like this, where when a request comes into the API server,
1305220	1309960	first, the API server would, you know, make sure you're allowed to do what you're trying to do in the first place,
1309960	1314700	to roll back and make sure, you know, you are a person or service account.
1314920	1319360	It would then pass it to the mutating admission controllers to see whether there's any changes they would like to make
1319360	1321100	and if they are happy with it.
1321540	1325940	Once all the changes are made, make sure it still meets, you know, the object schema validation.
1326160	1328780	So, make sure it's still a valid resource.
1329000	1333960	And then finally, passes it to the validating admission controls to make sure there are...
1333960	1341080	or run all the final checks and make sure it's still a valid piece of thing to save before saving it into etcd,
1341480	1344820	at which point the controller managers and schedulers can start, you know,
1344820	1347820	using that data to make the changes within the cluster as needed.
1349620	1352200	Now, it is possible to create your own admission controls,
1352200	1356640	but there are a whole bunch of inbuilt admission controllers that are there by default,
1356640	1358720	which can be enabled at the API server.
1358960	1361260	Now, there are a whole bunch of security admission controls as well,
1361260	1363640	but most of these tend not to be enabled by default.
1363640	1368780	So, it's usually manually need to be enabled in the API server if it's a self-managed cluster.
1369300	1372060	So, for example, pod security policy, which we're going to be talking about later,
1372060	1373440	or always pull images.
1374460	1380660	Always pull images is essentially a mutating controller where as pod specifications come through,
1380660	1386440	it makes sure that the image policy is set to always as opposed to the default of if not present.
1386940	1392140	This is especially important in like multi-tenant clusters where if it's set to if not present
1392140	1396880	and you have two separate tenants where one has permissions over an image and the other one doesn't,
1396880	1402520	the one that doesn't have permission over that image could use that image without having the relevant permissions.
1402520	1405020	If that image has been pre-pulled onto the node.
1405200	1410680	However, if the image pull policy is set to always, it would only allow the deployment of that pod
1410680	1415640	if that tenant could pull that image in the first place, which is kind of a way to prevent them using images
1415640	1416880	they're not permitted to.
1417700	1420240	And there's a whole bunch of admission controls like this.
1420320	1424700	In fact, having mutating and validating admission controllers, custom ones,
1424700	1428720	require the inbuilt admission controls for the same thing being enabled.
1428720	1438820	So there was an assessment we did a while back where we actually ran into OPA as a mutating admission controller
1438820	1441140	and within the network, it was the flat network.
1441140	1444320	So we could talk to the web front end for it.
1444320	1449920	And we found it in the cluster because the pod we had compromised had over-provisioned groups.
1449940	1455120	So we could start reading resources within the cluster because they had read-only access.
1455120	1460820	We could have also guessed the service name because it was just OPA.OPA, but that's boring.
1461760	1465880	Now, because it was a mutating admission controller, if we could get policies into that,
1465880	1468780	we could change the specification of pods in the cluster.
1468920	1472400	And luckily for us, there was no authentication for OPA configured.
1473140	1479480	So a quick policy pushed up to OPA and waiting for pods meant that we could suddenly start backdooring pods
1479480	1483780	in other areas of the cluster and start moving around the cluster in that manner.
1486000	1490240	Now, when talking about Kubernetes security, one thing that you kind of can't forget is the fact that
1490240	1491980	they are using containers under the hood.
1492460	1495480	And therefore, things like container breakouts do apply here.
1495480	1501020	Now, I won't be going into too much detail about, you know, all the ways that are possible for container breakouts,
1501020	1504580	but a couple of common ones are like, you know, leveraging host volume mounts,
1504580	1508160	where, you know, folders from the host file system have been mounted into the container
1508160	1513300	and you can kind of use those to add like a SSH key if you have someone's home folder
1513300	1519240	or if file run is mounted talking to the Docker socket if you're using Docker or all sorts of things.
1519680	1524540	There's also, you know, kernel exploits like DirtyCow if, you know, it's an out-of-date kernel version
1524540	1527300	or also overly permissive kernel capability.
1527300	1531480	So if, you know, the privilege flag has been used, you have the ability to load kernel modules.
1531480	1534380	So you can kind of load a kernel module, get onto the host that way.
1534380	1542480	And once you have that container breakout set up, because typically one node deploys model pods
1542480	1546080	on that same node, as soon as you've broken out one pod, you suddenly have access
1546080	1548980	to other pods shared on that node.
1549260	1553120	Now to help mitigate container breakouts, there's a pod security policy once you have
1553120	1556060	the admission controller enabled for such thing.
1556680	1560760	Excuse me. And that kind of helps mitigate a lot of container breakout attacks
1561740	1565880	by limiting what you can actually do with, you know, volumes, your user IDs that you're allowed
1565880	1568620	to run with, the kernel capabilities and so on and so forth.
1569380	1573680	Now, the way these are used is you would create a PSP or pod security policy.
1573860	1576340	And then through RBAC, you would assign that to your users.
1576340	1579600	And those users, when deploying the pod, would have to follow it, you know,
1579600	1584240	one of those PSPs if they... yeah.
1585040	1588400	Now, one of the things I typically recommend with this is, you know, using the system
1589260	1593560	authenticated group where you create the most restrictive PSP, you know, you can,
1593560	1596400	and then assign that to system authenticator at that point, you know, that should apply
1596400	1601880	cluster-wide. And then where more permissive pod security policies are required,
1601880	1605120	those can be, you know, assigned when and where they're needed.
1608280	1613020	On the same level, if you can't, you know, if you can't prevent container breakout
1613020	1616320	for whatever reason, and an attacker somehow does still manages it,
1616320	1619880	you can at least start controlling where pods are within a cluster
1620380	1623620	to kind of mitigate the impact of that compromise.
1623620	1626940	So let's say in the cluster, there are pods that have, you know,
1626940	1629900	just a standard basic brochureware applications.
1629920	1633620	There are ones that handle massive business critical data,
1633620	1636060	and all sorts of new management pods.
1636400	1640980	You can start categorizing nodes to, you know, whole different types of pods.
1640980	1644700	So you can have a whole bunch of nodes allocated to the brochureware applications,
1644700	1648120	all the stuff you don't really care about, a whole bunch of nodes allocated to,
1648120	1651780	you know, the business sensitive pods that handle very critical data that
1651780	1656340	you really don't want compromised, another set of nodes from the management plane,
1656340	1657560	and so on and so forth.
1657600	1660700	And these can kind of be enforced through tents and tolerations,
1660700	1664480	where you can apply these to the nodes and pods, and then a pod is only limited
1664480	1668940	where it can be deployed to based on these restrictions.
1669500	1672220	And then these can also be applied to the namespace layer,
1672220	1676760	where you can say that all pods within this namespace need to follow these tolerations.
1676860	1680600	And that way you can kind of, from a cluster administrative perspective,
1680600	1685040	manage what the impact could be of a container breakout should it were to happen.