Hello, everyone. So, please take a seat, grab a coffee, close the door. Do you hear me?
No, I'm sorry. It's a pre-recorded session. So, I can't hear you at the moment. As you
are listening, just record I'll be on holiday in the south of France in my swimming costume.
So, give me a question in the chat room. I'll be pleased to answer you at the same time.
So, hello, everybody. I'm Nicolas. Just a few words to present myself. I'm an independent
tester, security editor. I work for a decade in this cyber security industry.
Now, I've built my own company and I work in the red team of a financial institution in France.
So, as you see and heard, I'm French. So, sorry for my French. Thank you.
We'll talk about automating security operations. What does it mean to you and to me today?
Everyone has his own definition. For me, security operations could be related to
a bunch of activities. For now, please consider at least these activities. I will talk about
penetration testing, security control assessment, CTI, and DFI operations,
security compliance, and all about code review and web application testing.
We will see why and how to add automation within this activity.
And we'll talk about Patrol. Patrol is an open source framework for automating and orchestrating
tasks and security operations. It provides a solution to get a continuous and full stack
overview of your cyber exposure. The solution, sorry, lets you to define your assets,
policy, and the scan you want to perform. The scans give you findings. And all these findings
are then collected, analyzed, and aggregated within a unique database. We developed several
engines and connectors to existing security tools to assess risk on various security domains.
The idea is to get a risk overview from IP to data level. The scans could be start one shot,
scheduled on, started on the regular basis. And the final goal is to get a continuous monitoring
of our assets and our security posture on all stacks. All the findings, the findings could be
a confirmed vulnerability, a suspicious change in our systems, or a suspicious activity over
internet. These findings are then contextualized and tracked over the scans. And that's all for
me. Now, of course, it's not a tool presentation. I just want to tell you about security operation
and about automating SecOps. As I told you before, I work in a sys service since five years.
And from my windows, there are two major factors that lead current evolution of the IT landscape.
These are acceleration and diversification. This applies to assets, threats, and, by the way,
security incidents involved. We see an acceleration of digital program and diversification of assets.
So, thanks to the digital transformation program, we see an explosion of IT project. The information
systems are more and more open to the world, and then more and more open to hostilities.
We have to deal with new technologies every day. New product, new framework, new library,
and all these technologies are updated every day. We also see changes in the software delivery
processes. Remember a few years ago, when it was about four to five months of production per year.
Today, thanks to the hype of DevOps activities, we can see multiple go live in a day.
And a go live means a new application, and, by the way, new vulnerabilities or
new exposed things. The threats are growing too. The number of CVEs is growing year after year.
It's a metric. I don't know if it's a good metric to talk about the number of CVEs, but
it could be representative. But the attackers, they do a great job.
There are more and more, and they are more and more organized and efficient.
From a different point of view, we have to cover a quickly changing IT landscape,
and at the end of the day, it's increasingly hard to get a realistic, comprehensive,
and sufficiently updated vision of our cyber risk exposition.
We also have to face another problem. It's the talent shortage problem. We have not enough
people to do the job at the moment. Lots of tasks are repetitive, and this leads people to
lose their motivation and leave the team. We definitely believe it's time to adapt our cyber
defense and we have to adapt the way we do our job and we monitor our cyber security posture.
To face these challenges in the team, we try to manage security incidents with two goals.
The first one is for the red team. It's about identifying vulnerabilities on our assets
before attacker two. And for the blue team, it's to identify indicators of compromissions,
which could be past, current, or future of a potential security incidents. And to do this,
we have to keep us updated from many, many things. The first one is to keep us updated from the
continuous transformation of our assets. It could be more or less considered as shadow IT in big
companies. We have to keep us updated from the infosec knowledge database, the new research
publications, the talks talking about the new vulnerability or new way to detect certain things
or to exploit the vulnerability. And all the security news and the spectrum of threat scenario
is changing also every day. We have to manage lots of feeds of information every day.
And finally, we found that scanning our assets is not efficient anymore. We have to monitor
external resources to detect leaks, attack signals, and to understand how third parties see
our security posture. And for day-to-day work, it could be very hard to manage all of this
information. And the cyber security industry, it's a race against the clock.
And the third aspect we have to tackle is the window of exposure problem. It's all about
our reactivity. Today, we know that attackers will attack us, not just because we have a bank,
we are a gas industry, we are talking plenty. No, it's just because we are on the internet and
new vulnerabilities are found everywhere. It's just increasing the likelihood of attack scenarios.
So the windows of exposure is a real problem and could be handled with priority. We said
that we have to detect and fix the vulnerability and suspicious activities as soon as possible.
So facing the challenges, we think about automation and orchestration.
Just a quick reminder, my definitions of automation, it's setting a single task to run.
And orchestration, it's about automating a lot of things at once. It's about coordination and
management of automated tasks. Before we go, let's start our experience. A few months ago,
I set up a Kubernetes cluster with default configuration exposed to the internet. It
was unfiltered. Maybe you see what will be the next thing. Only 24 hours after, I was hacked.
CryptoMiner was deployed on my cluster and my server starts to mine some cryptocurrencies.
Forensic definitely thinks I was not targeted because I was near Nicolas.
Just because probably a scanner identified that a non-secure service was exposed on the public IP
and the attacker automatically deployed his payload on my server. So I don't blame him.
I don't blame him. He's doing his job. He's doing a great job for this point of view.
The fact we have to remember today, it's not that I'm a cheap DevOps, no. We have to accept
that attackers do automation and better than us in their field.
So why automating SecOps? The first thing is, for a defensive point of view, is to do more checks,
to cover a larger and diversified scope, to cover a bigger perimeter of assets and make more control
on each stack. And the second thing is to do it more often. It could be
continuous checks. It could be very useful to reduce the windows of exposure,
to reduce the delay in discovering and fixing a security incident.
The third thing I would like to say, it's about efficiency. As I told you before,
we have to face a big problem, a time-shortened problem.
The idea is to reduce the time affected to low-value-adding tasks, to focus on more complex
security cases. And for doing this, we have to automate the simple task. It's also a way to
reduce and manage costs and to start for low KPIs. It could be also very useful to
help you in your compliance and benchmark activities, to define and expedite the same
control on a subset of assets and do it continuously to see the trends and how far
you are compliant with your security standards.
So, at this time of this presentation, you should be all convinced of automation.
So, there are several downsides we have to discuss. Of course, there are limits. It doesn't
cover all of the risks in itself. If you automate your control, you will have an increasing number
of alerts to manage, an increasing number of false positives. We found also that it's very useless,
inefficient to found functional vulnerabilities. We also have to qualify and contextualize all
the findings we have, we have found. And about the TCO, yes, automating, we don't automate things
by magic. We use tools that orchestrate all the tools. So, we have also to manage and exploit
the tools. At the end of the day, a tool is a tool. And it's very useless without a cyber
defense strategy. So, if you don't have a strategy, don't try to automate things.
By the way, we decided to build a platform for automating and orchestrating the SecOps.
Because we wanted to improve our level of manageriality and to become more efficient to
adapt our work. The core concept is to efficiently moving from reactive to predictive, more or less
predictive security posture with the benefits from the power of automation.
And also, we decided to use... to don't develop our tools, but to use in priority the best of
built tools. The great tools exist, but they're not addressing all the stacks at the same level.
Issues of quality are very efficient to scan for vulnerabilities, for misconfiguration
on infrastructure and your cloud services. But the web application scanners and the
container security assessment and the anti-malware modules are not sufficient enough.
We found that we can have only one tool to cover all the security control we have to assess.
We found that we have to support scan policy, which are realistic from an attacker point of
view. And the idea was to take the benefit from the best part of
several security tools, making it easier to define a scan policy and to plan it.
And PathWall... I will talk about PathWall a little bit. PathWall is composed in two
independent types of applications. The first one on the left is the manager,
the front-end application, where you have your dashboard, you manage your assets,
you define your scans, you have your findings, and you try to manage the engines,
which are the micro-applications that perform the scan. All these applications are open source
and developed with Python. All features are reachable through the web UI or the REST API.
The PathWall engines are the probes. They are the micro-applications that perform the scans,
parse, analyze, and format the findings into a unique and pivoting format.
This could be deployed on several separate servers. We can scale it that way.
For example, you can deploy probes on your internet, on your internal network,
and probes on your administration or DMZ, which are your restricted networks.
And it could be the binding of your endpoint. And these PathWall engines are...
Sorry. They scan the assets. For example, we developed an engine for Nmap.
We don't really develop Nmap, but we made a connector to Nmap. And that way, we can perform
on the same assets security scans for using Nexus, Nmap, OpenVAS, Qualys,
and other security tools from the same cockpit. And all the findings have the same look.
And we can compare and track all the findings on these assets issued by the
separate engines we use. A bit deeper. PathManager, as I told, it's the front-end application.
Here, you define your assets and a group of assets. You can also define the scans policy.
You can schedule scans and manage the scan regime.
PathWall engines, it's the second component.
The PathWall engines are the connectors with the scanner which could scan the
assets, which could be on the internet or your internal network.
The PathWall engines could be also linked to an external scanning service.
Or a link to your CTI repeater. You can also create the tickets or
inject alerts or raise alerts to your DFI system. You can also inject the data in your sim
or on your ELK if you want to analyze or to make alerts on a different way.
As today, we developed a large range of engines in various domains. For each engine, we create
a Docker image including the tool needed and the REST API to deal with them. So, you don't have
to install tools, dependencies, or manage the system requirements. It's just as simple as a
Docker pool. It's true, except from several engines, like NMAP with the Docker image,
do not embed the scanner. But it could be linked to your instance.
So, on this slide, we see a lot of engines, a lot of various domains. The idea of PathWall,
it's a framework. You can build your tool to the detection of the control you have to perform.
I don't see many companies that use PathWall with all of these engines. It's more or less
separated between the vulnerability assessment or the data lakes or the static code analysis
or dynamic code analysis. We also have a lot of ideas for the nature engines regarding
gravity management, CTI, web application scanners, and so on. So, the engines are also open source
and we accept any contribution to create or to give us ideas to develop any engine.
Maybe it's time to talk about use cases. The first one is... I come from the right team. I
always do the same. The first step of the test is to perform the recon activities. So, we
search for subdomains. We always have IP. We try to discover the port, the open port. We
fingerprint these services. We search for vulnerabilities and so on. For this, we use
several tools with the same, more or less, the same settings on your assets. So, with PathWall,
we use PathWall in our security assessment to do this as quick as possible and to do this
continuously. The second use case is to examine the source code and the running web applications
for security defects. It's to be involved in the CI-CD pipeline.
So, on each commit, we take on the code repository. We are able to clone the project
and start a static code analysis using the one-day dependency checks or the
for the GS dependencies. And the code is built in and so on. And once the web application is
deployed on the staging environment or on the predictions, we can also orchestrate
autonomous scan using Arachne, ZAP, and NIC2.
PathWall is available through the REST API, and we also developed PathWall for PyClient,
which is a Python library. So, it's very easy to integrate with all those security tools.
The third one is about efficient preparation scenario. We use it to search for early signs
of malicious domains and the websites present. The idea is to search for suspicious domains
or type of cross-query domain. And once we identify them, we can monitor them to look
for changes. Are they still parked? Are they should certificate? What is the web application
look like? Are they new exposed services? And so on. And if we have any suspicious change on the
attacker's assets, we express an alert and we manage them.
The third use case is code leaks on GitHub. Many, many, many times,
DevOps or IT people are leaking something on GitHub because, I don't know, it's easy,
we don't really know how to use it. And we don't really know that it's a public repo. It's public.
So, we want to search for leaked internal resources code. API keys, password. And we
developed just a simple scrapper on GitHub to monitor our keywords and our patterns
to just detect that we don't have leaked any security system script or API keys
We will talk about use cases, but we found that when we automate all these security tools,
we can address a lot of use cases. So I want to talk about it.
Just step back a bit. If you orchestrate the security tools, you will perform more control
and do it more often. It will result in more findings. And it will result to more alerts.
And the security dashboard will look like a Christmas tree. That's very, very, very soon.
So, well, all these events are relevant, but we have to prioritize these things.
And it's the key challenge today, because we have the capacity to detect things,
to find out the false positives. We have the technology and the experience,
but we don't have the time to manage every, every alert. So we have to prioritize.
And if you permit, I will share my morning routine with the CSF team I work in.
When a new variability is discovered every, every morning, we talk about this and we share questions.
The first thing is when a new variability is discovered, we talk about the CVSS best score.
We talk about if we are vulnerable or not. Are we exposed on the internet with this
vulnerability? This vulnerability has been identified on a critical asset.
Are we aware of any functional exploit with this vulnerability? Is there any patch or
compensation measure available? Are there any likelihood catalysts? Is this
vulnerability exploited in the world? What is the media hype level? Has the
vulnerability been exploited with relevant threat actors? And we ask for the CTI team.
We also have the question, are we already found? And the team is in charge to investigate and
research us if we can. And the next question is, are we really able to detect exploitation of this
vulnerability? And at the end of the day, at the end of my morning, sorry, the managers say,
okay, we have enough data to initiate a crisis and to treat this in priority.
It's definitely a teamwork. It's not just become within the CTI team. It's really a teamwork.
All the IT and business service lines are involved.
The second thing is that vulnerability metadata are not static. They are continuously evolving
over the time. Everything changes when a new patch is available, when a new exploit is placed,
and when a new security resource is available. One event could change the way to manage a security
incident. And as you remember, we start using the CVSS BASC score. And we want to know today
if the CVSS BASC score is sufficient enough to be a primary factor of discrimination.
Just a quick reminder of the CVSS scoring. CVSS scoring, there are three vectors. The BASC score,
which represents the internal characteristics of the vulnerability. We also have a temporal
vector, which represents the characteristics of the vulnerabilities that change over time.
And we have another component about the environmental, which represents the
characteristics of the vulnerability within the client context. CVSS is the norm, is the standard
adopted. But the CVSS BASC score is usually provided, but the temporal and the environmental
scores are on our behalf. We have clues. We have information from many parts, which are not
always on the same page. It's just a score, a BASC score. For example, we have FunFact,
which was scored at 5, and Spectre was scored at minus 5. So we just have the CVSS BASC score
and the primary discriminator. We hope to see if it's sufficient enough to do our job.
So just go deeper with it. We found that we have to manage multiple criteria for prioritization.
And we have to manage the CVSS BASC score about the batch availability, the age of the vulnerability.
We have to manage the discovery ease and the detection ease. All of this data are available
from various sites or feeds of information. But all of these are publicly and in quite good
quality. For the criteria of prioritization, it's all about the threats, about the temporal metrics.
It's about the exploit availability, exploit maturity, and the ease of exploitation.
Also, the threat intensity is very useful to know because it's a sign of the maturity and
the likelihood of occurrence of the risk. With the MITRE ATT&CK and the CTI feeds
information, it could be useful to know about the threat relevancy. Sorry,
it's very hard to say that in French, remember. And it's excluded by monitors, protectors, or not.
And the third pillar, it's about the assets in itself, the renewable assets.
Is the asset critical or not? What is the exposure of the asset? Are we
vulnerable from an asset exposed from the internet or a restricted network? And what
about the distribution of the vulnerability? If we have a vulnerability with a high CVSS
without a functional exploit on the restricted network, it's not the real priority. But
a vulnerability with a standard middle CVSS, but exposed on the internet with an exploit available
and a large distribution, it's the top one priority. So we take all these criteria for
pre-stabilization and we have to make decisions to check if we have to manage this as quick as
possible or not. The first thing is, okay, it's very useful. We have to ask for immediate
correction and we start a crisis... we open the crisis room.
The second is, okay, it's urgent. We ask for an immediate correction and we
assess that the correction is efficient. The third action is, okay, it's a vulnerability.
We know about this. Apply the fix in the next patching campaign, but no more.
Finally, if the vulnerability matches no top criteria, okay, apply a fix if possible,
but we don't have any intention of this. We have to choose our battle.
The top of vulnerability, we don't manage this. For that, we are working on the new tool,
which is a battle here. The open sourcing of the... this application is currently in discussion. I
hope it will be the case. The idea of the battle here is to manage all these metrics.
And we use... we massively use the CVE search and the app of CVE tools, which are
open source tools released by the circle. The idea is to grab some... to collect and clean data,
like CVE, CPE, cross-references, to create and update vulnerability metadata
from this... from the NVD, the exploit DB, packet store, meta-exploit, Talos,
the enable DB, and so on. And to compute a vulnerability prioritization rating using
the vulnerability metadata. And the asset criticality and exposure. And this sector
are known by platform manager. Because in platform manager, we know that
we found a vulnerability on an asset exposed on the internet, on the internet network. And
we know about the distribution too. And the idea is to provide platform manager a rating
from the vulnerability found using any scanner as well. We can also use the battle here to track
changing on the changes, sorry, on vulnerability like CVSS, exploit known, and so on. And to
perform alerting, like if we found that... if we found a vulnerability on the monitored product or
product versions, okay, we send automatically an email, we send an event, we send a slack,
we open a Jira ticket, and so on. And finally, the idea is also to share feeds with using
platform and CTI. So we will talk about this one later. I reached the end of the presentation.
So if we can just have a step back and talk about all automating, let's say your seconds,
it's quite possible. The idea is to have a cost effective activities. We start to rational the
tool integration, the product licenses, and all the things needed to deploy and use
the various security scanners, we have to. The second one is to provide turnkey solutions.
Every component is available. So document images is very easy to use and they're deployed. We
provide templates for scan policy and so on. The third one is a patrol and all of the
tool galaxy is open source and easily customizable to your specific needs. And we
have documentation, we also have to improve the documentation, but everything is available.
And globally, it's a full stack and continuous assessment. The idea is to have a 360 degree
overview on your assets to perform a real time assessment with relevant data to keep you updated
from every source where we can. Finally, it's made with love by Express.
We and all the community, the patrol community is composed by security.
And finally, if we can summarize with two things, for big companies,
it's quite an opportunity to aggregate findings from different existing tools you already have
in place. And for newcomers and small organizations, you can bring new
capacities to quickly improve your security. So what next?
We have a roadmap, of course. We are working hard to improve our integration with
all the tools, especially two tools, DeHype, which is a security incident response,
and Umbroader, which is an IT automation and security compliance tool. All of them are
open source projects, very, very mature projects.
And we definitely want to have more integration with this.
We also try to improve the Python client API. We are currently redesigning the front end
of the platform manager. We are also testing endlessly new use cases in debugging, improving
quality and security, global security of the platform. And we also are building an enterprise
solution. But it's an open source project, and it will be always the case.
Contribution is really, really needed. So if you have the ability to test it and give us
feedback, it would be very, very great for us. And if you want to contribute or just to push
new issues, we will be very happy to have this. So we are at the end of the presentation.
So if you have any questions, please do it in the chat room. And finally,
I know it's a lot late to say that, but I really want to thank the DEF CON organization.
Thank you for accepting my talk. And thank you for you guys and girls to have attended this
session. Thank you. Thank you very much. And go for questions. Thank you.