We had this lightning talk available. The speaker didn't turn up. He had some issues. So we asked
a bunch of people, but then no one turned up. So we are, so Kunal will be speaking about a tool
called DataSplit. It's an open source intelligence framework. Okay, so over to you, Kunal.
Hello, good evening, guys. Audible? Yeah, hello. Yeah, what is? Okay. Okay, so we're
here. Hello, better? A little like this. Yeah. Okay, so here I'm to present DataSplit, an open
source intelligence tool. So we are the main three contributors to DataSplit. Right? So what is
DataSplit? Okay. Okay. Better now? Okay. So it's an automated
automation. It's a little bit more complicated. It's a little bit more complicated. It's
OSINT framework. So basically we have four major modules called domain, email, username, and IP.
So you feed in any of these things like domain test.com, email ID, an IP address, or username,
and then it fetches these informations from a lot of data sources that we have configured behind
all the modules. It's completely written in Python, and currently it runs either on a console
or you can export the output that's there on a JSON format. So one of the major things for
the major use case for this would be a pen test, probably, because let's say for a domain you get
a list of subdomains. You might be interested in, you know, finding which of those domains might be
vulnerable, right? So why data spread? So we have a lot of information that's available open source,
something like username, passwords that might be accidentally posted to GitHub, then address,
email ID, phone numbers, credentials, all this information, you know, might be available
accidentally or on purpose sometimes.
All right, so real time. So as you can see, these are some screenshots of how the information
is available. If you do a Google search for, let's say, a site, Pastebin, you'll find some
database leaks, you know, credentials that someone's done and then posted them online.
Then you can either do an index of search, you know, use those Google search terms to find
open indexes of different websites, then finding credentials on GitHub.
All right, so finding credentials on GitHub.
Then, you know, accidentally post. Then, so as I mentioned, we have four main components called
domain OSINT. So these are basically the four main scripts that you use to actually trigger the
various data sources. Domain OSINT, email OSINT, IP OSINT, and username OSINT. Then the work in
progress for us is, you know, company scoping, basically, you know, making a whole profile for a
company. Active modules, actually passing that information to active attack modules. Then CVE
data aggregation, the CVE data aggregation, the CVE data aggregation, the CVE data aggregation, the CVE data
aggregation, the CVE data aggregation, the CVE data aggregation, the CVE data aggregation. So these are all the
auto-generated reports. So as I mentioned, we have multiple data sources. So as you can see from domain,
we have about 10, 15 odd data sources. So this is how the data flows. From domain, you can get a lot of
email IDs from the email harvester module. That gets passed to email module. That, again, can fetch a lot
of information from these data sources. So it gets, let's say, enumerated usernames. So from those
usernames, you can pass it down to get details, profile, and then you can get a lot of information from
pictures hashtags the conversation that the user has been posting and similarly from a domain you
can get ip addresses which you can actually pass down to find out which all ports are open and the
services that are accessible right so how do you set it up so we have two modes one is a manual
mode which includes a git clone which is used to set up a standalone mode as a standalone tool
in itself and the second way uh so how do you do it you do a git clone install the requirements
there's a config.py that holds all your api keys because these data sources that we have
configured most of them might require an api key for you to configure to actually
uh use them then uh we have a structured uh structured uh you know folders basically
holding each of your scripts so as you can see on the right we have folders called domain emails ip
and then username so the domain has uh scripts specific to the domain osint and then we also
have
support pip recently so it can be installed as a python package and be used as a library in your
projects then we also have a docker implementation you just do a docker pull and you have a data
spread running then how do you write modules it's been recently converted to a framework
so there's a single file that you call the template.py in each folder it has three major
functions a banner a main and an output so banner displays some text about the module main does the
logic behind uh
uh uh you know the module some you know the processing that needs to be done and finally
the data returned from the main is actually what gets passed to the output for printing on console
in a you know human readable way then so we have a very nice uh documentation a very step-by-step
documentation on how to set things up in both manual and automated ways and then how to writing
your own modules and uh then the work in progress currently we support json we are planning to
support json we are planning to support json we are planning to support json we are planning to
support html reports and email uh text files that can be you know passed to another tools like
emails and subdomain lists okay so the demo just to add to that uh the documentation also have
step-by-step uh you know guidelines on how to generate your api keys how to sign up for
different services which we use here yeah it tells you exactly how which things you have to copy and
put back into basically the sites you need to visit how to sign up for them where to get an
api key and paste it in the config.py yeah so
is it visible at the back yeah yeah so as i am
domain domain no that's fine okay so uh let's say we have an ip address you just pass it to
the ipos in framework you can see it first searches for shodan so shodan has no information
then virus total whether or not this ip address was listed as a you know compromised domain and
then some who has information to get the as uh asn the autonomous system number to actually find
out more uh related uh ip ranges for this so before we go ahead like suppose if you have got an email
address okay you're tracing someone and you've got an email address what kind of what kind of
information would you like to collect about any pointers
yeah anything else you would like to like find out about a person you've got an email id of a
person yeah
yeah good sorry what was that true name good exactly yeah so what we're trying to do with data
exploit is you know it's it's no rocket science we're just the things which you'll do manually
going on one site and finding out information we're just collecting bunch of such sites calling
those apis getting it sorted like it goes in an automated manner there is a difference uh which
data split brings in comparison to other tools because a lot of other tools you know uh they
yeah so a lot of other tools you know they are really good but one thing which i really miss in
other tools is like they do not have an automated approach you have to manually pick and use things
sometimes you don't really know what modules you should use i mean for a person who do not
you do ocean on a daily basis it becomes very difficult yeah so that's the problem we are
trying to solve here we're trying to do everything in an automated manner so even if you do not know
anything about ocean just configure your config configure pi and it will be fine so that's the
it will do the things for you yeah so it gathers the relevant information displays them all you
know very clear text format so as he mentioned so let's say you get a domain test.com and you need
to find out what all information is available we have all this information punk spider the who
the dns records that are there the email address harvested related to that domain so these email
addresses you might be interested so let's say you find uh the combination of first name last
name so you can figure out what kind of format the organization uses to actually name their
employees then you can you know actually brute
force them so after this then you get uh so basically where all this domain has been mentioned
on github so let's say files where you know they might be username passwords or aws keys let's say
suppose so you can see the files the specific file where this domain name was seen and
okay and then uh some page links then the paste bin so as you can see for test.com you can
have something like a email id and then a password
so probably this email id password will mostly work on different websites that you find related to this email id so uh finding things on github and paste bin is something which i always do in my pen test you know so whenever i'm doing a pen test i come across any domain name i always make sure that i search these things on github and paste bin so you never know sometimes you might
directly get a db connection string or you might get a domain domain credential so you don't even need to explore a vulnerability so it's pretty easy for me so yeah yeah then uh
So that domain then gets searched on Shodan
for, you know, the ports that are open, as you can see.
So you might find something like open,
an SSH open on an IP address related to test.com.
So that might be exploitable.
And then all that information, AD443,
what services are running, what's the geolocation for that.
And then, so that's, like, a lot of information.
Then the subdomains that are there.
As I mentioned, this might be interesting
for a pentester guy to actually find subdomains
that are there and might be exploitable.
So not only pentester, but even if you are running a company,
you are managing security for your company,
you may check your parameter.
Sometimes what happens is there are some legacy subdomains,
like abcd.xyz.com, and you are not aware of them.
So most of the time, legacy code is very much vulnerable to things,
and it gets very easy to pop up a shell out of it.
So it's probably worth a while checking all the domains
which are lying around.
And then we also source that domain
for any WikiLeaks documents that are related to that domain.
Then ZoomEye, similar to Shodan,
gets all that IP-related information,
the services that are running,
the header information that gets returned from those IPs.
Then the next module we have is an email.
Yeah, it's the wrong one.
Pause the username.
Yeah.
So let's say a username.
You get a username.
And so Keybase is a very good source of information
where a user actually has to verify his own accounts.
So the information you get here is actually the profiles
that it belongs to the same user.
So you won't find data that is mismatching
or, let's say, false positives.
So here, this might be a very good information
to find out all that accounts that are listed for this user.
So let's say you found this username previously
on, let's say, a pastebin
and has a username-password combination
in a pastebin document.
You can go ahead on these sites and try that out.
So obviously, that opens up a very good opportunity for you.
And another thing which this could be used is
a lot of people verify their devices
and they leave their device name as they are.
So what happens is you can actually use those device names
to launch a phishing campaign
or trick the user to do something stupid.
So as I mentioned, we also allow you to...
Sorry, yeah, question.
So it's like, we're not grabbing from anywhere else.
The person generally verifies their devices on Keybase.
So the Keybase have an open API.
You don't even need authentication for that.
So it's pretty easy to get it out.
Sure.
You're doing a lot of stuff with, like, public dumps
and all the cross-correlation.
How are you verifying that any of this data is still fresh?
Okay, so yeah.
Yeah.
That's a valid thing.
I mean, so as of now,
we are completely relying on information which exists online.
But I guess I'm just curious, like,
are you doing active checks or are you just...
As of now, no.
As of now, no.
Because, you know, we thought of doing that,
but it becomes really tricky to verify
because we will have to do a lot of correlation.
We have to do match images and stuff.
Yeah.
So you make, like, a best effort.
Yeah.
Right now, we are making the best effort,
but we are trying to work on those things.
Yeah.
Follow-up question.
Are you doing anything to actively notify users
that you're leveraging...
Exactly.
So if you...
We'll be coming on the roadmap.
Those things are in the roadmap,
so you can schedule the things.
I'm writing it out.
Okay.
No, we don't have, as of now.
That's awesome that it's on your roadmap.
That's fine.
Yeah.
So as I mentioned,
we also support DataSploit as a library.
So let's say if you do a simple import DataSploit,
you have it available to use in your own Python project.
And let's say if you do a DataSploit,
I'll just show a quick demo for it.
.username.
That's our main module for...
Oh, sorry.
Username.
And then let's say the module that's in there
is called username underscore git scrape.
Right?
And then we have a main function for each module
that needs to be implemented.
And then you pass in the actual username.
Let's say I'll pass in my colleagues.
Oh, this is not going to use it.
That might, yeah.
That might use it.
Yeah.
Just single it.
Single it.
Oh, so user...
Typo.
User...
What's happening to your keyboard?
I don't know.
You messed it up.
You just messed it up.
Okay, so he hasn't defined any API keys for it,
but let's say if he had,
you'll have that data returned to you
in a native Python object,
probably a dictionary, a list,
whatever, you know, is relevant to that module.
So for git scrape,
you'll probably find the repositories that the user have,
and in those, you'll find the commits.
So you can find out which all are the most active,
you know, repositories that the user contributes to,
and probably in that, you can find something,
you know, similar like a password username combination
that might be there.
Yeah.
And then...
The last one might be...
So similarly,
so rather than, you know, using the consolidated script
called domain OSINT or even email OSINT,
the individual modules themselves are callable.
You can just say a Python domain,
then the specific...
the specific module that you need to call
domain-subdomains.py,
and then just pass in the domain name that's there,
you'll find all the specific subdomains that are there.
Quick question.
Yeah.
What are you using for your subdomain integration?
What are your sources for your subdomains?
Okay, so for subdomain,
right now we are using DNS dumpster,
we are using NetGraph,
and we are using certificate transparency from Google.
And we plan to introduce LDS walking later on,
but that's in the plan.
I have a question.
Yeah.
Do you guys ever run into problems with, like,
connection limitations?
Yeah, yeah.
So most of the APIs might have a throttling there,
so they obviously return, you know,
rate limit exceeded, something like that.
And here's this one script called emailOSINT.py.
No?
Okay, I'm sorry.
Sorry, was that your...
what you were asking?
Are you making an effort to subvert that?
So we can't,
but obviously we can, you know,
get in touch with the, you know,
specific...
Yeah, exactly.
So about the emailOSINT,
this is what we do.
Yeah, so...
So the very first thing,
it will check for some basic things.
For example, if you are running a phishing campaign,
it will tell you whether it's a disposable email or not,
whether this domain can receive emails or not.
So if you are trying to send emails,
it will tell you those things.
It will tell you search information from Clearbit.
It will tell you a bunch of information
about the person, the name,
the tentative location,
or the profile, or whatever things.
So as mentioned by you,
so for an email, you get the true name that's there,
the usernames that are there.
Then it goes to full contact.
We are not relying on just one source.
There are other sources called full contact.
We're using them.
It tells you the name.
It tells you the designation of the person,
social profiles.
We have got, like, 500 profiles,
which we check for.
But we are trying to get more.
And then Twitter,
and it will download all the images
from these respective places.
What you can do is you can use these images
to do Google reverse image search and stuff.
So you can get more information.
Then it goes and checks on have I been pawned,
whether this email ID was a part of a compromise or not.
So as you see, my email ID was part of three compromises.
So what you can do is you can go back to, you know,
other places where you can get these database dumps.
You might get correct thread hashes or passwords.
And then there might be chances
that I'm using the same password somewhere else.
So it gets an easy win.
It checks for paste on other places.
So right now my API is expired.
So, yeah.
And then skip documents.
So that's probably what you were asking, right?
Slide shares.
When API send us a forbidden.
Slides.
Any slides which I published
and, you know, associated subdomains for me.
So this is the information
which you are getting from email ID.
I think that's it.
That's what we have for the demonstration.
We have a Twitter account called DataSplit
where we post in tweets and announcement about this.
The same feed goes into Facebook.
Facebook also.
This is a small roadmap.
Which we think of.
We plan to.
We are not lazy.
We'll try to get this coded.
It's like an ad hoc tool.
So you run it as and when you want.
So probably you want to integrate it with automated tools.
So you want, let's say,
incremental change that's happening
to find out the difference between a previous run
and a next run, right?
Yep.
Yeah.
I don't know if this is a dumb question or not.
Do you plan to integrate
any kind of a multi-go platform or anything like that?
Yeah, we can.
I mean, we can.
What we can do.
I mean, there's no.
There's no point writing the same code for multi-go
because any of you are writing in our tool.
What we can do is
we can write code which generates an output
which could be directly used by multi-go.
I mean, that would be a better way.
Otherwise, there's no point reinventing the wheel, right?
So, yeah, we can work on that line.
So far, the tool has been covered on multiple places,
but I don't want to brag about it right now.
Yeah.
But the point what I wanted to raise
is how you can contribute.
You can test the tool.
You can write blogs about the tool.
You can promote the tool.
So now, since it's a framework,
we, you know, encourage you to actually
go through the framework that's there
and actually implement the function.
So, you know, write modules, send us a pull request.
We'll try to integrate it there.
So, we are the core contributors.
I'm Shubham.
So, Dhanushu, Kunal, and Nuthan is missing.
He's in the other room.
And if you've got any questions,
please shoot.
That's it.
Thanks.
Thank you.