start	end	text
640	8000	The first time the Packet Hacking Village ran a talks track was at DEF CON 21 in 2013.
8720	15980	Each year since then, Mike Rago and Chet Hosmer has given a talk at the Packet Hacking Village.
16040	23340	However, they have not given the opening talk at the Packet Hacking Village until now.
23380	29460	It is my pleasure to introduce to you, old friends and supporters, Mike Rago and Chet Hosmer
29460	34720	with their talk, Media Analysis of Disinformation Campaigns.
37330	40670	Hi everyone and welcome to DEF CON 28.
41010	45730	We are going to be talking today about media analysis of disinformation campaigns.
45730	50170	I'm Chet Hosmer. I'm going to do a quick introduction of the talk and then I'm going to turn it over to Mike Rago
50710	53650	who is going to kind of dive into this topic a little bit deeper.
53650	55930	But I want to at least set the stage for you.
55930	59270	First of all, Mike and I have worked together for many, many years.
59270	69370	We created a book together called Data Hiding where some of the information here is relevant or related to the work that we've done in data hiding and steganography.
69370	73210	So, I want to kind of give you just a quick introduction to the talk today.
74630	79370	So, the threat. You know, what are we worried about here with these disinformation campaigns?
79370	82370	And we're going to focus a lot on media today.
82370	87750	In other words, multimedia content that is included as part of these disinformation campaigns.
87750	94990	So, today the internet and social media is filled with a lot of multimedia content.
95070	100690	Most of this is relatively harmless and in many cases can be quite useful.
101290	108490	These multimedia events basically communicate knowledge in the forms of training and information sharing.
108490	111810	New ideas, new concepts, even new thinking.
111850	114000	And of course, pictures and videos of our pets.
114000	132720	However, a growing percentage of multimedia content have been manipulated in order to deceive and change the way we think, people think, act, react, believe, and can ultimately cause harm because they're fake, because they're untrue.
132720	145560	The simple question then is, how do we tell the difference between real and fake, fact and fiction, and I guess ultimately, good and evil?
146400	152600	So, with that quick introduction, let me turn it over to my good friend, Mike Rago to kind of take you through the next section.
152600	159140	So, we thought we would begin by talking a little bit about the different types of disinformation campaigns.
159140	164300	There's certainly a variety of disinformation campaigns that we see in today's world.
164360	178560	Certainly with COVID or the coronavirus being top of mind for many folks, there's a variety of disinformation that is inspiring mass hysteria across people all around the world, including here in the United States.
178560	194730	And that's inspired by a variety of media, video, images of people suffering from the coronavirus, remedies that are available for it, cities or locations that have been heavily impacted by the coronavirus.
195280	202100	And as a result, inspiring a lot of concerns and risks associated with that.
202100	225980	In addition, there continues to be a huge uptick in political disinformation as well. We've seen with politics in Brazil, here in the United States, and other locations all around the world, where we have different candidates spreading different forms of disinformation, as well as various groups and organizations and individuals also inspiring that as well.
226780	234800	Some of the less familiar ones that we see out there, but still very impactful, are things like catfish fake rallies.
234800	243620	This is a technique used through social media to actually inspire a rally, one that hasn't been planned yet.
243620	253800	And so someone will create a fictional event and post it across Facebook, across Twitter, other social media outlets.
253800	266440	And then in doing so, to garner attention around it, the creator will then send things like private messages to other social media users, asking them to attend and repost it.
266820	277700	And then this is eventually turned over to a larger group whose message is inspired by this, to have them also promote it to a broad base of users.
277700	290860	The intent behind this type of thing is to actually form a rally in some cases, and in doing so, support a specific narrative supported by a large organization who supports that narrative.
290860	306380	And the underlying tone being, in a way, kind of manipulating people and really getting them to join a particular rally, all based on an original fake type of catfish.
306380	321380	And then other things that we see also involve malware. So in the case of malware, we've seen things such as phishing or vishing attacks, where fake audio is used to impersonate an executive.
321380	334920	It may be spliced together, different pieces of audio from recordings that have been found online and elsewhere, to create a message that is sent to one or more executives, someone in the accounting department,
334920	345300	or to customers of that company or that organization, informing them that, say, the account information that needs to be paid has changed.
345300	353780	The bank information has changed and inspired them to make payment to a particular account different from the original they have on file.
353800	360320	So a lot of these information campaigns inspire a lot of different narratives and a lot of different results.
360320	369980	Again, kind of our focus here in this particular presentation is actually taking a closer look at the media tied to a lot of those disinformation campaigns.
372820	380180	In terms of political disinformation, here's just one of many examples that are out there.
380180	391570	In this particular one, a congressman actually tweeted a fake photo of Obama meeting with the Iranian President Rouhani, when actually this was an altered image.
391820	404920	And the original image was really of Obama meeting with former Indian Prime Minister Manmohan Singh to talk about nuclear laws and things of that nature.
404920	421360	And so it was soon discovered that not only as a result of the photo, but furthermore, the placement of these people at different points in time was that they didn't actually meet at the time at which this photo was representative of them actually meeting together.
421360	425740	So based on the timeline, these two people were not in the same place at the same time.
425820	433580	But furthermore, upon further analysis, it was determined that this image was actually fake and that someone had modified it from the original.
436580	448220	Some of the research that we've done involves leveraging many of the cool capabilities available within AWS, such as recognition and the ML built into that.
448220	458480	And in doing our research, we've created a number of systems that allow for deep analysis around these types of things.
458480	478180	For example, taking this particular image and running it through AWS's recognition and machine learning, based on some modifications we made, actually allows you to identify some very interesting characteristics to help identify when there are altered forms of media, in this case an image.
478180	487440	For example, in terms of identifying items and objects within the photo, provides the ability to understand context.
487440	497960	And in this case, you can see that there are individuals that are identified, various flags, accessories, person wearing a pin or a tie.
497960	510400	And those things become more important in building out the broader picture and trying to understand when an altered piece of media, in this case an image, is actually used.
513320	525720	In addition, it can also bring out facial attributes, where you can see and understand expressions, and in addition, understand what people are wearing within the photo.
525720	534840	And in addition, things such as whether they're happy, they're sad, and other types of characteristics related to their facial attributes.
539390	554290	If we go back to the original photo, the one from which the altered image or the fake image was actually created from, and we run that through the tool as well, we can further identify other characteristics that we can compare to the fake photo.
554490	566770	In this case, we additionally identify other characteristics, not only beyond the fact that there's a different individual in the photo than in the fake photo, but also characteristics of that image.
566770	584710	Interesting aspects of this image was a different color turban that the individual is actually wearing, and how that was actually... pieces of that were left over in the fake photo when splicing in a different individual within the photo itself.
584710	603850	In addition, there were kind of blocky components of the fake photo related to the individual that was spliced over and into that fake image, whereby structures of the face were along the line, and some other characteristics that came out of some of the analysis too.
606470	634830	So, interesting aspects related to this. In looking at traditional media and trying to understand some of the metadata and characteristics of those images involved looking at metadata and really understanding the photographer or the source of that image or what camera was used to take that photo, and even sometimes some of the location information included in that metadata.
634830	657750	But as we've demonstrated over the last number of years, actually the last roughly 20 years that we've been presenting across DEF CON, Sky Talks, Wall of Sheep, and other conferences as well, is that, as we know today, social media largely strips off a lot of this metadata, and furthermore, recompresses media, whether that be an image, audio, or video.
657750	666690	And through that recompression, actually further morphs and changes the content of that actual media.
667310	686070	And therefore, really in today's world, getting beyond the metadata and really looking at many of the characteristics we've shown so far becomes increasingly important in terms of doing this deeper analysis of altered media, in this case, images of which we talked about up until now.
691150	700210	We stood up a different ML type of environment within AWS that leverages video analysis, too.
700330	713330	We took a video from a presentation we did in DEF CON 25, where Dr. Phil Tully and myself presented a picture is worth a thousand words, literally.
713330	720410	And so that presentation garnered the attention of CNN at the time, and so we did a TV interview.
720450	727950	And so I took that video and ran it through the video analysis tool to see what type of things it could identify.
729250	741770	AWS recognition and the other things that are built into many of the capabilities allow you to understand things such as what objects are in the video, but furthermore, how often do they show up?
741770	745570	And where in the video timeline do they show up?
745570	760250	A lot of these so-called forensic data points can be really helpful and really important in terms of building out and understanding analysis around disinformation campaigns in general,
760250	777470	and the types of things within those images that may identify the source of that image, understanding and placement of people or objects at different points in time, a video that may claim to be recorded in New York when actually objects in the video are located in San Francisco.
777470	780710	So there's a lot of interesting characteristics that can come out of this.
785330	790910	One of those things is being able to look in more detail at those objects.
790910	804270	And in doing so, you can see here that it'll pull out and identify a plethora of different objects, how frequently they show up, and where in the video timeline do they show up?
804270	807990	And this can also be really helpful from a forensic standpoint.
807990	817010	Traditional methods, we would take a look at an image or we would take a look at a video and try to identify interesting characteristics kind of using that human analysis.
817170	832570	Now is the ability to actually pull that out in an automated way, leveraging machine learning and leveraging all of that deep analysis to bring out a lot of those characteristics across OCR, NLP, and many other things.
832750	835930	And this also does work with images as well.
835930	839530	Remember that a video is just comprised of multiple images.
843090	847670	It'll also leverage things like optical character recognition or OCR.
847670	852350	And this allows you to pull out words from objects identified within the video.
852350	862570	This could be things like a license plate or a sign or a logo on the side of a vehicle or a building or something else, even a logo on somebody's T-shirt.
862570	876870	These all become really, really important in terms of understanding all the data and the analysis of that to understand things that have inspired the disinformation campaign and characteristics of that video or that image.
879790	886710	One important aspect to this, though, is that old school methods still apply.
886710	909010	And what we mean by that is old school methods of analyzing images, analyzing things, and understanding when there are different types of text within an image is that with old school methods, we could actually take a look at a black background and identify where there's embedded text within that.
909010	918330	So if there's only one or two shade difference for that text compared to the black background, can OCR actually identify that?
918330	923730	What we found is that actually it does not do a very good job of identifying that.
923730	936230	So blending kind of those old school, a little bit more manual analysis techniques with current day aspects of OCR combined really provide a much better strategy overall.
937350	942530	But that said, AWS's Textract is actually super awesome.
942890	950910	There are a lot of features and capabilities in here that would take a long time trying to accomplish using old school manual methods.
950990	962570	So actually using this to perform the OCR analysis can allow you to bring out a lot of the aforementioned objects and text and things like that within an image.
962570	967090	And as a result, help you identify those.
967090	974310	But if there are minor subtleties, such as in this case, text on a black background, it may miss that.
975590	988490	And that's something that we actually tested using AWS Textract is that when taking this type of image and putting it into the tool, it simply just didn't identify any text within that particular image.
991320	996520	But again, kudos to AWS. I mean, Textract is a fantastic tool.
997560	1005620	In addition, we can also do video analysis and extract the audio. And in doing so, there's a lot of interesting data points there.
1005740	1018960	The speech recognition built into what AWS provides and what we build out in the tool set allows you to identify those key phrases from the transcript of what is being mentioned during that video.
1018960	1030420	From here, you can identify the frequency at which much of that occurs, such as how often are certain phrases or words used throughout the video.
1030700	1041400	And in analyzing that, understanding what type of lingo does that individual normally use versus what's actually being said in the video.
1044790	1065080	Also, the speech recognition can identify other characteristics, such as people's names and things that are mentioned within the video, dates, countries, quantities of items, and things of that nature too.
1065080	1078120	So what may be really helpful is that if there are multiple individuals talking and some of it's in the background, some of it does have the capability to pick up on some of that as well and really provide analytics around that.
1078120	1085460	And also in the context of where that audio or that word or that phrase was mentioned in the timeline.
1087540	1099160	So bringing this all together, the video analysis and going beyond just deep fakes in general, disinformation is more than deep fakes.
1099160	1107540	And what we mean by that is there are a lot of other attributes we've talked about so far related to a video that can also be key.
1107540	1120980	So although there's some great AI around understanding and identifying deep fakes, also all of this additional analysis is super important into understanding the fake narrative.
1120980	1137860	These attributes, as we've mentioned, that drive these false narratives may be individuals that are in the video or objects, any type of identified words or lingo, and facial expressions and even human characteristics.
1138200	1147180	And this is particularly important in today's world of social media where metadata is less and less found across the different forms of media.
1151230	1160430	In addition, if we kind of take a look at the old school steganography problem, is there something embedded in this image?
1160750	1172790	The big challenge that's always existed is, well, I don't have the original file before that data was embedded in that image, so it becomes more difficult to identify that.
1172790	1187090	I can't do a hash of the original and a hash of the modified image with the hidden data in it because I don't have the original image. Same thing goes for a diff or other types of techniques that you could use.
1187090	1198330	But although hashing may not work in these circumstances, ML, NLP, OCR, facial and object recognition, all of these things do actually work particularly well.
1199330	1213490	And understanding the behavioral changes of a social media account or website over time is also super, super helpful in understanding the timeline and behavioral changes across that social media account.
1214570	1233470	And also, in addition, is the source of the additional media and being able to collect that over the timeline and actually run that through the machine learning to build out a profile for that individual, for that account. What are the normal behaviors and do we see a deviation of that?
1235030	1242370	So social media accounts can change over time, especially when they're taken over.
1242370	1253470	And so in this particular case here, this is just one of numerous examples where an innocent individual had their account taken over and turned into a zombie account.
1253470	1264390	And whoever took over that account then used it to inspire disinformation campaigns by posting a lot of disinformation around the coronavirus.
1264690	1279190	And over that time frame, characteristics of how that account changed was not only the background, but also the name, the language, and a variety of other things in terms of the type of content posted.
1279190	1295290	So monitoring this over a timeline can help gather a much more broad corpus of media to really identify characteristics of the disinformation, characteristics of how it's been weaponized, and characteristics of their TTPs.
1295290	1297390	And with that, I'll turn it over to Chet.
1299070	1302530	So let's take a look at some sample fake images.
1302530	1312330	I know we're so interested in audio and video and other things, but remember, videos within multimedia are just made up of images and audio clips, right?
1312330	1317530	So typically JPEGs and MP3s within, let's say, an MP4 kind of environment.
1317530	1327850	But let's take a look at some images and we'll talk about how we break those down and how we actually approach the process of detecting anomalies within.
1327850	1331550	Before we do that, we should take a look at a few fakes to see what they actually look like.
1332210	1337230	So here's one. This is actually a really good one because it was actually produced by a studio.
1337230	1341950	So this was actually produced by Paramount Pictures in the Baywatch movie.
1341950	1348190	So we've got Zac Efron and Dwayne Johnson here kind of walking away from a beach scene.
1348210	1354150	In this particular clip, neither one was actually in the shot together.
1354150	1358270	They were separate and they were pasted together with this particular background.
1358270	1360430	So we're going to take a look at that in a minute to see.
1360430	1366230	Now you notice this particular image with Dwayne Johnson's tattoos and that kind of stuff.
1366230	1371450	You would expect us to be able to get some false positives, let's say, within those particular areas.
1371450	1374090	We're going to talk about how we can actually avoid those.
1374530	1381570	This is a really old one. This was the cover of National Geographic magazine back in February 1982.
1381570	1386410	And I bring this one up because this again was produced by a professional organization.
1386410	1394310	What they've done is move the pyramids closer together and added the camel riders to the foreground of this particular image.
1394310	1400590	They received significant amounts of criticism for their modification of this image.
1400590	1408190	This happens a lot, right? When photographers are taking photographs of scenes, they want to enhance their impact.
1408190	1413270	In this one, the impact and the enhancement may not have been that bad.
1413270	1418290	In other words, again, the fact versus fiction issue.
1418290	1425470	But it's still the same thing. They've tried to enhance this image to give us an impression that isn't true.
1426690	1429270	Here's one that's a bit more serious.
1429370	1432290	This was during the time of Hurricane Sandy.
1432290	1441070	And someone put together this particular image of the Statue of Liberty in the foreground and Hurricane Sandy approaching New York Harbor.
1441070	1453510	Obviously, this didn't happen. However, it caused a bit of panic because this ended up as a photograph on a couple of the evening news segments and obviously on the Internet.
1453510	1458390	So people were looking at this and thinking, wow, this is right on top of us. What should we do?
1458530	1466450	So again, these kinds of fakes can have impact on people's lives as we take a look at them if we believe them to be true.
1466450	1469390	Many enhancements were made to this particular image.
1469390	1474230	Obviously, this is a really good one. Obviously, this is a real image of a flying saucer in the desert.
1474330	1479870	We'll talk about this one as we actually walk through the demonstration to talk about how this was in fact created.
1479930	1483810	But again, there's many different ways that we can create these.
1483970	1494330	This is a really good one. This is a selfie of a pilot flying at 300 miles an hour, sticking his head out the cockpit window and taking a selfie of himself.
1494330	1500830	And you notice that it has the plane, the engine, and many different parts of this were actually factored in.
1500830	1504750	The glasses, the tie, flying in the wind, all of that kind of stuff.
1504750	1510770	And we'll talk about how this was actually done and actually break this down for you in a couple of minutes.
1510870	1513610	So some things a little bit more serious.
1513610	1520910	Obviously, this is Vladimir Putin supposedly whispering over Donald Trump's ear, kind of giving him some advice.
1520910	1525590	And obviously, this never happened as well. So we're going to break this one down for you and kind of show you.
1525590	1531510	So this is becoming a little bit more political in nature and how they're being used in this regard.
1531510	1538670	It's a really good one was created with machine learning to basically bring these two photographs together and make them look realistic.
1539510	1549790	Another political photo with Vladimir Putin shaking the hands of Donald Trump with Merkel in the foreground as well and a lot of folks here.
1549790	1558170	And obviously, we're going to talk about how this image was created and show you where some of the modifications have been made to this particular image.
1559830	1565130	So what we want to talk about is data hiding versus deep fakes.
1565130	1573750	So in other words, we've done a lot of work in data hiding detection, steganography detection in all forms, whether it's images, multimedia, etc.
1573810	1578490	But how does data hiding relate to deep fake?
1578490	1581390	In other words, how do these two things actually come together?
1581390	1589330	Let's take a look at some of the aspects of steganography first, and then we'll talk a little bit about deep fakes.
1590070	1598070	So the etymology of steganography, it comes from the Greek roots steganos for covered and graphi for writing, yielding covered writing.
1598070	1600870	So steganography is all about covered writing.
1600870	1605770	And some of the first known uses of this happened 4,000 years ago.
1605770	1613630	So this has been a technique that has been used in warfare for pretty much ever in order to be able to conceal information and communicate information.
1613630	1619990	So remember, the difference between encryption and steganography is an important thing to remember.
1620050	1621630	We're going to talk about that.
1621670	1624030	The story of Demardus is one of my favorites.
1624030	1631810	Demardus was exiled in Persia, and while he was there, he received news that there was going to be an invasion of Greece.
1631810	1635930	He wanted to get back in the good graces of Sparta, so he wanted to get words to them.
1635930	1643730	So he took the tablet of the day, which was a wax tablet where there was wax covered over a wood background.
1643730	1648570	That's the stylus that you see at the top, so it's kind of like an iPad of the day.
1648570	1655410	And you would actually write your message within the wax and hand it off.
1655410	1668570	Well, since he knew that if he tried to get a message out to Sparta this way, they would detect it as the guarders of the gates of where he was exiled would certainly have picked this up.
1668570	1678370	So what he did was he peeled the wax off the tablet and he engraved the message, the secret message, in the wood of the tablet and then passed it by the centuries of the day.
1678750	1681270	He was able to get that by and get the message to them.
1681270	1686850	So it's the physical form of covered writing, of hiding a message underneath something else.
1686850	1694550	In this case, the cover was wax and the writing instrument was a wax tablet that was in fact created.
1695750	1704530	We take a look at images. This is one of my favorites that I use in many of my presentations because it's an image that everybody knows.
1704530	1710410	It has a couple characteristics that make it really a terrible image to use for hiding information.
1710890	1717790	The image on the left is the original. The image on the right has a fairly significant word document embedded in it.
1717930	1723590	And the reason it's a terrible image is it's one that we all know. We have an expectation of what this is going to look like.
1723590	1728250	We've all seen this image. It's probably one of the most known images in the world.
1728610	1735130	The second is that it has very limited colors in most of the dark shades. Those are not the kinds of things that you use to hide information.
1735130	1742770	However, even so, these two images side by side, the original and the fake, look pretty good.
1743270	1747530	So what if we take a look at these two images in a little different way?
1747530	1753250	In other words, let's render these images differently to see if we can actually see a difference between those.
1753290	1761730	Since this is actually a true color image, the data hiding occurred in the least significant bit of the RGB spectrum.
1761730	1766150	So let's take a look at that from that point of view and see what we get.
1766150	1771970	This is again the original image on the left and you notice if we render just the least significant bit,
1771970	1780730	we find there is in fact information in that least significant bit in the original image that basically is providing the shape of the image.
1780730	1785150	Even though it was only a single color, in this case the red LSB value.
1785150	1792250	If I look at the red LSB value in the image that was staggered, in other words, data was hidden in it,
1792250	1798890	we see the information is completely different because that least significant bit was used to hide the information.
1798890	1806890	So how we render the image and how we actually look at these things is what leads us to the ability to be able to detect the difference between,
1806890	1812130	in this case, real and fake, within a staggered realm.
1812130	1818210	So how do we do this when it's not steganography, when it's actually a fake image?
1818330	1827870	So it all comes down to the concept of intelligent feature extraction and classification so that we can actually train a machine
1827870	1831490	in order to be able to do this using machine learning techniques.
1831490	1840250	But if we don't understand the kinds of features that we need to extract that make a difference between real and fake, it's not going to work.
1840250	1848330	So let's take one of the images that I've showed you so far. We'll take the image of Vladimir Putin kind of giving the thumbs up to Trump and Merkel.
1848990	1852150	Was this a real image? Let's take a detailed look at this.
1852150	1856850	So how do we do that? The first thing we want to do is break the image up into small pieces.
1856890	1864050	Obviously, you break it up into smaller just to kind of illustrate how we do this, we break it up into frames within that particular image.
1864050	1873350	Once we do that, we take each individual frames, for example, the one that is around the hand, the thumbs up from Vladimir Putin.
1873570	1884350	We take that center pixel and we take a look at that center pixel, we want to basically compare it to the adjacent pixels to see if we can find any anomalies
1884350	1889250	within those adjacent pixels that don't occur in the normal image.
1889250	1898930	So this is how we actually are doing this. We actually take that center pixel and we do this throughout the entire image in comparing those adjacent pixels.
1898930	1911430	So we first create a baseline of what normal is across the image and then we're looking for areas of the image that don't conform to that normal behavior between these adjacent pixels that are around it.
1911430	1920850	In order to do that, what we can do then is create a set of sample images that are labeled either fake or real in this particular case.
1920950	1932810	And then we can extract those features that we're interested in, take those features and put them into a set of pandas, in other words, panel data, which is basically kind of like a spreadsheet.
1932810	1942270	And scrub them, basically kind of actually make sure that we have no real outstanding outliers and that kind of stuff and actually scrub that data a little bit.
1942330	1950810	And then what we do is we actually run that through a learning engine like Scikit-Learn or TensorFlow, whatever you want to use.
1950810	1961290	And basically create a model of what is normal. This is supervised learning, obviously, so we're going to basically know when we start this out what's good and bad.
1961290	1964470	The issue is that this is a very interactive process.
1965170	1977970	So once we do this, we have to create the model and we run through this process over and over again until we get both the feature extraction right and we tune the learning engine to learn properly.
1977970	1985450	So at least we can get the images that we use as the model, in other words as the sample images, to come out correctly.
1985450	1988950	In other words, to be able to detect the difference between real and fake.
1988950	2002350	Then what we do is take that to the next level and we bring in images that are real and fake and run them into the model to see if we can identify those.
2002350	2009850	So we do the same feature extraction, we run those through whatever model we have, and then we can actually visualize the results.
2009850	2016430	In other words, what areas of the image are we finding anomalies that we don't find in the rest of the image.
2016430	2022510	So it's kind of a simple take on what we do, but just kind of give you an idea of how this actually functions and works.
2022770	2027690	So with that, you might be saying, OK, well, I'm from Missouri, show me how this works.
2027690	2034110	So the next thing I want to do is kind of give you a quick demonstration of a tool that we've developed that actually puts this into practice.
2035410	2038150	So let's put the machine learning engine to work.
2038150	2047910	I'm actually going to show you a demonstration here of a tool called PX that we've developed in order to be able to do this kind of active analysis of these potential images.
2047990	2053110	So the first thing we're going to do is select an image that we want to actually analyze, that pilot selfie that I've already shown you.
2053670	2059250	And what we're going to do is first come up and create the baseline of that particular image.
2059350	2065950	And then what we're going to do is we're going to go ahead and perform the analysis of that image in order to identify areas of the image that are anomalous.
2065950	2073470	So that we can actually pinpoint areas of the image that we believe were altered in order to create this fake.
2073950	2079050	So you can kind of see, this is what it looks like. The yellow dots basically indicate the areas that we've identified.
2079050	2083210	So you can see that there was multiple components of this image. The tie was added, the glasses were added.
2083210	2086590	Obviously, the pilot was added, you know, to that.
2086590	2091990	You'll notice that the engine actually contains some analysis as well.
2091990	2096670	And the reason for that is because they wanted to make it look like the engine was running, not on some runway.
2097010	2099150	Let's take a look at another one real quick.
2099610	2104130	In this particular case, we'll take a look at the set of Ron and Dwayne Johnson.
2104570	2114970	We'll do the same thing, create the baseline, go ahead and perform the analysis of that image and see if we can identify areas of how that was pieced into that background by Paramount Studios.
2115730	2119610	And of course, we can kind of see the edges of that.
2119610	2129010	You notice the rest of the image is right. So there's edges in the other parts of the image that are not showing up, but only the ones that actually interact with the background in a specific way.
2129390	2136530	Also, there was adjustments to the hair, the eyes, that kind of thing that were modified in order to be able to make this image look better.
2136770	2145090	So that's the second one that we wanted to take a quick look at so you can kind of see how we can actually identify those areas where the image was, in fact, altered.
2145790	2149210	So let's take a look at another one in this particular case.
2149210	2156570	We're going to go ahead and actually look at something just a bit more interesting, in my opinion, that's potentially a bit more dangerous.
2156950	2167790	This was the image that I showed you where Putin was shaking hands with Trump, which obviously never occurred, with Merkel in the background as well.
2167790	2172910	So what we're going to do is basically take a look and see if we can identify areas that that particular image was altered.
2172910	2179230	Obviously, exactly what we expected here, the thumbs up and the hand was actually added to that particular image.
2179230	2183610	So we get all kinds of anomalies being detected around that.
2183610	2188810	So you can kind of almost see exactly how that was actually added to this particular image.
2188810	2197130	And DML is picking this up as an anomaly based on the extracted features that I talked about earlier that we use.
2197130	2202350	This is a really interesting one. This is the Trump and Putin image.
2202350	2211410	And this one's interesting mainly because there's some features here that we weren't expecting to discover that we actually discovered as part of this particular image.
2211410	2215450	So what we expected to see was where the two images were put together.
2215450	2221670	We can kind of see how they were blended together, but even though it's done really well using AI in order to integrate these two.
2221670	2225030	The other thing we noticed was we found anomalies within Putin's eye.
2225030	2227290	We didn't understand what that was initially.
2227290	2232210	And then we realized later that he wears contacts to enhance the color of his eyes.
2232210	2234750	And we're picking up that anomaly there.
2234750	2239450	In the case of Donald Trump, he has caps on his front teeth to enhance the smile.
2239450	2241850	And therefore we're picking up some of those anomalies as well.
2241850	2250110	So not just anomalies that were actually created by the alterations, but anomalies that were in the original image as well that were in fact dramatic.
2250650	2252410	And then the last one is kind of a fun one.
2252410	2258270	We'll take a look at this image with the flying saucer.
2258270	2262310	This is a little bit larger image, so it's going to take a second to load.
2262310	2267330	So I kind of fast forwarded the analysis part so you didn't have to wait for that.
2267390	2269670	And when we analyzed this one, we noticed there's nothing.
2269670	2272450	We see no changes in this particular image whatsoever.
2272450	2275610	No anomalies that are there no matter how much we zoom in.
2275790	2279290	And that's because this particular image was real.
2279290	2287310	So in other words, either the photograph that was taken was taking up an object that was there to appear like a flying saucer.
2287310	2294410	Or this was a computer generated image where there are no anomalies that we can detect within the particular image.
2294410	2299490	So this can happen when we're actually analyzing images like that.
2300290	2307430	Next I want to kind of walk you through a cool way that we capture these images from social media.
2307630	2314590	One of the questions you might have is how do we actually capture these live videos or live images within the internet?
2314590	2322150	We built a special application that allows us to create bounding boxes around specific areas within the world.
2322210	2331130	In this particular case, I've selected Orlando, Florida, in order to be able to kind of track some of the tweets that are occurring during the rover launch.
2331370	2333770	And I thought this would be a little more fun than something negative.
2333770	2335350	So hopefully you'll enjoy this.
2335350	2341450	And so what we're going to do is actually we selected that bounding box area, 100 miles, for two hours.
2341450	2343910	Obviously I'm only going to play about a minute of this for you.
2343910	2350090	And we're going to start to capture the tweets that occurred along with the images that were posted along with those tweets.
2350530	2356930	So really dramatic. You can kind of see that not only do we capture the tweets, but we also capture the emojis that are there.
2356930	2359730	We convert them to text so you can see what they are.
2359790	2367030	Obviously we have a log of everything that occurred, as well as all the images and all of the text are associated with it.
2367550	2374950	This tool allows us then to pinpoint any area in the contrary of the world so that we can actually capture live tweets.
2374950	2379230	Many of you may not know, but Twitter will pull down tweets that are offensive, etc.
2379230	2381470	But we can capture them because they're here live.
2381470	2384090	We're capturing these as they occur.
2384090	2386330	In this case, everything is pretty benign.
2386330	2398010	But in certain cases, we want to be able to capture events that are occurring and be able to capture those images and videos that have been altered as they occur.
2398010	2402130	So that we can then perform the analysis of them with the Phelps tool.
2402830	2410690	With that quick introduction of our research and development activities as they are associated with media analysis of disinformation campaigns,
2410690	2415310	we want to thank you for tuning in and always supporting DEF CON.
2415310	2418010	A lot of the innovations in the world happen here.
2418010	2420030	I'm so glad that you were able to make it.
2420290	2424890	If you want to reach out to Mike or I, those are our email addresses.
2424890	2426790	It's probably the best way. We're also on LinkedIn.
2426790	2428490	Obviously, you can find us there.
2428950	2435450	Have a great rest of the conference and hopefully you've learned a lot from this particular event.
2435450	2442470	And hopefully this introduction to media analysis of disinformation campaigns has been worthwhile to you.
2442630	2444090	Have a great rest of the week.