WEBVTT 00:07.820 --> 00:17.720 Um, when you leave this tent, if I could get you to exit through any other door, except the one you came in through right there. 00:17.820 --> 00:22.660 Because people come in through that door, please leave through any of the other doors. 00:22.660 --> 00:23.380 Thank you very much. 00:23.380 --> 00:26.060 Also, as you know, the tent is one hour behind. 00:26.060 --> 00:29.540 All of the other speaker rooms are operating on schedule. 00:29.540 --> 00:35.160 If you want to see the next talk in Apollo, I suggest that you start queuing a half an hour before the talk starts. 00:35.160 --> 00:39.640 If you're here in this room, odds are you will be able to stay for the next talk. 00:39.640 --> 00:41.980 We haven't had to empty this room all day today. 00:42.820 --> 00:44.300 That's all I have. 00:44.300 --> 00:52.020 Also, please don't take any photos during a presentation at DEF CON without the consent of the speaker in advance. 00:52.020 --> 00:53.840 That's a new rule that's come to... 00:53.840 --> 00:54.460 what? 00:55.440 --> 01:01.560 Oh, no photos of the crowd or anybody else unless you have consent of the people you're photographing. 01:02.520 --> 01:03.380 That's all. 01:03.380 --> 01:04.100 Are you ready? 01:04.620 --> 01:05.000 Okay. 01:05.000 --> 01:05.900 Let's give a hand for Christian. 01:05.900 --> 01:06.540 Thank you. 01:09.930 --> 01:10.790 Thank you. 01:12.010 --> 01:22.450 So I assume everybody here knows and loves encryption, but I think steganography is much cooler because in that case, the adversary doesn't know not only what you're sending, but also that you're sending a message at all. 01:22.550 --> 01:24.090 So it's in some sense much better. 01:25.350 --> 01:34.450 And what I'm trying to talk about specifically today is, well, in steganography, you always have to have some kind of cover message, the message the adversary sees that you're sending on the surface. 01:34.790 --> 01:37.950 And what I'm going to use as the cover is natural language. 01:38.030 --> 01:42.450 More common covers are images and sound, but we're going to use natural language here. 01:43.770 --> 01:44.810 Okay. 01:45.150 --> 01:49.010 So what have people done so far for natural language steganography? 01:49.250 --> 02:05.890 Well, Peter Wehner, Chapman, and DeVita did something where they essentially by hand wrote some context-free grammars, and then these context-free grammars were to decode to natural language sentences, entire texts, and depending on which productions you would choose in the context-free grammar, 02:05.890 --> 02:07.690 it would encode a different message. 02:07.910 --> 02:13.850 The problem with that is obviously that you have to construct these large context-free grammars, and both the sender and receiver have to have them. 02:14.630 --> 02:26.570 So one thing they did then, they automatically extracted syntactic templates from text and essentially took the volumes of Shakespeare and produced a grammar that would produce text that looked like it came from Shakespeare. 02:26.570 --> 02:30.150 The problem is, it was semantically terrible. 02:31.150 --> 02:34.210 Syntactically correct, somewhat, but semantically terrible. 02:34.710 --> 02:39.850 Chapman and DeVita improved on that by doing synonym placement on existing text. 02:39.850 --> 02:45.390 So you would take some existing text and just substitute some of the words in that text by synonyms. 02:45.390 --> 02:57.910 Synonyms are words with the same meaning, and thereby you get a semantically, syntactically, coherent, and rhetorically correct text, except that you have to come up with good synonyms, and you have to come up with this good original text that you're going to substitute on. 03:00.750 --> 03:08.810 So there are lots of problems with these approaches, and in particular with the synonym placement approach, you have things that are synonyms in some contexts but not in others. 03:08.810 --> 03:14.730 For example, I can say, I eat a sandwich, I devour a sandwich, but I cannot say, I devour. 03:14.730 --> 03:16.430 Devour requires a direct object. 03:16.670 --> 03:18.930 So a synonym is not always a synonym. 03:21.010 --> 03:33.430 And of course, if you need this original text that you're going to do semantic substitution on, if you use the same text twice, the adversary will say, wait a second, I just saw the same text with minimal variations, and you're exposed. 03:33.670 --> 03:39.430 If the adversary can spot the original text, well, differential text, or knowing discovered text. 03:39.430 --> 03:43.050 So these are problems with the existing approaches prior to this work. 03:43.750 --> 03:45.750 What do these problems really come from? 03:46.250 --> 03:51.270 Well, first of all, automatic generation of cinematically and rhetorically correct text is very difficult on its own. 03:51.490 --> 03:54.090 If you ever try to write a paper, it's hard for you. 03:54.250 --> 03:59.210 Now imagine how hard it is for a computer, especially with all the synchronicities in language. 03:59.870 --> 04:03.930 What all of these previous approaches have tried to do is they try to mimic correct text. 04:04.730 --> 04:14.610 And as in all steganographic approaches, you have some kind of cover object that the adversary expects, and if you deviate from this expectation, that's giving the attacker an attack vector. 04:15.110 --> 04:20.570 And if you're trying to mimic correct text, well, you better produce really correct text, otherwise the adversary can possibly spot you. 04:21.010 --> 04:26.590 So what we really want is something where incorrect text is OK. 04:27.350 --> 04:36.070 We want something where the adversary, when he gets the text, doesn't expect that this text makes perfect sense, is syntactically and semantically perfectly correct. 04:36.750 --> 04:44.610 Ideally, also, we can solve the other problem, that if the adversary gets the original text or some database, he can't say, oh, this other text should not exist. 04:44.610 --> 04:46.150 It's just a minor variation. 04:46.170 --> 04:50.030 It should exist together with whatever you used to generate your cover from. 04:50.690 --> 04:57.570 And as I said, there should be errors in there that we can exploit for steganographic hiding of the hidden message. 04:59.470 --> 05:03.950 Well, I'm sure you've all read the title, so you know what I'm heading for. 05:04.190 --> 05:06.390 Here's some example from Babelfish. 05:07.070 --> 05:10.030 Took a site from a German Linux camp site. 05:10.530 --> 05:17.570 In German it reads, keine Sorge, sie sind alle handsam und beantworten auf freiwillig Fragengründen das Thema Linux und geben gerne einen Klein-MIG-Innovator open source. 05:17.810 --> 05:19.490 OK, nobody understood that one, right? 05:20.450 --> 05:26.770 If you translate this to English by hand, you get something like, don't worry, they're all tame and will also readily answer questions regarding the topic Linux. 05:26.770 --> 05:29.110 I'm glad to give a small glimpse into the world of open source. 05:29.590 --> 05:47.330 Now, if you feed this into Babelfish, I don't know, I assume most of you have used this translation at some point or other, say on Heise or something, you get a concern, it are all not handsome and also readily questions approximately around the topic Linux and give gladly a small idea into the world of the open source. 05:49.430 --> 05:52.270 OK, so first of all, the not is kind of in the wrong place. 05:52.270 --> 05:55.630 It's not a concern as opposed to it's a concern that they're not tame. 05:56.510 --> 06:04.890 It didn't translate handsome at all because it wasn't in the dictionary and it adds an article, the, in front of open source and I don't know why it does this strange capitalization. 06:04.970 --> 06:11.850 But you can see there are plenty of errors in here and still, this text, you might get some idea of what this event was about, right? 06:11.850 --> 06:19.050 So it's still useful for humans, it's still perfectly plausible that somebody sends you this, say you don't speak German and he sends you a rough translation. 06:19.890 --> 06:21.950 So, let's see. 06:22.330 --> 06:23.950 How does machine translation work? 06:23.950 --> 06:29.130 Well, machine translation, first of all, there are lots of low quality encoders out there. 06:31.490 --> 06:35.290 They can, there's never a perfect translation anyway, right? 06:35.610 --> 06:40.750 If you translate from one language to the other, there are multiple variations of how you can express it in the target language. 06:40.870 --> 06:48.870 So, if there is a variation in the translation, it's OK, even if you have a human translation and even voiceless machine translation. 06:50.850 --> 06:54.530 What current machine translation systems do is they are statistical. 06:54.530 --> 06:58.830 They have a language model, a translation model, which maps words and sentences. 06:58.850 --> 07:03.330 They have some idea of syntactic and semantic rules, mostly do pattern matching. 07:03.470 --> 07:13.270 They always ignore all of the context, so they look at one sentence at a time, at least all of the ones that we found, which can have some pretty bad results. 07:13.270 --> 07:21.930 Say somebody talks about pipelines and seals, and suddenly talks about sea lions in the middle of the translation, because the translation picked the wrong word in the dictionary to translate seals. 07:23.570 --> 07:26.970 So there's plenty of errors in machine translation. 07:29.870 --> 07:39.830 Now, one other thing I should say, even if these errors weren't there, even if machine translation were perfect, as I said before, there's always some room in natural language, so we're not just exploiting an error in current software. 07:40.630 --> 07:43.610 So, how does the basic protocol work? 07:43.610 --> 07:50.310 Well, we assume that Alice wants to send a message to Bob, and, well, as usual, they have some shared secret. 07:50.310 --> 07:56.450 In our case, what kind of specific translation engines they're going to use, and other things that define the configuration of the system. 07:56.770 --> 07:59.450 And Alice first has to select some source text. 07:59.690 --> 08:05.710 And in our case, it can be from a public news source, so you could just take text from BBC, the Bible, whatever you want. 08:05.750 --> 08:07.290 You could write some on your own. 08:08.130 --> 08:10.570 It doesn't have to be secret for the system to work in general. 08:10.570 --> 08:14.770 It can have advantages, as you'll see towards the end of the talk. 08:15.450 --> 08:29.890 So, given this cover source text, the translator configuration, and the secret message, Alice gives all of this to our tool, Lit, which, by the way, is available both as a PHP demo and in code at the URL at the bottom of the slides. 08:29.950 --> 08:31.670 Don't go all there at once and try the demo. 08:31.670 --> 08:33.890 The server is a really, really slow machine. 08:33.890 --> 08:37.070 It's processing PHP, and you'll just kill it. 08:37.890 --> 08:40.050 However, it works if you don't try it all at once. 08:42.330 --> 08:48.570 Anyway, so Alice gives this to the Lost in Translation system, and as you would expect, gets back a cover translation. 08:48.790 --> 09:03.290 She sends that cover translation together with either the cover source text or a reference to it to Bob, and Bob just feeds his shared secret, again, the cover source text and the cover translation to his copy of the Lit system and gets back the secret message. 09:03.290 --> 09:04.570 So far, so easy. 09:06.730 --> 09:08.590 So how does the encoding really work? 09:08.590 --> 09:13.390 First, we take our cover source text and run it through several commercial and custom-generated translation engines. 09:13.390 --> 09:27.610 In our case, since we weren't really up to the task of writing our own machine translation software, well, we go online, we query Babelfish, we query Google, we query SysTran and a couple of other engines that are out there and get back translations. 09:29.350 --> 09:34.690 Then, after we get these translations, we want actually more than just two or three variations because then the bit rate would be very low. 09:34.690 --> 09:40.690 So what we do is we add additional variations to the sentences, making slight modifications that are presumably plausible for S.S. 09:40.690 --> 09:46.450 translation errors, possibly even fixing translation errors and we get even more sentences out of that. 09:47.790 --> 09:55.390 As we do so, whenever we add these variations, we compute probabilities that this is actually still a valid translation. 09:55.390 --> 10:09.870 And at the end, once we have lots of sentences, lots of translations for a given sentence and probabilities, we build a Huffman tree and then we walk through this Huffman tree based on the secret message we're trying to encode. 10:10.550 --> 10:14.090 So the picture says more than I can speak into the microphone. 10:14.490 --> 10:20.010 Again, cover source text goes into commercial translators, which ones are again selected by the configuration. 10:20.010 --> 10:22.850 You can add new ones if you are so inclined to the tool. 10:22.850 --> 10:26.270 Then we do some post-processing to get even more sentences. 10:27.770 --> 10:32.610 Then we have some model that will tell us how likely are these sentences actually good translations. 10:32.810 --> 10:35.050 Based on these probabilities, we build a Huffman tree. 10:35.290 --> 10:39.470 And then based on the secret message, we select which translation we're going to emit. 10:39.850 --> 10:43.790 We do this on a sentence-by-sentence basis because the translators work like that. 10:44.930 --> 10:50.790 Eventually we either run off cover source text and cut the secret message or we run off secret message and pad with random bits. 10:51.030 --> 10:52.090 Very simple, right? 10:52.090 --> 10:53.670 Now what does the decoder do? 10:53.670 --> 10:55.050 Well, pretty much the same thing. 10:55.050 --> 11:01.130 Except that of course, at the end, we get the cover translation in. 11:01.130 --> 11:02.490 We still built the Huffman tree. 11:02.710 --> 11:06.550 And now we essentially go upwards and say, okay, which were the bits that were leading to the sentence in this Huffman tree. 11:06.910 --> 11:08.410 And voila, we got our secret message. 11:09.950 --> 11:12.610 To give you some idea of what the post-passes look like. 11:13.270 --> 11:14.730 Some of them, very simple one. 11:15.110 --> 11:20.410 What we found when we looked at translations, one of the things that are often wrong is that they pick the wrong kind of article. 11:20.410 --> 11:21.930 In English, definite versus indefinite. 11:21.930 --> 11:23.370 In German, the wrong gender. 11:24.790 --> 11:25.490 And so on. 11:25.490 --> 11:27.430 So that's one thing we can play with. 11:27.430 --> 11:31.210 Or some words do not have an article, should have one and vice versa. 11:31.230 --> 11:32.670 So that's something we can do. 11:34.310 --> 11:36.230 Also prepositions are very tricky. 11:36.230 --> 11:38.590 Should it be to, should it be towards, should it be from? 11:38.970 --> 11:40.910 So we can play with them a little bit. 11:41.350 --> 11:47.510 And again, we did some manual shuffling of what are prepositions that are close to each other that are likely to be mistakes. 11:48.110 --> 11:50.630 And then that's where the probabilities come from. 11:51.070 --> 11:55.550 And of course what you can do is if you have a dictionary that lists you how often a given word occurs in a language. 11:55.550 --> 12:00.530 You can say, okay, this word is one of the ten million words that nobody ever knows. 12:00.550 --> 12:01.830 So likely the translator didn't know it. 12:01.830 --> 12:04.070 So we leave it in the original language even if it was translated. 12:04.070 --> 12:04.830 Or vice versa. 12:04.830 --> 12:05.930 We have it in our dictionary. 12:05.950 --> 12:06.770 Babafish didn't. 12:06.770 --> 12:07.210 Go us. 12:07.210 --> 12:08.450 We can fix this error. 12:08.990 --> 12:12.570 Right, so again you get two variations just by doing this little change. 12:12.570 --> 12:17.570 And you can have in the archimetry additional ways for encoding information. 12:18.750 --> 12:22.150 Another post-pass that I like very much is semantic substitution. 12:22.310 --> 12:26.410 We talked earlier about synonyms and how hard they are to get and that you have to construct them by hand. 12:26.470 --> 12:33.690 Now if we have a translation, this gives us a way to construct synonyms in a way that we call second-order semantic substitution. 12:34.190 --> 12:36.890 So what we do is we have a German word, say, flach. 12:37.110 --> 12:41.050 And the original machine translator translated it to say flat. 12:41.430 --> 12:43.090 Okay, that's one choice. 12:43.130 --> 12:45.830 Now if we look at our dictionary, we would have had other words. 12:45.830 --> 12:50.130 Tabular, vapid, even, smooth, plain, plain, shallow, and so on and so on. 12:50.270 --> 12:54.170 Now some of these are semantically rather far away from flat. 12:54.330 --> 12:56.690 So we would not just pick a random one out of these. 12:56.690 --> 12:57.370 But what can we do? 12:57.370 --> 12:59.730 We can translate flat back into German. 13:00.250 --> 13:02.170 We get flach, eben, and glatt. 13:02.650 --> 13:05.090 So these are also again quite close now in German. 13:05.530 --> 13:07.950 And if we translate these words now back into English. 13:08.190 --> 13:09.670 See it's going back and forth here. 13:09.670 --> 13:14.630 Then it says, okay, eben goes to even, glatt goes to plain, plain and smooth and so on. 13:14.630 --> 13:16.990 So there's some of these words which kind of form a cluster. 13:17.310 --> 13:19.950 A cluster that shows that they're semantically quite close. 13:19.950 --> 13:22.070 And we can do that because we've got these two languages. 13:22.410 --> 13:30.590 And now assuming that the original translator might have taken a little bit of context from the current sentence and chosen a good translation in flat, we get good semantic substitutions here. 13:30.790 --> 13:36.770 Now if the original translator chose badly, well, then we're not going to do much damage anyway because it was a bad choice. 13:37.010 --> 13:38.970 So it's again very plausible to do this. 13:38.970 --> 13:45.090 And the number of witnesses you pick for the semantic substitution pass you can configure and it's one of these configuration parameters. 13:45.290 --> 13:46.850 Just like, do you want to use this at all? 13:50.730 --> 13:54.030 And the tool itself, you can plug in new modules if you feel so inclined. 13:54.030 --> 13:57.390 You just have to study machine translation, figure out what is actually a plausible error. 14:02.230 --> 14:04.630 So I'm sure you all want to see what it looks like, right? 14:04.970 --> 14:07.290 So let's take an original sentence in German. 14:07.290 --> 14:12.230 In dieser Zeit soll festgestellt werden, ob die Schüler die richtige Schule gewählt haben und ob sie ihren Fähigkeiten entspricht. 14:12.270 --> 14:12.910 Oh God. 14:13.210 --> 14:19.890 In Google, this looks like, in this time it is to be determined whether the pupil selected the correct school and whether it corresponds to its abilities. 14:19.950 --> 14:21.550 Quite a good translation, actually. 14:21.990 --> 14:29.230 Now, LinguaTek, another engine that we're using, gives whether the pupils have chosen the right school and whether it corresponds to its abilities shall be found out at this time. 14:29.670 --> 14:41.010 If you use Lit, and this encodes 8 bits, so that's quite a bit of information, quite a number of choices that we had here, you get, in this time, it is to be determined whether the pupil selected the correct school and whether it corresponds to its abilities. 14:41.250 --> 14:46.710 That's not better, but it's not that much worse that you could say, well, there couldn't be a translator which does this. 14:46.830 --> 14:51.670 And yes, you might notice the closeness to Google, because in this case it selected Google as the basis. 14:51.890 --> 14:57.990 But now assume you've got a couple of hundred translation packages out there, and you don't know which one was chosen. 14:58.350 --> 15:00.890 You don't have this comparison directly at hand. 15:00.890 --> 15:04.750 Maybe we'll see later how we can even hide the original text, so you don't have that one. 15:04.750 --> 15:07.190 You don't have the base translation, you just get one translation. 15:07.670 --> 15:12.350 And you can say, yeah, well, maybe a human wouldn't have produced that, only a first class in English. 15:12.570 --> 15:17.310 But, hey, hard to tell that there was really something going on. 15:18.370 --> 15:19.590 Give you another example. 15:19.910 --> 15:21.490 This is from a film review. 15:21.490 --> 15:27.710 Der marokkanische Film Windhorse erzählt die Geschichte zweier unterschiedlicher Generationen angehorener Männer, die durch Marokko reisen. 15:27.990 --> 15:31.250 Auf dem Weg suchen sie nach dem einzigen, was ihnen wichtig ist, dem Sinn des Lebens. 15:31.890 --> 15:34.810 Okay, I'm not going to read the Google and Linguatech translations. 15:35.170 --> 15:41.790 However, what this gives us is the Marokkan film Windhorse tells a story from men belonging to two different generations who travel through Morocco. 15:41.950 --> 15:45.850 They are looking for the only one which is important to them on the way, the sense of a life. 15:46.370 --> 15:48.790 Now, the best translation would leave out the Ö. 15:49.510 --> 15:51.970 Well, Google and Linguatech put a ö in there. 15:51.970 --> 15:54.410 And actually, lit, as one of the choices, don't put anything. 15:54.930 --> 15:57.510 Well, we included the string lit here, so we got this one. 15:57.750 --> 16:03.870 But as you can see, it might even improve the quality of the given translation, and at least it's not necessarily a significant difference. 16:06.030 --> 16:07.710 So what are the advantages again? 16:07.930 --> 16:15.750 We can hide information within the limits of machine translation, and as machine translation improves, our system will also automatically improve in what it's generating. 16:16.810 --> 16:20.850 We avoid this generation problem by mimicking the results of an imperfect transformation. 16:21.210 --> 16:23.830 So we don't have to produce correct text. 16:23.850 --> 16:26.570 We can get away with producing slightly incorrect text. 16:26.710 --> 16:31.310 And what is also interesting, if you have a larger document with some rhetorical structure, we get that. 16:31.310 --> 16:33.310 The translation gets the rhetorical structure. 16:33.410 --> 16:44.770 If you use any of the other generation systems, well, if you're lucky, you get one sentence correct, but you're never going to get a large book or article correct that way, in terms of rhetorical structure and so on. 16:45.450 --> 16:50.150 We have a secret key that can be used to configure the system. 16:50.370 --> 16:52.110 And to that, you can have it very small. 16:52.110 --> 16:56.390 You can just say, OK, I'm using Babel and Fish and Google and semantic substitution with two witnesses. 16:56.550 --> 16:57.990 Or you can make it more complicated. 16:57.990 --> 17:02.390 You can say, I've got this machine translation system, and it was trained on this corpus. 17:02.810 --> 17:04.050 Machine translations are statistical. 17:04.050 --> 17:13.030 So they are typically done in a way that you have some large parallel corpus of sentences in English and in German, and the machine translation is supposed to learn what a good translation is. 17:13.390 --> 17:16.070 And if you have a different corpus, you get a different machine translator. 17:16.250 --> 17:19.610 So you can make this entire corpus part of your secret, if you so want to. 17:20.170 --> 17:22.710 So you get lots of possible variations there. 17:23.670 --> 17:28.550 And again, for now, we can also say that the cover text was from a public source. 17:28.550 --> 17:32.710 You don't have to write a new one each time, take a new image, or whatever else you used to do in the cover. 17:33.730 --> 17:34.970 What are the disadvantages? 17:35.390 --> 17:38.390 Well, the bitrate is not really high still, right? 17:39.610 --> 17:45.290 One of the big disadvantages of this text is always very robust compared to, say, binary data like images. 17:45.670 --> 17:53.150 And as I said multiple times before, we need to transmit both the source text, or some reference at least to it, and the translation for the decoder to work. 17:54.790 --> 17:57.830 Well, let's try to address these two issues. 18:03.120 --> 18:05.220 So what can we do to increase the bitrate? 18:05.260 --> 18:07.680 Well, we can create more machine translation systems. 18:08.260 --> 18:14.160 I'm sure the people who are doing machine translation will do that for us anyway, but that's one way of doing it. 18:14.860 --> 18:19.700 We can, as I said, create new corpus to train existing machine translation implementations. 18:19.700 --> 18:20.920 And that's actually quite easy. 18:20.920 --> 18:27.520 If I've got a parallel corpus of 20,000 words, I can select any subset of this corpus and get another corpus. 18:28.020 --> 18:34.300 So that's easy if I have the full source of a machine translation system and can train it myself. 18:34.900 --> 18:40.260 And, of course, you can write additional post-passes or pre-passes that will make plausible modifications to sentences. 18:42.860 --> 18:46.520 So what we've done so far is we have a prototype implementation. 18:46.680 --> 18:48.880 And that, of course, means we have limited dictionaries. 18:49.080 --> 18:53.500 We did not write anything that would give us knowledge about grammar or semantics of a language. 18:53.500 --> 18:56.720 And we only have a couple of translation engines available to us. 18:57.120 --> 19:03.040 Still, what we get is a bitrate of something like 0.082 bits per bit transmitted. 19:04.100 --> 19:05.700 Now, this is for normal text. 19:05.700 --> 19:11.520 If I just gzip the text, I get a much better bitrate, which is 0.022, which is a bit more aligned with... 19:11.520 --> 19:17.040 If I take a PNG or JPEG image and hide something in it, well, I could compress this and get a bit... 19:20.860 --> 19:24.700 Quickly, some minor variation that will become interesting soon. 19:25.360 --> 19:28.900 We could use this scheme not only for steganography, but also for watermarking. 19:29.280 --> 19:34.340 Often, watermarking and steganographic systems are related, so this is quite natural to do. 19:35.040 --> 19:40.440 In watermarking, what we want to do is we want to be able to read some embedded mark from just the translation in this case. 19:40.440 --> 19:42.560 We don't want to need the original text. 19:43.300 --> 19:44.540 So, what can we do here? 19:44.700 --> 19:50.960 Well, we could use some keytash of a certain set of sentences and take the least significant bit of that, and that's going to be our marked bit. 19:50.960 --> 20:01.320 And essentially, when we pick our sentences in the translation, we're going to go through all of the sentences, compute all the keytash, and torture it until we get a sentence that actually gives us the marked bit. 20:01.320 --> 20:04.680 Since we have lots of variations, we would statistically succeed most of the time. 20:07.140 --> 20:11.240 And of course, we use multiple sentences if we need multiple marked bits. 20:12.060 --> 20:13.760 Which sentence are we going to pick? 20:13.780 --> 20:19.600 Well, we could just have some other secret mechanism that gives us sentence 5, 10, 17. 20:20.960 --> 20:24.140 Or just at the beginning, but this makes the watermark kind of fragile. 20:24.140 --> 20:27.840 The adversary could just insert a sentence at the beginning and suddenly all of our offsets are wrong. 20:28.160 --> 20:38.320 What is a better technique is to say, we have some arbitrary mechanism to rank sentences, all of the sentences in the translation, and that just gives us some total ordering. 20:38.320 --> 20:44.800 We take the sentences that have the scores lowest on this ordering, and the sentences after that in the document are the ones that are going to hold the marked bits. 20:44.980 --> 20:48.820 This way, an insertion or deletion attack will not necessarily succeed in destroying the mark. 20:53.480 --> 20:55.880 It wouldn't be DEF CON if we didn't have some attacks, right? 20:56.260 --> 20:57.800 So what can we do about attacks? 20:57.940 --> 21:02.180 Well, first of all, there is the possibility of somebody spotting obvious inconsistencies. 21:02.180 --> 21:09.480 The most easy one that comes to mind is, say you've got a translation which in the original text contains the same sentence twice, and you translate it in different ways. 21:09.680 --> 21:11.300 A machine translation would never do that. 21:11.760 --> 21:15.100 It's a deterministic machine, and now suddenly it does it not. 21:15.360 --> 21:18.100 So, that's in some sense a simple bug. 21:18.100 --> 21:25.340 In the LIT system, you just have to keep track of which sentences have you translated so far, and if you see the same sentence again, well, you're not going to use that to encode anything. 21:25.340 --> 21:27.760 You have to omit the sentence you omitted the first time. 21:28.220 --> 21:35.600 Similar, there are certain mistakes that machine translators make, like translate the plural of foot as foots. 21:36.040 --> 21:37.860 Should be feet, obviously, right? 21:38.860 --> 21:46.080 If you pick two different machine translators, where one makes this mistake and the other one doesn't, and you alternate between them, you get inconsistent mistakes in your translation. 21:46.080 --> 21:54.620 And the reader will wonder, okay, so a human might make this mistake once but not another time, but a machine translator should make mistakes consistently. 21:54.680 --> 21:57.180 So that's another possible source of bugs. 21:57.900 --> 22:04.120 Which you would fix by, obviously, writing some kind of post-pass or processing that makes sure that these don't happen if you can spot them. 22:04.120 --> 22:16.740 In general, essentially one problem that most steganographic systems have, somebody could construct a new statistical model that says all of the translation systems in the world obey this model, except for this one. 22:16.860 --> 22:21.780 That applies the same to image steganography or something with audio files. 22:22.100 --> 22:25.420 Somebody has a model, and you deviate from it, and you shouldn't. 22:25.420 --> 22:26.860 How can you defeat that? 22:27.020 --> 22:32.620 Well, let's assume, just for fairness, that we also know about that model. 22:32.620 --> 22:40.240 If we know about that model, we can easily make our encoder aware of that model by sorting out sentences in the translation process that don't obey the model. 22:40.420 --> 22:42.240 And then we can't be detected anymore. 22:42.880 --> 22:50.800 What is interesting is that creating such a model is in some sense equivalent of improving statistical machine translation because, well, what is this machine translator? 22:50.800 --> 22:53.920 It is a model of how one sentence corresponds to another. 22:54.700 --> 22:56.480 And in the end, it's an arms race. 22:56.480 --> 22:58.160 Whoever got the better model wins. 22:58.160 --> 23:06.140 If the attacker and the defender have the same quality of machine translation model, then the defender will win because the attacker will have to say, oh, it's plausible. 23:09.560 --> 23:11.460 Finally, the most fun part. 23:11.460 --> 23:15.660 How can we avoid transmitting the original document for the system? 23:16.860 --> 23:19.880 Remember the least significant bit ID for watermarking? 23:20.440 --> 23:24.500 Now, what if we take H bits in each sentence? 23:25.340 --> 23:31.220 The first H bits that tell us how many of these least significant bits in the hash are going to actually contain a message. 23:31.960 --> 23:34.720 So, I'm a sender, some receiver. 23:34.820 --> 23:36.220 We have some small constant H. 23:36.220 --> 23:42.960 When I pick my sentences, I try to pick one where the first two bits, say, encode the number one. 23:42.960 --> 23:44.580 That means I'm going to transmit one bit. 23:44.580 --> 23:47.700 And then the third bit is going to transmit the actual message bit. 23:47.700 --> 23:59.100 Or, if I have more variations available for this given sentence, I could transmit one where it has the first two bits set, meaning the number three, and then follow three bits in this least significant hash that contain the actual message. 23:59.600 --> 24:07.520 This is a statistical way, just based on the fact that hashes are pseudorandom, that I can encode the message by just transmitting the translation. 24:07.980 --> 24:22.120 And, well, I might fail to find one of the translations in the variation that allow me to encode this, but I'm unlikely to fail, statistically speaking, depending on my choice of H and how many translations I've got available for a given sentence. 24:22.660 --> 24:33.140 And, well, since I can assume that I will fail eventually, what we can do is just add forward error correction to the scheme, and then we can correct these errors that are statistically bound to occur. 24:36.900 --> 24:37.780 Conclusions? 24:40.880 --> 24:45.300 Translation-based steganography is a promising new approach for text steganography, in my opinion. 24:45.300 --> 25:01.240 The bitrate that we can achieve is lower than that of systems that operate on binary data, but we have the advantages, and we can defeat statistical attacks if the underlying machine-translating model the attacker is using is made public to us. 25:04.360 --> 25:05.480 Questions? 25:08.730 --> 25:10.450 Can't have no questions. 25:23.770 --> 25:26.570 I give the slides to the DEF CON goons. 25:26.570 --> 25:30.170 They are... I can put them on the website, obviously. 25:32.450 --> 25:40.170 And there is a paper at Information Hiding Workshop 2005 which also describes the scheme, except for this last bit with the forward error correction. 25:41.550 --> 25:41.750 Yeah ? 25:49.000 --> 25:54.940 Okay, so the question was if we intend to add support for languages that don't use the Roman character set. 25:54.940 --> 25:55.980 Yeah. 25:56.840 --> 26:00.200 Well, as soon as we get... I mean, the system itself uses Unicode. 26:00.280 --> 26:04.840 So if you give us a translation system that translates English to Chinese, you can use it. 26:04.840 --> 26:15.640 The problem is, if you take languages that are not inter-European anymore, the distance between the languages is so big that existing machine translation systems just cannot cope with it anymore. 26:15.640 --> 26:22.900 You just get total garbage, and it's no longer plausible for someone to send a translation of a Chinese document to English that is total garbage to someone else. 26:22.900 --> 26:29.420 The system would still work, but you lose the plausibility because you need a certain level of correctness of the machine translator to make it plausible. 26:41.160 --> 26:43.480 Okay, how do we detect tampering in the message? 26:43.480 --> 26:44.480 Well, we don't. 26:44.680 --> 26:46.260 This is a steganographic system. 26:46.260 --> 26:49.020 Typically, the goal is not to detect tampering in the message. 26:49.540 --> 26:54.280 You could, of course, just add a hash at the end or something or sign the resulting message. 26:54.420 --> 27:02.920 But the idea is the adversary doesn't even know that you're using it, so that the adversary tries to actively disrupt your communication is something that's completely out of the goals of steganography. 27:20.520 --> 27:22.700 No, it does not use symmetric key cryptography. 27:22.700 --> 27:24.800 There's no cryptography involved here at all. 27:27.100 --> 27:31.100 The shared key... we assume it's transmitted out of band up front. 27:31.100 --> 27:33.440 By some means, we don't care. 27:34.000 --> 27:35.440 We just assume it exists. 27:35.440 --> 27:37.120 The two parties have it at the beginning. 27:37.180 --> 27:41.620 They have to know that they're going to use steganography anyway, otherwise the guy probably wonders why he gets this translation. 27:44.910 --> 27:48.230 No, we don't care about public key infrastructure or anything like that. 27:57.430 --> 27:58.610 No more questions? 28:02.220 --> 28:02.820 Okay. 28:02.820 --> 28:06.820 Well, I originally told the DEF CON goons that I would give a half an hour talk and they gave me a one hour slot. 28:08.700 --> 28:18.540 So, however, what I can do very quickly is very, very quickly talk a little bit about some other project of mine that the DEF CON goons didn't want to talk about. 28:18.540 --> 28:22.460 I don't know if you've heard about Glunet. 28:22.460 --> 28:24.820 It's a glue peer-to-peer framework. 28:25.140 --> 28:31.660 Just like Phenet tries to do anonymous file sharing with running on Linux, BSD, Solaris, and so on. 28:31.660 --> 28:34.560 It's written in C code, so portability is not as given. 28:35.920 --> 28:40.160 Goals, as I said, anonymity, confidentiality, so all of the transactions are encrypted. 28:40.760 --> 28:46.500 Deniability, you're supposed to not... even if I break into your computer, you should be able to say, I didn't have these files on my machine. 28:46.500 --> 28:50.220 We're trying to make economic sense that people have incentives for sharing. 28:50.220 --> 28:51.740 And, of course, it should be efficient. 28:52.480 --> 29:00.760 And, very quickly, the upcoming, hopefully this year, 0.7 release will have a very nice plugin architecture. 29:01.060 --> 29:05.440 We can have transport so we can send either fragments like UDP or streaming. 29:05.840 --> 29:07.960 We'll have improved resource accounting. 29:07.960 --> 29:11.840 We can have, as Ian Clark just said earlier, a friend-to-friend or a peer-to-peer topology. 29:11.840 --> 29:13.140 You have the choice. 29:14.160 --> 29:18.080 Cryptographic-wise, we use AIS, SHA, and RSA. 29:18.680 --> 29:24.680 And if you want to look at the new nice features in file sharing, look for the cryptography behind k-blocks. 29:24.760 --> 29:30.240 We can associate arbitrary metadata with files so you can have ad descriptions to your search results. 29:30.260 --> 29:36.040 Or, in particular, previews so you can see before you actually download the file some preview of the contents. 29:36.720 --> 29:40.580 We get quite a big speed-up from using 32k blocks. 29:40.580 --> 29:43.940 And we have a nice, shiny, new, blade-based user interface. 29:44.800 --> 29:46.180 We're currently at pre-5. 29:46.180 --> 29:48.180 I hope the next one will be 0.7.0. 29:48.980 --> 29:52.920 And if you care to learn more about GNU Net, this is the information. 29:53.340 --> 29:57.760 I see us as a friendly competitor to Freenet, which you've heard about just before. 30:00.320 --> 30:01.120 Questions? 30:05.100 --> 30:07.180 Right, yeah, that was one of the things, right? 30:07.180 --> 30:09.160 Can you use steganography with GNU Net? 30:09.840 --> 30:10.540 Theoretically, yes. 30:10.540 --> 30:16.400 We've got this transport-layer abstraction which essentially says, okay, if you can send smoke signals to the other guy, we can transfer messages. 30:16.540 --> 30:19.220 The problem is, look at the bitrate again. 30:19.220 --> 30:22.100 If you want to do peer-to-peer with that kind of bitrate, you have a first problem. 30:22.100 --> 30:26.200 The second problem is, the steganographic encoder is quite, quite slow. 30:26.480 --> 30:31.700 So, not only would you use extreme amounts of bandwidth, but also your CPU is just going to crash and burn. 30:32.260 --> 30:36.880 And the third problem is, you need this shared secret and these machine translators in the background. 30:36.880 --> 30:44.360 And usability-wise, we want to go for a slightly larger user base than the people who would actually be capable of setting this thing up. 30:45.260 --> 30:50.480 Oh, and LIT is written in PHP and GNU is in C, so there would be some bridging code that would be also non-trivial. 30:50.800 --> 30:52.280 So, theoretically, yes. 30:52.280 --> 30:53.980 In practice, no. 30:57.440 --> 31:03.680 And besides, why would you want to do anonymous file sharing if you use steganography so nobody even knows that you're transmitting anything? 31:11.360 --> 31:12.680 No more questions? 31:13.500 --> 31:17.180 Okay, well, if you want to see me in private, I'll be somewhere outside.