[00:07.820 --> 00:16.220]  Um, when you leave this tent, if I could get you to exit through any other door, except the one
[00:16.220 --> 00:21.080]  you came in through right there. Because people come in through that door, please leave through
[00:21.080 --> 00:26.060]  any of the other doors. Thank you very much. Also, as you know, the tent is one hour behind.
[00:26.060 --> 00:30.940]  All of the other speaker rooms are operating on schedule. If you want to see the next talk in
[00:30.940 --> 00:35.740]  Apollo, I suggest that you start queuing a half an hour before the talk starts. If you're here
[00:35.740 --> 00:40.680]  in this room, odds are you will be able to stay for the next talk. We haven't had to empty this
[00:40.680 --> 00:47.960]  room all day today. That's all I have. Also, please don't take any photos during a presentation
[00:47.960 --> 00:53.840]  at DEF CON without the consent of the speaker in advance. That's a new rule that's come to...
[00:53.840 --> 01:00.560]  what? Oh, no photos of the crowd or anybody else unless you have consent of the people you're
[01:00.560 --> 01:06.540]  photographing. That's all. Are you ready? Okay. Let's give a hand for Christian. Thank you.
[01:09.930 --> 01:16.410]  Thank you. So I assume everybody here knows and loves encryption, but I think steganography is
[01:16.410 --> 01:20.850]  much cooler because in that case, the adversary doesn't know not only what you're sending, but
[01:20.850 --> 01:25.750]  also that you're sending a message at all. So it's in some sense much better. And what I'm
[01:25.750 --> 01:29.810]  trying to talk about specifically today is, well, in steganography, you always have to have
[01:29.810 --> 01:34.450]  some kind of cover message, the message the adversary sees that you're sending on the surface.
[01:34.790 --> 01:39.570]  And what I'm going to use as the cover is natural language. More common covers are images and
[01:39.570 --> 01:46.810]  sound, but we're going to use natural language here. Okay. So what have people done so far
[01:46.810 --> 01:52.590]  for natural language steganography? Well, Peter Wehner, Chapman, and DeVita did something
[01:52.590 --> 01:58.650]  where they essentially by hand wrote some context-free grammars, and then these context-free
[01:58.650 --> 02:03.490]  grammars were to decode to natural language sentences, entire texts, and depending on which
[02:03.490 --> 02:07.690]  productions you would choose in the context-free grammar, it would encode a different message.
[02:07.910 --> 02:11.770]  The problem with that is obviously that you have to construct these large context-free grammars,
[02:11.770 --> 02:16.030]  and both the sender and receiver have to have them. So one thing they did then, they
[02:16.650 --> 02:21.730]  automatically extracted syntactic templates from text and essentially took the volumes of
[02:21.730 --> 02:26.050]  Shakespeare and produced a grammar that would produce text that looked like it came from
[02:26.050 --> 02:32.810]  Shakespeare. The problem is, it was semantically terrible. Syntactically correct, somewhat,
[02:32.810 --> 02:38.670]  but semantically terrible. Chapman and DeVita improved on that by doing synonym placement
[02:38.670 --> 02:44.030]  on existing text. So you would take some existing text and just substitute some of the words
[02:44.030 --> 02:48.130]  in that text by synonyms. Synonyms are words with the same meaning, and thereby you get
[02:48.290 --> 02:53.490]  a semantically, syntactically, coherent, and rhetorically correct text, except that you
[02:53.490 --> 02:56.690]  have to come up with good synonyms, and you have to come up with this good original text
[02:56.690 --> 03:02.870]  that you're going to substitute on. So there are lots of problems with these approaches,
[03:02.870 --> 03:07.250]  and in particular with the synonym placement approach, you have things that are synonyms
[03:07.250 --> 03:12.970]  in some contexts but not in others. For example, I can say, I eat a sandwich, I devour a sandwich,
[03:12.970 --> 03:18.090]  but I cannot say, I devour. Devour requires a direct object. So a synonym is not always
[03:18.230 --> 03:24.890]  a synonym. And of course, if you need this original text that you're going to do semantic
[03:24.890 --> 03:29.090]  substitution on, if you use the same text twice, the adversary will say, wait a second,
[03:29.090 --> 03:35.110]  I just saw the same text with minimal variations, and you're exposed. If the adversary can spot
[03:35.110 --> 03:40.190]  the original text, well, differential text, or knowing discovered text. So these are problems
[03:40.190 --> 03:43.050]  with the existing approaches prior to this work.
[03:43.750 --> 03:48.030]  What do these problems really come from? Well, first of all, automatic generation of
[03:48.030 --> 03:51.790]  cinematically and rhetorically correct text is very difficult on its own. If you ever
[03:51.790 --> 03:56.910]  try to write a paper, it's hard for you. Now imagine how hard it is for a computer, especially
[03:56.910 --> 04:01.410]  with all the synchronicities in language. What all of these previous approaches have
[04:01.410 --> 04:07.210]  tried to do is they try to mimic correct text. And as in all steganographic approaches, you
[04:07.210 --> 04:12.610]  have some kind of cover object that the adversary expects, and if you deviate from this expectation,
[04:12.610 --> 04:16.510]  that's giving the attacker an attack vector. And if you're trying to mimic correct text,
[04:16.510 --> 04:20.570]  well, you better produce really correct text, otherwise the adversary can possibly spot you.
[04:21.010 --> 04:29.010]  So what we really want is something where incorrect text is OK. We want something where
[04:29.010 --> 04:33.710]  the adversary, when he gets the text, doesn't expect that this text makes perfect sense,
[04:33.710 --> 04:38.050]  is syntactically and semantically perfectly correct. Ideally, also, we can solve the other
[04:38.050 --> 04:43.350]  problem, that if the adversary gets the original text or some database, he can't say, oh, this
[04:43.350 --> 04:47.350]  other text should not exist. It's just a minor variation. It should exist together with
[04:47.350 --> 04:53.190]  whatever you used to generate your cover from. And as I said, there should be errors in there
[04:53.190 --> 04:57.570]  that we can exploit for steganographic hiding of the hidden message.
[04:59.470 --> 05:04.730]  Well, I'm sure you've all read the title, so you know what I'm heading for. Here's some
[05:04.730 --> 05:11.230]  example from Babelfish. Took a site from a German Linux camp site. In German it reads,
[05:11.230 --> 05:15.110]  keine Sorge, sie sind alle handsam und beantworten auf freiwillig Fragengründen das Thema Linux
[05:15.110 --> 05:19.490]  und geben gerne einen Klein-MIG-Innovator open source. OK, nobody understood that one, right?
[05:20.450 --> 05:24.090]  If you translate this to English by hand, you get something like, don't worry, they're all tame
[05:24.090 --> 05:27.770]  and will also readily answer questions regarding the topic Linux. I'm glad to give a small glimpse
[05:27.770 --> 05:32.890]  into the world of open source. Now, if you feed this into Babelfish, I don't know, I assume
[05:32.890 --> 05:36.970]  most of you have used this translation at some point or other, say on Heise or something,
[05:36.970 --> 05:41.990]  you get a concern, it are all not handsome and also readily questions approximately around
[05:41.990 --> 05:47.330]  the topic Linux and give gladly a small idea into the world of the open source.
[05:49.430 --> 05:53.630]  OK, so first of all, the not is kind of in the wrong place. It's not a concern as opposed to
[05:53.630 --> 05:59.010]  it's a concern that they're not tame. It didn't translate handsome at all because it wasn't in the
[05:59.010 --> 06:03.590]  dictionary and it adds an article, the, in front of open source and I don't know why it does this
[06:03.590 --> 06:08.090]  strange capitalization. But you can see there are plenty of errors in here and still, this text,
[06:08.090 --> 06:13.130]  you might get some idea of what this event was about, right? So it's still useful for humans,
[06:13.130 --> 06:16.870]  it's still perfectly plausible that somebody sends you this, say you don't speak German
[06:16.870 --> 06:23.950]  and he sends you a rough translation. So, let's see. How does machine translation work?
[06:23.950 --> 06:29.130]  Well, machine translation, first of all, there are lots of low quality encoders out there.
[06:31.490 --> 06:36.870]  They can, there's never a perfect translation anyway, right? If you translate from one language
[06:36.870 --> 06:40.750]  to the other, there are multiple variations of how you can express it in the target language.
[06:40.870 --> 06:46.550]  So, if there is a variation in the translation, it's OK, even if you have a human translation
[06:46.550 --> 06:52.810]  and even voiceless machine translation. What current machine translation systems do
[06:52.810 --> 06:57.750]  is they are statistical. They have a language model, a translation model, which maps words
[06:57.750 --> 07:03.330]  and sentences. They have some idea of syntactic and semantic rules, mostly do pattern matching.
[07:03.470 --> 07:08.110]  They always ignore all of the context, so they look at one sentence at a time, at least all of the
[07:08.110 --> 07:14.610]  ones that we found, which can have some pretty bad results. Say somebody talks about pipelines
[07:14.610 --> 07:18.750]  and seals, and suddenly talks about sea lions in the middle of the translation, because the translation
[07:18.750 --> 07:26.970]  picked the wrong word in the dictionary to translate seals. So there's plenty of errors in machine translation.
[07:29.870 --> 07:33.870]  Now, one other thing I should say, even if these errors weren't there, even if machine translation
[07:33.870 --> 07:37.790]  were perfect, as I said before, there's always some room in natural language, so we're not just
[07:37.790 --> 07:43.610]  exploiting an error in current software. So, how does the basic protocol work?
[07:43.610 --> 07:50.310]  Well, we assume that Alice wants to send a message to Bob, and, well, as usual, they have some shared secret.
[07:50.310 --> 07:54.810]  In our case, what kind of specific translation engines they're going to use, and other things that define
[07:54.810 --> 08:01.110]  the configuration of the system. And Alice first has to select some source text. And in our case, it can be
[08:01.110 --> 08:05.710]  from a public news source, so you could just take text from BBC, the Bible, whatever you want.
[08:05.750 --> 08:10.570]  You could write some on your own. It doesn't have to be secret for the system to work in general.
[08:10.570 --> 08:14.770]  It can have advantages, as you'll see towards the end of the talk.
[08:15.450 --> 08:21.570]  So, given this cover source text, the translator configuration, and the secret message, Alice gives all of this
[08:21.570 --> 08:28.830]  to our tool, Lit, which, by the way, is available both as a PHP demo and in code at the URL at the bottom
[08:28.830 --> 08:33.890]  of the slides. Don't go all there at once and try the demo. The server is a really, really slow machine.
[08:33.890 --> 08:40.050]  It's processing PHP, and you'll just kill it. However, it works if you don't try it all at once.
[08:42.330 --> 08:48.570]  Anyway, so Alice gives this to the Lost in Translation system, and as you would expect, gets back a cover translation.
[08:48.790 --> 08:54.190]  She sends that cover translation together with either the cover source text or a reference to it to Bob,
[08:54.190 --> 09:01.170]  and Bob just feeds his shared secret, again, the cover source text and the cover translation to his copy of the Lit system
[09:01.170 --> 09:04.570]  and gets back the secret message. So far, so easy.
[09:06.730 --> 09:11.610]  So how does the encoding really work? First, we take our cover source text and run it through several commercial
[09:11.610 --> 09:17.270]  and custom-generated translation engines. In our case, since we weren't really up to the task of writing our own
[09:17.270 --> 09:23.730]  machine translation software, well, we go online, we query Babelfish, we query Google, we query SysTran
[09:23.730 --> 09:27.610]  and a couple of other engines that are out there and get back translations.
[09:29.350 --> 09:32.890]  Then, after we get these translations, we want actually more than just two or three variations
[09:32.890 --> 09:36.970]  because then the bit rate would be very low. So what we do is we add additional variations to the sentences,
[09:36.970 --> 09:43.230]  making slight modifications that are presumably plausible for S.S. translation errors, possibly even fixing translation errors
[09:43.990 --> 09:46.450]  and we get even more sentences out of that.
[09:47.790 --> 09:55.390]  As we do so, whenever we add these variations, we compute probabilities that this is actually still a valid translation.
[09:55.390 --> 10:04.110]  And at the end, once we have lots of sentences, lots of translations for a given sentence and probabilities,
[10:04.110 --> 10:09.870]  we build a Huffman tree and then we walk through this Huffman tree based on the secret message we're trying to encode.
[10:10.550 --> 10:14.090]  So the picture says more than I can speak into the microphone.
[10:14.490 --> 10:20.010]  Again, cover source text goes into commercial translators, which ones are again selected by the configuration.
[10:20.010 --> 10:22.850]  You can add new ones if you are so inclined to the tool.
[10:22.850 --> 10:26.270]  Then we do some post-processing to get even more sentences.
[10:27.770 --> 10:32.610]  Then we have some model that will tell us how likely are these sentences actually good translations.
[10:32.810 --> 10:35.050]  Based on these probabilities, we build a Huffman tree.
[10:35.290 --> 10:39.470]  And then based on the secret message, we select which translation we're going to emit.
[10:39.850 --> 10:43.790]  We do this on a sentence-by-sentence basis because the translators work like that.
[10:44.930 --> 10:50.790]  Eventually we either run off cover source text and cut the secret message or we run off secret message and pad with random bits.
[10:51.030 --> 10:52.090]  Very simple, right?
[10:52.090 --> 10:53.670]  Now what does the decoder do?
[10:53.670 --> 10:55.050]  Well, pretty much the same thing.
[10:55.050 --> 11:01.130]  Except that of course, at the end, we get the cover translation in.
[11:01.130 --> 11:02.490]  We still built the Huffman tree.
[11:02.710 --> 11:06.550]  And now we essentially go upwards and say, okay, which were the bits that were leading to the sentence in this Huffman tree.
[11:06.910 --> 11:08.410]  And voila, we got our secret message.
[11:09.950 --> 11:12.610]  To give you some idea of what the post-passes look like.
[11:13.270 --> 11:14.730]  Some of them, very simple one.
[11:15.110 --> 11:20.410]  What we found when we looked at translations, one of the things that are often wrong is that they pick the wrong kind of article.
[11:20.410 --> 11:21.930]  In English, definite versus indefinite.
[11:21.930 --> 11:23.370]  In German, the wrong gender.
[11:24.790 --> 11:25.490]  And so on.
[11:25.490 --> 11:27.430]  So that's one thing we can play with.
[11:27.430 --> 11:31.210]  Or some words do not have an article, should have one and vice versa.
[11:31.230 --> 11:32.670]  So that's something we can do.
[11:34.310 --> 11:36.230]  Also prepositions are very tricky.
[11:36.230 --> 11:38.590]  Should it be to, should it be towards, should it be from?
[11:38.970 --> 11:40.910]  So we can play with them a little bit.
[11:41.350 --> 11:47.510]  And again, we did some manual shuffling of what are prepositions that are close to each other that are likely to be mistakes.
[11:48.110 --> 11:50.630]  And then that's where the probabilities come from.
[11:51.070 --> 11:55.550]  And of course what you can do is if you have a dictionary that lists you how often a given word occurs in a language.
[11:55.550 --> 12:00.530]  You can say, okay, this word is one of the ten million words that nobody ever knows.
[12:00.550 --> 12:01.830]  So likely the translator didn't know it.
[12:01.830 --> 12:04.070]  So we leave it in the original language even if it was translated.
[12:04.070 --> 12:04.830]  Or vice versa.
[12:04.830 --> 12:05.930]  We have it in our dictionary.
[12:05.950 --> 12:06.770]  Babafish didn't.
[12:06.770 --> 12:07.210]  Go us.
[12:07.210 --> 12:08.450]  We can fix this error.
[12:08.990 --> 12:12.570]  Right, so again you get two variations just by doing this little change.
[12:12.570 --> 12:17.570]  And you can have in the archimetry additional ways for encoding information.
[12:18.750 --> 12:22.150]  Another post-pass that I like very much is semantic substitution.
[12:22.310 --> 12:26.410]  We talked earlier about synonyms and how hard they are to get and that you have to construct them by hand.
[12:26.470 --> 12:33.690]  Now if we have a translation, this gives us a way to construct synonyms in a way that we call second-order semantic substitution.
[12:34.190 --> 12:36.890]  So what we do is we have a German word, say, flach.
[12:37.110 --> 12:41.050]  And the original machine translator translated it to say flat.
[12:41.430 --> 12:43.090]  Okay, that's one choice.
[12:43.130 --> 12:45.830]  Now if we look at our dictionary, we would have had other words.
[12:45.830 --> 12:50.130]  Tabular, vapid, even, smooth, plain, plain, shallow, and so on and so on.
[12:50.270 --> 12:54.170]  Now some of these are semantically rather far away from flat.
[12:54.330 --> 12:56.690]  So we would not just pick a random one out of these.
[12:56.690 --> 12:57.370]  But what can we do?
[12:57.370 --> 12:59.730]  We can translate flat back into German.
[13:00.250 --> 13:02.170]  We get flach, eben, and glatt.
[13:02.650 --> 13:05.090]  So these are also again quite close now in German.
[13:05.530 --> 13:07.950]  And if we translate these words now back into English.
[13:08.190 --> 13:09.670]  See it's going back and forth here.
[13:09.670 --> 13:14.630]  Then it says, okay, eben goes to even, glatt goes to plain, plain and smooth and so on.
[13:14.630 --> 13:16.990]  So there's some of these words which kind of form a cluster.
[13:17.310 --> 13:19.950]  A cluster that shows that they're semantically quite close.
[13:19.950 --> 13:22.070]  And we can do that because we've got these two languages.
[13:22.410 --> 13:26.110]  And now assuming that the original translator might have taken a little bit of context from the current sentence
[13:26.110 --> 13:30.590]  and chosen a good translation in flat, we get good semantic substitutions here.
[13:30.790 --> 13:35.290]  Now if the original translator chose badly, well, then we're not going to do much damage anyway
[13:35.290 --> 13:36.770]  because it was a bad choice.
[13:37.010 --> 13:38.970]  So it's again very plausible to do this.
[13:38.970 --> 13:42.850]  And the number of witnesses you pick for the semantic substitution pass you can configure
[13:42.850 --> 13:45.090]  and it's one of these configuration parameters.
[13:45.290 --> 13:46.850]  Just like, do you want to use this at all?
[13:50.730 --> 13:54.030]  And the tool itself, you can plug in new modules if you feel so inclined.
[13:54.030 --> 13:57.390]  You just have to study machine translation, figure out what is actually a plausible error.
[14:02.230 --> 14:04.630]  So I'm sure you all want to see what it looks like, right?
[14:04.970 --> 14:07.290]  So let's take an original sentence in German.
[14:07.290 --> 14:10.410]  In dieser Zeit soll festgestellt werden, ob die Schüler die richtige Schule gewählt haben
[14:10.410 --> 14:12.230]  und ob sie ihren Fähigkeiten entspricht.
[14:12.270 --> 14:12.910]  Oh God.
[14:13.210 --> 14:17.590]  In Google, this looks like, in this time it is to be determined whether the pupil selected the correct school
[14:17.590 --> 14:19.890]  and whether it corresponds to its abilities.
[14:19.950 --> 14:21.550]  Quite a good translation, actually.
[14:21.990 --> 14:26.130]  Now, LinguaTek, another engine that we're using, gives whether the pupils have chosen the right school
[14:26.130 --> 14:29.230]  and whether it corresponds to its abilities shall be found out at this time.
[14:29.670 --> 14:33.110]  If you use Lit, and this encodes 8 bits, so that's quite a bit of information,
[14:33.110 --> 14:34.730]  quite a number of choices that we had here,
[14:34.730 --> 14:38.630]  you get, in this time, it is to be determined whether the pupil selected the correct school
[14:38.630 --> 14:41.010]  and whether it corresponds to its abilities.
[14:41.250 --> 14:44.790]  That's not better, but it's not that much worse that you could say,
[14:44.790 --> 14:46.710]  well, there couldn't be a translator which does this.
[14:46.830 --> 14:51.670]  And yes, you might notice the closeness to Google, because in this case it selected Google as the basis.
[14:51.890 --> 14:55.710]  But now assume you've got a couple of hundred translation packages out there,
[14:55.710 --> 14:57.990]  and you don't know which one was chosen.
[14:58.350 --> 15:00.890]  You don't have this comparison directly at hand.
[15:00.890 --> 15:04.750]  Maybe we'll see later how we can even hide the original text, so you don't have that one.
[15:04.750 --> 15:07.190]  You don't have the base translation, you just get one translation.
[15:07.670 --> 15:10.010]  And you can say, yeah, well, maybe a human wouldn't have produced that,
[15:10.010 --> 15:12.350]  only a first class in English.
[15:12.570 --> 15:17.310]  But, hey, hard to tell that there was really something going on.
[15:18.370 --> 15:19.590]  Give you another example.
[15:19.910 --> 15:21.490]  This is from a film review.
[15:21.490 --> 15:27.710]  Der marokkanische Film Windhorse erzählt die Geschichte zweier unterschiedlicher Generationen angehorener Männer, die durch Marokko reisen.
[15:27.990 --> 15:31.250]  Auf dem Weg suchen sie nach dem einzigen, was ihnen wichtig ist, dem Sinn des Lebens.
[15:31.890 --> 15:34.810]  Okay, I'm not going to read the Google and Linguatech translations.
[15:35.170 --> 15:41.790]  However, what this gives us is the Marokkan film Windhorse tells a story from men belonging to two different generations who travel through Morocco.
[15:41.950 --> 15:45.850]  They are looking for the only one which is important to them on the way, the sense of a life.
[15:46.370 --> 15:48.790]  Now, the best translation would leave out the Ö.
[15:49.510 --> 15:51.970]  Well, Google and Linguatech put a ö in there.
[15:51.970 --> 15:54.410]  And actually, lit, as one of the choices, don't put anything.
[15:54.930 --> 15:57.510]  Well, we included the string lit here, so we got this one.
[15:57.750 --> 16:01.110]  But as you can see, it might even improve the quality of the given translation,
[16:01.110 --> 16:03.870]  and at least it's not necessarily a significant difference.
[16:06.030 --> 16:07.710]  So what are the advantages again?
[16:07.930 --> 16:11.010]  We can hide information within the limits of machine translation,
[16:11.010 --> 16:15.750]  and as machine translation improves, our system will also automatically improve in what it's generating.
[16:16.810 --> 16:20.850]  We avoid this generation problem by mimicking the results of an imperfect transformation.
[16:21.210 --> 16:23.830]  So we don't have to produce correct text.
[16:23.850 --> 16:26.570]  We can get away with producing slightly incorrect text.
[16:26.710 --> 16:31.310]  And what is also interesting, if you have a larger document with some rhetorical structure, we get that.
[16:31.310 --> 16:33.310]  The translation gets the rhetorical structure.
[16:33.410 --> 16:37.490]  If you use any of the other generation systems, well, if you're lucky, you get one sentence correct,
[16:37.490 --> 16:44.770]  but you're never going to get a large book or article correct that way, in terms of rhetorical structure and so on.
[16:45.450 --> 16:50.150]  We have a secret key that can be used to configure the system.
[16:50.370 --> 16:52.110]  And to that, you can have it very small.
[16:52.110 --> 16:56.390]  You can just say, OK, I'm using Babel and Fish and Google and semantic substitution with two witnesses.
[16:56.550 --> 16:57.990]  Or you can make it more complicated.
[16:57.990 --> 17:02.390]  You can say, I've got this machine translation system, and it was trained on this corpus.
[17:02.810 --> 17:04.050]  Machine translations are statistical.
[17:04.050 --> 17:09.830]  So they are typically done in a way that you have some large parallel corpus of sentences in English and in German,
[17:09.830 --> 17:13.030]  and the machine translation is supposed to learn what a good translation is.
[17:13.390 --> 17:16.070]  And if you have a different corpus, you get a different machine translator.
[17:16.250 --> 17:19.610]  So you can make this entire corpus part of your secret, if you so want to.
[17:20.170 --> 17:22.710]  So you get lots of possible variations there.
[17:23.670 --> 17:28.550]  And again, for now, we can also say that the cover text was from a public source.
[17:28.550 --> 17:32.710]  You don't have to write a new one each time, take a new image, or whatever else you used to do in the cover.
[17:33.730 --> 17:34.970]  What are the disadvantages?
[17:35.390 --> 17:38.390]  Well, the bitrate is not really high still, right?
[17:39.610 --> 17:45.290]  One of the big disadvantages of this text is always very robust compared to, say, binary data like images.
[17:45.670 --> 17:50.530]  And as I said multiple times before, we need to transmit both the source text, or some reference at least to it,
[17:50.530 --> 17:53.150]  and the translation for the decoder to work.
[17:54.790 --> 17:57.830]  Well, let's try to address these two issues.
[18:03.120 --> 18:05.220]  So what can we do to increase the bitrate?
[18:05.260 --> 18:07.680]  Well, we can create more machine translation systems.
[18:08.260 --> 18:14.160]  I'm sure the people who are doing machine translation will do that for us anyway, but that's one way of doing it.
[18:14.860 --> 18:19.700]  We can, as I said, create new corpus to train existing machine translation implementations.
[18:19.700 --> 18:20.920]  And that's actually quite easy.
[18:20.920 --> 18:27.520]  If I've got a parallel corpus of 20,000 words, I can select any subset of this corpus and get another corpus.
[18:28.020 --> 18:34.300]  So that's easy if I have the full source of a machine translation system and can train it myself.
[18:34.900 --> 18:40.260]  And, of course, you can write additional post-passes or pre-passes that will make plausible modifications to sentences.
[18:42.860 --> 18:46.520]  So what we've done so far is we have a prototype implementation.
[18:46.680 --> 18:48.880]  And that, of course, means we have limited dictionaries.
[18:49.080 --> 18:53.500]  We did not write anything that would give us knowledge about grammar or semantics of a language.
[18:53.500 --> 18:56.720]  And we only have a couple of translation engines available to us.
[18:57.120 --> 19:03.040]  Still, what we get is a bitrate of something like 0.082 bits per bit transmitted.
[19:04.100 --> 19:05.700]  Now, this is for normal text.
[19:05.700 --> 19:11.520]  If I just gzip the text, I get a much better bitrate, which is 0.022, which is a bit more aligned with...
[19:11.520 --> 19:17.040]  If I take a PNG or JPEG image and hide something in it, well, I could compress this and get a bit...
[19:20.860 --> 19:24.700]  Quickly, some minor variation that will become interesting soon.
[19:25.360 --> 19:28.900]  We could use this scheme not only for steganography, but also for watermarking.
[19:29.280 --> 19:34.340]  Often, watermarking and steganographic systems are related, so this is quite natural to do.
[19:35.040 --> 19:40.440]  In watermarking, what we want to do is we want to be able to read some embedded mark from just the translation in this case.
[19:40.440 --> 19:42.560]  We don't want to need the original text.
[19:43.300 --> 19:44.540]  So, what can we do here?
[19:44.700 --> 19:50.960]  Well, we could use some keytash of a certain set of sentences and take the least significant bit of that, and that's going to be our marked bit.
[19:50.960 --> 20:01.320]  And essentially, when we pick our sentences in the translation, we're going to go through all of the sentences, compute all the keytash, and torture it until we get a sentence that actually gives us the marked bit.
[20:01.320 --> 20:04.680]  Since we have lots of variations, we would statistically succeed most of the time.
[20:07.140 --> 20:11.240]  And of course, we use multiple sentences if we need multiple marked bits.
[20:12.060 --> 20:13.760]  Which sentence are we going to pick?
[20:13.780 --> 20:19.600]  Well, we could just have some other secret mechanism that gives us sentence 5, 10, 17.
[20:20.960 --> 20:24.140]  Or just at the beginning, but this makes the watermark kind of fragile.
[20:24.140 --> 20:27.840]  The adversary could just insert a sentence at the beginning and suddenly all of our offsets are wrong.
[20:28.160 --> 20:38.320]  What is a better technique is to say, we have some arbitrary mechanism to rank sentences, all of the sentences in the translation, and that just gives us some total ordering.
[20:38.320 --> 20:44.800]  We take the sentences that have the scores lowest on this ordering, and the sentences after that in the document are the ones that are going to hold the marked bits.
[20:44.980 --> 20:48.820]  This way, an insertion or deletion attack will not necessarily succeed in destroying the mark.
[20:53.480 --> 20:55.880]  It wouldn't be DEF CON if we didn't have some attacks, right?
[20:56.260 --> 20:57.800]  So what can we do about attacks?
[20:57.940 --> 21:02.180]  Well, first of all, there is the possibility of somebody spotting obvious inconsistencies.
[21:02.180 --> 21:09.480]  The most easy one that comes to mind is, say you've got a translation which in the original text contains the same sentence twice, and you translate it in different ways.
[21:09.680 --> 21:11.300]  A machine translation would never do that.
[21:11.760 --> 21:15.100]  It's a deterministic machine, and now suddenly it does it not.
[21:15.360 --> 21:18.100]  So, that's in some sense a simple bug.
[21:18.100 --> 21:25.340]  In the LIT system, you just have to keep track of which sentences have you translated so far, and if you see the same sentence again, well, you're not going to use that to encode anything.
[21:25.340 --> 21:27.760]  You have to omit the sentence you omitted the first time.
[21:28.220 --> 21:35.600]  Similar, there are certain mistakes that machine translators make, like translate the plural of foot as foots.
[21:36.040 --> 21:37.860]  Should be feet, obviously, right?
[21:38.860 --> 21:46.080]  If you pick two different machine translators, where one makes this mistake and the other one doesn't, and you alternate between them, you get inconsistent mistakes in your translation.
[21:46.080 --> 21:54.620]  And the reader will wonder, okay, so a human might make this mistake once but not another time, but a machine translator should make mistakes consistently.
[21:54.680 --> 21:57.180]  So that's another possible source of bugs.
[21:57.900 --> 22:04.120]  Which you would fix by, obviously, writing some kind of post-pass or processing that makes sure that these don't happen if you can spot them.
[22:04.120 --> 22:16.740]  In general, essentially one problem that most steganographic systems have, somebody could construct a new statistical model that says all of the translation systems in the world obey this model, except for this one.
[22:16.860 --> 22:21.780]  That applies the same to image steganography or something with audio files.
[22:22.100 --> 22:25.420]  Somebody has a model, and you deviate from it, and you shouldn't.
[22:25.420 --> 22:26.860]  How can you defeat that?
[22:27.020 --> 22:32.620]  Well, let's assume, just for fairness, that we also know about that model.
[22:32.620 --> 22:40.240]  If we know about that model, we can easily make our encoder aware of that model by sorting out sentences in the translation process that don't obey the model.
[22:40.420 --> 22:42.240]  And then we can't be detected anymore.
[22:42.880 --> 22:50.800]  What is interesting is that creating such a model is in some sense equivalent of improving statistical machine translation because, well, what is this machine translator?
[22:50.800 --> 22:53.920]  It is a model of how one sentence corresponds to another.
[22:54.700 --> 22:56.480]  And in the end, it's an arms race.
[22:56.480 --> 22:58.160]  Whoever got the better model wins.
[22:58.160 --> 23:06.140]  If the attacker and the defender have the same quality of machine translation model, then the defender will win because the attacker will have to say, oh, it's plausible.
[23:09.560 --> 23:11.460]  Finally, the most fun part.
[23:11.460 --> 23:15.660]  How can we avoid transmitting the original document for the system?
[23:16.860 --> 23:19.880]  Remember the least significant bit ID for watermarking?
[23:20.440 --> 23:24.500]  Now, what if we take H bits in each sentence?
[23:25.340 --> 23:31.220]  The first H bits that tell us how many of these least significant bits in the hash are going to actually contain a message.
[23:31.960 --> 23:34.720]  So, I'm a sender, some receiver.
[23:34.820 --> 23:36.220]  We have some small constant H.
[23:36.220 --> 23:42.960]  When I pick my sentences, I try to pick one where the first two bits, say, encode the number one.
[23:42.960 --> 23:44.580]  That means I'm going to transmit one bit.
[23:44.580 --> 23:47.700]  And then the third bit is going to transmit the actual message bit.
[23:47.700 --> 23:55.120]  Or, if I have more variations available for this given sentence, I could transmit one where it has the first two bits set, meaning the number three,
[23:55.120 --> 23:59.100]  and then follow three bits in this least significant hash that contain the actual message.
[23:59.600 --> 24:07.520]  This is a statistical way, just based on the fact that hashes are pseudorandom, that I can encode the message by just transmitting the translation.
[24:07.980 --> 24:15.460]  And, well, I might fail to find one of the translations in the variation that allow me to encode this,
[24:15.460 --> 24:22.120]  but I'm unlikely to fail, statistically speaking, depending on my choice of H and how many translations I've got available for a given sentence.
[24:22.660 --> 24:29.700]  And, well, since I can assume that I will fail eventually, what we can do is just add forward error correction to the scheme,
[24:29.700 --> 24:33.140]  and then we can correct these errors that are statistically bound to occur.
[24:36.900 --> 24:37.780]  Conclusions?
[24:40.880 --> 24:45.300]  Translation-based steganography is a promising new approach for text steganography, in my opinion.
[24:45.300 --> 24:50.060]  The bitrate that we can achieve is lower than that of systems that operate on binary data,
[24:51.300 --> 25:01.240]  but we have the advantages, and we can defeat statistical attacks if the underlying machine-translating model the attacker is using is made public to us.
[25:04.360 --> 25:05.480]  Questions?
[25:08.730 --> 25:10.450]  Can't have no questions.
[25:23.770 --> 25:26.570]  I give the slides to the DEF CON goons.
[25:26.570 --> 25:30.170]  They are... I can put them on the website, obviously.
[25:32.450 --> 25:40.170]  And there is a paper at Information Hiding Workshop 2005 which also describes the scheme, except for this last bit with the forward error correction.
[25:41.550 --> 25:41.750]  Yeah?
[25:49.000 --> 25:54.940]  Okay, so the question was if we intend to add support for languages that don't use the Roman character set.
[25:54.940 --> 26:00.200]  Yeah. Well, as soon as we get... I mean, the system itself uses Unicode.
[26:00.280 --> 26:04.840]  So if you give us a translation system that translates English to Chinese, you can use it.
[26:04.840 --> 26:08.880]  The problem is, if you take languages that are not inter-European anymore,
[26:08.880 --> 26:15.640]  the distance between the languages is so big that existing machine translation systems just cannot cope with it anymore.
[26:15.640 --> 26:22.900]  You just get total garbage, and it's no longer plausible for someone to send a translation of a Chinese document to English that is total garbage to someone else.
[26:22.900 --> 26:29.420]  The system would still work, but you lose the plausibility because you need a certain level of correctness of the machine translator to make it plausible.
[26:41.160 --> 26:43.480]  Okay, how do we detect tampering in the message?
[26:43.480 --> 26:44.480]  Well, we don't.
[26:44.680 --> 26:46.260]  This is a steganographic system.
[26:46.260 --> 26:49.020]  Typically, the goal is not to detect tampering in the message.
[26:49.540 --> 26:54.280]  You could, of course, just add a hash at the end or something or sign the resulting message.
[26:54.420 --> 26:56.700]  But the idea is the adversary doesn't even know that you're using it,
[26:56.700 --> 27:02.920]  so that the adversary tries to actively disrupt your communication is something that's completely out of the goals of steganography.
[27:20.520 --> 27:24.800]  No, it does not use symmetric key cryptography. There's no cryptography involved here at all.
[27:27.100 --> 27:31.100]  The shared key... we assume it's transmitted out of band up front.
[27:31.100 --> 27:33.440]  By some means, we don't care.
[27:34.000 --> 27:37.120]  We just assume it exists. The two parties have it at the beginning.
[27:37.180 --> 27:41.620]  They have to know that they're going to use steganography anyway, otherwise the guy probably wonders why he gets this translation.
[27:44.910 --> 27:48.230]  No, we don't care about public key infrastructure or anything like that.
[27:57.430 --> 27:58.610]  No more questions?
[28:02.220 --> 28:06.820]  Okay. Well, I originally told the DEF CON goons that I would give a half an hour talk and they gave me a one hour slot.
[28:08.700 --> 28:18.540]  So, however, what I can do very quickly is very, very quickly talk a little bit about some other project of mine that the DEF CON goons didn't want to talk about.
[28:18.540 --> 28:24.820]  I don't know if you've heard about Glunet. It's a glue peer-to-peer framework.
[28:25.140 --> 28:31.660]  Just like Phenet tries to do anonymous file sharing with running on Linux, BSD, Solaris, and so on.
[28:31.660 --> 28:34.560]  It's written in C code, so portability is not as given.
[28:35.920 --> 28:40.160]  Goals, as I said, anonymity, confidentiality, so all of the transactions are encrypted.
[28:40.760 --> 28:46.500]  Deniability, you're supposed to not... even if I break into your computer, you should be able to say, I didn't have these files on my machine.
[28:46.500 --> 28:50.220]  We're trying to make economic sense that people have incentives for sharing.
[28:50.220 --> 28:51.740]  And, of course, it should be efficient.
[28:52.480 --> 29:00.760]  And, very quickly, the upcoming, hopefully this year, 0.7 release will have a very nice plugin architecture.
[29:01.060 --> 29:05.440]  We can have transport so we can send either fragments like UDP or streaming.
[29:05.840 --> 29:07.960]  We'll have improved resource accounting.
[29:07.960 --> 29:11.840]  We can have, as Ian Clark just said earlier, a friend-to-friend or a peer-to-peer topology.
[29:11.840 --> 29:13.140]  You have the choice.
[29:14.160 --> 29:18.080]  Cryptographic-wise, we use AIS, SHA, and RSA.
[29:18.680 --> 29:24.680]  And if you want to look at the new nice features in file sharing, look for the cryptography behind k-blocks.
[29:24.760 --> 29:30.240]  We can associate arbitrary metadata with files so you can have ad descriptions to your search results.
[29:30.260 --> 29:36.040]  Or, in particular, previews so you can see before you actually download the file some preview of the contents.
[29:36.720 --> 29:40.580]  We get quite a big speed-up from using 32k blocks.
[29:40.580 --> 29:43.940]  And we have a nice, shiny, new, blade-based user interface.
[29:44.800 --> 29:48.180]  We're currently at pre-5. I hope the next one will be 0.7.0.
[29:48.980 --> 29:52.920]  And if you care to learn more about GNU Net, this is the information.
[29:53.340 --> 29:57.760]  I see us as a friendly competitor to Freenet, which you've heard about just before.
[30:00.320 --> 30:01.120]  Questions?
[30:05.100 --> 30:07.180]  Right, yeah, that was one of the things, right?
[30:07.180 --> 30:09.160]  Can you use steganography with GNU Net?
[30:09.840 --> 30:12.740]  Theoretically, yes. We've got this transport-layer abstraction which essentially says,
[30:12.740 --> 30:16.400]  okay, if you can send smoke signals to the other guy, we can transfer messages.
[30:16.540 --> 30:19.220]  The problem is, look at the bitrate again.
[30:19.220 --> 30:22.100]  If you want to do peer-to-peer with that kind of bitrate, you have a first problem.
[30:22.100 --> 30:26.200]  The second problem is, the steganographic encoder is quite, quite slow.
[30:26.480 --> 30:31.700]  So, not only would you use extreme amounts of bandwidth, but also your CPU is just going to crash and burn.
[30:32.260 --> 30:36.880]  And the third problem is, you need this shared secret and these machine translators in the background.
[30:36.880 --> 30:44.360]  And usability-wise, we want to go for a slightly larger user base than the people who would actually be capable of setting this thing up.
[30:45.260 --> 30:50.480]  Oh, and LIT is written in PHP and GNU is in C, so there would be some bridging code that would be also non-trivial.
[30:50.800 --> 30:53.980]  So, theoretically, yes. In practice, no.
[30:57.440 --> 31:03.680]  And besides, why would you want to do anonymous file sharing if you use steganography so nobody even knows that you're transmitting anything?
[31:11.360 --> 31:12.680]  No more questions?
[31:13.500 --> 31:17.180]  Okay, well, if you want to see me in private, I'll be somewhere outside.