1
00:00:07,820 --> 00:00:17,720
Um, when you leave this tent, if I could get you to exit through any other door, except the one you came in through right there.

2
00:00:17,820 --> 00:00:22,660
Because people come in through that door, please leave through any of the other doors.

3
00:00:22,660 --> 00:00:23,380
Thank you very much.

4
00:00:23,380 --> 00:00:26,060
Also, as you know, the tent is one hour behind.

5
00:00:26,060 --> 00:00:29,540
All of the other speaker rooms are operating on schedule.

6
00:00:29,540 --> 00:00:35,160
If you want to see the next talk in Apollo, I suggest that you start queuing a half an hour before the talk starts.

7
00:00:35,160 --> 00:00:39,640
If you're here in this room, odds are you will be able to stay for the next talk.

8
00:00:39,640 --> 00:00:41,980
We haven't had to empty this room all day today.

9
00:00:42,820 --> 00:00:44,300
That's all I have.

10
00:00:44,300 --> 00:00:52,020
Also, please don't take any photos during a presentation at DEF CON without the consent of the speaker in advance.

11
00:00:52,020 --> 00:00:53,840
That's a new rule that's come to...

12
00:00:53,840 --> 00:00:54,460
what?

13
00:00:55,440 --> 00:01:01,560
Oh, no photos of the crowd or anybody else unless you have consent of the people you're photographing.

14
00:01:02,520 --> 00:01:03,380
That's all.

15
00:01:03,380 --> 00:01:04,100
Are you ready?

16
00:01:04,620 --> 00:01:05,000
Okay.

17
00:01:05,000 --> 00:01:05,900
Let's give a hand for Christian.

18
00:01:05,900 --> 00:01:06,540
Thank you.

19
00:01:09,930 --> 00:01:10,790
Thank you.

20
00:01:12,010 --> 00:01:22,450
So I assume everybody here knows and loves encryption, but I think steganography is much cooler because in that case, the adversary doesn't know not only what you're sending, but also that you're sending a message at all.

21
00:01:22,550 --> 00:01:24,090
So it's in some sense much better.

22
00:01:25,350 --> 00:01:34,450
And what I'm trying to talk about specifically today is, well, in steganography, you always have to have some kind of cover message, the message the adversary sees that you're sending on the surface.

23
00:01:34,790 --> 00:01:37,950
And what I'm going to use as the cover is natural language.

24
00:01:38,030 --> 00:01:42,450
More common covers are images and sound, but we're going to use natural language here.

25
00:01:43,770 --> 00:01:44,810
Okay.

26
00:01:45,150 --> 00:01:49,010
So what have people done so far for natural language steganography?

27
00:01:49,250 --> 00:02:05,890
Well, Peter Wehner, Chapman, and DeVita did something where they essentially by hand wrote some context-free grammars, and then these context-free grammars were to decode to natural language sentences, entire texts, and depending on which productions you would choose in the context-free grammar,

28
00:02:05,890 --> 00:02:07,690
it would encode a different message.

29
00:02:07,910 --> 00:02:13,850
The problem with that is obviously that you have to construct these large context-free grammars, and both the sender and receiver have to have them.

30
00:02:14,630 --> 00:02:26,570
So one thing they did then, they automatically extracted syntactic templates from text and essentially took the volumes of Shakespeare and produced a grammar that would produce text that looked like it came from Shakespeare.

31
00:02:26,570 --> 00:02:30,150
The problem is, it was semantically terrible.

32
00:02:31,150 --> 00:02:34,210
Syntactically correct, somewhat, but semantically terrible.

33
00:02:34,710 --> 00:02:39,850
Chapman and DeVita improved on that by doing synonym placement on existing text.

34
00:02:39,850 --> 00:02:45,390
So you would take some existing text and just substitute some of the words in that text by synonyms.

35
00:02:45,390 --> 00:02:57,910
Synonyms are words with the same meaning, and thereby you get a semantically, syntactically, coherent, and rhetorically correct text, except that you have to come up with good synonyms, and you have to come up with this good original text that you're going to substitute on.

36
00:03:00,750 --> 00:03:08,810
So there are lots of problems with these approaches, and in particular with the synonym placement approach, you have things that are synonyms in some contexts but not in others.

37
00:03:08,810 --> 00:03:14,730
For example, I can say, I eat a sandwich, I devour a sandwich, but I cannot say, I devour.

38
00:03:14,730 --> 00:03:16,430
Devour requires a direct object.

39
00:03:16,670 --> 00:03:18,930
So a synonym is not always a synonym.

40
00:03:21,010 --> 00:03:33,430
And of course, if you need this original text that you're going to do semantic substitution on, if you use the same text twice, the adversary will say, wait a second, I just saw the same text with minimal variations, and you're exposed.

41
00:03:33,670 --> 00:03:39,430
If the adversary can spot the original text, well, differential text, or knowing discovered text.

42
00:03:39,430 --> 00:03:43,050
So these are problems with the existing approaches prior to this work.

43
00:03:43,750 --> 00:03:45,750
What do these problems really come from?

44
00:03:46,250 --> 00:03:51,270
Well, first of all, automatic generation of cinematically and rhetorically correct text is very difficult on its own.

45
00:03:51,490 --> 00:03:54,090
If you ever try to write a paper, it's hard for you.

46
00:03:54,250 --> 00:03:59,210
Now imagine how hard it is for a computer, especially with all the synchronicities in language.

47
00:03:59,870 --> 00:04:03,930
What all of these previous approaches have tried to do is they try to mimic correct text.

48
00:04:04,730 --> 00:04:14,610
And as in all steganographic approaches, you have some kind of cover object that the adversary expects, and if you deviate from this expectation, that's giving the attacker an attack vector.

49
00:04:15,110 --> 00:04:20,570
And if you're trying to mimic correct text, well, you better produce really correct text, otherwise the adversary can possibly spot you.

50
00:04:21,010 --> 00:04:26,590
So what we really want is something where incorrect text is OK.

51
00:04:27,350 --> 00:04:36,070
We want something where the adversary, when he gets the text, doesn't expect that this text makes perfect sense, is syntactically and semantically perfectly correct.

52
00:04:36,750 --> 00:04:44,610
Ideally, also, we can solve the other problem, that if the adversary gets the original text or some database, he can't say, oh, this other text should not exist.

53
00:04:44,610 --> 00:04:46,150
It's just a minor variation.

54
00:04:46,170 --> 00:04:50,030
It should exist together with whatever you used to generate your cover from.

55
00:04:50,690 --> 00:04:57,570
And as I said, there should be errors in there that we can exploit for steganographic hiding of the hidden message.

56
00:04:59,470 --> 00:05:03,950
Well, I'm sure you've all read the title, so you know what I'm heading for.

57
00:05:04,190 --> 00:05:06,390
Here's some example from Babelfish.

58
00:05:07,070 --> 00:05:10,030
Took a site from a German Linux camp site.

59
00:05:10,530 --> 00:05:17,570
In German it reads, keine Sorge, sie sind alle handsam und beantworten auf freiwillig Fragengründen das Thema Linux und geben gerne einen Klein-MIG-Innovator open source.

60
00:05:17,810 --> 00:05:19,490
OK, nobody understood that one, right?

61
00:05:20,450 --> 00:05:26,770
If you translate this to English by hand, you get something like, don't worry, they're all tame and will also readily answer questions regarding the topic Linux.

62
00:05:26,770 --> 00:05:29,110
I'm glad to give a small glimpse into the world of open source.

63
00:05:29,590 --> 00:05:47,330
Now, if you feed this into Babelfish, I don't know, I assume most of you have used this translation at some point or other, say on Heise or something, you get a concern, it are all not handsome and also readily questions approximately around the topic Linux and give gladly a small idea into the world of the open source.

64
00:05:49,430 --> 00:05:52,270
OK, so first of all, the not is kind of in the wrong place.

65
00:05:52,270 --> 00:05:55,630
It's not a concern as opposed to it's a concern that they're not tame.

66
00:05:56,510 --> 00:06:04,890
It didn't translate handsome at all because it wasn't in the dictionary and it adds an article, the, in front of open source and I don't know why it does this strange capitalization.

67
00:06:04,970 --> 00:06:11,850
But you can see there are plenty of errors in here and still, this text, you might get some idea of what this event was about, right?

68
00:06:11,850 --> 00:06:19,050
So it's still useful for humans, it's still perfectly plausible that somebody sends you this, say you don't speak German and he sends you a rough translation.

69
00:06:19,890 --> 00:06:21,950
So, let's see.

70
00:06:22,330 --> 00:06:23,950
How does machine translation work?

71
00:06:23,950 --> 00:06:29,130
Well, machine translation, first of all, there are lots of low quality encoders out there.

72
00:06:31,490 --> 00:06:35,290
They can, there's never a perfect translation anyway, right?

73
00:06:35,610 --> 00:06:40,750
If you translate from one language to the other, there are multiple variations of how you can express it in the target language.

74
00:06:40,870 --> 00:06:48,870
So, if there is a variation in the translation, it's OK, even if you have a human translation and even voiceless machine translation.

75
00:06:50,850 --> 00:06:54,530
What current machine translation systems do is they are statistical.

76
00:06:54,530 --> 00:06:58,830
They have a language model, a translation model, which maps words and sentences.

77
00:06:58,850 --> 00:07:03,330
They have some idea of syntactic and semantic rules, mostly do pattern matching.

78
00:07:03,470 --> 00:07:13,270
They always ignore all of the context, so they look at one sentence at a time, at least all of the ones that we found, which can have some pretty bad results.

79
00:07:13,270 --> 00:07:21,930
Say somebody talks about pipelines and seals, and suddenly talks about sea lions in the middle of the translation, because the translation picked the wrong word in the dictionary to translate seals.

80
00:07:23,570 --> 00:07:26,970
So there's plenty of errors in machine translation.

81
00:07:29,870 --> 00:07:39,830
Now, one other thing I should say, even if these errors weren't there, even if machine translation were perfect, as I said before, there's always some room in natural language, so we're not just exploiting an error in current software.

82
00:07:40,630 --> 00:07:43,610
So, how does the basic protocol work?

83
00:07:43,610 --> 00:07:50,310
Well, we assume that Alice wants to send a message to Bob, and, well, as usual, they have some shared secret.

84
00:07:50,310 --> 00:07:56,450
In our case, what kind of specific translation engines they're going to use, and other things that define the configuration of the system.

85
00:07:56,770 --> 00:07:59,450
And Alice first has to select some source text.

86
00:07:59,690 --> 00:08:05,710
And in our case, it can be from a public news source, so you could just take text from BBC, the Bible, whatever you want.

87
00:08:05,750 --> 00:08:07,290
You could write some on your own.

88
00:08:08,130 --> 00:08:10,570
It doesn't have to be secret for the system to work in general.

89
00:08:10,570 --> 00:08:14,770
It can have advantages, as you'll see towards the end of the talk.

90
00:08:15,450 --> 00:08:29,890
So, given this cover source text, the translator configuration, and the secret message, Alice gives all of this to our tool, Lit, which, by the way, is available both as a PHP demo and in code at the URL at the bottom of the slides.

91
00:08:29,950 --> 00:08:31,670
Don't go all there at once and try the demo.

92
00:08:31,670 --> 00:08:33,890
The server is a really, really slow machine.

93
00:08:33,890 --> 00:08:37,070
It's processing PHP, and you'll just kill it.

94
00:08:37,890 --> 00:08:40,050
However, it works if you don't try it all at once.

95
00:08:42,330 --> 00:08:48,570
Anyway, so Alice gives this to the Lost in Translation system, and as you would expect, gets back a cover translation.

96
00:08:48,790 --> 00:09:03,290
She sends that cover translation together with either the cover source text or a reference to it to Bob, and Bob just feeds his shared secret, again, the cover source text and the cover translation to his copy of the Lit system and gets back the secret message.

97
00:09:03,290 --> 00:09:04,570
So far, so easy.

98
00:09:06,730 --> 00:09:08,590
So how does the encoding really work?

99
00:09:08,590 --> 00:09:13,390
First, we take our cover source text and run it through several commercial and custom-generated translation engines.

100
00:09:13,390 --> 00:09:27,610
In our case, since we weren't really up to the task of writing our own machine translation software, well, we go online, we query Babelfish, we query Google, we query SysTran and a couple of other engines that are out there and get back translations.

101
00:09:29,350 --> 00:09:34,690
Then, after we get these translations, we want actually more than just two or three variations because then the bit rate would be very low.

102
00:09:34,690 --> 00:09:40,690
So what we do is we add additional variations to the sentences, making slight modifications that are presumably plausible for S.S.

103
00:09:40,690 --> 00:09:46,450
translation errors, possibly even fixing translation errors and we get even more sentences out of that.

104
00:09:47,790 --> 00:09:55,390
As we do so, whenever we add these variations, we compute probabilities that this is actually still a valid translation.

105
00:09:55,390 --> 00:10:09,870
And at the end, once we have lots of sentences, lots of translations for a given sentence and probabilities, we build a Huffman tree and then we walk through this Huffman tree based on the secret message we're trying to encode.

106
00:10:10,550 --> 00:10:14,090
So the picture says more than I can speak into the microphone.

107
00:10:14,490 --> 00:10:20,010
Again, cover source text goes into commercial translators, which ones are again selected by the configuration.

108
00:10:20,010 --> 00:10:22,850
You can add new ones if you are so inclined to the tool.

109
00:10:22,850 --> 00:10:26,270
Then we do some post-processing to get even more sentences.

110
00:10:27,770 --> 00:10:32,610
Then we have some model that will tell us how likely are these sentences actually good translations.

111
00:10:32,810 --> 00:10:35,050
Based on these probabilities, we build a Huffman tree.

112
00:10:35,290 --> 00:10:39,470
And then based on the secret message, we select which translation we're going to emit.

113
00:10:39,850 --> 00:10:43,790
We do this on a sentence-by-sentence basis because the translators work like that.

114
00:10:44,930 --> 00:10:50,790
Eventually we either run off cover source text and cut the secret message or we run off secret message and pad with random bits.

115
00:10:51,030 --> 00:10:52,090
Very simple, right?

116
00:10:52,090 --> 00:10:53,670
Now what does the decoder do?

117
00:10:53,670 --> 00:10:55,050
Well, pretty much the same thing.

118
00:10:55,050 --> 00:11:01,130
Except that of course, at the end, we get the cover translation in.

119
00:11:01,130 --> 00:11:02,490
We still built the Huffman tree.

120
00:11:02,710 --> 00:11:06,550
And now we essentially go upwards and say, okay, which were the bits that were leading to the sentence in this Huffman tree.

121
00:11:06,910 --> 00:11:08,410
And voila, we got our secret message.

122
00:11:09,950 --> 00:11:12,610
To give you some idea of what the post-passes look like.

123
00:11:13,270 --> 00:11:14,730
Some of them, very simple one.

124
00:11:15,110 --> 00:11:20,410
What we found when we looked at translations, one of the things that are often wrong is that they pick the wrong kind of article.

125
00:11:20,410 --> 00:11:21,930
In English, definite versus indefinite.

126
00:11:21,930 --> 00:11:23,370
In German, the wrong gender.

127
00:11:24,790 --> 00:11:25,490
And so on.

128
00:11:25,490 --> 00:11:27,430
So that's one thing we can play with.

129
00:11:27,430 --> 00:11:31,210
Or some words do not have an article, should have one and vice versa.

130
00:11:31,230 --> 00:11:32,670
So that's something we can do.

131
00:11:34,310 --> 00:11:36,230
Also prepositions are very tricky.

132
00:11:36,230 --> 00:11:38,590
Should it be to, should it be towards, should it be from?

133
00:11:38,970 --> 00:11:40,910
So we can play with them a little bit.

134
00:11:41,350 --> 00:11:47,510
And again, we did some manual shuffling of what are prepositions that are close to each other that are likely to be mistakes.

135
00:11:48,110 --> 00:11:50,630
And then that's where the probabilities come from.

136
00:11:51,070 --> 00:11:55,550
And of course what you can do is if you have a dictionary that lists you how often a given word occurs in a language.

137
00:11:55,550 --> 00:12:00,530
You can say, okay, this word is one of the ten million words that nobody ever knows.

138
00:12:00,550 --> 00:12:01,830
So likely the translator didn't know it.

139
00:12:01,830 --> 00:12:04,070
So we leave it in the original language even if it was translated.

140
00:12:04,070 --> 00:12:04,830
Or vice versa.

141
00:12:04,830 --> 00:12:05,930
We have it in our dictionary.

142
00:12:05,950 --> 00:12:06,770
Babafish didn't.

143
00:12:06,770 --> 00:12:07,210
Go us.

144
00:12:07,210 --> 00:12:08,450
We can fix this error.

145
00:12:08,990 --> 00:12:12,570
Right, so again you get two variations just by doing this little change.

146
00:12:12,570 --> 00:12:17,570
And you can have in the archimetry additional ways for encoding information.

147
00:12:18,750 --> 00:12:22,150
Another post-pass that I like very much is semantic substitution.

148
00:12:22,310 --> 00:12:26,410
We talked earlier about synonyms and how hard they are to get and that you have to construct them by hand.

149
00:12:26,470 --> 00:12:33,690
Now if we have a translation, this gives us a way to construct synonyms in a way that we call second-order semantic substitution.

150
00:12:34,190 --> 00:12:36,890
So what we do is we have a German word, say, flach.

151
00:12:37,110 --> 00:12:41,050
And the original machine translator translated it to say flat.

152
00:12:41,430 --> 00:12:43,090
Okay, that's one choice.

153
00:12:43,130 --> 00:12:45,830
Now if we look at our dictionary, we would have had other words.

154
00:12:45,830 --> 00:12:50,130
Tabular, vapid, even, smooth, plain, plain, shallow, and so on and so on.

155
00:12:50,270 --> 00:12:54,170
Now some of these are semantically rather far away from flat.

156
00:12:54,330 --> 00:12:56,690
So we would not just pick a random one out of these.

157
00:12:56,690 --> 00:12:57,370
But what can we do?

158
00:12:57,370 --> 00:12:59,730
We can translate flat back into German.

159
00:13:00,250 --> 00:13:02,170
We get flach, eben, and glatt.

160
00:13:02,650 --> 00:13:05,090
So these are also again quite close now in German.

161
00:13:05,530 --> 00:13:07,950
And if we translate these words now back into English.

162
00:13:08,190 --> 00:13:09,670
See it's going back and forth here.

163
00:13:09,670 --> 00:13:14,630
Then it says, okay, eben goes to even, glatt goes to plain, plain and smooth and so on.

164
00:13:14,630 --> 00:13:16,990
So there's some of these words which kind of form a cluster.

165
00:13:17,310 --> 00:13:19,950
A cluster that shows that they're semantically quite close.

166
00:13:19,950 --> 00:13:22,070
And we can do that because we've got these two languages.

167
00:13:22,410 --> 00:13:30,590
And now assuming that the original translator might have taken a little bit of context from the current sentence and chosen a good translation in flat, we get good semantic substitutions here.

168
00:13:30,790 --> 00:13:36,770
Now if the original translator chose badly, well, then we're not going to do much damage anyway because it was a bad choice.

169
00:13:37,010 --> 00:13:38,970
So it's again very plausible to do this.

170
00:13:38,970 --> 00:13:45,090
And the number of witnesses you pick for the semantic substitution pass you can configure and it's one of these configuration parameters.

171
00:13:45,290 --> 00:13:46,850
Just like, do you want to use this at all?

172
00:13:50,730 --> 00:13:54,030
And the tool itself, you can plug in new modules if you feel so inclined.

173
00:13:54,030 --> 00:13:57,390
You just have to study machine translation, figure out what is actually a plausible error.

174
00:14:02,230 --> 00:14:04,630
So I'm sure you all want to see what it looks like, right?

175
00:14:04,970 --> 00:14:07,290
So let's take an original sentence in German.

176
00:14:07,290 --> 00:14:12,230
In dieser Zeit soll festgestellt werden, ob die Schüler die richtige Schule gewählt haben und ob sie ihren Fähigkeiten entspricht.

177
00:14:12,270 --> 00:14:12,910
Oh God.

178
00:14:13,210 --> 00:14:19,890
In Google, this looks like, in this time it is to be determined whether the pupil selected the correct school and whether it corresponds to its abilities.

179
00:14:19,950 --> 00:14:21,550
Quite a good translation, actually.

180
00:14:21,990 --> 00:14:29,230
Now, LinguaTek, another engine that we're using, gives whether the pupils have chosen the right school and whether it corresponds to its abilities shall be found out at this time.

181
00:14:29,670 --> 00:14:41,010
If you use Lit, and this encodes 8 bits, so that's quite a bit of information, quite a number of choices that we had here, you get, in this time, it is to be determined whether the pupil selected the correct school and whether it corresponds to its abilities.

182
00:14:41,250 --> 00:14:46,710
That's not better, but it's not that much worse that you could say, well, there couldn't be a translator which does this.

183
00:14:46,830 --> 00:14:51,670
And yes, you might notice the closeness to Google, because in this case it selected Google as the basis.

184
00:14:51,890 --> 00:14:57,990
But now assume you've got a couple of hundred translation packages out there, and you don't know which one was chosen.

185
00:14:58,350 --> 00:15:00,890
You don't have this comparison directly at hand.

186
00:15:00,890 --> 00:15:04,750
Maybe we'll see later how we can even hide the original text, so you don't have that one.

187
00:15:04,750 --> 00:15:07,190
You don't have the base translation, you just get one translation.

188
00:15:07,670 --> 00:15:12,350
And you can say, yeah, well, maybe a human wouldn't have produced that, only a first class in English.

189
00:15:12,570 --> 00:15:17,310
But, hey, hard to tell that there was really something going on.

190
00:15:18,370 --> 00:15:19,590
Give you another example.

191
00:15:19,910 --> 00:15:21,490
This is from a film review.

192
00:15:21,490 --> 00:15:27,710
Der marokkanische Film Windhorse erzählt die Geschichte zweier unterschiedlicher Generationen angehorener Männer, die durch Marokko reisen.

193
00:15:27,990 --> 00:15:31,250
Auf dem Weg suchen sie nach dem einzigen, was ihnen wichtig ist, dem Sinn des Lebens.

194
00:15:31,890 --> 00:15:34,810
Okay, I'm not going to read the Google and Linguatech translations.

195
00:15:35,170 --> 00:15:41,790
However, what this gives us is the Marokkan film Windhorse tells a story from men belonging to two different generations who travel through Morocco.

196
00:15:41,950 --> 00:15:45,850
They are looking for the only one which is important to them on the way, the sense of a life.

197
00:15:46,370 --> 00:15:48,790
Now, the best translation would leave out the Ö.

198
00:15:49,510 --> 00:15:51,970
Well, Google and Linguatech put a ö in there.

199
00:15:51,970 --> 00:15:54,410
And actually, lit, as one of the choices, don't put anything.

200
00:15:54,930 --> 00:15:57,510
Well, we included the string lit here, so we got this one.

201
00:15:57,750 --> 00:16:03,870
But as you can see, it might even improve the quality of the given translation, and at least it's not necessarily a significant difference.

202
00:16:06,030 --> 00:16:07,710
So what are the advantages again?

203
00:16:07,930 --> 00:16:15,750
We can hide information within the limits of machine translation, and as machine translation improves, our system will also automatically improve in what it's generating.

204
00:16:16,810 --> 00:16:20,850
We avoid this generation problem by mimicking the results of an imperfect transformation.

205
00:16:21,210 --> 00:16:23,830
So we don't have to produce correct text.

206
00:16:23,850 --> 00:16:26,570
We can get away with producing slightly incorrect text.

207
00:16:26,710 --> 00:16:31,310
And what is also interesting, if you have a larger document with some rhetorical structure, we get that.

208
00:16:31,310 --> 00:16:33,310
The translation gets the rhetorical structure.

209
00:16:33,410 --> 00:16:44,770
If you use any of the other generation systems, well, if you're lucky, you get one sentence correct, but you're never going to get a large book or article correct that way, in terms of rhetorical structure and so on.

210
00:16:45,450 --> 00:16:50,150
We have a secret key that can be used to configure the system.

211
00:16:50,370 --> 00:16:52,110
And to that, you can have it very small.

212
00:16:52,110 --> 00:16:56,390
You can just say, OK, I'm using Babel and Fish and Google and semantic substitution with two witnesses.

213
00:16:56,550 --> 00:16:57,990
Or you can make it more complicated.

214
00:16:57,990 --> 00:17:02,390
You can say, I've got this machine translation system, and it was trained on this corpus.

215
00:17:02,810 --> 00:17:04,050
Machine translations are statistical.

216
00:17:04,050 --> 00:17:13,030
So they are typically done in a way that you have some large parallel corpus of sentences in English and in German, and the machine translation is supposed to learn what a good translation is.

217
00:17:13,390 --> 00:17:16,070
And if you have a different corpus, you get a different machine translator.

218
00:17:16,250 --> 00:17:19,610
So you can make this entire corpus part of your secret, if you so want to.

219
00:17:20,170 --> 00:17:22,710
So you get lots of possible variations there.

220
00:17:23,670 --> 00:17:28,550
And again, for now, we can also say that the cover text was from a public source.

221
00:17:28,550 --> 00:17:32,710
You don't have to write a new one each time, take a new image, or whatever else you used to do in the cover.

222
00:17:33,730 --> 00:17:34,970
What are the disadvantages?

223
00:17:35,390 --> 00:17:38,390
Well, the bitrate is not really high still, right?

224
00:17:39,610 --> 00:17:45,290
One of the big disadvantages of this text is always very robust compared to, say, binary data like images.

225
00:17:45,670 --> 00:17:53,150
And as I said multiple times before, we need to transmit both the source text, or some reference at least to it, and the translation for the decoder to work.

226
00:17:54,790 --> 00:17:57,830
Well, let's try to address these two issues.

227
00:18:03,120 --> 00:18:05,220
So what can we do to increase the bitrate?

228
00:18:05,260 --> 00:18:07,680
Well, we can create more machine translation systems.

229
00:18:08,260 --> 00:18:14,160
I'm sure the people who are doing machine translation will do that for us anyway, but that's one way of doing it.

230
00:18:14,860 --> 00:18:19,700
We can, as I said, create new corpus to train existing machine translation implementations.

231
00:18:19,700 --> 00:18:20,920
And that's actually quite easy.

232
00:18:20,920 --> 00:18:27,520
If I've got a parallel corpus of 20,000 words, I can select any subset of this corpus and get another corpus.

233
00:18:28,020 --> 00:18:34,300
So that's easy if I have the full source of a machine translation system and can train it myself.

234
00:18:34,900 --> 00:18:40,260
And, of course, you can write additional post-passes or pre-passes that will make plausible modifications to sentences.

235
00:18:42,860 --> 00:18:46,520
So what we've done so far is we have a prototype implementation.

236
00:18:46,680 --> 00:18:48,880
And that, of course, means we have limited dictionaries.

237
00:18:49,080 --> 00:18:53,500
We did not write anything that would give us knowledge about grammar or semantics of a language.

238
00:18:53,500 --> 00:18:56,720
And we only have a couple of translation engines available to us.

239
00:18:57,120 --> 00:19:03,040
Still, what we get is a bitrate of something like 0.082 bits per bit transmitted.

240
00:19:04,100 --> 00:19:05,700
Now, this is for normal text.

241
00:19:05,700 --> 00:19:11,520
If I just gzip the text, I get a much better bitrate, which is 0.022, which is a bit more aligned with...

242
00:19:11,520 --> 00:19:17,040
If I take a PNG or JPEG image and hide something in it, well, I could compress this and get a bit...

243
00:19:20,860 --> 00:19:24,700
Quickly, some minor variation that will become interesting soon.

244
00:19:25,360 --> 00:19:28,900
We could use this scheme not only for steganography, but also for watermarking.

245
00:19:29,280 --> 00:19:34,340
Often, watermarking and steganographic systems are related, so this is quite natural to do.

246
00:19:35,040 --> 00:19:40,440
In watermarking, what we want to do is we want to be able to read some embedded mark from just the translation in this case.

247
00:19:40,440 --> 00:19:42,560
We don't want to need the original text.

248
00:19:43,300 --> 00:19:44,540
So, what can we do here?

249
00:19:44,700 --> 00:19:50,960
Well, we could use some keytash of a certain set of sentences and take the least significant bit of that, and that's going to be our marked bit.

250
00:19:50,960 --> 00:20:01,320
And essentially, when we pick our sentences in the translation, we're going to go through all of the sentences, compute all the keytash, and torture it until we get a sentence that actually gives us the marked bit.

251
00:20:01,320 --> 00:20:04,680
Since we have lots of variations, we would statistically succeed most of the time.

252
00:20:07,140 --> 00:20:11,240
And of course, we use multiple sentences if we need multiple marked bits.

253
00:20:12,060 --> 00:20:13,760
Which sentence are we going to pick?

254
00:20:13,780 --> 00:20:19,600
Well, we could just have some other secret mechanism that gives us sentence 5, 10, 17.

255
00:20:20,960 --> 00:20:24,140
Or just at the beginning, but this makes the watermark kind of fragile.

256
00:20:24,140 --> 00:20:27,840
The adversary could just insert a sentence at the beginning and suddenly all of our offsets are wrong.

257
00:20:28,160 --> 00:20:38,320
What is a better technique is to say, we have some arbitrary mechanism to rank sentences, all of the sentences in the translation, and that just gives us some total ordering.

258
00:20:38,320 --> 00:20:44,800
We take the sentences that have the scores lowest on this ordering, and the sentences after that in the document are the ones that are going to hold the marked bits.

259
00:20:44,980 --> 00:20:48,820
This way, an insertion or deletion attack will not necessarily succeed in destroying the mark.

260
00:20:53,480 --> 00:20:55,880
It wouldn't be DEF CON if we didn't have some attacks, right?

261
00:20:56,260 --> 00:20:57,800
So what can we do about attacks?

262
00:20:57,940 --> 00:21:02,180
Well, first of all, there is the possibility of somebody spotting obvious inconsistencies.

263
00:21:02,180 --> 00:21:09,480
The most easy one that comes to mind is, say you've got a translation which in the original text contains the same sentence twice, and you translate it in different ways.

264
00:21:09,680 --> 00:21:11,300
A machine translation would never do that.

265
00:21:11,760 --> 00:21:15,100
It's a deterministic machine, and now suddenly it does it not.

266
00:21:15,360 --> 00:21:18,100
So, that's in some sense a simple bug.

267
00:21:18,100 --> 00:21:25,340
In the LIT system, you just have to keep track of which sentences have you translated so far, and if you see the same sentence again, well, you're not going to use that to encode anything.

268
00:21:25,340 --> 00:21:27,760
You have to omit the sentence you omitted the first time.

269
00:21:28,220 --> 00:21:35,600
Similar, there are certain mistakes that machine translators make, like translate the plural of foot as foots.

270
00:21:36,040 --> 00:21:37,860
Should be feet, obviously, right?

271
00:21:38,860 --> 00:21:46,080
If you pick two different machine translators, where one makes this mistake and the other one doesn't, and you alternate between them, you get inconsistent mistakes in your translation.

272
00:21:46,080 --> 00:21:54,620
And the reader will wonder, okay, so a human might make this mistake once but not another time, but a machine translator should make mistakes consistently.

273
00:21:54,680 --> 00:21:57,180
So that's another possible source of bugs.

274
00:21:57,900 --> 00:22:04,120
Which you would fix by, obviously, writing some kind of post-pass or processing that makes sure that these don't happen if you can spot them.

275
00:22:04,120 --> 00:22:16,740
In general, essentially one problem that most steganographic systems have, somebody could construct a new statistical model that says all of the translation systems in the world obey this model, except for this one.

276
00:22:16,860 --> 00:22:21,780
That applies the same to image steganography or something with audio files.

277
00:22:22,100 --> 00:22:25,420
Somebody has a model, and you deviate from it, and you shouldn't.

278
00:22:25,420 --> 00:22:26,860
How can you defeat that?

279
00:22:27,020 --> 00:22:32,620
Well, let's assume, just for fairness, that we also know about that model.

280
00:22:32,620 --> 00:22:40,240
If we know about that model, we can easily make our encoder aware of that model by sorting out sentences in the translation process that don't obey the model.

281
00:22:40,420 --> 00:22:42,240
And then we can't be detected anymore.

282
00:22:42,880 --> 00:22:50,800
What is interesting is that creating such a model is in some sense equivalent of improving statistical machine translation because, well, what is this machine translator?

283
00:22:50,800 --> 00:22:53,920
It is a model of how one sentence corresponds to another.

284
00:22:54,700 --> 00:22:56,480
And in the end, it's an arms race.

285
00:22:56,480 --> 00:22:58,160
Whoever got the better model wins.

286
00:22:58,160 --> 00:23:06,140
If the attacker and the defender have the same quality of machine translation model, then the defender will win because the attacker will have to say, oh, it's plausible.

287
00:23:09,560 --> 00:23:11,460
Finally, the most fun part.

288
00:23:11,460 --> 00:23:15,660
How can we avoid transmitting the original document for the system?

289
00:23:16,860 --> 00:23:19,880
Remember the least significant bit ID for watermarking?

290
00:23:20,440 --> 00:23:24,500
Now, what if we take H bits in each sentence?

291
00:23:25,340 --> 00:23:31,220
The first H bits that tell us how many of these least significant bits in the hash are going to actually contain a message.

292
00:23:31,960 --> 00:23:34,720
So, I'm a sender, some receiver.

293
00:23:34,820 --> 00:23:36,220
We have some small constant H.

294
00:23:36,220 --> 00:23:42,960
When I pick my sentences, I try to pick one where the first two bits, say, encode the number one.

295
00:23:42,960 --> 00:23:44,580
That means I'm going to transmit one bit.

296
00:23:44,580 --> 00:23:47,700
And then the third bit is going to transmit the actual message bit.

297
00:23:47,700 --> 00:23:59,100
Or, if I have more variations available for this given sentence, I could transmit one where it has the first two bits set, meaning the number three, and then follow three bits in this least significant hash that contain the actual message.

298
00:23:59,600 --> 00:24:07,520
This is a statistical way, just based on the fact that hashes are pseudorandom, that I can encode the message by just transmitting the translation.

299
00:24:07,980 --> 00:24:22,120
And, well, I might fail to find one of the translations in the variation that allow me to encode this, but I'm unlikely to fail, statistically speaking, depending on my choice of H and how many translations I've got available for a given sentence.

300
00:24:22,660 --> 00:24:33,140
And, well, since I can assume that I will fail eventually, what we can do is just add forward error correction to the scheme, and then we can correct these errors that are statistically bound to occur.

301
00:24:36,900 --> 00:24:37,780
Conclusions?

302
00:24:40,880 --> 00:24:45,300
Translation-based steganography is a promising new approach for text steganography, in my opinion.

303
00:24:45,300 --> 00:25:01,240
The bitrate that we can achieve is lower than that of systems that operate on binary data, but we have the advantages, and we can defeat statistical attacks if the underlying machine-translating model the attacker is using is made public to us.

304
00:25:04,360 --> 00:25:05,480
Questions?

305
00:25:08,730 --> 00:25:10,450
Can't have no questions.

306
00:25:23,770 --> 00:25:26,570
I give the slides to the DEF CON goons.

307
00:25:26,570 --> 00:25:30,170
They are... I can put them on the website, obviously.

308
00:25:32,450 --> 00:25:40,170
And there is a paper at Information Hiding Workshop 2005 which also describes the scheme, except for this last bit with the forward error correction.

309
00:25:41,550 --> 00:25:41,750
Yeah ?

310
00:25:49,000 --> 00:25:54,940
Okay, so the question was if we intend to add support for languages that don't use the Roman character set.

311
00:25:54,940 --> 00:25:55,980
Yeah.

312
00:25:56,840 --> 00:26:00,200
Well, as soon as we get... I mean, the system itself uses Unicode.

313
00:26:00,280 --> 00:26:04,840
So if you give us a translation system that translates English to Chinese, you can use it.

314
00:26:04,840 --> 00:26:15,640
The problem is, if you take languages that are not inter-European anymore, the distance between the languages is so big that existing machine translation systems just cannot cope with it anymore.

315
00:26:15,640 --> 00:26:22,900
You just get total garbage, and it's no longer plausible for someone to send a translation of a Chinese document to English that is total garbage to someone else.

316
00:26:22,900 --> 00:26:29,420
The system would still work, but you lose the plausibility because you need a certain level of correctness of the machine translator to make it plausible.

317
00:26:41,160 --> 00:26:43,480
Okay, how do we detect tampering in the message?

318
00:26:43,480 --> 00:26:44,480
Well, we don't.

319
00:26:44,680 --> 00:26:46,260
This is a steganographic system.

320
00:26:46,260 --> 00:26:49,020
Typically, the goal is not to detect tampering in the message.

321
00:26:49,540 --> 00:26:54,280
You could, of course, just add a hash at the end or something or sign the resulting message.

322
00:26:54,420 --> 00:27:02,920
But the idea is the adversary doesn't even know that you're using it, so that the adversary tries to actively disrupt your communication is something that's completely out of the goals of steganography.

323
00:27:20,520 --> 00:27:22,700
No, it does not use symmetric key cryptography.

324
00:27:22,700 --> 00:27:24,800
There's no cryptography involved here at all.

325
00:27:27,100 --> 00:27:31,100
The shared key... we assume it's transmitted out of band up front.

326
00:27:31,100 --> 00:27:33,440
By some means, we don't care.

327
00:27:34,000 --> 00:27:35,440
We just assume it exists.

328
00:27:35,440 --> 00:27:37,120
The two parties have it at the beginning.

329
00:27:37,180 --> 00:27:41,620
They have to know that they're going to use steganography anyway, otherwise the guy probably wonders why he gets this translation.

330
00:27:44,910 --> 00:27:48,230
No, we don't care about public key infrastructure or anything like that.

331
00:27:57,430 --> 00:27:58,610
No more questions?

332
00:28:02,220 --> 00:28:02,820
Okay.

333
00:28:02,820 --> 00:28:06,820
Well, I originally told the DEF CON goons that I would give a half an hour talk and they gave me a one hour slot.

334
00:28:08,700 --> 00:28:18,540
So, however, what I can do very quickly is very, very quickly talk a little bit about some other project of mine that the DEF CON goons didn't want to talk about.

335
00:28:18,540 --> 00:28:22,460
I don't know if you've heard about Glunet.

336
00:28:22,460 --> 00:28:24,820
It's a glue peer-to-peer framework.

337
00:28:25,140 --> 00:28:31,660
Just like Phenet tries to do anonymous file sharing with running on Linux, BSD, Solaris, and so on.

338
00:28:31,660 --> 00:28:34,560
It's written in C code, so portability is not as given.

339
00:28:35,920 --> 00:28:40,160
Goals, as I said, anonymity, confidentiality, so all of the transactions are encrypted.

340
00:28:40,760 --> 00:28:46,500
Deniability, you're supposed to not... even if I break into your computer, you should be able to say, I didn't have these files on my machine.

341
00:28:46,500 --> 00:28:50,220
We're trying to make economic sense that people have incentives for sharing.

342
00:28:50,220 --> 00:28:51,740
And, of course, it should be efficient.

343
00:28:52,480 --> 00:29:00,760
And, very quickly, the upcoming, hopefully this year, 0.7 release will have a very nice plugin architecture.

344
00:29:01,060 --> 00:29:05,440
We can have transport so we can send either fragments like UDP or streaming.

345
00:29:05,840 --> 00:29:07,960
We'll have improved resource accounting.

346
00:29:07,960 --> 00:29:11,840
We can have, as Ian Clark just said earlier, a friend-to-friend or a peer-to-peer topology.

347
00:29:11,840 --> 00:29:13,140
You have the choice.

348
00:29:14,160 --> 00:29:18,080
Cryptographic-wise, we use AIS, SHA, and RSA.

349
00:29:18,680 --> 00:29:24,680
And if you want to look at the new nice features in file sharing, look for the cryptography behind k-blocks.

350
00:29:24,760 --> 00:29:30,240
We can associate arbitrary metadata with files so you can have ad descriptions to your search results.

351
00:29:30,260 --> 00:29:36,040
Or, in particular, previews so you can see before you actually download the file some preview of the contents.

352
00:29:36,720 --> 00:29:40,580
We get quite a big speed-up from using 32k blocks.

353
00:29:40,580 --> 00:29:43,940
And we have a nice, shiny, new, blade-based user interface.

354
00:29:44,800 --> 00:29:46,180
We're currently at pre-5.

355
00:29:46,180 --> 00:29:48,180
I hope the next one will be 0.7.0.

356
00:29:48,980 --> 00:29:52,920
And if you care to learn more about GNU Net, this is the information.

357
00:29:53,340 --> 00:29:57,760
I see us as a friendly competitor to Freenet, which you've heard about just before.

358
00:30:00,320 --> 00:30:01,120
Questions?

359
00:30:05,100 --> 00:30:07,180
Right, yeah, that was one of the things, right?

360
00:30:07,180 --> 00:30:09,160
Can you use steganography with GNU Net?

361
00:30:09,840 --> 00:30:10,540
Theoretically, yes.

362
00:30:10,540 --> 00:30:16,400
We've got this transport-layer abstraction which essentially says, okay, if you can send smoke signals to the other guy, we can transfer messages.

363
00:30:16,540 --> 00:30:19,220
The problem is, look at the bitrate again.

364
00:30:19,220 --> 00:30:22,100
If you want to do peer-to-peer with that kind of bitrate, you have a first problem.

365
00:30:22,100 --> 00:30:26,200
The second problem is, the steganographic encoder is quite, quite slow.

366
00:30:26,480 --> 00:30:31,700
So, not only would you use extreme amounts of bandwidth, but also your CPU is just going to crash and burn.

367
00:30:32,260 --> 00:30:36,880
And the third problem is, you need this shared secret and these machine translators in the background.

368
00:30:36,880 --> 00:30:44,360
And usability-wise, we want to go for a slightly larger user base than the people who would actually be capable of setting this thing up.

369
00:30:45,260 --> 00:30:50,480
Oh, and LIT is written in PHP and GNU is in C, so there would be some bridging code that would be also non-trivial.

370
00:30:50,800 --> 00:30:52,280
So, theoretically, yes.

371
00:30:52,280 --> 00:30:53,980
In practice, no.

372
00:30:57,440 --> 00:31:03,680
And besides, why would you want to do anonymous file sharing if you use steganography so nobody even knows that you're transmitting anything?

373
00:31:11,360 --> 00:31:12,680
No more questions?

374
00:31:13,500 --> 00:31:17,180
Okay, well, if you want to see me in private, I'll be somewhere outside.