1
00:00:00,000 --> 00:00:06,460
The next speaker has been working in the computer security field for over 20 years.

2
00:00:06,680 --> 00:00:12,740
The focus of his talk will be walking through the process of how he used the scientific method

3
00:00:12,740 --> 00:00:18,060
to conduct the research that led to his 2011 insulin pump findings.

4
00:00:18,740 --> 00:00:25,880
And a fun fact, SC Magazine named him one of the top influential IT security thinkers in 2013.

5
00:00:26,300 --> 00:00:27,240
I'm blushing.

6
00:00:31,020 --> 00:00:35,640
Now here to present his talk, Using the Scientific Method in Security Research,

7
00:00:35,940 --> 00:00:37,820
ladies and gentlemen, Jay Ratcliffe.

8
00:00:40,800 --> 00:00:41,560
Thank you.

9
00:00:43,960 --> 00:00:48,680
So one of the things that I really am passionate about and wanted to talk about today was

10
00:00:49,540 --> 00:00:53,740
there's a really big problem that we have in the cybersecurity community.

11
00:00:54,160 --> 00:00:56,800
And it's that we're not taken seriously outside of our own circles.

12
00:00:57,420 --> 00:00:59,840
And one of the reasons we're not taken seriously outside,

13
00:01:00,000 --> 00:01:04,840
outside of our own circles, is we kind of ignore a lot of what other science fields do,

14
00:01:05,620 --> 00:01:07,260
particularly in the area of research.

15
00:01:08,120 --> 00:01:09,900
Research is done in a certain way.

16
00:01:10,420 --> 00:01:15,880
And sciences, traditional sciences like chemistry, mathematics, medicine,

17
00:01:16,600 --> 00:01:20,780
have methods on how to do this, statistically provable methods.

18
00:01:21,820 --> 00:01:25,460
In computer security, we kind of just ignore them, right?

19
00:01:25,600 --> 00:01:28,900
We grab a box, we pop it, we claim victory,

20
00:01:29,120 --> 00:01:29,980
and we run.

21
00:01:30,000 --> 00:01:31,020
We move on to the next conference.

22
00:01:31,720 --> 00:01:34,320
And that's really amateurish.

23
00:01:34,420 --> 00:01:39,700
When we look from, when people look from the outside into our area, it kind of looks

24
00:01:39,700 --> 00:01:40,540
like child's play.

25
00:01:41,740 --> 00:01:47,080
So one of the things that I was pushed to do by a friend of mine, which was, she has

26
00:01:47,080 --> 00:01:48,600
a PhD in clinical psychology.

27
00:01:49,280 --> 00:01:53,340
And she's like, you should try and use more traditional research methods.

28
00:01:53,700 --> 00:01:59,220
And actually follow methodology, so that way your research will be taken more seriously

29
00:01:59,220 --> 00:02:02,060
when you present it, especially in the medical world.

30
00:02:03,260 --> 00:02:10,980
So when I looked at my insulin pump as a type 1 diabetic, I started to create a method on

31
00:02:10,980 --> 00:02:15,660
how I wanted to do that research and how I was going to conduct that research to fall

32
00:02:15,660 --> 00:02:17,500
in line with that scientific method.

33
00:02:20,260 --> 00:02:23,000
You know, computer security industry is very young.

34
00:02:23,240 --> 00:02:28,360
We've only been around for, you know, this is the 25th DEFCON, so probably not more

35
00:02:28,360 --> 00:02:29,200
than 25 years.

36
00:02:29,200 --> 00:02:33,800
And when we compare that to other fields of science, we know that we haven't been around

37
00:02:33,800 --> 00:02:34,220
that long.

38
00:02:34,940 --> 00:02:38,360
We often get looked down upon for many reasons.

39
00:02:39,960 --> 00:02:47,960
Not just our behavior at parties, the stickers and our acting and our behavior, but also

40
00:02:47,960 --> 00:02:49,560
the way we conduct ourselves in our research.

41
00:02:50,300 --> 00:02:54,520
And we really fail at using traditional research methods where we need to get back to it.

42
00:02:56,480 --> 00:02:58,860
Now this is the scientific method.

43
00:02:59,200 --> 00:03:02,460
I actually pulled this from a sixth grade science book.

44
00:03:03,620 --> 00:03:07,620
Most of us that have gone through education process kind of recognize this.

45
00:03:07,960 --> 00:03:11,060
We learned about this very, very, very long time ago.

46
00:03:11,740 --> 00:03:17,420
This is where I went back to to try and figure out how to root my research in these types

47
00:03:17,420 --> 00:03:17,880
of methods.

48
00:03:20,280 --> 00:03:24,940
As I go through each of these stages, I'm going to give you real examples from real

49
00:03:24,940 --> 00:03:26,080
devices that I've worked on.

50
00:03:26,900 --> 00:03:29,040
We're going to step through each phase of the process.

51
00:03:29,200 --> 00:03:34,880
To kind of give you an idea of where my mind went and what actions I took for each step

52
00:03:34,880 --> 00:03:36,260
of that scientific method.

53
00:03:36,960 --> 00:03:42,420
And I think it's important to know how strict I was to that method to accomplish my research.

54
00:03:45,640 --> 00:03:47,520
Stage one is purpose.

55
00:03:49,220 --> 00:03:51,020
What do we want to research and why?

56
00:03:51,900 --> 00:03:54,960
This one we have pretty clear, right?

57
00:03:55,060 --> 00:03:57,420
We want to make devices more secure.

58
00:03:57,620 --> 00:03:59,020
We want to find security.

59
00:03:59,020 --> 00:04:00,580
We want to find security vulnerabilities in these devices.

60
00:04:01,300 --> 00:04:02,480
And what do we want to do?

61
00:04:02,540 --> 00:04:03,160
What's the purpose?

62
00:04:04,280 --> 00:04:05,940
Well, we want to get root access.

63
00:04:06,980 --> 00:04:08,440
I don't think that needs to change.

64
00:04:08,640 --> 00:04:12,800
That's the purpose of a lot of our research and it's the goal of a lot of our research

65
00:04:12,800 --> 00:04:16,380
is to go after these devices and to get elevated privileges.

66
00:04:17,000 --> 00:04:20,300
To get access to data that we shouldn't have access to.

67
00:04:20,640 --> 00:04:24,160
And maybe do things that we shouldn't do to patients.

68
00:04:24,620 --> 00:04:26,620
To see how safe these patients are.

69
00:04:30,440 --> 00:04:35,060
Stage two, research and observations, is really where we get into a lot of substance

70
00:04:35,060 --> 00:04:35,740
of the research.

71
00:04:38,140 --> 00:04:42,560
Now I'm going to walk you through what I do when I am researching an item.

72
00:04:43,100 --> 00:04:44,700
Now some researchers do some of this.

73
00:04:44,800 --> 00:04:45,800
Some researchers don't.

74
00:04:45,840 --> 00:04:47,180
Some researchers do more than this.

75
00:04:47,400 --> 00:04:48,840
But this is just an example.

76
00:04:50,220 --> 00:04:53,680
So one of the first things I do is I look for the manual.

77
00:04:54,500 --> 00:04:57,660
Now this is kind of surprising because tech people don't like manuals.

78
00:04:57,660 --> 00:05:02,380
But you have to remember, in a lot of these devices, whether we're talking about medical

79
00:05:02,380 --> 00:05:05,600
devices, IOT devices, we're not experts.

80
00:05:06,160 --> 00:05:07,660
I am not a registered nurse.

81
00:05:08,200 --> 00:05:10,680
I do not know how an infusion pump works perfectly.

82
00:05:11,400 --> 00:05:15,060
I do know how an insulin pump works because I'm a diabetic.

83
00:05:15,700 --> 00:05:20,640
But there's a slew of devices out there that I don't know how they work.

84
00:05:20,700 --> 00:05:22,100
I don't know how they function.

85
00:05:22,480 --> 00:05:25,600
And the only way to learn how to do that is to read the manual.

86
00:05:25,860 --> 00:05:26,140
So I'm going to look at the manual.

87
00:05:27,660 --> 00:05:30,200
There's also a wealth of data in the manual.

88
00:05:30,700 --> 00:05:35,520
These manuals are very detailed about how a nurse should interact with the device and

89
00:05:35,520 --> 00:05:37,360
how a patient interacts with the device.

90
00:05:38,000 --> 00:05:42,560
And this can tell you a lot of the user interface that is going on.

91
00:05:43,720 --> 00:05:46,280
Also, it tells you how the device is programmed.

92
00:05:47,020 --> 00:05:53,680
On one particular device, a kidney dialysis machine, there's a password option.

93
00:05:54,120 --> 00:05:57,060
And the manual says, please set the password.

94
00:05:57,660 --> 00:06:01,260
So one, one, one, one, one.

95
00:06:04,080 --> 00:06:04,580
That's it.

96
00:06:04,660 --> 00:06:05,520
Those were the directions.

97
00:06:05,880 --> 00:06:08,060
That was the password that was going to be set on that device.

98
00:06:08,180 --> 00:06:09,560
You were not to change that password.

99
00:06:09,780 --> 00:06:11,840
You were not to use any other password other than that.

100
00:06:12,460 --> 00:06:15,300
I found that kind of interesting, but that was right in the manual.

101
00:06:17,220 --> 00:06:20,280
The other thing that these manuals have are troubleshooting sections.

102
00:06:21,820 --> 00:06:26,000
Just like when we do system administration and we do troubleshooting, that's how we find

103
00:06:26,000 --> 00:06:27,100
a lot of security vulnerabilities.

104
00:06:27,660 --> 00:06:33,420
I think that looking in the manual and reading the troubleshooting section clues me in to

105
00:06:33,420 --> 00:06:34,720
where vulnerabilities could be.

106
00:06:35,420 --> 00:06:36,340
And that's great.

107
00:06:38,500 --> 00:06:42,000
There are also some really juicy tidbits in the manual.

108
00:06:42,770 --> 00:06:48,720
The FCC mandates that there's information that needs to be published in the manual that

109
00:06:48,720 --> 00:06:53,180
gives you a lot of information if you want to go after a wireless device.

110
00:06:53,970 --> 00:06:55,260
So for example, the frequency.

111
00:06:55,720 --> 00:06:57,540
The amount of power being used.

112
00:06:57,660 --> 00:06:58,940
The type of modulation.

113
00:06:59,500 --> 00:07:01,140
All of those things are in the manual.

114
00:07:01,560 --> 00:07:03,300
You don't have to go hunting for them.

115
00:07:03,580 --> 00:07:05,600
It's not a treasure hunt to go find them.

116
00:07:05,680 --> 00:07:07,240
It's printed right there.

117
00:07:09,400 --> 00:07:13,660
The other thing that's very interesting to me is that these manuals can be surprisingly

118
00:07:13,660 --> 00:07:14,480
hard to get.

119
00:07:15,760 --> 00:07:18,680
They are often not on the manufacturer's website.

120
00:07:19,000 --> 00:07:21,940
You have to either go to eBay and buy them.

121
00:07:22,480 --> 00:07:24,580
You can buy the CDs that come with them.

122
00:07:24,980 --> 00:07:26,660
You can go to doctor's offices.

123
00:07:27,200 --> 00:07:27,640
You can go to the doctor's office.

124
00:07:27,640 --> 00:07:29,000
You can dumpster dive for them.

125
00:07:29,180 --> 00:07:32,820
Because most doctor's offices don't keep all those manuals around.

126
00:07:33,220 --> 00:07:36,520
And they don't consider them private data.

127
00:07:36,960 --> 00:07:38,160
It's not a patient record.

128
00:07:38,380 --> 00:07:39,700
It's a user's manual.

129
00:07:39,960 --> 00:07:41,720
We've all thrown out user's manuals.

130
00:07:41,800 --> 00:07:45,860
But these user's manuals are ones that we need to do our research.

131
00:07:48,180 --> 00:07:51,140
The second step I go through is patent research.

132
00:07:52,020 --> 00:07:56,860
Google has a great repository of patents that tell you exactly.

133
00:07:56,860 --> 00:07:59,120
How the device works.

134
00:08:00,180 --> 00:08:04,260
So in a lot of cases if a manufacturer is going to use an obfuscation method.

135
00:08:04,660 --> 00:08:07,560
Or a made up cryptography method.

136
00:08:07,740 --> 00:08:08,840
They will patent it.

137
00:08:09,180 --> 00:08:13,360
So they'll give you the exact directions on where the bits are shifting and how they're

138
00:08:13,360 --> 00:08:13,640
shifting.

139
00:08:14,100 --> 00:08:15,480
It's really quite handy.

140
00:08:17,460 --> 00:08:21,720
One of the things that you have to do when you get a patent is you give up that secrecy.

141
00:08:22,080 --> 00:08:26,020
You publish exactly what that device does and how it functions.

142
00:08:26,860 --> 00:08:30,580
And very often, especially with diabetic devices that I've looked at.

143
00:08:31,240 --> 00:08:37,640
I've learned all the dirty details about how that device communicates and how it works.

144
00:08:38,040 --> 00:08:39,680
And the protocols that it uses.

145
00:08:41,200 --> 00:08:42,400
They are wordy.

146
00:08:42,420 --> 00:08:43,560
They're written by lawyers.

147
00:08:43,780 --> 00:08:45,420
And they are very boring to read.

148
00:08:45,580 --> 00:08:49,600
But after you get the hang of it you figure out where the good pieces are and where the

149
00:08:49,600 --> 00:08:50,380
boring pieces are.

150
00:08:52,520 --> 00:08:54,960
Another thing that happens a lot in the medical world.

151
00:08:55,240 --> 00:08:55,960
Is that.

152
00:08:55,960 --> 00:08:58,660
Companies purchase ideas from other companies.

153
00:08:59,200 --> 00:09:02,280
So I might be working with one device as my target.

154
00:09:02,460 --> 00:09:07,140
But they have purchased patent rights from other manufacturers.

155
00:09:07,320 --> 00:09:10,120
So that way they can coexist in the same ecosphere.

156
00:09:10,640 --> 00:09:13,900
So you have to kind of look at some of these cross branded concepts.

157
00:09:14,160 --> 00:09:16,520
Whether these devices intercommunicate or not.

158
00:09:17,200 --> 00:09:20,400
And I might iterate through this multiple times to look through the patents.

159
00:09:20,620 --> 00:09:22,460
And kind of hunt down what I'm looking for.

160
00:09:25,960 --> 00:09:28,820
User knowledge is another area that I love.

161
00:09:29,220 --> 00:09:36,080
A lot of these devices, especially very personal ones, neural stimulators, insulin pumps.

162
00:09:36,600 --> 00:09:41,800
Ones that the individual patient interacts with have user forms.

163
00:09:42,200 --> 00:09:44,960
And these user forms complain all the time.

164
00:09:45,380 --> 00:09:49,120
If I hit this button and this button it ends up deleting something.

165
00:09:49,440 --> 00:09:52,160
Or it doesn't work right if I use this function.

166
00:09:53,020 --> 00:09:54,100
Well what are those?

167
00:09:55,300 --> 00:09:55,920
Bugs.

168
00:09:55,920 --> 00:09:59,360
Those are flaws in the operating system of these medical devices.

169
00:10:00,070 --> 00:10:04,100
So by going through and lurking in these communities and forms.

170
00:10:04,260 --> 00:10:08,980
You can get a head start on a lot of where these problems on these devices are.

171
00:10:10,210 --> 00:10:14,940
You also get an idea of the tips and tricks of what users are doing.

172
00:10:15,950 --> 00:10:19,280
So the manual might have a five step process to do a certain thing.

173
00:10:19,480 --> 00:10:22,340
But the users have figured out how to do it in two.

174
00:10:23,500 --> 00:10:24,740
They've figured out a hack.

175
00:10:24,740 --> 00:10:26,700
To make things faster.

176
00:10:27,200 --> 00:10:28,800
That is a flaw.

177
00:10:29,060 --> 00:10:32,320
That is something that might be leveraged in your research.

178
00:10:33,200 --> 00:10:37,900
You also get an idea of what dumb configurations look like.

179
00:10:37,960 --> 00:10:42,420
And default configurations like setting the password to 1111.

180
00:10:44,660 --> 00:10:49,060
One of my favorite people to talk to in the medical industry are nurses.

181
00:10:50,040 --> 00:10:51,740
Nurses are hackers.

182
00:10:52,680 --> 00:10:58,720
They know exactly what to do and exactly how to get the fastest and best patient care

183
00:10:58,720 --> 00:10:59,820
out of what they're given.

184
00:11:00,340 --> 00:11:04,060
And that often is not printed in the manual.

185
00:11:04,500 --> 00:11:07,980
They know how to do things one minute faster.

186
00:11:08,180 --> 00:11:09,420
Three steps faster.

187
00:11:09,600 --> 00:11:10,660
Because that's their job.

188
00:11:11,380 --> 00:11:13,980
So I'm always talking to those front line people.

189
00:11:14,120 --> 00:11:18,680
The nurses and the practitioners using the devices to figure out exactly how they're

190
00:11:18,680 --> 00:11:19,160
using them.

191
00:11:23,100 --> 00:11:25,260
So now we're going to get into some dangerous area.

192
00:11:26,160 --> 00:11:27,220
Software and firmware.

193
00:11:29,100 --> 00:11:32,000
Collecting and harvesting all of this as much as possible is great.

194
00:11:32,440 --> 00:11:33,880
Go to FTP sites.

195
00:11:35,160 --> 00:11:38,760
They often, manufacturers have the firmware posted on their sites.

196
00:11:38,860 --> 00:11:39,680
And you can grab them.

197
00:11:40,400 --> 00:11:43,520
But here's what I'm going to give you a big, huge warning about.

198
00:11:44,860 --> 00:11:45,700
Don't open them.

199
00:11:46,880 --> 00:11:47,720
Collect them.

200
00:11:47,720 --> 00:11:48,700
Harvest them.

201
00:11:49,000 --> 00:11:53,900
It is not time yet to go poking around and looking at them and opening them.

202
00:11:54,300 --> 00:11:59,440
And one of the reasons for that is we're not, we haven't looked at the legal liability yet.

203
00:11:59,600 --> 00:12:01,300
We're still in reconnaissance mode.

204
00:12:01,520 --> 00:12:03,400
We're still in data collection mode.

205
00:12:03,700 --> 00:12:08,820
If you go and start opening and playing with this right now, you could violate the DMCA.

206
00:12:09,300 --> 00:12:12,620
You could violate different copyright laws.

207
00:12:13,080 --> 00:12:15,500
You could violate the CFAA.

208
00:12:15,900 --> 00:12:17,180
And you don't want to do that.

209
00:12:17,180 --> 00:12:20,660
It's very hard to hack these devices from a jail cell.

210
00:12:21,400 --> 00:12:26,260
So I highly recommend that you just harvest these, collect them, and not open them at

211
00:12:26,260 --> 00:12:26,700
this point.

212
00:12:28,140 --> 00:12:34,500
You can use Google Foo to get zips, binaries, and executables off the website by specifying

213
00:12:34,940 --> 00:12:36,960
the individual manufacturer domain name.

214
00:12:38,640 --> 00:12:40,040
I do a lot of stuff on eBay.

215
00:12:40,260 --> 00:12:44,840
And I get CDs, DVDs, manuals, floppy disks.

216
00:12:45,380 --> 00:12:45,940
Yes.

217
00:12:45,940 --> 00:12:51,340
Floppy disks, because a lot of these medical devices still use 3 1⁄2 inch floppy disks

218
00:12:51,340 --> 00:12:53,000
to hold their configurations and data.

219
00:12:54,140 --> 00:12:57,060
So you could purchase all that stuff, and you kind of don't know what you're getting.

220
00:12:57,200 --> 00:12:59,140
It's kind of like a mystery box.

221
00:13:00,880 --> 00:13:05,780
You have to pay attention to the EULAs and click-through agreements to figure out where,

222
00:13:05,860 --> 00:13:08,480
if or not you're violating the law.

223
00:13:09,720 --> 00:13:12,080
And you also want to make note of certain platforms.

224
00:13:12,760 --> 00:13:14,800
The example I have here is Medtronic.

225
00:13:14,800 --> 00:13:23,460
All of their insulin devices use a Java platform to push and configure all those devices.

226
00:13:23,940 --> 00:13:31,960
And those Java libraries, later on, I discovered were unobfuscated and completely open to me.

227
00:13:32,220 --> 00:13:37,740
And that was one of the big steps that led me to be successful in my research with the

228
00:13:37,740 --> 00:13:38,360
Medtronic devices.

229
00:13:42,240 --> 00:13:44,700
We also looked at chips and datasheets.

230
00:13:44,800 --> 00:13:47,900
And this can be a little bit harder before we have the device.

231
00:13:48,160 --> 00:13:50,220
Remember, we haven't even purchased a device.

232
00:13:50,320 --> 00:13:51,220
We don't have a device yet.

233
00:13:51,380 --> 00:13:56,440
We are in reconnaissance mode just looking at all the information available for the device.

234
00:13:57,100 --> 00:14:00,900
So sometimes this can be hard without actually opening the device and looking at the chips.

235
00:14:02,180 --> 00:14:09,280
But news releases are great, because if Medtronic goes and decides on a certain processor,

236
00:14:09,640 --> 00:14:14,340
for example, an MSP430 chip, they'll put out a press release saying,

237
00:14:14,340 --> 00:14:17,520
hey, we just got a big contract with this medical device vendor, and they're going to

238
00:14:17,520 --> 00:14:19,860
be using our brand new microprocessor.

239
00:14:20,480 --> 00:14:20,920
Congratulations.

240
00:14:21,500 --> 00:14:25,280
You now know exactly what microprocessor is being used in your target device.

241
00:14:25,980 --> 00:14:33,240
You can download those datasheets and start to think about what attack platforms and what

242
00:14:33,240 --> 00:14:35,700
attack vectors you can use in your research.

243
00:14:36,500 --> 00:14:40,660
The example I have here is that there's a chip company called Freescale, and there's

244
00:14:40,660 --> 00:14:42,380
an insulin pump maker called Omnipod.

245
00:14:42,460 --> 00:14:44,060
And they did a joint press release.

246
00:14:44,060 --> 00:14:47,720
They did a joint press release saying that Omnipod is going to be using Freescale chips

247
00:14:47,720 --> 00:14:48,700
in all of their products.

248
00:14:49,140 --> 00:14:50,700
And I was like, sweet.

249
00:14:50,900 --> 00:14:51,960
I don't have to crack one open.

250
00:14:52,120 --> 00:14:55,600
I know exactly what processor is being used, and I know exactly what memory is being used,

251
00:14:55,700 --> 00:14:57,380
because they put it in the press release.

252
00:14:57,880 --> 00:15:00,220
So there's a lot of free information out there that you can get.

253
00:15:01,640 --> 00:15:04,400
You can also do some searching on LinkedIn.

254
00:15:05,280 --> 00:15:10,000
Does the engineering lead for the device company have a particular background in a chip?

255
00:15:10,840 --> 00:15:12,500
They usually list it on their resume.

256
00:15:12,500 --> 00:15:17,620
Whether they're a Texas Instrument chip, or if they're a Bluetooth engineer, or if

257
00:15:17,620 --> 00:15:23,820
they're an Ant engineer, those types of things, you can obviously make a conclusion, well,

258
00:15:24,520 --> 00:15:29,540
if they're experts on that particular chip set or that particular framework, they're

259
00:15:29,540 --> 00:15:30,660
probably using it in that device.

260
00:15:32,180 --> 00:15:36,700
The datasheets for these chips really give you an idea of what pins you need to tap.

261
00:15:37,360 --> 00:15:42,480
It limits down what other types of tools you need, and it gives you an idea of what functions,

262
00:15:42,480 --> 00:15:45,080
and capacities these devices have.

263
00:15:45,980 --> 00:15:49,260
Some of these chips have AES encryption built right onto them.

264
00:15:49,460 --> 00:15:50,520
Some of them don't.

265
00:15:50,800 --> 00:15:55,080
So you can kind of see what security options are available just by looking at those sheets.

266
00:15:58,960 --> 00:16:01,480
The last step in reconnaissance that I use is legal.

267
00:16:02,540 --> 00:16:05,940
At this point, you should think about getting some legal guidance, right?

268
00:16:06,240 --> 00:16:11,100
You've done a ton of research, you've got a lot of data, and if you make a mistake here,

269
00:16:12,480 --> 00:16:15,380
you're going to get serious fine or jail, and nobody wants that.

270
00:16:15,560 --> 00:16:16,600
That's not fun.

271
00:16:17,760 --> 00:16:19,720
In this case, I really love the EFF.

272
00:16:20,040 --> 00:16:21,540
I've used them multiple times.

273
00:16:22,060 --> 00:16:23,540
They will give you free consultation.

274
00:16:24,180 --> 00:16:26,040
They will help connect you to the right people.

275
00:16:26,940 --> 00:16:29,400
What I have them do is draw me a legal box.

276
00:16:30,560 --> 00:16:34,860
That legal box tells me what is safe to research and what is not safe to research.

277
00:16:35,840 --> 00:16:40,400
So one of the things that I could not research, even though I had it, was firmware.

278
00:16:40,400 --> 00:16:46,020
At the time that I did my research, going into the firmware was a very clear violation

279
00:16:46,020 --> 00:16:46,820
of the DMCA.

280
00:16:47,500 --> 00:16:50,020
And it was not something that I wanted to play around with at all.

281
00:16:50,640 --> 00:16:56,560
So I said, I am not going to do anything with firmware in my research because I want

282
00:16:56,560 --> 00:16:57,200
to stay out of jail.

283
00:16:58,980 --> 00:17:03,940
So they will give you guidance on if you'd say, I've got all this reconnaissance, what

284
00:17:03,940 --> 00:17:05,420
can I do and what can I not do?

285
00:17:05,740 --> 00:17:09,500
This is why it's really important to not go into those binaries.

286
00:17:09,500 --> 00:17:11,680
To go into that firmware once you get it.

287
00:17:11,920 --> 00:17:16,620
Because you can't put it back in the box and return it.

288
00:17:16,840 --> 00:17:19,960
Once you start, you violated it.

289
00:17:20,340 --> 00:17:23,720
And you can't go backwards legally.

290
00:17:24,340 --> 00:17:31,320
So be very careful to do your collecting, then the legal stuff, then dig into it.

291
00:17:35,400 --> 00:17:37,940
Stage three is the hypothesis stage.

292
00:17:38,260 --> 00:17:38,880
Now.

293
00:17:38,880 --> 00:17:42,480
This is generally where computer security research falls apart.

294
00:17:42,940 --> 00:17:47,180
Because at this point, we're so excited and so giddy that we can pop a box because we

295
00:17:47,180 --> 00:17:49,700
have enough information that we just go out and do it.

296
00:17:52,700 --> 00:17:54,120
That's not what I chose to do.

297
00:17:55,000 --> 00:18:02,200
The first thing is you need to be able to replicate these problems and you need to be

298
00:18:02,200 --> 00:18:03,500
able to do it in an incredible way.

299
00:18:04,340 --> 00:18:08,240
The example I gave is I said, you know, I was really confused about this at first.

300
00:18:08,320 --> 00:18:08,640
And I said.

301
00:18:08,880 --> 00:18:13,600
Well, if you collected a bunch of data on patients, couldn't you just do all the correlations

302
00:18:13,600 --> 00:18:17,960
and figure out what correlated and then be like, ah, I found something that correlated.

303
00:18:18,300 --> 00:18:19,360
And they were like, no.

304
00:18:20,240 --> 00:18:25,020
I think that my friend termed it as that would be hack amateurish bullshit.

305
00:18:26,560 --> 00:18:32,020
She's like, you have to go into the research with a very specific idea and be able to prove

306
00:18:32,020 --> 00:18:33,720
that idea right or wrong.

307
00:18:34,820 --> 00:18:36,200
So I thought about that.

308
00:18:36,300 --> 00:18:38,860
And I thought about how do I be as specific as possible.

309
00:18:38,880 --> 00:18:45,440
In computer security research to do this, it was a little bit overwhelming, but here's

310
00:18:45,440 --> 00:18:46,240
what I ended up doing.

311
00:18:47,460 --> 00:18:49,200
So I wear a glucose monitor.

312
00:18:49,780 --> 00:18:51,280
It's attached to me on my belly.

313
00:18:51,500 --> 00:18:58,180
I change it out every 14 days and it tells my phone, it tells this little pager device

314
00:18:58,180 --> 00:18:59,980
what my blood sugar is every five minutes.

315
00:19:00,660 --> 00:19:04,120
And the little transmitter runs on a watch battery for a year.

316
00:19:04,780 --> 00:19:05,220
Okay.

317
00:19:05,560 --> 00:19:08,220
Now I know all this because of all the recon I've done.

318
00:19:08,220 --> 00:19:09,880
And all the observations I've done.

319
00:19:10,680 --> 00:19:14,860
I also know that I don't have the ability to set time or date on this.

320
00:19:15,380 --> 00:19:17,860
And that the ID is hard coded.

321
00:19:18,020 --> 00:19:20,260
It's printed right on the back and it's not changeable.

322
00:19:21,500 --> 00:19:22,780
So let's think about this.

323
00:19:22,980 --> 00:19:24,740
I've got these observations that I've made.

324
00:19:25,380 --> 00:19:26,760
One year on a watch battery.

325
00:19:27,580 --> 00:19:29,200
No ability to set time.

326
00:19:29,980 --> 00:19:31,080
Hard coded ID.

327
00:19:32,680 --> 00:19:35,220
So what kind of hypothesis can I draw from that?

328
00:19:36,360 --> 00:19:37,800
First thing I thought was.

329
00:19:38,220 --> 00:19:42,940
there's no way we can do encryption. Encryption takes a lot of horsepower. It takes a lot

330
00:19:42,940 --> 00:19:47,860
of battery. If that little old watch battery is going to last a year, it's certainly not

331
00:19:47,860 --> 00:19:55,900
doing key exchanges and AES encryption. So my hypothesis is that no encryption is being

332
00:19:55,900 --> 00:20:04,460
used. My second hypothesis is that since you can't set the time on the device and there's

333
00:20:04,460 --> 00:20:10,120
no element of time, how would you prevent a replay attack? Because you don't know if

334
00:20:10,120 --> 00:20:16,460
that packet has already been taken or already not. So I have another hypothesis. I think

335
00:20:16,460 --> 00:20:21,860
that there's a chance that there's no replay attack prevention like you would have in a

336
00:20:21,860 --> 00:20:32,360
TCP sequence. At this point, we can build experiments, and we can try and figure out

337
00:20:32,360 --> 00:20:34,440
if our hypothesis is true or not.

338
00:20:34,440 --> 00:20:40,220
or false. See how this is very different from our traditional model of how we do computer

339
00:20:40,220 --> 00:20:44,460
security research where we just go, well, I don't think it uses encryption, so I'm just

340
00:20:44,460 --> 00:20:50,960
going to blast it, fuzz everything, and figure out if it's broken. This has more methodology

341
00:20:50,960 --> 00:20:57,020
to it. It has more science to it to be able to go through and build an experiment on how

342
00:20:57,020 --> 00:21:05,340
to do that. So we think about how do we build experiments to prove the hypothesis or not

343
00:21:05,340 --> 00:21:12,200
that we're using encryption. And knowledge is more important than the results. Remember,

344
00:21:12,960 --> 00:21:18,360
my goal here is to prove if we're using encryption or not and if replay attacks are preventable

345
00:21:18,360 --> 00:21:23,100
or not. I've not mentioned getting root. I've not mentioned taking control of the device.

346
00:21:23,380 --> 00:21:25,840
I'm actually looking at these two specific things.

347
00:21:26,080 --> 00:21:26,240
Okay.

348
00:21:28,180 --> 00:21:33,300
So one of the experiments that I did was I used something very similar to the Japanese

349
00:21:33,300 --> 00:21:39,600
Purple Code. So when we broke the Japanese Purple Code in World War II, what we did was

350
00:21:39,600 --> 00:21:46,560
we injected information to be able to see it when it was encrypted. The example, the

351
00:21:46,560 --> 00:21:52,740
famous example, is that we sent a false transmission that said Midway Island has a water condenser

352
00:21:52,740 --> 00:21:56,060
problem. And then we picked up a message from the Japanese.

353
00:21:56,240 --> 00:22:04,320
It said AF has water problem. AF, now we know, is Midway Island. And that's one of the

354
00:22:04,320 --> 00:22:10,120
reasons that we were successful in turning around the war in the Battle of Midway. So

355
00:22:10,120 --> 00:22:15,260
here we can inject known data as input and then look at the output of that data to see

356
00:22:15,260 --> 00:22:23,040
how it changes and validating that we're collecting the right data. So for example, I know the

357
00:22:23,040 --> 00:22:25,620
transmitter ID for my little gizmo that I wear.

358
00:22:26,240 --> 00:22:31,880
So I'm looking for that transmitter ID in the transmission. I know it's part of the transmission.

359
00:22:32,060 --> 00:22:36,240
Otherwise, it wouldn't be able to distinguish me from other devices. And I have three or

360
00:22:36,240 --> 00:22:41,680
four of these. So I can start to look at, see what patterns develop and where that ID

361
00:22:41,680 --> 00:22:42,500
is going to be.

362
00:22:44,080 --> 00:22:48,460
Now one thing that's important to note here is this is very time consuming. You have to

363
00:22:48,460 --> 00:22:52,620
collect all those packets and you have to analyze them. You have to sort them and you

364
00:22:52,620 --> 00:22:55,180
have to look at how they are.

365
00:22:55,840 --> 00:22:56,200
Okay.

366
00:22:56,200 --> 00:23:03,020
And where they line up. And small changes make a big difference in how we see that data.

367
00:23:03,460 --> 00:23:09,600
So for example, on my little transmitter gizmo, I know that the first set of data

368
00:23:09,600 --> 00:23:14,200
is always the same in every single transmission. If it was encrypted, would it look like that?

369
00:23:15,020 --> 00:23:15,300
No.

370
00:23:16,140 --> 00:23:25,840
So that proves my hypothesis that we're not using encryption. Being patient here is going

371
00:23:25,840 --> 00:23:26,180
to reward you for your hard work.

372
00:23:26,180 --> 00:23:31,880
So I recorded all of my transmitter values and what my blood sugar was and then I went

373
00:23:31,880 --> 00:23:35,740
through all of the packets to try and figure out exactly how it was transmitting that data

374
00:23:35,740 --> 00:23:36,560
and figuring it out.

375
00:23:41,020 --> 00:23:45,700
So the next stage is the fun stage and that's more the analysis and exploitation phase.

376
00:23:48,020 --> 00:23:52,060
So now we know what the data looks like, we know where it is and we know how devices

377
00:23:52,060 --> 00:23:56,160
respond to it. So the next thing we need to next thing we can do is we can network

378
00:23:56,160 --> 00:24:03,480
build a device to spoof one side of the connection. So what I did is I used a transmitter, a Yardstick

379
00:24:03,480 --> 00:24:09,920
One, to be able to transmit that ID and whatever I think that blood sugar value should be.

380
00:24:10,320 --> 00:24:15,300
So I was able to replicate what one side of that connection looked like and make that

381
00:24:15,300 --> 00:24:22,480
device see that data value. This is also very time consuming because there's a lot of little

382
00:24:22,480 --> 00:24:28,060
details, especially when you transmit data, on what the preamble looks like, what the

383
00:24:28,060 --> 00:24:32,040
end, the checksum looks like. There's lots of different pieces that you need to put that

384
00:24:32,040 --> 00:24:39,340
puzzle together. And it requires really expert knowledge. This is also an area where tools

385
00:24:39,340 --> 00:24:44,220
become more popular. When we look at things like Bluetooth, they have specific protocols

386
00:24:44,220 --> 00:24:49,940
that say where those things are and how they're going to line up in that packet structure.

387
00:24:50,400 --> 00:24:50,560
Okay.

388
00:24:54,080 --> 00:25:00,180
One thing that I note is that I'm very meticulous about it working every time. When you do a

389
00:25:00,180 --> 00:25:07,100
demo of a vulnerability in front of a manufacturer, it better work. And it better work ten times

390
00:25:07,100 --> 00:25:11,880
out of ten. Because if it doesn't work even once, you lose almost all your credibility.

391
00:25:13,020 --> 00:25:17,880
So it's not good enough for it to just work once. It has to be reliably reproducible.

392
00:25:18,440 --> 00:25:21,940
And this is the thing that is very important that we don't do.

393
00:25:22,480 --> 00:25:28,260
Is one person might go out and do some great research on a particular program, but we don't

394
00:25:28,260 --> 00:25:33,280
see a lot of people replicating that research. And going back and saying, oh, you know what,

395
00:25:33,340 --> 00:25:37,720
I thought that was really cool. I want to replicate that person's research and do exactly

396
00:25:37,720 --> 00:25:44,640
what he did and maybe find some more. Or maybe validate it and prove it to be true. Right

397
00:25:44,640 --> 00:25:50,500
now, almost all of our research is single sourced. Which means one person did one research,

398
00:25:50,740 --> 00:25:52,000
published it one time.

399
00:25:52,480 --> 00:26:00,060
And we accept those values as being completely true. No other science does that. Every other

400
00:26:00,060 --> 00:26:06,460
science, there are other people that do the exact same research or 98% of the exact same

401
00:26:06,460 --> 00:26:11,200
research with slight modifications to validate that research and to make sure that it's true.

402
00:26:12,720 --> 00:26:17,560
So it's got to be reliably reproducible. You have to have a very good documentation

403
00:26:17,560 --> 00:26:21,660
trail on how you did that. And it has to work every time.

404
00:26:21,660 --> 00:26:27,100
And you have to know how it works and why. I can't tell you how many presentations that

405
00:26:27,100 --> 00:26:32,520
I've been to where a researcher's been like, and then I loaded up Metasploit and I pointed

406
00:26:32,520 --> 00:26:38,000
at the device and I clicked this button and boom, I have Shell. That's not research.

407
00:26:39,820 --> 00:26:44,660
That's great. I'm glad you could do that. That has a value. But it is not research.

408
00:26:44,880 --> 00:26:51,460
You don't know how that exploit worked. That level of detail is what you need when you

409
00:26:51,460 --> 00:26:56,800
go to a manufacturer and you say, I found something wrong with your device. And if

410
00:26:56,800 --> 00:27:00,880
you go, yeah, I bought a tool and I pointed it at your device, and it popped it. They're

411
00:27:00,880 --> 00:27:04,180
going to be like, great, I have somebody I can sue now. You can leave.

412
00:27:05,440 --> 00:27:05,840
Crowd 1-1

413
00:27:05,840 --> 00:27:10,040
They're not going to help you. You need to know exactly what's going on behind the scenes

414
00:27:10,040 --> 00:27:11,800
in order to have that kind of credibility.

415
00:27:12,900 --> 00:27:13,720
Crowd 2-2

416
00:27:15,600 --> 00:27:16,000
Crowd 4-4

417
00:27:16,000 --> 00:27:20,640
The other thing I think about in the analysis stage is what's the nightmare scenario. What's

418
00:27:20,640 --> 00:27:21,440
the worst case scenario?

419
00:27:21,460 --> 00:27:25,900
scenario, what can I do to this device that would keep people up at night? And the user

420
00:27:25,900 --> 00:27:32,900
knowledge that we talked about with those user community forms, nurses, those manuals

421
00:27:32,900 --> 00:27:38,000
that we looked at, those are the places where we're going to see where those things come

422
00:27:38,000 --> 00:27:44,280
out at. And the most obvious thing might not be the worst. So, for example, you could turn

423
00:27:44,280 --> 00:27:51,240
an insulin pump off. And that's bad, right? Diabetics need insulin. But that is not the

424
00:27:51,240 --> 00:27:55,440
worst thing you can do. The worst thing you can do is change the therapy settings. So

425
00:27:55,440 --> 00:27:59,540
instead of getting one unit of insulin, I get ten units of insulin. And then two hours

426
00:27:59,540 --> 00:28:06,800
later when I'm on a floor in a coma, I have no idea what happened. That's the nightmare

427
00:28:06,800 --> 00:28:10,860
scenario, not turning off the device. So the most obvious thing might not be

428
00:28:10,860 --> 00:28:16,600
the worst. And that's where that real specialized knowledge comes in. And this is similar to

429
00:28:16,600 --> 00:28:21,080
building hypothesis theory. It's very similar to threat modeling that we do. In

430
00:28:21,080 --> 00:28:23,780
in, you know, kind of traditional computer security spaces,

431
00:28:24,140 --> 00:28:25,800
where we look at different threat actors

432
00:28:25,800 --> 00:28:29,120
and we try and figure out exactly what they are going to do

433
00:28:29,120 --> 00:28:31,300
and how we can prevent them from working.

434
00:28:31,920 --> 00:28:33,520
In this case, we're doing it backwards.

435
00:28:33,640 --> 00:28:35,580
We're trying to figure out exactly what the worst thing is

436
00:28:35,580 --> 00:28:37,200
and how can I make that happen.

437
00:28:38,680 --> 00:28:41,240
And remember, please, please, please, please,

438
00:28:41,320 --> 00:28:44,320
these devices are very serious, okay?

439
00:28:45,500 --> 00:28:49,220
Every time I experimented with my insulin pump on vulnerabilities,

440
00:28:49,220 --> 00:28:51,740
I removed it from me, okay?

441
00:28:52,380 --> 00:28:56,800
You know, I'm not crazy, but these devices are very serious.

442
00:28:57,120 --> 00:28:59,540
They can kill you. They can keep people alive.

443
00:29:00,020 --> 00:29:03,420
So be safe. Don't do anything haphazardly like,

444
00:29:03,480 --> 00:29:05,660
well, I just took this up to me and then I'll just run this exploit.

445
00:29:06,080 --> 00:29:07,320
Probably a bad idea.

446
00:29:12,740 --> 00:29:15,020
You know, one of the more controversial

447
00:29:15,020 --> 00:29:18,540
and heavily talked about things is disclosure and publishing.

448
00:29:20,880 --> 00:29:23,180
This is a topic that I take very seriously

449
00:29:24,120 --> 00:29:27,120
because it's not so much as it's difficult, but it is scary.

450
00:29:28,000 --> 00:29:30,660
When you present this to a medical device manufacturer,

451
00:29:31,020 --> 00:29:33,280
you could be threatening to put them out of business.

452
00:29:34,740 --> 00:29:36,980
If their device gets recalled by the FDA,

453
00:29:37,240 --> 00:29:38,520
that could be the end of them,

454
00:29:38,560 --> 00:29:40,020
if they're a small device manufacturer.

455
00:29:41,000 --> 00:29:42,720
So they are going to do everything possible

456
00:29:42,720 --> 00:29:44,520
to protect themselves in that case.

457
00:29:45,660 --> 00:29:47,860
You also have to realize that, like I said,

458
00:29:47,880 --> 00:29:48,560
these devices,

459
00:29:49,220 --> 00:29:49,880
they keep people alive.

460
00:29:50,440 --> 00:29:51,720
They keep people healthy.

461
00:29:52,800 --> 00:29:54,160
That is really important.

462
00:29:54,800 --> 00:29:56,280
And when you attack that,

463
00:29:56,940 --> 00:29:58,780
when you go after that and say,

464
00:29:58,880 --> 00:30:00,880
I can do something bad with this,

465
00:30:01,840 --> 00:30:03,200
you're going to get some backlash.

466
00:30:04,120 --> 00:30:04,600
Right?

467
00:30:04,700 --> 00:30:06,460
Because people take that very personally.

468
00:30:07,440 --> 00:30:10,380
I have a very big stack of emails from parents

469
00:30:10,380 --> 00:30:12,460
that did not like my research at all

470
00:30:13,340 --> 00:30:14,580
because they said,

471
00:30:14,700 --> 00:30:17,080
I'm showing people how to do bad things to their kids.

472
00:30:18,300 --> 00:30:19,200
I'm showing them how to do bad things to their kids.

473
00:30:19,200 --> 00:30:20,480
I'm showing them how to be terrorists.

474
00:30:21,520 --> 00:30:22,320
And I'm not.

475
00:30:22,520 --> 00:30:24,100
I'm making these devices better,

476
00:30:24,600 --> 00:30:26,040
but that's the first thing

477
00:30:26,040 --> 00:30:27,320
because they want to protect their children.

478
00:30:27,420 --> 00:30:28,880
They want to protect the people that they love.

479
00:30:29,520 --> 00:30:30,820
So you have to keep that in mind.

480
00:30:33,740 --> 00:30:35,780
Also, your career might be at risk.

481
00:30:37,020 --> 00:30:40,660
Nobody likes somebody that is a little bit too maverick.

482
00:30:41,920 --> 00:30:43,660
When I left, I didn't leave.

483
00:30:44,660 --> 00:30:46,180
After my Medtronic research,

484
00:30:46,380 --> 00:30:47,460
I did not leave that company.

485
00:30:47,540 --> 00:30:48,860
I was fired from that company.

486
00:30:50,740 --> 00:30:52,240
And I was fired from that company

487
00:30:52,240 --> 00:30:54,460
because they wanted full control

488
00:30:54,460 --> 00:30:56,200
over who I talked to,

489
00:30:56,320 --> 00:30:57,220
what I talked about,

490
00:30:57,340 --> 00:30:58,100
and what I researched.

491
00:30:58,560 --> 00:30:59,900
And I was uncomfortable with that

492
00:31:00,400 --> 00:31:01,880
because if it were up to them,

493
00:31:02,540 --> 00:31:04,140
they wouldn't have published that research,

494
00:31:04,360 --> 00:31:05,660
and we wouldn't know anything

495
00:31:05,660 --> 00:31:07,240
about insulin pump vulnerabilities.

496
00:31:08,160 --> 00:31:09,620
Well, at least not at that point.

497
00:31:10,980 --> 00:31:12,740
So you're really putting yourself out there.

498
00:31:13,500 --> 00:31:15,280
It's a heavy personal investment.

499
00:31:16,180 --> 00:31:17,160
It's a year and a half

500
00:31:17,160 --> 00:31:19,180
of research for each of the two of us.

501
00:31:19,200 --> 00:31:24,340
insulin pumps that I've worked on. And when you spend that much time honing something down and

502
00:31:24,340 --> 00:31:32,780
making it perfect, when you present it and somebody says, that's horrible, you really take it

503
00:31:32,780 --> 00:31:37,520
personally. There's an emotional connection to that research. It's one of the reasons that we

504
00:31:37,520 --> 00:31:45,940
try and find an intermediary to disclose through. ICS-CERT, Department of Homeland Security, the FDA,

505
00:31:45,940 --> 00:31:51,420
all very viable options. Maybe you shouldn't, if you're doing this research, go to the manufacturer

506
00:31:51,420 --> 00:32:00,040
directly. I always, always, always, always go through one of those three agencies. Because we

507
00:32:00,040 --> 00:32:08,200
have a standardized process at Rapid7 on how we do that. We publish, we notify ICS-CERT, we wait

508
00:32:08,200 --> 00:32:15,540
10 days, we notify the company, we wait 45 days, then we go public. That's our standard procedure.

509
00:32:16,280 --> 00:32:22,140
Sometimes that changes. If we see a manufacturer genuinely needs more time and they are genuinely

510
00:32:22,140 --> 00:32:28,260
working hard, we will give them more time. We gave Johnson & Johnson six months. Because they

511
00:32:28,260 --> 00:32:35,280
needed that time to move their giant ship of a company to get aligned. But they were very genuine

512
00:32:35,280 --> 00:32:40,260
and they were very forthcoming about, yes, these are vulnerabilities and we want to notify our

513
00:32:40,260 --> 00:32:45,780
patients and we want to do the right thing. When a company is acting like that, I have no problem

514
00:32:45,780 --> 00:32:45,920
at all. I'm going to go public. I'm going to go public. I'm going to go public. I'm going to go

515
00:32:45,920 --> 00:32:47,660
public. I'm going to go public. I'm going to go public at all. Because that's

516
00:32:47,660 --> 00:32:54,060
what's important. We went out with a message that didn't freak patients out. And that's really

517
00:32:54,060 --> 00:32:59,820
important when we're talking about these types of devices. I also in disclosure want to loop in

518
00:32:59,820 --> 00:33:08,300
the EFF and legal as soon as possible. You can expect to be bullied. You can expect to get

519
00:33:08,300 --> 00:33:15,460
cease and desist letters. You can expect to be sued. So be careful what you ask for when you

520
00:33:15,460 --> 00:33:15,900
get into this situation.

521
00:33:15,900 --> 00:33:21,660
research. You might find really cool things, but you also might increase your stress levels

522
00:33:21,660 --> 00:33:30,200
and shave years off your life. The next thing I'll say is difficult, but it's true, which

523
00:33:30,200 --> 00:33:37,280
is don't expect them to do anything. You might find something, you might pop shell, you might

524
00:33:37,280 --> 00:33:42,400
think it's the worst thing in the world, and they might think, whatever. We're not going

525
00:33:42,400 --> 00:33:48,660
to do a thing. We're not going to patch that. It's not important to us. And that might blow

526
00:33:48,660 --> 00:33:55,680
your mind, but it happens. So don't necessarily get your hopes up that something drastic is

527
00:33:55,680 --> 00:33:59,140
going to change and the manufacturer is going to be like, whoa, we can get a patch out for

528
00:33:59,140 --> 00:34:04,960
that next week. More often than not, it's a very slow process, and sometimes the process

529
00:34:04,960 --> 00:34:10,680
is nothing happens. Medical companies are big and highly regulated, and it makes it

530
00:34:10,680 --> 00:34:11,360
very confusing.

531
00:34:11,940 --> 00:34:16,200
We're not talking about a game on an iPhone that you can just patch and push an update

532
00:34:16,200 --> 00:34:21,840
to the app store. It's a lot more complicated than that. And the desire might be there,

533
00:34:21,920 --> 00:34:29,160
but other factors impact that. So, for example, Johnson & Johnson. We think, notify your

534
00:34:29,160 --> 00:34:36,080
clients. Shoot out an e-mail. This is a five-minute process. Johnson & Johnson's process was

535
00:34:36,080 --> 00:34:41,340
to send a registered letter to every customer and every doctor. That's quite a lot of work.

536
00:34:41,340 --> 00:34:45,160
That's quite a lot of production. And it was very eye-opening for Rapid7 and myself

537
00:34:45,160 --> 00:34:50,600
to be like, wow, you do have a process. You want to follow it. I totally want to respect

538
00:34:50,600 --> 00:34:58,700
that. But mail? Like stamps? For realsies? No. And it's true. I still have my letter

539
00:34:58,700 --> 00:35:03,780
that they sent, because I was a patient. That's how I got the device. And they sent out a

540
00:35:03,780 --> 00:35:11,320
letter, and we had to time our public disclosure to when the first person got the first letter

541
00:35:11,340 --> 00:35:15,440
because we didn't want it to be after they got the letter because that would be confusing.

542
00:35:15,800 --> 00:35:18,900
We didn't want it to be before the letter because that would be confusing to patients

543
00:35:18,900 --> 00:35:25,040
as well. So we had to kind of be like, okay, did you send it first class? Like, how exactly

544
00:35:25,040 --> 00:35:30,320
does that work? And it was very eye-opening. So you have to consider these other factors

545
00:35:30,860 --> 00:35:33,080
because they are unique to the medical industry.

546
00:35:35,420 --> 00:35:39,980
Relationships are the key to this. I have a very good friend now at Johnson & Johnson.

547
00:35:40,080 --> 00:35:41,040
His name is Colin Morgan.

548
00:35:41,340 --> 00:35:48,520
He's very active in our community. He's given talks at DEF CON and B-Sides. And he works

549
00:35:48,520 --> 00:35:53,400
at Johnson & Johnson and heads up their security. And I have a great relationship with him now.

550
00:35:54,280 --> 00:36:01,200
And we've helped. And the reputation as a researcher helps that company. It helps that

551
00:36:01,200 --> 00:36:05,320
company understand security so that the next time they get a vulnerability reported to

552
00:36:05,320 --> 00:36:11,320
them, they know what to do and what to expect. And word spreads quickly. If you do get a

553
00:36:11,340 --> 00:36:18,560
sue, an unethical, crappy job of disclosing, you will get sued a lot more. Right? But

554
00:36:18,560 --> 00:36:23,460
if you behave yourself and you're cooperative with companies, you get a reputation of being

555
00:36:23,960 --> 00:36:29,120
on their side, of being helpful. And that really makes things go a lot easier.

556
00:36:32,560 --> 00:36:37,340
So the talk before this kind of demonstrated a little bit of speaking publicly might be

557
00:36:37,340 --> 00:36:41,320
very tricky. There are legal issues. There are relationship issues. There are legal

558
00:36:41,340 --> 00:36:46,320
issues. Sometimes companies are like, hey, yeah, I'd prefer you didn't go to that DEF CON

559
00:36:46,320 --> 00:36:51,420
thing and tell 27,000 people we have a big vulnerability in our device. That could crush

560
00:36:51,420 --> 00:36:56,000
a relationship. So you have to be very careful about how often you speak and where you speak

561
00:36:56,000 --> 00:37:02,060
and what positioning you're taking in those talks. You really also have to think about

562
00:37:02,060 --> 00:37:09,320
something else. What's there to gain? Are you doing this to make yourself a rock star?

563
00:37:10,260 --> 00:37:11,320
Or is this really helpful?

564
00:37:11,340 --> 00:37:16,340
Is it helping the cause? Is going up and giving a presentation about how you popped root on

565
00:37:16,340 --> 00:37:24,100
an MRI machine, does that help raise awareness? Is that really moving the ball forward on

566
00:37:24,100 --> 00:37:29,320
how we're doing things? Is that what we need? And I think that there's a lot to think about

567
00:37:29,320 --> 00:37:36,440
when you do a presentation about medical devices and IoT devices in our space and what it is

568
00:37:36,440 --> 00:37:39,020
that's going to make an impact on the community and make things helpful.

569
00:37:42,340 --> 00:37:46,800
There is a second part to this publishing thing that I think is really really important

570
00:37:47,420 --> 00:37:55,840
and that I really really really want our community to take hold of. Being wrong is just as important

571
00:37:55,840 --> 00:38:03,780
as being right. What if my hypothesis was they don't use encryption and I grabbed the

572
00:38:03,780 --> 00:38:08,860
packets and they are all random? And I would use statistical analysis on them and it's

573
00:38:08,860 --> 00:38:13,660
it is totally random. It's definitely encrypted. It's definitely highly obfuscated. No packet

574
00:38:13,660 --> 00:38:21,560
is the same. In that case, I'm wrong. I'm just flat out wrong. In our community, you

575
00:38:21,560 --> 00:38:27,440
know what happens at this stage? We throw it away. Nobody cares about a device being

576
00:38:27,440 --> 00:38:31,880
secure. How many talks do you hear about, hey, I did a bunch of research on this medical

577
00:38:31,880 --> 00:38:36,900
device. You know what? Turns out it was really good. They have passwords. They use encryption.

578
00:38:36,900 --> 00:38:44,740
They got good key storage. I was impressed. My research concluded that I would feel comfortable

579
00:38:44,740 --> 00:38:53,680
using this product. You know what? Every other science does that. When they do an experiment

580
00:38:53,680 --> 00:38:58,180
and they do research and it doesn't line up with their hypothesis, they publish it anyway

581
00:38:58,180 --> 00:39:03,720
because it adds to the collection of knowledge. It adds to the community of knowledge of saying,

582
00:39:03,860 --> 00:39:06,820
hey, we have examples that show the

583
00:39:06,820 --> 00:39:15,460
counterpoint to this, that they are successful. The example I'll give is I did a blog post

584
00:39:15,460 --> 00:39:22,980
about the company Dexcom, which I wear a sensor for. And they pushed a firmware update. It's

585
00:39:22,980 --> 00:39:27,820
the first time I've ever seen a medical device do a, where they had the customer push a firmware

586
00:39:27,820 --> 00:39:32,540
update to their own devices. You could download it off their website. You plug in a USB cable

587
00:39:32,540 --> 00:39:36,800
to this little pager device that takes the blood sugar, and it pushes new firmware.

588
00:39:36,800 --> 00:39:43,280
To the processor. I was like, we've never seen this before. I'm sure this is going

589
00:39:43,280 --> 00:39:48,160
to be full of holes. Like this is going to be crap. So I downloaded the binary. I went

590
00:39:48,160 --> 00:39:55,700
through all my recon steps. And you know what I found? Signed firmware. I found encryption

591
00:39:55,700 --> 00:40:01,720
being used. I found updated transmission so that way, you know, there was more security

592
00:40:01,720 --> 00:40:05,400
for replay attacks. I was impressed.

593
00:40:06,800 --> 00:40:11,300
So I wrote a blog article and I called them out and I said, hey, I do security research

594
00:40:11,300 --> 00:40:16,060
and I was expecting to find a mess. And what I've actually found was that this manufacturer

595
00:40:16,060 --> 00:40:23,580
did a pretty good job. Was it perfect? No. But you know what? Good on them. Good for

596
00:40:23,580 --> 00:40:28,220
doing what's right and actually listening to the security community and advancing that.

597
00:40:29,020 --> 00:40:34,440
I never see that. And I really would like to see more. I would love to see people's

598
00:40:34,440 --> 00:40:36,780
research where they were like, you know, I tried this thing and it didn't work. But there's

599
00:40:36,800 --> 00:40:43,880
work out. That's valuable. Those techniques and that knowledge help everybody in the community

600
00:40:44,360 --> 00:40:49,220
figure out how to be better researchers and how to have a bigger collection of knowledge.

601
00:40:50,960 --> 00:40:55,360
And it gives us examples when we go to these conferences and when we talk to these vendors

602
00:40:55,360 --> 00:41:01,160
about who is doing it right instead of just focusing on how everybody is doing it wrong.

603
00:41:02,520 --> 00:41:08,760
We have to look to other fields of science and model what they are doing, because they

604
00:41:08,760 --> 00:41:13,820
have been doing it for hundreds of years. They have crafted something that's been extremely

605
00:41:13,820 --> 00:41:19,760
successful and we can't just be like, ha that's trash, that don't apply to us. We will just

606
00:41:19,760 --> 00:41:25,500
do our own thing. We have to learn and we have to take that into consideration.

607
00:41:28,060 --> 00:41:29,220
Just kind of a conclusion.

608
00:41:30,260 --> 00:41:31,140
I want you toème

609
00:41:31,160 --> 00:41:36,540
to do more than just wildly pop devices as fast as you can and pen test them. I want

610
00:41:36,540 --> 00:41:41,220
you to follow a method. Even if that method is out of your sixth grade science book, it's

611
00:41:41,220 --> 00:41:48,300
very valid. That's what sciences use. It's what we should be using. And lastly, I want

612
00:41:48,300 --> 00:41:53,980
you to publish. Even if you think you found nothing, your methods and your results help

613
00:41:53,980 --> 00:42:00,000
build the community. It helps make our research better and it makes our research look more

614
00:42:00,000 --> 00:42:05,740
credible from the outside, which is desperately something that we need. Especially in the

615
00:42:05,740 --> 00:42:11,180
medical field. Because that's what they do for a living. And when we come in there with

616
00:42:11,180 --> 00:42:16,740
our wild haircuts and, you know, our hoodies on, and we're like, hey, we popped your device,

617
00:42:17,220 --> 00:42:21,960
we're not going to make ground. We're not going to succeed at making the world a safer

618
00:42:21,960 --> 00:42:29,520
place. We have to use some of their methodologies in order to make an impression on them. And

619
00:42:29,520 --> 00:42:29,980
that is what I want you to do.

620
00:42:30,000 --> 00:42:35,760
That is all I have. If you guys have any questions, we have, like, ten minutes. I'd

621
00:42:35,760 --> 00:42:36,400
love to take some more.

622
00:42:46,340 --> 00:42:50,620
Hmm? Oh, we do? All right. So we have time. So if you have a question, come up to the

623
00:42:50,620 --> 00:42:53,460
microphone. Don't be bashful.

624
00:42:54,520 --> 00:42:58,400
Did Johnson & Johnson end up patching? Did they actually end up patching or they just

625
00:42:58,400 --> 00:42:58,900
sent a letter?

626
00:43:00,000 --> 00:43:04,280
So this is what we ended up doing. One of the reasons it took six months is their firmware

627
00:43:04,280 --> 00:43:10,800
is unupdateable. You know, a lot of these medical device manufacturers hard code the

628
00:43:10,800 --> 00:43:15,000
device so it's not updateable. Because that was secure when they built it 20 years ago.

629
00:43:16,120 --> 00:43:22,300
There is two workarounds that can be done. One completely resolves the solution so you

630
00:43:22,300 --> 00:43:27,440
can turn the wireless off for the device. And that fixes the problem. There's no more

631
00:43:27,440 --> 00:43:28,080
wireless vector.

632
00:43:29,120 --> 00:43:33,420
So people that are very concerned could do that. And then we also published two additional

633
00:43:33,420 --> 00:43:39,080
kind of remediation steps. One is that you can turn on alerting. So that way you get

634
00:43:39,080 --> 00:43:45,640
alerted any time that you get medicine. So that way if you had an unauthorized dosing

635
00:43:45,640 --> 00:43:50,260
of medicine, you would know about it. And the second is for children to lower the maximum

636
00:43:50,260 --> 00:43:56,240
dose. Because on an insulin pump, it will only give you so much a day. So you can lower

637
00:43:56,240 --> 00:43:57,620
that amount to be more realistic.

638
00:43:58,080 --> 00:44:04,320
And that means any attack might not get into the life-threatening area. It would still

639
00:44:04,320 --> 00:44:11,160
be bad, but easily resolvable. So one of the important things is that Rapid7 and Johnson

640
00:44:11,160 --> 00:44:17,260
& Johnson came out with the exact same language and the exact same press release on what those

641
00:44:17,260 --> 00:44:22,760
solutions were. So we wanted patients to feel very secure and not to think, Rapid7 says

642
00:44:22,760 --> 00:44:27,680
this and Johnson & Johnson says this, and they totally don't align. Because we wanted

643
00:44:28,080 --> 00:44:31,540
the message to go out the same way, and we wanted patients to feel very safe and secure.

644
00:44:33,620 --> 00:44:34,100
Yep.

645
00:44:34,500 --> 00:44:38,740
You mentioned that our community doesn't always do a good job of reproducing other

646
00:44:38,740 --> 00:44:43,520
researchers' results. When a researcher publishes a proof-of-concept and people are

647
00:44:43,520 --> 00:44:48,160
able to run it on their own, isn't that an effective way of reproducing results?

648
00:44:48,560 --> 00:44:52,960
Not really. Because I think that those, you run that proof-of-concept against devices

649
00:44:52,960 --> 00:44:58,020
that are totally configured differently. And there's no methodology there. Your proof-of-concept

650
00:44:58,080 --> 00:45:03,700
code, it's like popping a shell. It's a Metasploit module. I don't think that that's reproducing

651
00:45:03,700 --> 00:45:07,720
research. I don't think that that's adding to the collectiveness. It's testing a tool,

652
00:45:07,900 --> 00:45:14,560
but it's not adding to the pool of knowledge that is security research the same way that

653
00:45:15,040 --> 00:45:17,000
research is done in other fields.

654
00:45:17,640 --> 00:45:22,520
I agree with you on that. But it is, you have to agree that it's a way, method of verification

655
00:45:22,520 --> 00:45:25,840
that the research that they published was, in fact, valid.

656
00:45:26,100 --> 00:45:27,140
So that verification...

657
00:45:27,820 --> 00:45:28,060
Yeah.

658
00:45:28,080 --> 00:45:31,380
That verification is different than kind of expanding on the research.

659
00:45:31,460 --> 00:45:32,040
Right, yes, of course.

660
00:45:32,100 --> 00:45:36,580
Right. So yeah, I do agree that it validates that. It proves that that research data point

661
00:45:36,580 --> 00:45:41,240
is true, but it doesn't expand upon it. And that's really more what I'm looking for, or

662
00:45:41,240 --> 00:45:42,260
what I think should be looked for.

663
00:45:45,900 --> 00:45:52,320
I'm wondering about the repository of the publications that it could be your, part of

664
00:45:52,320 --> 00:45:58,060
your literature review. So it's like when you do some research, you have to see what

665
00:45:58,060 --> 00:46:00,600
other people's has been done. You don't want to do the same thing.

666
00:46:00,780 --> 00:46:01,080
Right.

667
00:46:01,180 --> 00:46:06,880
So I noticed you mentioned a couple of spots that you're collecting your raw material

668
00:46:06,880 --> 00:46:15,820
from, like Google and Patent. Do you actually step into a repository of the literature

669
00:46:15,820 --> 00:46:22,140
available in that specific field that will be helpful for your research? And also, are

670
00:46:22,140 --> 00:46:28,040
you contributing your, this research result to the community? If there is, you know, if

671
00:46:28,040 --> 00:46:31,380
there is a repository of those literatures, so it can benefit other people?

672
00:46:31,780 --> 00:46:37,100
Yes. You know, I think that the way that we do that is very, very more, is much more in

673
00:46:37,100 --> 00:46:43,840
line with doing security talk, like doing talks. So when I did the talk in 2011, and

674
00:46:43,840 --> 00:46:47,920
subsequent talks that I do, I talk about the methodologies that I used in my research,

675
00:46:48,120 --> 00:46:50,340
so that way other researchers can use those methods.

676
00:46:50,940 --> 00:46:56,000
We don't really have a great way, a great centralized location to publish research.

677
00:46:56,760 --> 00:46:57,960
You know, and I think that's...

678
00:46:57,960 --> 00:46:58,140
Right.

679
00:46:58,140 --> 00:47:03,320
That's a weakness, right? We don't have a scientific journal of how to, of, of

680
00:47:04,120 --> 00:47:10,140
security research. I've been asked to publish in medical journals some of the security research

681
00:47:10,140 --> 00:47:14,500
I've done. And the thing that always gets pushed back to me is it's not scientific enough.

682
00:47:15,160 --> 00:47:19,800
I didn't use a big enough sample size. I'm not using traditional statistical methodologies.

683
00:47:19,940 --> 00:47:25,160
So I need to up my game in order to get into some of these medical journals and get published

684
00:47:25,160 --> 00:47:25,960
in those areas.

685
00:47:26,520 --> 00:47:27,860
And it's something that...

686
00:47:27,860 --> 00:47:27,940
Yeah.

687
00:47:27,940 --> 00:47:32,520
That is a challenge for me, you know, to, to do better on that. And sometimes I get

688
00:47:32,520 --> 00:47:36,120
overwhelmed looking at scientific research articles because I can't keep up with what

689
00:47:36,120 --> 00:47:37,220
some of these doctors are doing.

690
00:47:37,380 --> 00:47:41,820
Yeah, it's like giant amount of work and you have to do all the repeats and collect

691
00:47:41,820 --> 00:47:46,800
your data and to be convincing and, you know, that's just like the beginning. You get into

692
00:47:46,800 --> 00:47:47,740
the medical area.

693
00:47:47,960 --> 00:47:48,280
Exactly.

694
00:47:48,440 --> 00:47:48,580
Yeah.

695
00:47:48,700 --> 00:47:49,700
You have to work your way up.

696
00:47:49,700 --> 00:47:53,480
So, yeah. Yeah, I, I'm, I will, I, you know, I think that in future research I'd like

697
00:47:53,480 --> 00:47:57,920
to do more white papers to kind of build up some more of that, that...

698
00:47:57,920 --> 00:47:58,100
Yeah.

699
00:47:58,100 --> 00:48:02,180
...publishable material. But we'll see how that goes.

700
00:48:02,920 --> 00:48:03,680
Great. Thank you.

701
00:48:03,800 --> 00:48:04,120
Yep.

702
00:48:05,340 --> 00:48:09,120
Thank you very much for your talk. Just follow up to the first question, the previous

703
00:48:09,120 --> 00:48:13,900
question. Do you think we as a community should have a journal of, you know, cybersecurity

704
00:48:15,260 --> 00:48:18,360
instead of just, say, publishing blog posts or YouTube videos?

705
00:48:18,540 --> 00:48:23,460
Yeah. I think that's, I think that would be great. You know, I, here's the, here's the

706
00:48:23,460 --> 00:48:27,900
other struggle. There's proof of concept.

707
00:48:27,900 --> 00:48:39,000
Okay. I can't take that into the medical world. Can't. Right? I love our community.

708
00:48:39,600 --> 00:48:44,520
Sometimes it drives me a little nuts. Right? Because we take things too flippantly. You

709
00:48:44,520 --> 00:48:49,560
know, we do need something that I can go into the medical world or the legal world and be

710
00:48:49,560 --> 00:48:54,940
like, no, seriously, this is a credible journal. You can publish in this. What does it stand

711
00:48:54,940 --> 00:48:55,120
for?

712
00:48:55,240 --> 00:48:57,700
Uh, I don't really, it's, no worries.

713
00:48:57,700 --> 00:49:03,060
Letters don't mean anything. You know, so it, sometimes we shoot ourselves in the foot.

714
00:49:03,280 --> 00:49:08,500
We do have methods to publish and then we choose to be a little more immature about

715
00:49:08,500 --> 00:49:13,360
them than we need to be. That's tough. Because we like to have fun. Everybody wants to have

716
00:49:13,360 --> 00:49:16,780
fun. But there's some times that we need to tone it down a little bit, I think.

717
00:49:20,040 --> 00:49:20,920
Any other questions?

718
00:49:21,840 --> 00:49:23,620
Where else do you give the speech?

719
00:49:24,920 --> 00:49:27,100
Uh, I've given this talk at DerbyCon.

720
00:49:27,700 --> 00:49:32,220
And, uh, I've given it a couple B-sides. You know, I try to get out as much as I can to

721
00:49:32,220 --> 00:49:37,360
do talks. Uh, but, you know, it, it depends on my travel schedule and my work schedule.

722
00:49:37,800 --> 00:49:38,200
So...

723
00:49:38,760 --> 00:49:43,640
I also spend a lot of time now focused on speaking outside of our community. So I try

724
00:49:43,640 --> 00:49:47,720
to go to, uh, I was at a diabetic technology conference earlier this year. I've been to

725
00:49:47,720 --> 00:49:54,060
geriatric conferences. I've been to different medical and legal conferences to try and spread

726
00:49:54,060 --> 00:49:56,920
our knowledge out and to get more people involved in that.

727
00:50:02,760 --> 00:50:09,700
Yeah. Yeah, absolutely. Alright, thank you guys very much.