start	end	text
0	4000	Welcome, everybody, to my talk, and thank you to Biohacking Village for having me.
4020	7240	The title of today is, Why is Healthcare Security Hard?
8260	12540	My name is Seth Carmody. I spent eight years at the FDA.
12540	16360	I started off as a device reviewer, and then I transitioned
16360	20180	to tech policy, and then I recently left the agency and joined
21180	23100	a security tech startup.
25300	28160	When we start to talk about healthcare economics,
28160	32120	we need to first move past the struggles that security
32120	37140	folks face, the frustration and the friction, the sticking points,
37240	40020	a lack of funding, a lack of traction, and this problem
40020	44200	is too big kind of thoughts. And once you
44200	48360	are able to move past that, you start to observe some patterns, and what you realize is that
48360	52080	these patterns are really just people. It's a collection of people
52080	56000	making decisions, and those decisions are based on incentives.
56000	60040	And the dude said, you look for the person who benefits. And indeed,
60040	64600	once you start to unpack why people in the healthcare space make the decisions they do,
64600	68020	especially around security, you start to understand the tension
68020	72820	I felt as a regulator, and now still feel as a tech supplier.
73280	76360	This talk is really a call to action,
76360	80040	and this is the beginnings of shining a light, if only briefly,
80040	83960	on the economic forces to provoke questions and really
83960	88040	to ultimately prioritize where we spend our career capital. Since our time
88040	92190	is precious, we want to spend it on the most impactful endeavors.
97180	99360	Professor Ross Anderson wrote a paper
99360	103440	from around 2001, what was called
103440	107680	Why Information Security is Hard. I didn't know at the time
107680	111600	when I was titling my talk for the Biohacking Village, but it's clear now after
111600	115480	researching this talk why the titles
115480	119180	align. In that paper, which as I understand launched
119180	123420	formal research into security economics, Anderson described three market forces that make
123420	127220	information security hard. He talks about network effects, such as
127220	132280	the race to be first to market, today we launch, tomorrow we fix.
132360	135720	He talks about the low marginal costs to produce software
135720	139540	and that necessitates that software be sold on its value
139540	143420	and not its cost. And lastly, technical lock-in, the software
143420	147300	builder, the value of the software builder's company is equal
147300	151360	to the total lock-in of all their users. So while I am presumably
151360	155480	two decades behind, Professor Anderson states in his reconfigure
155480	159500	of security engineering, the prominent textbook that he's authored,
159500	163560	that medical devices specifically is an area that needs additional research. And today
163560	167520	I intend to show the healthcare version of his key economic insights
167520	170280	that make healthcare security hard.
173200	176320	Whenever we start to talk about hard problems and things
176320	180320	that make it easier, I tend to draw on things that I know well and I know
180320	184280	the security space, I know the chemistry space very well, and
184280	188180	I start to think of catalysis. And the general concept is that
188180	192280	the universe says if you want to travel or go from point A to point
192280	196520	B, that's going to cost you, and in catalysis we're talking about energy.
196580	200240	But if you add a catalyst, it's like a discount from the universe, like a
200240	204520	wormhole, thereby allowing you to reach your destination using fewer resources.
204920	208260	And if we consider the significant resource
208260	212180	constraints in healthcare security, if we want to be successful,
212180	215780	we have to seek out the economic catalysts.
216100	220460	So let's pose a thought experiment, and this might be helpful
221520	224400	in decomposing a giant problem of healthcare
224400	228320	security into some bite-sized chunks. So let's consider the
228320	232200	following thought experiment given a specific resource time. The baseline
232200	236200	projection on the bottom there basically says
236780	240380	to reach some reasonable goal of, say, 90% safety, which is
240380	244380	fairly arbitrary, but let's start there. Considering our current trajectory and current
244380	248620	progress could take anywhere from 15 to 55 years.
249320	252220	And this is the tension I feel. I felt it as a regulator
252220	256500	being responsible for the progress of medical device cybersecurity. I felt it as a technology
256500	260600	I feel it now even as a technology vendor responsible
260600	264080	for making security easier. It's the same problem and the same
264080	269080	operating incentives. And I also still feel this sense of urgency.
269260	272720	Maybe nothing catastrophic will happen again, but maybe
272720	276660	my concern is overblown, but this is the exact point of being
276660	281400	prepared for a cyber event. You prepare for the worst and you hope for the best.
281400	284780	It would be perfectly acceptable to me to over-prepare,
284780	287720	to never need to reap the fruits of preparedness,
287720	292660	but whatever your viewpoint is on urgency and preparedness, hopefully we can agree
292660	296580	that regression isn't acceptable. And without a hard look and
296580	300400	reckoning with the security economics, I fear that the probability that progress languishes
300400	304900	or diminishes is too high. So if we can effectively identify the catalysts,
304900	308620	we can compress the timeline to a decade or so without any additional
308620	312580	effort. And when I envisioned this talk, I really
312580	316520	wanted to talk about supply chain and liability together because they're literally
316520	320840	the one-two punch of security economics. And when I say liability,
322080	324000	I mean that
324460	328500	when people produce technology, that there's some culpability to
328500	332140	the security debt that's ingrained in that piece of technology.
332620	336640	But first, I felt like it was sort of outside the
336640	340880	bounds of this talk because we had some real problems in supply chain to solve first,
340880	344860	which we'll talk about throughout the talk. The topics
344860	348860	of equipment phase-out, legacy equipment, and simulation exercises are important
348860	351960	but these topics won't be covered today.
354120	358420	I like to frame the concept of healthcare supply chain
358420	362540	economics within the supply chain. So that's the first
362540	366560	problem that we're going to tackle. And it's a useful
366560	370340	framer for all the discussions that will happen.
370340	374000	It's kind of low fidelity. Maybe it's only one example. There are plenty
374000	378400	of permutations on the supply chain. But here I think we have at least
378400	382320	some of the major players represented. There are certainly some simplifications
382320	386380	here and parts missing. The general flow is
386380	390280	that if we consider that there are a bunch of builders and
390280	394460	buyers relationships within the supply chain, we can start to tease apart some of the operating
394460	398520	incentives. So tech, from big to small
398520	402460	to security to general purpose tech, are supplying medical
402460	406100	device manufacturers who act as buyers in this regard
406100	410220	who then reassemble these pieces of technology into
411420	414760	a medical device or some medical technology.
414760	418420	This is specific to medical devices. But they're building it
418420	422280	to achieve some intended effect. It could be a pacemaker. It could be
422280	425840	an insulin pump. It could be a linear accelerator.
425840	429820	And in general, we'll just assume that the medical
429820	433080	device manufacturer who's regulated principally by the FDA but there are other
435380	437800	regulatory forces that
437800	441360	they're selling to principally a healthcare delivery organization or hospital system
441360	445280	is a buyer. The hospital then facilitates the
445740	448960	interaction of the device with the doctor and then the doctor delivers
449720	453760	the therapy of that device to the patient. And all of these stakeholders
453760	457440	when the supply chain economics have a role.
458080	461740	But the thing is, is that
462720	466500	healthcare's core competency is healthcare, not security.
466500	469780	And therein lies sort of the first principle
469780	474260	of healthcare supply chain economics and represents a
474260	478000	significant barrier to overcome. So
478000	481800	we typically hear about sort of where the security debt
481800	485320	sort of builds up. We hear about what hospitals are doing
485880	490020	to secure medical devices and other technologies. And just yesterday, IBM
490020	493980	released a report that basically said healthcare is lagging in all
493980	497900	key indicators within security. They spend less. They take more
497900	502120	time to resolve things. They use less technology to solve a security problem.
502120	505860	And this is indicative exactly of the thing that I'm talking about. It's because
505860	510220	healthcare's core competency as it should be is healthcare, not security.
510220	514560	And this is a core economic principle that specialization
514560	518600	is efficient. You wouldn't want hospitals to be security experts
518600	521900	because a jack of all trades is a master of none. So
521900	526320	if you try to make healthcare security experts,
526320	530520	healthcare stakeholders of all varieties, security experts, you get worse
530520	534500	healthcare and inadequate security. And furthermore, trying to make everyone
534500	538580	security experts is the uncatalyzed path. It will cost the most,
538580	542340	will take the longest, and if we take this path, I doubt that we will ever
542340	544100	collectively succeed.
546540	550100	Make no mistake, secure tech is hard to build.
550580	554400	So if we just back one step up from
554400	558280	the healthcare delivery organization up into the manufacturer and its supply chain,
558280	562620	from the manufacturer perspective, historically there has been a lack of out-of-the-box
562620	566480	secure by design solutions. If MDMs, as I like to call them,
566480	570360	needed an operating system, a wireless chip, a protocol, MDMs
570360	574900	were under similar constraints as HDOs. In order to build life-saving technology,
574900	577340	they had to use what was there.
579360	582460	And because of the economics that Professor Anderson outlines
582460	586320	in his 2000 paper, the components that the MDMs pulled from
586320	590320	technology vendors' shelves weren't built with security in mind.
590460	594220	Therefore, the security debt was passed to the manufacturer and in turn passed down the chain.
594220	598020	And now that security has emerged as top of mind for regulators like the FDA
598020	602560	and some leading HDOs, device manufacturers are at an inflection point
602560	606600	where they need to buy secure components to build and maintain secure devices.
606700	610400	And this transition is hampered by the security economic truth of healthcare.
610400	613300	Healthcare's core competency is healthcare as it should be.
613540	618300	Don't get me wrong, absolutely, device manufacturers should be building some security engineering
618300	622780	capacity. These folks are the guides to the security universe and you need them.
622780	627680	To what extent that capacity is built really depends on each individual company.
627880	631080	Yet despite the receptivity of device manufacturers to transition
631080	635600	and build security capacity, every product security person I've ever met,
635600	638760	even the ones that have experienced tremendous success,
638760	642760	have struggled in some capacity and scale to get the organization and individual
642760	646940	business units to do what's necessary for security for reasons that we'll
646940	651560	discuss throughout the talk. Since the transition is arduous,
651560	656260	security folks within the device manufacturers need the tech communities help.
656520	658500	Device manufacturers need as many out-of-the-box
659180	663280	secure-by-design solutions as they can get
663280	667160	and make no mistake, secure-by-design, leveraging zero-trust
667160	671100	architectures, while not the topic of my talk, is a prerequisite
671100	675600	for everything that follows. We can now confront the reality
675600	679720	and magnitude of the challenge before us because despite the economic reality,
679720	682180	the healthcare supply chain must be secured.
685440	690000	We have a saying in chemistry that the best model system is the real thing and you won't know
690000	693840	until you try, which means there is no amount of testing that can accurately
693840	698220	model real life. I'm not the only one that thinks this way.
698560	701860	The story of the DERAC-25 is a classic case study
701860	706100	for lawyers, computer scientists, medical physicists, and so on.
706100	709200	The case study is exquisitely described by Nancy Levinson's work
709660	712620	in a few places. There's an appendix A of her book,
712620	717640	Safeware, Systems, Safety, and Computers, and I think there's a separate paper as well.
718180	721180	If you're not familiar with the story, a linear
721180	725780	accelerator experiences failures over and under, irradiates, and kills some patients.
726040	729100	It's a nightmare scenario for every
729100	732340	manufacturer, and which they seek to avoid.
732860	736980	And the story is probably so cliche and cherry-picked that eyes glaze over every time it's mentioned,
736980	741200	but I'd like to use it here to describe two key messages. In particular, there's an FDA
741200	744940	quote, which is basically
746140	748920	giving feedback to the manufacturer on
748920	752420	their corrective action plan, or a plan to fix the issues.
752720	757140	The FDA reviewer
757140	760820	says the following, we are in the position of saying that the proposed
760820	764940	corrective action plan can reasonably be expected to correct the deficiency
764940	769140	for which they were developed. We cannot say that we are reasonably confident
769140	773040	about the safety of the entire system to prevent or minimize exposure from
773040	776960	other fault conditions. And if I can speak for this FDA employee, I
776960	780800	think what they're trying to say is what Donald Rumsfeld said so well. There are
780800	784980	known knowns and unknown unknowns. They're just properties of the system
784980	788900	which we cannot accurately predict. And this gets to the second point,
788900	792580	that there's this underlying devotion and subscription, even today, to
792580	796980	extrapolating the rate of component failure to the rate of system failure.
797160	800520	And when we rely on deterministic values for rates of system
800520	804700	failures, history has shown us that those risk models break down. And simply
804700	808700	put, the classic probability times severity model is an oversimplified model
808700	811900	that trades tractability for fidelity.
812420	816700	This exact model breakdown is further exemplified by Richard
816700	820980	Feynman and his Roger Commission work. On January 28, 1986,
820980	824580	73 seconds after liftoff, the space shuttle Challenger broke apart, killing
824580	829120	seven crew members. In the resulting investigation of the Rogers Commission,
829120	832700	member and Nobel Laureate Richard Feynman, a theoretical physicist,
832700	836600	excoriated NASA management for extrapolating component failure
836600	840360	rates to the entire shuttle system. Essentially, managers had
840360	844520	concluded that through the use of component failure data,
844520	848300	that if the shuttle launched once daily for 300 years, NASA would experience
848300	852300	exactly one catastrophic failure, or one failure
852300	856020	in 100,000 launches. Feynman's instincts told him otherwise
856020	860260	and conversations with engineers resulted in a more realistic figure of
860260	864360	one failure in 100 launches. Don't get me wrong, device manufacturers
864360	868420	absolutely need to test to reduce risk. However, you will not know
868420	872320	the extent until it's marketed. That's a fact. That's why the bar for marketing
872320	876300	is not 100% safety and 100% effectiveness. The legal
876300	880160	bar is reasonable assurance of safety and effectiveness, and in
880160	884240	complex systems, with emergent behavior, you need to collect post-market
884240	887860	data, which they do. It's just sub-optimal.
891480	895440	Data is important, especially if the data indicate a near miss.
895460	899280	Also, as part of the Rogers Commission, Feynman had to explain to NASA what a
899280	903800	safety factor was. Some early tests of the booster rocket's O-rings
903800	906560	resulted in the O-ring burning a third of the way through.
906560	910000	NASA managers recorded this result as demonstrating that the
910560	914180	O-rings had a safety factor of three, and Feynman had to explain
914180	918880	that the safety factor is, in fact, zero. To paraphrase
918880	922640	Feynman's example, if a 1,000-pound truck drove across a bridge
922640	926400	designed to bear 3,000 pounds and a crack appeared in a beam
926400	930520	even just a third of the way through, the safety factor is now zero. The bridge
930520	934420	is defective. There was no safety factor at all, even though the bridge
934420	938720	did not actually collapse. The data for the O-ring was there.
943180	946480	To further build on this idea of systems
946480	950260	and uncertainty in systems, as a regulator
950260	954340	I had a front-row seat to quite a few economic externalities for security
954340	958200	from routine vulnerability disclosures to global cyber attacks.
958400	962180	What has been the impact of these events? We have lots of disclosures with
962320	966140	a dash of coordination and a pinch of public. We've got WannaCry and
966140	969820	Petya and a handful of large impact vulnerabilities that have us questioning
969820	974360	why did I choose this field? As a
974660	978380	manufacturer, it would be perfectly rational to
979340	982300	the event of a disclosure dropping my stock price
982300	986460	but ultimately recovering. You might read it with a sigh of relief.
986460	990420	I'm glad that's over. What would be the incentive to change
990420	993980	in that case? It would be very easy to play the event off as a black swan or
993980	996620	an outlier instead of a larger symptom
997540	1001940	of an underlying issue or a cause to look under the hood or look into the
1001940	1005980	field and see if there's anything else there. And furthermore, because healthcare's
1005980	1009900	job is healthcare, hospitals still need to buy the technology and those darn
1009900	1013980	patients just won't spontaneously get better. It wouldn't surprise me
1013980	1017660	if there were negligible impacts to device and technology sales as a result of taxes
1017660	1022080	potent as WannaCry. So what should
1022080	1026460	we do when our crashes, our visible events are collectively shrugged off?
1027960	1029500	Especially when folks know that there's
1030040	1033040	nothing really that prevents a WannaCry 2.0.
1035540	1038160	And this is the perfect job of
1038160	1042200	the regulator. If the market fails to deliver the correct incentives, this is a primary example
1042200	1045940	of the role of the regulator. If you look at the history of the FDA and think about why the
1045940	1050100	FDA exists, it's about resolving information asymmetry for the consumer.
1050100	1054000	The average person cannot assess the safety and efficacy of a drug or device
1054000	1057960	and even if they could, it would be economically inefficient, not to mention ridiculous, that if
1058620	1062400	before you ate or started chemo or passed out from a cardiac event,
1062400	1066140	you had to evaluate the safety and efficacy of a product you needed.
1066160	1069980	So you need an impartial body to see what's in the meat or
1069980	1074040	evaluate the claim of a drug or device. And therein lies, again, the security
1074040	1078140	economics of healthcare. Healthcare is healthcare, not security. So you
1078140	1081980	are then reliant, as a consumer and patient,
1081980	1085980	you're reliant on the entire supply chain to deliver healthcare and deliver security.
1085980	1090180	It just doesn't work efficiently. It doesn't scale. It doesn't solve the problem on a timeline
1090180	1094040	that we desire. Nicholas Taleb, the author of
1094040	1097900	The Black Swan, offers us another brilliant explanation for the challenge of
1097900	1102280	healthcare's transition to security. The idea of a Black Swan event or a low-probability
1102280	1106460	event is often disregarded as possible, if regarded at all,
1106460	1110380	and therefore isn't present in our risk models. Now, Taleb was talking about
1110380	1114480	the uncertainty present in financial systems, but the concept applies directly
1114480	1118380	to healthcare as well. We're so concerned with the probabilities, and indeed
1118380	1122240	it is the foundation of risk management, that we've convinced ourselves
1122240	1127200	that a security event is a Black Swan event, not an everyday occurrence.
1131910	1134070	In 1990, the GAO released
1134350	1137530	a report the FDA knew of less than 1%
1137530	1141590	deaths, serious injuries, or equipment failures that occurred in hospitals.
1141590	1145410	And at the time, reporting from hospitals to manufacturers about
1145410	1149290	these events was voluntary, so the law was amended to make it compulsory for user
1149290	1153390	facilities and hospitals to report to device manufacturers and FDA
1153390	1157650	under certain conditions. And the result was a significant uptick in reporting.
1159890	1161430	Separately, in 1999,
1161430	1165250	the Institute of Medicine released a shocking report that the third leading cause of death
1165250	1168170	in the U.S. was about 250,000 deaths.
1168330	1173330	It was absolutely shocking.
1175870	1177150	This study
1177150	1181210	touched off a wave of efforts in hospitals to corral and reduce those
1181210	1185270	errors, and I've seen recent reports into 2020
1185270	1189270	that those deaths due to preventable errors are significantly reduced to around
1189270	1193210	22,000. Regardless of the number, these two pieces of information got me
1193210	1197050	thinking, how many close calls are there? And I wondered, what could be gained by
1197050	1201110	leveraging the technology and connectedness to automatically capture,
1201110	1205130	aggregate, and analyze the data about the performance of the device? In the age
1205130	1208950	of technology, it's absolutely possible and cheap to assess the performance of the device
1208950	1212770	for the betterment of healthcare. I mean, the internet figured out a way to
1212770	1216930	monetize my online behaviors. Why not do something else useful with it?
1217870	1221210	Imagine how difficult it was for the Therac-25 manufacturer
1221830	1225070	AECL to piece together the puzzle as it emerged.
1225070	1229150	We enjoyed the benefits of hindsight, but I imagine in the moment it was quite vexing.
1229150	1232510	You've got people dying, and you can't even reproduce the error, but if
1233180	1236630	you could further leverage the technology
1237120	1241370	already on board to capture the events, black box style,
1241370	1245050	pool and analyze the data, maybe it becomes far easier and far cheaper
1245050	1249170	to respond meaningfully to the real world. We need to
1249170	1253790	pay more attention at the system level. We need to look deeper. We need to collect data.
1253970	1257170	I believe that in the real world, everyday devices experience security
1257170	1260330	events. I mean, events as broadly as possible.
1260710	1265310	People may not be dying, but I'm sure that there are near misses all the time.
1265310	1269090	And if we can show the data, then that creates the incentive to build
1269090	1273090	secure healthcare
1273090	1277250	technology from the ground up. Using secure by design components from the tech
1277250	1281250	industry as well as helps us answer the questions of how
1281250	1285230	much security is adequate. So let's measure it.
1285490	1289330	Heisenberg said it best. He changed the outcome by measuring it. Now he meant it for subatomic
1289330	1292090	particles, but we'll take it here for changing the
1293110	1295770	outcomes of healthcare for the better.
1297190	1301390	And honestly, in order to get across the finish line for security, no security
1301390	1305130	event perceived to be a black swan will get us across the finish line. It will be the
1305130	1308990	innumerable cygnets that are hiding in the weeds of everyday data.
1308990	1312270	And today I learned that cygnets were baby swans.
1315690	1317170	Costanza said that he needed
1317170	1321210	to learn risk management because it was on his resume. Let's not fall
1321210	1325030	into the trap of using oversimplified risk management models because we
1325030	1329030	build our careers on it. The uncertainty of behavior complex system demands
1329030	1333070	the adoption of new, higher fidelity models that are plugged into the real
1333070	1337030	world. For as Feynman put it, mother nature cannot be fooled.
1337110	1340150	And in doing so, we'll begin to reveal the late economic incentives
1340970	1345010	not just for secure by design, but I imagine a host of
1345010	1348770	important clinical items we'll be able to tease out
1348770	1352550	efficiently if failures are related to human factors,
1352950	1356950	humidity, cyber security, bad boards, a software bug, or if it's a Tuesday
1356950	1360930	in April during a full moon, you'll have the data right there. And as we look
1360930	1364690	towards the future of healthcare where outcomes determine who gets paid,
1364690	1368950	it will be the only way. Certainly, tremendous progress has
1368950	1372890	been made, and the groundwork for catalysis and acceleration has been made possible
1372890	1376890	by this progress. You only need to look to the FDA and international regulators
1376890	1380130	recent policies in this space, and quite frankly, the efforts
1380810	1384730	of security champions across the supply chain really to understand how
1384730	1388850	deeply passionate healthcare is about fixing all sorts of problems, including security.
1388850	1392750	We can supercharge their efforts by turning ineffectual black swans into
1392750	1396990	effective everyday economic incentives. Thank you, and I'd be happy to take
1396990	1397970	any questions.