start	end	text
0	8180	So our next speaker is from NYU Tandon School of Engineering, from the Digital Future Lab.
9100	16780	His focus for this talk will be using engineering and AI to build the next gen of smart scopes
16780	19340	capable of diagnostic support.
19700	26660	And fun little fact, on his spare time, he does 3D print graffiti artists in the style
26660	28660	of green army men.
33260	41380	So presenting their talk, microscopes are stupid, I agree, here is Lou Auguste.
46700	51600	Hello everybody, good morning, good morning, thanks for coming to see this speech on microscopes
51600	52580	are stupid.
52900	58680	My name is Lou Auguste, I work at NYU Tandon School of Engineering in their future lab,
58680	64200	so it's really, I'm an entrepreneur in residence there, they incubate my company, been running
64200	68020	it there for three years now, and we work on solving this problem.
68160	72040	So just a little bit before we get started, we've won some awards over the last couple
72040	72600	of years.
72600	76120	We won the ASME award for best hardware prototype in 2015.
76540	80620	Last year we were nominated for science startup of the year by this competition out in Germany
80620	82280	called Falling Walls.
82400	88160	We won the Indian government's endowment for science research last year, and this year
88160	92100	we're finalists for Qualcomm's prize for design in India.
92420	99100	I published my methods in the British Medical Journal in 2015 on how to use smartphones
99100	106320	to create whole slide images, and I did a pre-clinical validation study of that information
106880	111920	in 2016, which I presented at USCAP, which is the United States and Canada Anatomical
111920	113480	Pathologist Society.
113940	117560	And this year we received IRB approval to do our clinical validation study, which I'm
117560	118960	writing up right now.
120180	122720	So yes, microscopes are stupid.
122720	124460	I said it, and that's it.
124560	129260	If you look at this microscope here, it's such a dinosaur that it's not even connected
129260	130240	to the internet.
133360	137740	So basically the problem is microscopes are dependent on the operator, and who's operating
137740	138840	that microscope.
139100	143500	And the problem that we're facing is as we look out at more and more of the healthcare
143500	149040	landscape in the world, we find that there are things called healthcare deserts.
149040	154260	These are places in America, the developing world, where options for healthcare don't
154260	155200	exist.
155400	156860	And it's not that they don't have microscopes.
156860	157940	They have microscopes.
157940	160860	They don't have anyone that could help them do diagnosis.
161680	167980	So this is very true, as I said, in Kentucky, just as true as it is in Haiti, just as true
167980	169220	as it is in India.
170760	175220	So, before we go forward, I want to give a shout out to my boy, Anton van Leeuwenhoek.
176260	178980	He's like the G of microscopes.
179020	185980	So basically, he was born in the 1500s, and 40 years before he was born, they had created
185980	187620	the compound microscope.
188500	193360	Leeuwenhoek was a tradesman, and he was actually into glassworking.
193360	195180	He didn't have any advanced degrees.
195180	197120	He didn't have a PhD.
197120	199740	So he was eliminated from a lot of the science community.
200120	205740	But what he did was he created, with his knowledge of glassblowing or glassmaking, he created
205740	207260	this microscope here.
207360	211880	And this microscope had a magnification of over 200 times magnification.
212200	216140	Up to that point, they had already invented the compound microscope, which is very similar
216140	218660	to what we see today in any laboratory.
218760	223120	But that only had a magnification of 20 to 30 times magnification.
223640	227400	So van Leeuwenhoek was definitely one of the first disruptors.
227460	232540	And he's also considered the father of microbiology, because of what he was able to see through
232540	234280	his microscope that he created.
234440	239380	And in fact, it wasn't really until the 1800s that they made any advancements past what
239380	241120	he made with this microscope here.
242500	244640	So also, humans are stupid.
244680	251560	Because when we're given the task, humans get tired, and they miss things.
251560	254780	And diagnosis itself depends on memory.
255160	260480	So you probably can't see this right here, because the screen's a little bit contrasty.
260620	265460	But if you were looking for cancer on this thing, there's actually a monkey in the upper
265460	270480	right-hand corner of the lung that most people miss when they're looking for cancer.
270540	274260	So I'm not sure if you could see that here, but that's usually a good example of how people
274260	278900	miss things that are obvious when they're doing something that's repetitive.
279560	285180	So a little history on telepathology, which is sort of what we do.
285420	287540	So this is 1986.
287560	291340	This was the first demonstration of a robotic microscope.
291600	295060	And you can only imagine the satisfaction that this man felt clutching those keys and
295060	299760	hearing those sweet clicking sounds as he moved the microscope around.
299980	304780	And really, the history of telepathology is sort of a history of the internet in many
304780	306080	ways as well.
306080	311320	Because as things improved with the internet, we were able to do more and more in terms
311320	316900	of sharing slides, and then virtually moving on to creating virtual slides and sharing
316900	318360	those slides over the cloud.
318540	322360	And now with things like TensorFlow, we're actually getting to the point where we can
322360	326140	deploy AI locally on the device.
326360	331260	And that's a big step, I think, that's happening right now in the telepathology industry.
332740	339440	So, really, the work that I've done over the last four years has been focused on helping
339440	343200	health outcomes in the developing world and solving the problems that they're facing.
343300	346900	And so one of the biggest problems we look at is this really simple equation.
346900	348940	Lost time equals lost lives.
349060	354340	So there's a worldwide shortage of pathologists or doctors able to make diagnosis.
354660	358240	And because of that, people can't get diagnosed.
358240	361800	And so if people can't get diagnosed, they can't begin treatment.
361860	366800	And the later you wait to begin treatment, the worse your health outcome is going to be.
368720	375480	So a lot of the research we've done is creating just simple ratios when we look and see how
375480	381120	many specialists there are in a country versus how many people there are.
381120	388360	And so you can see that in the US, you've got a ratio of basically 1 to 17,000.
389260	395360	But then if you look at a country like China, you've got 1 to 58,000.
395420	400860	And then if you look at a country like Haiti, you've got 1 to 1,375,000.
400920	407320	So you can see that there's just not enough doctors to make diagnosis for all these people.
407820	411580	And there's also things that happen when you're in one of these countries that has a small
411580	412680	number of doctors.
412920	416600	You can actually fix your price as one of those doctors, and your price can be a lot
416600	418640	higher than it would be in another country.
418860	421700	And so not only are those people not able to get access, but they have to pay a lot
421700	423960	more money for access to diagnosis.
424840	431200	So the College of American Pathologists states that between 60% and 70% of all diseases need
431340	436360	a diagnosis using some sort of cellular biology.
436360	442300	And that includes cancer, things like malaria, a lot of diseases that we know today.
443060	448240	Anyway, so another thing we've studied is this travel burden.
448540	451840	And a lot of people have done studies here in America.
451840	455040	And so we have three studies here, one from Kentucky, one from Illinois, and one from
455040	455960	Georgia.
456240	462680	And basically all these studies stated that people that lived at a far distance away from
462680	467120	the primary diagnosis center, they would get staged for cancer much later.
467600	469540	And so that's a big thing that we're facing.
469640	474400	So like I said, if it takes you a long time to get a diagnosis, you're probably going
474400	477900	to get diagnosed much later when it's going to be much more difficult to cure.
479160	483840	So to give you an example of one of the patients that I saw when I was down in Haiti.
484020	487660	So we had this woman come about three years ago and present herself.
487660	489000	And she had a breast mask.
489560	495060	And so we had a surgeon at the hospital who was able to do a biopsy for her, but there
495060	499400	was no one available that could look at her specimen once we had it.
500020	505440	So we gave her her specimen in a jar, and we gave her instructions on how to get to
505440	506660	the nearest laboratory.
506920	511000	And so that laboratory is about six, seven hours away, and she never made that visit.
511320	515880	So a couple of years later, she came back in, and this time the mask had grown so large
515880	517960	that it actually had ulcerated out of her breast.
517960	523760	So now you see we definitely can say, okay, well, this without a doubt is going to be
523760	525060	breast cancer right here.
525340	528940	But when we first saw her, when she first presented herself, when she only had a small
528940	533560	mask, we could have used diagnostic testing to prove that it was breast cancer at that
533560	537140	point, and consequently we could have started treatment much earlier.
537500	540340	And that's sort of an example of what happens.
540760	546760	Also it's important to note that the cost of testing, diagnostic testing, in Haiti is
546760	548440	$50 per test.
548580	552800	So that's like a really important number to kind of remember.
553440	558660	So the previous solutions were like to travel to Port-au-Prince, find a lab, wait a few
558660	560520	months and hopefully get a diagnosis.
561220	565660	What we were doing before, a long time ago, was collecting all the samples, flying them
565660	570100	to America, getting a lab to process them in the United States, sending all the results
570100	573360	back and praying that we could find all the people that we had tested while we were down
573360	574020	there.
575060	579980	Another chance is like the doctor operates based on symptoms, and then really what mostly
579980	582620	ends up happening is people die of unknown diseases.
583400	586320	So that's pretty heavy, so I'm going to chill out for a second.
589360	594100	But we're like, we're working to solve this problem because we know we can.
594100	597280	And we know we can use technology to solve this problem.
597700	602360	And so what we've done is we've been focusing on building human-machine teams.
602940	608480	So the vision really is, and it's something that the WHO sort of shares with us as well,
608480	610940	is to create more healthcare workers.
610940	616660	Because the paradigm of having a hospital, having a doctor that we know and well love
616660	620700	here in America or in other developed nations, it's just not going to work in a lot of these
620700	621340	countries.
621340	626440	So we have to take people that are willing to go to school for maybe six months to a
626440	631520	year, train them at a skill, that could be a skill like becoming a lab tech, and then
631520	636120	give them the technology they need in order to share those slides so that they could make
636120	637360	more diagnosis.
638120	639560	And so that's something we've been working on.
639560	644120	This is the team that I go down to Haiti with from time to time to do that.
645300	650040	So another big thing we've been looking at is, okay, so once we can build the capacity
652000	656800	for doctors or for healthcare workers to prepare specimens to upload those specimens to the
656800	659900	internet, well, who are they going to share those specimens with?
659900	663600	So normally what we did in the past was we shared them with doctors here in America.
663740	665900	But doctors in America are really busy.
666020	669420	They don't have enough time to look at cases from another hospital, especially if it's
669420	673100	not helping them on the bottom line, it's not helping them make more money or do whatever.
673700	677680	So what we've actually done is we're working with this team of doctors here in Tanzania.
678240	682660	And so these are all doctors, these are all pathologists, the women that you're seeing
682660	686040	here, and that's our friend Mumbek, he's also a doctor in Tanzania.
686040	692600	And what we've been working is in this hospital, Mumbek in Dar es Salaam, we're setting them
692600	695000	up with the technology that they need.
695260	699560	So remember when I said it was important, remember that it costs $50 to get a slide
699560	700840	diagnosed in Haiti.
700840	703780	Well, in Tanzania, the cost of reading a slide is only $12.
703900	709060	So already we're saving money for the people that are in Haiti by giving them this option.
709200	713060	And the doctors in Tanzania are only making about $700 per month.
713060	716540	So they are actually interested in supplementing their income.
717240	721280	And they also have an unused capacity, because they're sitting in traffic, and most of them
721280	722180	have drivers.
722180	725560	So when they're sitting in traffic, there's time for them, if they're on their tablet
725560	728620	or smartphone device, to actually be reviewing cases.
728900	735600	So our solution is really to actually send our cases from Haiti to Tanzania to decrease
735600	739400	the cost of testing, so the people in Haiti can afford to pay for testing, and they can
739400	740840	get a diagnosis quickly.
741340	744460	And then hopefully what we'll do is export the lab training program that we're developing
744460	749040	in Haiti, export that over to Tanzania, and then create more lab techs, more health workers,
749040	750760	that can work with what we're building.
751900	754000	So that's sort of where we're at.
754080	758580	And how we've been accomplishing this right now is we've been developing a system we call
758580	759440	SmartScope.
759880	762740	So this system basically has two components.
763260	768440	It attaches to a regular microscope, and over the eyepiece, we have a camera that just fits
768440	769600	on the eyepiece.
769600	772780	And so that'll fit up to an outer diameter of 44 millimeters.
772860	775920	So that'll cover most eyepieces that are out there.
776260	781760	And then we built this robotic slide holder, and it just attaches with two screws to the
781760	785880	side of the stage of the microscope.
786320	792080	So basically with these two pretty simple pieces of technology we've developed over
792080	796960	the last three years, we can take a standard microscope, and then we can turn it into an
796960	798040	IoT device.
798430	804500	And so what this is is something that we can deploy at a cost of about $1,000 to $2,000.
804900	809000	And on the other side, the companies that were previously building whole slide imaging
809000	814480	machines and dynamic microscopes, they were normally charging between $3,000 and $100,000
814480	816520	for one of these standalone devices.
816880	820500	And if you've worked in the developing world, you'll know that any sort of standalone device
820500	824840	that they have, there's a good likelihood that it's going to be broken by the time you
824840	826900	get back there the next time you come down.
826900	829360	And there's no one that's going to service these devices.
829540	835160	So we've actually created this technology really mimicking what we saw in 3D printers,
835160	838040	and we've created technology that you can actually repair yourself.
838440	841180	So I think that's a big change right now.
841340	845420	And then you can share it on your... it runs on the cloud, so you can share it on your
845420	848500	desktop, you can share it on your tablet, or you can even look at it on your smartphone.
850140	853920	So how we built it up to this point was we used the Intel Edison.
854280	859480	We thought that was a really good solution for us to start with, because it gave us a
859480	863060	lot of versatility, we could run a whole bunch of programs on it.
863060	868220	But the one drawback of working with the Edison was that it required us to have a computer
868220	869880	to run everything with.
869880	872940	So we couldn't sort of do things, everything on the device.
872940	878380	And that was sort of where we're at, and we're actually deploying these units into the field,
878380	881720	having sort of more and more use cases for it.
881720	885820	So we have hospitals in Indiana that are using it for immunohistochemistry stains.
885880	889380	They're just making sure, they're just using it for Q&A, for that kind of stuff.
889820	891260	But it's been good.
891260	894500	And so now we're looking at the next... what's next for us?
894500	895780	What's next in the future?
896240	899920	And so, I mean, I think this quote from Freeman Dyson is really good.
900200	904000	New directions in science are launched by new tools more than they are by new concepts.
904140	906940	The effect of concept-driven revolution is to explain old things in new ways.
906940	911420	The effect of tool-driven revolution is to discover new things that have to be explained.
911420	915940	So I think in a lot of ways, when Leeuwenhoek went on and developed a microscope
915940	921400	that could see at 200 times magnification, then he could explain microbiology.
921420	927100	For us, what we see in the vision is not only are we sort of working with this concept of people
927100	933340	that are going to be providing healthcare workers, we're also giving them the tools they need.
933380	935720	And we think these tools are powerful enough and cheap enough
935720	939540	that they can be deployed in every single laboratory anywhere in the world.
939540	944940	And we can create a whole global network, something that we like to call Skype for Microscopes.
945000	950620	And this is how we're going to be able to track new diseases as new diseases emerge into the world.
950620	953680	We're going to need to have these capabilities.
953680	955860	We can't just send an expert there right away.
955860	958220	There's time that it takes to transfer someone there.
958340	961180	But having a sort of a network that's really light,
961180	965780	we can create people around the world that can share this information with a centralized location.
966060	968240	And so I think that's sort of where we're at.
968240	971580	So I'll give you a little bit of a demo of how everything works.
972120	974660	Let me see, just press the play button there.
975740	979200	So this is our new website. It's launching later this month.
980020	986640	Basically, once you sign in, you can access any devices that you have permissions to access.
987120	988920	And so the devices are there.
989300	996500	And then these are invitations, so you can receive invitations to join a session by someone else that has a device.
997060	1002180	And then once we have everything sort of running, we have this interface here.
1002180	1004100	And you'll see the slide come up in a second.
1004660	1008360	And so what we're able to do is we're able to control the movement of the slide.
1008480	1011040	But we can also do things like add comments.
1011060	1013860	So right now we're adding a comment to this box here.
1014100	1017040	That could be sort of like anything you're seeing on the slide.
1017600	1020080	You can submit that comment and that's recorded.
1020340	1023100	We can invite people into the session, really simple.
1023100	1025180	So we're going to invite Carlos right now.
1025180	1026600	He's going to come in.
1026780	1027960	We're going to invite him.
1028100	1030620	You can see we have all the information available there.
1031040	1032420	That's running on the local host.
1033000	1034840	We have all the information on the slide there.
1034840	1037720	And then we can slide that away and then you can view the whole slide.
1038420	1044540	And so this is just a feed from a regular microscope, just from the eyepiece of the microscope.
1045160	1046900	So obviously, I mean, there's some imperfections.
1046900	1048300	You're seeing dirt and stuff like that.
1048300	1051620	But this is what you would be seeing if you were looking through that eyepiece anyway.
1051620	1056060	So it's a lot of, I mean, it's got clinical quality.
1056240	1058020	It's getting the job done.
1058020	1063940	And it really requires very little IT infrastructure to work in any way.
1064940	1067280	So, yeah, I mean, now you're viewing the slide.
1067280	1073000	And this is sort of an example of how we can look at different parts of the slide in order to determine a diagnosis.
1073380	1076860	And then if we see something we like, we can take an image of that.
1077060	1079100	We can add a comment to that image.
1079500	1082900	And then once we add the comment to the image, we'll save that as well.
1084040	1087240	And that just downloads to your computer, so you have a copy of it.
1087780	1093140	And the idea, basically, is once you close out of the session, what you'll be able to do is create a report.
1093420	1098580	And that report would just act as a single lab report, very similar to any sort of lab report you get from any hospital.
1098920	1103680	It would tell you who was in the session, who started the session, what hospital the session took place at.
1103940	1106780	And it would tell you if any sort of diagnosis was made.
1106780	1108900	And then it would give thumbnails of any images.
1109300	1111860	Okay, cool. So I've got about five minutes left.
1113460	1115840	So, let's finish with this guy.
1116940	1118960	And then we'll move on to the next slide.
1119200	1125360	So, that's sort of where we're at in terms of how we see connecting laboratories around the world together.
1125600	1130120	So let me talk to you a little bit about something that's called FlockMind, which I think is pretty interesting.
1130240	1131900	So this was a study that was done.
1131900	1137080	The doctor is like, some people don't like her, she's like sort of one of those figures.
1137200	1140000	But she did this study basically where she took pigeons.
1140140	1145360	And she trained the pigeons in order to make diagnosis on breast cancer using images from slides.
1145740	1151620	So the pigeons themselves had an 85% diagnostic accuracy when determining cancer.
1152420	1157360	But when they took the whole flock of pigeons, and then they actually took the result of the flock,
1157360	1159940	they found that it actually raised up to 98%.
1161120	1162880	So it was actually pretty good.
1162880	1167880	But the thing they weren't telling you when I read the article was that they only used a training set.
1167880	1171660	So they trained the pigeons on these images, and they just gave them the same training images again.
1171660	1174680	They never gave them a testing set, so they sort of tried to hide that data.
1174680	1175920	So I'm going to tell you that one.
1175920	1180280	But it's still really interesting that with a training set, they could have the pigeons make their diagnosis.
1181340	1185020	So that's essentially what we're trying to do right now with AI.
1185020	1192520	So we're using TensorFlow, and we're basically creating our own methods of detecting cancer.
1192520	1195280	So this is one in particular for cervical cancer.
1195280	1200800	And we're doing screening programs where we can determine what level of cervical cancer that people are facing.
1201180	1205100	And it's actually going really well. We're really happy.
1205100	1210100	And we're able to build a lot of data because we have our own slide scanners, which is a big thing that holds a lot of people in line.
1210860	1214280	And right now, we're working with Qualcomm, as I mentioned before.
1214280	1219760	And the reason why we're really excited about that is because we're trying to actually create a new device altogether.
1220040	1223500	This would be a standalone device that wouldn't need a computer or anything like that.
1223500	1227120	And we really see this as the future of microscopy in a lot of ways.
1227300	1230580	We can do everything that we were talking about before on this device.
1230580	1232980	We could control our motors. We could do everything we want there.
1233020	1235860	We could run our CV-based image stitching programs.
1236040	1237720	And we can use TensorFlow and AI.
1237720	1242180	And a lot of people are getting at us right now because they have been developing AI and they've been looking for platforms to run it on
1242180	1245220	that are so cheap that they can be deployed into the developing world.
1246040	1248200	And on the conclusion, that's what I'm going to say.
1248460	1251800	It's all going to be good. We're all going to be alright because we're working on solving this problem.
1252160	1253980	Thanks very much for your time today.