So our next speaker is from NYU Tandon School of Engineering, from the Digital Future Lab.
His focus for this talk will be using engineering and AI to build the next gen of smart scopes
capable of diagnostic support.
And fun little fact, on his spare time, he does 3D print graffiti artists in the style
of green army men.
So presenting their talk, microscopes are stupid, I agree, here is Lou Auguste.
Hello everybody, good morning, good morning, thanks for coming to see this speech on microscopes
are stupid.
My name is Lou Auguste, I work at NYU Tandon School of Engineering in their future lab,
so it's really, I'm an entrepreneur in residence there, they incubate my company, been running
it there for three years now, and we work on solving this problem.
So just a little bit before we get started, we've won some awards over the last couple
of years.
We won the ASME award for best hardware prototype in 2015.
Last year we were nominated for science startup of the year by this competition out in Germany
called Falling Walls.
We won the Indian government's endowment for science research last year, and this year
we're finalists for Qualcomm's prize for design in India.
I published my methods in the British Medical Journal in 2015 on how to use smartphones
to create whole slide images, and I did a pre-clinical validation study of that information
in 2016, which I presented at USCAP, which is the United States and Canada Anatomical
Pathologist Society.
And this year we received IRB approval to do our clinical validation study, which I'm
writing up right now.
So yes, microscopes are stupid.
I said it, and that's it.
If you look at this microscope here, it's such a dinosaur that it's not even connected
to the internet.
So basically the problem is microscopes are dependent on the operator, and who's operating
that microscope.
And the problem that we're facing is as we look out at more and more of the healthcare
landscape in the world, we find that there are things called healthcare deserts.
These are places in America, the developing world, where options for healthcare don't
exist.
And it's not that they don't have microscopes.
They have microscopes.
They don't have anyone that could help them do diagnosis.
So this is very true, as I said, in Kentucky, just as true as it is in Haiti, just as true
as it is in India.
So, before we go forward, I want to give a shout out to my boy, Anton van Leeuwenhoek.
He's like the G of microscopes.
So basically, he was born in the 1500s, and 40 years before he was born, they had created
the compound microscope.
Leeuwenhoek was a tradesman, and he was actually into glassworking.
He didn't have any advanced degrees.
He didn't have a PhD.
So he was eliminated from a lot of the science community.
But what he did was he created, with his knowledge of glassblowing or glassmaking, he created
this microscope here.
And this microscope had a magnification of over 200 times magnification.
Up to that point, they had already invented the compound microscope, which is very similar
to what we see today in any laboratory.
But that only had a magnification of 20 to 30 times magnification.
So van Leeuwenhoek was definitely one of the first disruptors.
And he's also considered the father of microbiology, because of what he was able to see through
his microscope that he created.
And in fact, it wasn't really until the 1800s that they made any advancements past what
he made with this microscope here.
So also, humans are stupid.
Because when we're given the task, humans get tired, and they miss things.
And diagnosis itself depends on memory.
So you probably can't see this right here, because the screen's a little bit contrasty.
But if you were looking for cancer on this thing, there's actually a monkey in the upper
right-hand corner of the lung that most people miss when they're looking for cancer.
So I'm not sure if you could see that here, but that's usually a good example of how people
miss things that are obvious when they're doing something that's repetitive.
So a little history on telepathology, which is sort of what we do.
So this is 1986.
This was the first demonstration of a robotic microscope.
And you can only imagine the satisfaction that this man felt clutching those keys and
hearing those sweet clicking sounds as he moved the microscope around.
And really, the history of telepathology is sort of a history of the internet in many
ways as well.
Because as things improved with the internet, we were able to do more and more in terms
of sharing slides, and then virtually moving on to creating virtual slides and sharing
those slides over the cloud.
And now with things like TensorFlow, we're actually getting to the point where we can
deploy AI locally on the device.
And that's a big step, I think, that's happening right now in the telepathology industry.
So, really, the work that I've done over the last four years has been focused on helping
health outcomes in the developing world and solving the problems that they're facing.
And so one of the biggest problems we look at is this really simple equation.
Lost time equals lost lives.
So there's a worldwide shortage of pathologists or doctors able to make diagnosis.
And because of that, people can't get diagnosed.
And so if people can't get diagnosed, they can't begin treatment.
And the later you wait to begin treatment, the worse your health outcome is going to be.
So a lot of the research we've done is creating just simple ratios when we look and see how
many specialists there are in a country versus how many people there are.
And so you can see that in the US, you've got a ratio of basically 1 to 17,000.
But then if you look at a country like China, you've got 1 to 58,000.
And then if you look at a country like Haiti, you've got 1 to 1,375,000.
So you can see that there's just not enough doctors to make diagnosis for all these people.
And there's also things that happen when you're in one of these countries that has a small
number of doctors.
You can actually fix your price as one of those doctors, and your price can be a lot
higher than it would be in another country.
And so not only are those people not able to get access, but they have to pay a lot
more money for access to diagnosis.
So the College of American Pathologists states that between 60% and 70% of all diseases need
a diagnosis using some sort of cellular biology.
And that includes cancer, things like malaria, a lot of diseases that we know today.
Anyway, so another thing we've studied is this travel burden.
And a lot of people have done studies here in America.
And so we have three studies here, one from Kentucky, one from Illinois, and one from
Georgia.
And basically all these studies stated that people that lived at a far distance away from
the primary diagnosis center, they would get staged for cancer much later.
And so that's a big thing that we're facing.
So like I said, if it takes you a long time to get a diagnosis, you're probably going
to get diagnosed much later when it's going to be much more difficult to cure.
So to give you an example of one of the patients that I saw when I was down in Haiti.
So we had this woman come about three years ago and present herself.
And she had a breast mask.
And so we had a surgeon at the hospital who was able to do a biopsy for her, but there
was no one available that could look at her specimen once we had it.
So we gave her her specimen in a jar, and we gave her instructions on how to get to
the nearest laboratory.
And so that laboratory is about six, seven hours away, and she never made that visit.
So a couple of years later, she came back in, and this time the mask had grown so large
that it actually had ulcerated out of her breast.
So now you see we definitely can say, okay, well, this without a doubt is going to be
breast cancer right here.
But when we first saw her, when she first presented herself, when she only had a small
mask, we could have used diagnostic testing to prove that it was breast cancer at that
point, and consequently we could have started treatment much earlier.
And that's sort of an example of what happens.
Also it's important to note that the cost of testing, diagnostic testing, in Haiti is
$50 per test.
So that's like a really important number to kind of remember.
So the previous solutions were like to travel to Port-au-Prince, find a lab, wait a few
months and hopefully get a diagnosis.
What we were doing before, a long time ago, was collecting all the samples, flying them
to America, getting a lab to process them in the United States, sending all the results
back and praying that we could find all the people that we had tested while we were down
there.
Another chance is like the doctor operates based on symptoms, and then really what mostly
ends up happening is people die of unknown diseases.
So that's pretty heavy, so I'm going to chill out for a second.
But we're like, we're working to solve this problem because we know we can.
And we know we can use technology to solve this problem.
And so what we've done is we've been focusing on building human-machine teams.
So the vision really is, and it's something that the WHO sort of shares with us as well,
is to create more healthcare workers.
Because the paradigm of having a hospital, having a doctor that we know and well love
here in America or in other developed nations, it's just not going to work in a lot of these
countries.
So we have to take people that are willing to go to school for maybe six months to a
year, train them at a skill, that could be a skill like becoming a lab tech, and then
give them the technology they need in order to share those slides so that they could make
more diagnosis.
And so that's something we've been working on.
This is the team that I go down to Haiti with from time to time to do that.
So another big thing we've been looking at is, okay, so once we can build the capacity
for doctors or for healthcare workers to prepare specimens to upload those specimens to the
internet, well, who are they going to share those specimens with?
So normally what we did in the past was we shared them with doctors here in America.
But doctors in America are really busy.
They don't have enough time to look at cases from another hospital, especially if it's
not helping them on the bottom line, it's not helping them make more money or do whatever.
So what we've actually done is we're working with this team of doctors here in Tanzania.
And so these are all doctors, these are all pathologists, the women that you're seeing
here, and that's our friend Mumbek, he's also a doctor in Tanzania.
And what we've been working is in this hospital, Mumbek in Dar es Salaam, we're setting them
up with the technology that they need.
So remember when I said it was important, remember that it costs $50 to get a slide
diagnosed in Haiti.
Well, in Tanzania, the cost of reading a slide is only $12.
So already we're saving money for the people that are in Haiti by giving them this option.
And the doctors in Tanzania are only making about $700 per month.
So they are actually interested in supplementing their income.
And they also have an unused capacity, because they're sitting in traffic, and most of them
have drivers.
So when they're sitting in traffic, there's time for them, if they're on their tablet
or smartphone device, to actually be reviewing cases.
So our solution is really to actually send our cases from Haiti to Tanzania to decrease
the cost of testing, so the people in Haiti can afford to pay for testing, and they can
get a diagnosis quickly.
And then hopefully what we'll do is export the lab training program that we're developing
in Haiti, export that over to Tanzania, and then create more lab techs, more health workers,
that can work with what we're building.
So that's sort of where we're at.
And how we've been accomplishing this right now is we've been developing a system we call
SmartScope.
So this system basically has two components.
It attaches to a regular microscope, and over the eyepiece, we have a camera that just fits
on the eyepiece.
And so that'll fit up to an outer diameter of 44 millimeters.
So that'll cover most eyepieces that are out there.
And then we built this robotic slide holder, and it just attaches with two screws to the
side of the stage of the microscope.
So basically with these two pretty simple pieces of technology we've developed over
the last three years, we can take a standard microscope, and then we can turn it into an
IoT device.
And so what this is is something that we can deploy at a cost of about $1,000 to $2,000.
And on the other side, the companies that were previously building whole slide imaging
machines and dynamic microscopes, they were normally charging between $3,000 and $100,000
for one of these standalone devices.
And if you've worked in the developing world, you'll know that any sort of standalone device
that they have, there's a good likelihood that it's going to be broken by the time you
get back there the next time you come down.
And there's no one that's going to service these devices.
So we've actually created this technology really mimicking what we saw in 3D printers,
and we've created technology that you can actually repair yourself.
So I think that's a big change right now.
And then you can share it on your... it runs on the cloud, so you can share it on your
desktop, you can share it on your tablet, or you can even look at it on your smartphone.
So how we built it up to this point was we used the Intel Edison.
We thought that was a really good solution for us to start with, because it gave us a
lot of versatility, we could run a whole bunch of programs on it.
But the one drawback of working with the Edison was that it required us to have a computer
to run everything with.
So we couldn't sort of do things, everything on the device.
And that was sort of where we're at, and we're actually deploying these units into the field,
having sort of more and more use cases for it.
So we have hospitals in Indiana that are using it for immunohistochemistry stains.
They're just making sure, they're just using it for Q&A, for that kind of stuff.
But it's been good.
And so now we're looking at the next... what's next for us?
What's next in the future?
And so, I mean, I think this quote from Freeman Dyson is really good.
New directions in science are launched by new tools more than they are by new concepts.
The effect of concept-driven revolution is to explain old things in new ways.
The effect of tool-driven revolution is to discover new things that have to be explained.
So I think in a lot of ways, when Leeuwenhoek went on and developed a microscope
that could see at 200 times magnification, then he could explain microbiology.
For us, what we see in the vision is not only are we sort of working with this concept of people
that are going to be providing healthcare workers, we're also giving them the tools they need.
And we think these tools are powerful enough and cheap enough
that they can be deployed in every single laboratory anywhere in the world.
And we can create a whole global network, something that we like to call Skype for Microscopes.
And this is how we're going to be able to track new diseases as new diseases emerge into the world.
We're going to need to have these capabilities.
We can't just send an expert there right away.
There's time that it takes to transfer someone there.
But having a sort of a network that's really light,
we can create people around the world that can share this information with a centralized location.
And so I think that's sort of where we're at.
So I'll give you a little bit of a demo of how everything works.
Let me see, just press the play button there.
So this is our new website. It's launching later this month.
Basically, once you sign in, you can access any devices that you have permissions to access.
And so the devices are there.
And then these are invitations, so you can receive invitations to join a session by someone else that has a device.
And then once we have everything sort of running, we have this interface here.
And you'll see the slide come up in a second.
And so what we're able to do is we're able to control the movement of the slide.
But we can also do things like add comments.
So right now we're adding a comment to this box here.
That could be sort of like anything you're seeing on the slide.
You can submit that comment and that's recorded.
We can invite people into the session, really simple.
So we're going to invite Carlos right now.
He's going to come in.
We're going to invite him.
You can see we have all the information available there.
That's running on the local host.
We have all the information on the slide there.
And then we can slide that away and then you can view the whole slide.
And so this is just a feed from a regular microscope, just from the eyepiece of the microscope.
So obviously, I mean, there's some imperfections.
You're seeing dirt and stuff like that.
But this is what you would be seeing if you were looking through that eyepiece anyway.
So it's a lot of, I mean, it's got clinical quality.
It's getting the job done.
And it really requires very little IT infrastructure to work in any way.
So, yeah, I mean, now you're viewing the slide.
And this is sort of an example of how we can look at different parts of the slide in order to determine a diagnosis.
And then if we see something we like, we can take an image of that.
We can add a comment to that image.
And then once we add the comment to the image, we'll save that as well.
And that just downloads to your computer, so you have a copy of it.
And the idea, basically, is once you close out of the session, what you'll be able to do is create a report.
And that report would just act as a single lab report, very similar to any sort of lab report you get from any hospital.
It would tell you who was in the session, who started the session, what hospital the session took place at.
And it would tell you if any sort of diagnosis was made.
And then it would give thumbnails of any images.
Okay, cool. So I've got about five minutes left.
So, let's finish with this guy.
And then we'll move on to the next slide.
So, that's sort of where we're at in terms of how we see connecting laboratories around the world together.
So let me talk to you a little bit about something that's called FlockMind, which I think is pretty interesting.
So this was a study that was done.
The doctor is like, some people don't like her, she's like sort of one of those figures.
But she did this study basically where she took pigeons.
And she trained the pigeons in order to make diagnosis on breast cancer using images from slides.
So the pigeons themselves had an 85% diagnostic accuracy when determining cancer.
But when they took the whole flock of pigeons, and then they actually took the result of the flock,
they found that it actually raised up to 98%.
So it was actually pretty good.
But the thing they weren't telling you when I read the article was that they only used a training set.
So they trained the pigeons on these images, and they just gave them the same training images again.
They never gave them a testing set, so they sort of tried to hide that data.
So I'm going to tell you that one.
But it's still really interesting that with a training set, they could have the pigeons make their diagnosis.
So that's essentially what we're trying to do right now with AI.
So we're using TensorFlow, and we're basically creating our own methods of detecting cancer.
So this is one in particular for cervical cancer.
And we're doing screening programs where we can determine what level of cervical cancer that people are facing.
And it's actually going really well. We're really happy.
And we're able to build a lot of data because we have our own slide scanners, which is a big thing that holds a lot of people in line.
And right now, we're working with Qualcomm, as I mentioned before.
And the reason why we're really excited about that is because we're trying to actually create a new device altogether.
This would be a standalone device that wouldn't need a computer or anything like that.
And we really see this as the future of microscopy in a lot of ways.
We can do everything that we were talking about before on this device.
We could control our motors. We could do everything we want there.
We could run our CV-based image stitching programs.
And we can use TensorFlow and AI.
And a lot of people are getting at us right now because they have been developing AI and they've been looking for platforms to run it on
that are so cheap that they can be deployed into the developing world.
And on the conclusion, that's what I'm going to say.
It's all going to be good. We're all going to be alright because we're working on solving this problem.
Thanks very much for your time today.