Welcome, everybody, to the Open Ventilator Remote Monitoring Project.
This was a project that we did. The open source community came together to create software to remotely monitor
do-it-yourself, rapidly manufactured ventilators during the COVID-19 pandemic.
So, by way of introduction, my name is Sam Cervantes.
I'm the Chief Technology Officer at MakerGear 3D printers based out of Cleveland, Ohio.
And also leading the project with me is Dave Guthrie.
He's the Medical Device Cybersecurity Program Manager at Mass General Brigham Healthcare.
So, we're going to give you a little description of our project, what we did, and how we went about it.
So, over to you, Dave.
Okay, so this has been the year of COVID.
We've all been impacted by this global pandemic.
And early on in this pandemic, there were fears of massive shortages of ventilators.
For example, Governor Cuomo of New York predicted there would be 26,000 people who could die from ventilator shortages.
In Italy, they stopped intubating patients over the age of 60.
And the New England Journal of Medicine predicted a need of anywhere from 160,000 to over a million.
So, there's a global pandemic.
Early on in this, Massachusetts has hit pretty hard by this as far as the United States is concerned.
Mass General Brigham is the largest healthcare provider in Massachusetts and has a significant need for ventilators.
So, Mass General Brigham reaches out to the tech community to develop new technologies, new processes, and has to be rapidly deployable in order to meet this predicted growth in rate of infections.
And you need to do this in a massively condensed timeline.
So, typical medical device development takes two to three years.
This needed to be done in two to three weeks. Crazy!
So, Mass General Brigham reaches out to the open-source community to develop rapid prototype ventilators.
Open-source community responds.
This is an example of just some of the resources that the open-source community has provided.
And Sam is going to go into this in more detail later in this presentation.
So, now you have the open-source community at the ready.
Working groups are starting to form to develop hardware systems, control software, and remote monitoring software.
All of these are being done in parallel under this condensed timeline.
So, what's the initial need and scope of this project?
So, you have this predicted ventilator shortage.
You have a need for these rapid prototype ventilators that the community is now building.
However, most of these ventilators lack built-in monitoring.
And clinical representatives need to have central monitoring and express this need to have central monitoring similar to central nursing station networks.
So, in a hospital, you typically would have a GE network or a Philips network or a Neon Codin network or some other monitoring network
where you have bedside physiological monitors going out.
And you have a network where you can go back to a central nursing station and oversee the data and the alarms of all of these patient rooms.
And you have ventilator systems that can then plug into these bedside monitors to jump onto this network.
And then you can monitor these ventilator systems as well.
Well, you can't do that with these rapid prototype ventilators.
So, you have these open source teams actively developing these ventilators.
And now you need to have remote monitoring and central monitoring of these systems.
That's where this team comes into play.
And it initiated in April 2nd of 2020.
So, what's the clinical request and the need for this remote monitoring software?
So, you have this rapid prototype ventilator groups.
They're not integrated into central monitoring traditionally with hospitals.
So, you need a method of monitoring of all your ventilators on a given floor or a given unit.
You have to have a central dashboard with visual learning that's easily interpretable.
You have to be able to ingest device data from all these ventilators.
So, not just your patient device data, but also device status data and alarm states and alarm data from ventilators if available.
And again, you have to do this in this wicked condensed timeline of two to three weeks.
And you shouldn't cause patient harm or interruption in patient care.
Because if you do that, you're defeating the purpose of helping clinical staff.
And instead, you're getting in the way and potentially causing patient harm.
So, this is where Sam's company, MakerGear, comes into play.
They have a 3D printing company.
And they have software that has the ability to integrate with physical hardware as well as providing a centralized dashboard.
So, this was the initial code base that was hacked and adapted by the remote monitoring team to develop this ventilator monitoring system.
So, now what happens is we're actively developing all of these rapid prototype ventilators.
And the remote monitoring team is working on integrating with these ventilator teams in order to build a central monitoring system.
However, simultaneously, hospitals are procuring ventilators left and right.
You're trying to build up your ventilator inventory from traditional commercial vendors.
And what happens is there was not a need to then deploy these rapid prototype ventilators.
Because you're sharing up your inventory of already off-the-shelf commercial ventilators.
And the problem, though, is a lot of these new ventilators may not have been the same model that you already built out integrations for.
And so, the amount of time to design these integrations was extremely costly, resource-intensive.
And you just can't do that in an emergency response situation.
So, you have new commercial ventilators that can't integrate into your central monitoring system.
On top of that, you have infection control restrictions because this is a global pandemic.
So, you have closed doors for infection control reasons.
You have ventilators inside of these rooms that can't be centrally monitored.
And now you have alarms going off with these ventilators.
Now clinical staff are having trouble hearing alarms through these closed doors.
So, this is where our team now pivots and we are requested to create a method for monitoring these commercial ventilators
in a central means that don't typically integrate with the other traditional central monitoring systems.
So, what's our plan of attack?
So, MGH Biomed, for example, researched off-the-shelf sound monitoring solutions.
They vectored in on one product, the SoundEar.
So, the SoundEar can deploy a one-to-one system per patient room per ventilator.
It's a self-contained consumer off-the-shelf system that can monitor sound pressure.
So, you can hook a SoundEar with a microphone up to the speaker on the ventilator that auditorily alerts the alarm.
And then when the sound pressure is exceeded, when that alarm goes off, the SoundEar can then trip.
So, we were then requested to integrate with this SoundEar and provide a central means for monitoring the SoundEar
and be able to create a central dashboard for when it trips.
And now, I'm going to pass it over to Sam to go into the architecture.
Thanks, Dave. So, I'm going to talk a little bit about the technical aspects of the project
and how we went about pulling this off.
And then also, later on, some of the future goals.
So, this right here, this is the user interface for the clinician.
This is the whole goal of the entire project right here.
So, this is what the nurse is going to see at the nursing station.
We've got one little box here for each ventilator.
So, we've got, in this case, 48 ventilators monitored from our single dashboard.
So, this is every ventilator on the entire floor of the hospital monitored from one central dashboard.
So, we've got three states of a ventilator right here that we can display.
The normal state, the happy state, is green.
That means we're able to ping the ventilator.
We're getting a good status report back and there's no alarm. All is good.
The second state is this gray strikethrough box right here.
That means we're not able to even connect with the ventilator.
So, a technician needs to go check on the ventilator.
So, there's a fault in the system somewhere.
The third state is this red box here.
And that will actually flash, you'll see in the live demo.
And that means an alarm is currently being triggered on the ventilator.
So, the nurse needs to go check on it.
So, at a glance, the nurse can see right here, okay, I need to go check on two different ventilators right here, room seven and room four.
I need to go check on those.
Okay, so how do we go about doing this?
Typical deployment process for software like this is two to three years.
We had two to three weeks.
COVID-19 emergency.
So, how do we go about pulling this off?
Basically, we had to pull all the rapid prototyping software development strategies.
We had to move fast and break things.
We rely heavily on the open source community and the maker communities.
Distributed development, we had software developers all over the country pushing to our GitHub repo.
We had to hack and fork existing products to get something out in just a few weeks.
We used things like 3D printing and quick turn PCBs off the shelf hardware to get something that we could very quickly snap together, bolt together, and ship a solution to our customers in just a very short amount of time.
We had to rely on remote lab testing.
And as much as we could, you know, source locally.
Component sourcing was a key thing.
We had to design our system around the components that we could actually source that were in stock and locally available.
We might have wanted to use a PLC or a custom circuit board, but we didn't have time.
So, we had to use things like off-the-shelf Raspberry Pis and Arduinos that we could get in quantities of thousands that were actually in stock.
So, basically, we had to move heaven and earth to ship something in a very short amount of time.
Like Dave said, the initial code base was hacked from MakerGear software.
So, MakerGear manufactures 3D printers. Been doing so for about 10 years out of Cleveland, Ohio.
And it just so happened that MakerGear had some software for remotely monitoring 3D printers.
And as it turns out, a 3D printer is an awful lot like a do-it-yourself ventilator.
A 3D printer has an Arduino and a Raspberry Pi, and a ventilator has an Arduino and a Raspberry Pi.
So, we were able to adapt and import the software to quickly meet the needs of the pandemic.
So, this is the overall architecture here. So, you'll see the dashboard is what we saw before.
The dashboard is the kind of central brains, if you will. All information flows towards the dashboard.
So, this is the clinician interface here. So, the dashboard is going to get its status data in real time from the ventilators.
You'll see three ventilators displayed down here. Each ventilator is in a different configuration.
And you'll see the cloud up here. So, the cloud is there only to serve up the application to the browser.
And the cloud doesn't store patient data. The cloud simply stores configuration data for the app.
A very limited amount of data. We don't send patient data to the cloud.
So, how does the dashboard get its information from each of the ventilators?
It was kind of a tricky problem for us because each ventilator... there are a number of different types of these DIY ventilators out there.
And we wanted to make a universal system to pull data from every ventilator.
Some ventilators are controlled by an Arduino. One of the MIT designs was controlled by an Arduino.
Some ventilators were controlled by a Raspberry Pi. I think the JamBest design used that.
And then some ventilators are a little different. So, we wanted to have a sound ear, a separate appliance to listen to the sound the ventilator is making.
And then trigger it a lot based on that.
In order to interface with all these different types of ventilators, we decided to put a Raspberry Pi in between each ventilator and the dashboard.
So, the Raspberry Pi could serve as a kind of universal network interface.
And on the Raspberry Pi, we have a library.
And so, we can write modules for this library to basically ingest data from any system.
So, the system we set up, the dashboard we set up, can basically take data from pretty much any device with any output capability.
If it's a serial out, if it's a SPI or a UART out, or whatever, analog out, digital, we can take that data from the ventilator and relay it to our dashboard here.
So, this is the overall architecture.
And the specific instance that we tested in the lab was this particular architecture here.
So, the ventilator is obviously going to be in the patient's room.
And then also in the patient's room is the SoundEar device, which listens for an audio alarm from the ventilator.
So, the ventilator is going to emit a beep, and you'll hear this in the demo.
It's going to emit a beep.
The SoundEar is going to detect that audio beep, that sound pressure.
If the sound pressure is above a certain level, it's going to send a signal to the Raspberry Pi, the network adapter.
And then the Raspberry Pi is going to then send the alarm to the dashboard that we saw earlier.
We should note here that we used the zero-tier software-defined network, which provides a very secure network in the hospital,
and even outside of the hospital if we need to, to provide connectivity between the Raspberry Pi and the interface here.
And of course, you've got the cloud here. And again, we're not storing patient data on the cloud.
The cloud is just there to serve up the static assets, the application, if you will, to the dashboard, and also very limited configuration information.
And again, this is what the nurse is going to see at his or her station right here at the dashboard.
Software stack, we're getting a little bit into the weeds here.
We tried to use as much open-source software as we could here. Ruby on Rails, reliable, secure, open-source, very quick to deploy.
Raspberry Pis and Arduinos, again, open-source, off-the-shelf, quick to deploy.
And then everything we did is open-source under a GPLv3 license, so you can pretty much use it for whatever purpose you'd like.
GPLv3 is a very permissive license. You can use it for commercial and or non-commercial.
We wanted hospitals to be able to use this very quickly, and you can see our GitHub link here below. Everything we have is up on GitHub.
We did a lot of testing. At-home testing, workbench testing, we did lab testing. We did not get to the point where we actually tested it in a hospital, but we did a lot of lab testing in a clinical lab.
And it actually worked pretty well. We got a very solid proof-of-concept alpha. We didn't quite make it through the beta phase, but we got a pretty good alpha prototype.
One of the lessons we learned was that the validation and the regulatory process was actually very involved, as you might imagine, from medical product.
A lot of us come from a typical software development background, but a medical device has a much more involved regulatory and approval documentation process.
We just wanted to give you a flavor of what that is right here. The main three documents are the requirements, which have to be formatted in a certain way, the architecture, which again needs to be formatted in a certain way, and then the hazard analysis, which is pretty involved.
Basically, for the hazard analysis, we had to brainstorm every possible way that this product might fail. It would either fail to communicate an alarm to the central station or fail in some other way.
Every method we predicted that it might fail, we had to put a contingency plan in place for. Very detailed, again. I just wanted to give you a brief overview of what this... for those of you who haven't done a medical product before, just a brief overview of what that involves.
It's a very rigorous process to prepare for FDA approval and hospital approval. Just a little taste of that right here.
Again, this is the detailed software architecture. If you want to take a look at that. All these documents are up on our GitHub wiki, so you can check those out there.
Another important thing here to do is to do a secure code review to just kind of get a good confidence level, since this is potentially a critical application that we don't want to fail.
The staff at MITRE did a very nice job and lent their support, so we're very grateful to them. They used an automated code service called VeriCode and also a manual code review. Still ongoing, but they were a big help with that.
Just to give you a taste of the automated code review.
This was a code review on, I think, a Ruby on Rails portion of the application. Basically, you see here, we don't have any severe alerts, but we had a couple of medium alerts.
We had some automated testing code that spit out automated test data, so we need to remove that before deployment. One little minor thing here.
A few things to fix. Manual code review. Again, just a few things. Nothing critical. A few things to fix. We just want to give a little insight into how we went about testing and validating our code.
We learned a lot in this project. We did have a few medical experts on our staff, both from the software development side and a lot of medical experts from the regulatory and healthcare field.
A lot of us were software developers, so these are some of the lessons we learned.
The most inspiring thing, I think, was we had a lot of world-class experts to come together very quickly to donate their time, their nights and weekends, to really come together and work for this good cause.
So that was really inspiring. The biggest hurdles we found were not necessarily the technology. As technologists, I don't want to say it was easy, but the technology was the easiest part in a lot of ways. The validation and approval were much more rigorous and time-involved than we anticipated.
All that being said, FDA approval is daunting, but it's not impossible. And there are experts out there that are willing to help, and that's what we found. A lot of really good people with a lot of deep FDA experience out there that are willing to help. So it's not undoable.
We found it's very important to be aware of HIPAA and personal identifiable information. So that's one reason we chose not to send patient data to the cloud. It's very important to be cognizant of HIPAA.
Another important lesson we learned was that hospitals are reluctant to use open source software without the strong backing of an organization to own the technical support and the liability aspect of it.
So in the future, we're looking for potentially a sponsor for the project to own that deployment and technical support aspect. And ultimately, we learned we didn't have a ventilator shortage in the U.S., which is great news. So ultimately, our software was not needed.
We're very happy that we could be there and rise to the nation's call as an insurance policy, but ultimately, there were plenty of ventilators, from what the experts tell us, in the U.S. to go around for people who needed them.
So FDA approval. We just wanted to give you a little flavor of kind of the difficulties we found in the FDA approval process. Big disclaimer, I'm not a lawyer or a medical expert, so please don't take us as medical advice or legal advice. Seek expert advice for your own project.
But it's just a high-level overview of some of the things I learned. In a normal timeframe, non-COVID, non-emergency timeframe, say last year, a typical approval process for a system like ours would be about two to three years. It's a lengthy submission process, and once you get that approval process, you can use your product for a long period of time.
During the emergency, for example, the COVID emergency, the FDA allows a couple types of new approvals. One is what's called EAU, Emergency Use Authorization. Typically, it can be fast-tracked in two to three months. It's much quicker, but EAU is only valid for the duration of the emergency.
So if we were to get EAU for our software system, we would not be able to use it after the COVID emergency. And then Compassionate Use is potentially even quicker. And that's a... I'm not an expert on that, but it's a slightly quicker process, even more limited in scope, even more limited in approval, but potentially quicker.
Again, not a legal expert, not a medical expert, but these are just some of the things that we learned in the process. There was even a little bit of discussion as to how to classify our software system. Is this a medical device? Is it a modification to an existing medical device? Or is it not a medical device? Is it a baby monitor or a pager or like a cell phone type device?
So we did have some ongoing debate about that.
So current status of the project. We've drafted up our validation documents for approval. We have a working demo proof of concept. I called it Alpha. It works. We tested it in the lab. We had a code review. We do not currently have a customer for the project. Our project is currently parked.
So if you have a potential use for the software, do let us know. We would love to see our software be used for a good purpose.
Future plans. One thing we discussed is additional audio alerting at the central nursing station that may require, in addition to the visual alerting that we already have, that may require a separate hardware appliance.
And potential AI machine learning to detect different types of audio alarms from different types of medical devices.
We're going to search continually for new customers.
Potential customers may be hospitals with limited resources.
Smaller hospitals in remote areas or developing countries.
And then potentially healthcare organizations with needs to monitor other devices, like pulse oximeters or things like that.
And we're continuing our search for a corporate sponsor, an entity to own the maintenance, installation, support, and liability. So we're just a group of distributed open source software developers.
We don't have an organization behind us.
And then potentially a future FDA application, again, if we find a customer for this.
So Dave's going to talk a little bit about the power and the possibilities.
Yeah, so when you think about this system, the real power is in the possibilities of all the integrations that this system can provide.
You have rapid prototype ventilator monitoring systems that we've discussed, such as bringing in device and patient data.
You have commercial ventilators that are not traditionally integrated with central monitoring systems that you can monitor through sound.
So you can get ventilator alarms from commercial ventilators.
And now with the sound monitoring, you can monitor any medical device alarm that is based in audio.
So you think about all the infusion pumps or other devices that you have all throughout your hospital or remote care sites that potentially do not have integrations into your central monitoring system.
This could be huge.
So this project has great potential to have a massive impact in the healthcare ecosystem.
Now for the fun part, we've got a live hardware demo.
So we're going to show you the system in operation.
So Dave in Massachusetts has what would be in the patient room.
On this picture is what's on the bench here in Massachusetts.
So the way you're going to see in the video is going to be from that camera that you see in the forefront of the picture.
So you have the speaker in the background next to the cyber lizard who has a microphone next to the speaker.
That microphone is plugged into the sound ear.
The sound ear is then going to calculate the sound pressure.
And it has what's called a sound buster port.
That sound buster port changes voltage if the audio alarm pressure is tripped.
So then the sound buster port comes out of the sound ear into this Raspberry Pi.
And then the Raspberry Pi itself will then go out through the network cable over to a switch that's on my bench here.
Out over the open internet through the encrypted zero tier tunnel over to Sam where he's running the dashboard.
And what I'm going to be doing also is running a ventilator recording of the audio alarm that we used to train this system.
So Dave, you've got the ventilator system. You've got the patient side system in Massachusetts.
I'm in Ohio. I've got the clinician dashboard.
On my clinician dashboard here, I can see a green box, which means I'm able to get a good signal from Dave's device in Massachusetts.
Again, all this is communicating over the zero tier secured network.
We're across the country. We might as well be across the floor in the same hospital.
But I'm getting a good signal back here from Dave's ventilator.
So Dave, if you want to go ahead and trigger your alarm there, we should see an alarm indicated here.
There we go.
The flashing red means the nurse at their dashboard needs to go in and check the patient room.
And as soon as we shut it off, the alarm should go off as well.
Okay, so that's our demo.
So like you see, it's a basic working system. It's fairly simple at this point, but we do have the capability to monitor up to 40 ventilators from one dashboard, and it's extensible.
So we have libraries that can easily incorporate any number of devices.
Pretty much any device that's got some kind of hardware, digital or analog output, we can tie this into our system.
Very, very adaptable system.
So just some acknowledgments for all of our developers. I was really honored to be a part of a team of so many people much smarter than myself.
Really fun and really great to work with this team.
David and I led the project, but we have a bunch of other open source contributors here. Everybody volunteered their time and effort, sacrificed their nights and weekends to really just, you know, contribute to the COVID cause.
Working to express my appreciation for everyone's time and effort here. Really inspired to see so many people come together on such a short time frame.
Thanks to everybody. Organizations here. I just want to thank all the organizations, GHS and FDA for their infrastructure and their funding.
Maker Gear for contributing development time from a number of engineers and also open sourcing the code base and contributing hardware resources.
Massachusetts General Hospital and their affiliates for everything they've done and expertise they've provided.
MITRE for its code review and consulting and just expertise and late night phone calls. Protocol Labs for special thanks for contributing some funding.
Sound Ear. So we contacted Sound Ear, one of the manufacturers of our sound interface device, and we asked them for help on a very short time frame and they were incredibly supportive.
Not only did they give us technical support, engineering consultation, they open sourced a portion of their code for us.
You'll see that on GitHub. So I cannot have... I cannot imagine a better response from a manufacturer in a time of crisis.
They were incredibly forthcoming and I cannot thank them enough for helping us to integrate their device into our system.
And last but not least, VeriCode for their automated security review.
So if you would like to become a contributor or customer or sponsor, you know, please reach out to us. You can find me or Dave on LinkedIn.
And I just want to give an extra shout out to everybody who worked on this project and an extra thanks to everyone. It was really inspiring to see so many people involved in this and in such a selfless way.
It really gives me hope for the future that we can really nip this thing in the bud. So thanks to everyone.