WEBVTT 00:00.000 --> 00:10.680 First of all, thank you so much to the organizers for having me here today live from home to talk to you today about some of my work with the All of Us Research Program. 00:10.960 --> 00:12.260 My name is Michelle Holko. 00:12.260 --> 00:19.600 I'm currently a Presidential Innovation Fellow, detailed to the NIH, to work with the program, specifically with digital health technologies. 00:19.600 --> 00:25.240 I'm going to talk to you about one of the projects I've been working on there since about December last year. 00:25.800 --> 00:29.120 So I'm going to give you first an introduction to the All of Us Research Program. 00:29.120 --> 00:36.480 I'm going to run pretty fast through those slides so that we can get to the digital health technologies portfolio, but more broadly in the program. 00:36.580 --> 00:45.520 And then I'm going to spend the most time talking about the Fitbit, bring your own device data analysis and participant demographics, which is the area that I'm going to be talking about most today. 00:45.860 --> 00:52.360 And then we'll close with some opportunity and considerations for digital health technologies in the All of Us Research Program. 00:52.540 --> 01:11.500 So just again, to give you a kind of an introduction to the program, in case you're not familiar with it, the All of Us Research Program grew out of the Precision Medicine Initiative, which was founded several years ago in the previous administration and has $1.5 billion of congressional dollars to get the program started and to continue running. 01:11.640 --> 01:21.320 And the mission really for the program is to accelerate health research and medical breakthroughs to promote individualized prevention, treatment, and care for all of us. 01:21.320 --> 01:26.060 There are three core values to the program that are really important to us. 01:26.060 --> 01:32.220 The first is to nurture relationships with one million or more participant partners from all walks of life for decades. 01:32.220 --> 01:36.340 And again, diversity is really critical to the program in that. 01:36.340 --> 01:47.060 To deliver the largest, richest biomedical data set ever, as well as making it easy to use to a wide range of researchers and to catalyze a robust ecosystem of researchers. 01:48.700 --> 01:55.060 So the All of Us Research Program truly is an innovative and ambitious research effort. 01:55.480 --> 02:04.640 And here are some reasons why really the diversity at the scale of a million people or more and focusing on, you know, capturing the diversity that makes America, America. 02:04.860 --> 02:06.880 Focusing on participants as partners. 02:06.880 --> 02:14.860 So not just, you know, taking data from them, but also giving back to them and continuing to engage with them throughout the length of the program. 02:14.920 --> 02:22.400 The longitudinal design of the program means that we have the ability to recontact participants, which we've already done in some cases around COVID. 02:22.920 --> 02:25.280 And multiple data types. 02:25.280 --> 02:39.600 So diversity in participants, but also diversity in data, including electronic health records data, survey data, baseline physical measurements, biospecimens, genomics, and digital health technologies, which is what I'm going to be talking about today. 02:40.160 --> 02:46.960 Creating a national open resource for all, which is broadly accessible to all researchers with open source software and tools. 02:46.960 --> 02:50.960 And building in security and privacy safeguards for all participants. 02:51.120 --> 02:52.720 So some current progress. 02:52.720 --> 02:56.000 We opened the doors nationally on May 6, 2018. 02:56.080 --> 03:07.220 So far, we've enrolled over 271,000 participants that have not only completed the first initial enrollment step, but have also completed what we call the full protocol. 03:07.280 --> 03:10.720 And this represents individuals from all 50 states. 03:10.860 --> 03:25.820 Furthermore, greater than 80% of these participants represent one or more of the underrepresented in biomedical research categories, which includes racial and ethnic minorities, you know, living in a rural region, and there are many more. 03:25.820 --> 03:28.640 I'm going to go through those in more detail a little bit later. 03:28.800 --> 03:32.800 And greater than 50% of the participants are racial and ethnic minorities. 03:33.000 --> 03:47.200 We built a significant infrastructure to support the program with over 100 plus academic groups working with us, the VA, the FQHCs, which are the federally qualifying healthcare centers, technology and community partners. 03:47.200 --> 03:52.340 We have over 320 clinics enrolling participants, and that number is still expanding. 03:52.420 --> 03:54.300 We also have a bilingual website. 03:54.300 --> 03:59.740 So the website is available in English and in Spanish, as well as the participant portal app and call center. 03:59.760 --> 04:07.900 The biobank at the Mayo Clinic supports 24-hour shipping and processing with capacity for 335 million plus files. 04:07.900 --> 04:11.000 And we also have an interactive mobile exhibit that travels the country. 04:11.000 --> 04:13.060 The picture is on the bottom right of the screen. 04:13.060 --> 04:23.860 Certainly not traveling during right now, but usually it is, and it's actually wonderful because it can reach, you know, more rural areas where we don't have partners. 04:24.460 --> 04:29.140 So here is just a snapshot of enrollment to date. 04:29.240 --> 04:34.560 So you can see enrollment over time in the graph on the right, and then the bubbles kind of tell you where we are right now. 04:34.560 --> 04:48.440 Like I said, we have 271,000 that have completed the initial steps of the program, which includes consenting, agreeing to share your EHR data, completing the first three surveys, providing physical measurements, and donating at least one bio assessment. 04:48.680 --> 04:56.640 We have many more than that, 353,000 who have completed some part of the program, but they haven't completed all of those kind of initial steps. 04:57.520 --> 04:59.220 This is what the current protocol looks like. 04:59.220 --> 05:04.400 The first step is to enroll, to provide your informed consent, and to authorize your EHR. 05:04.400 --> 05:09.660 It's really important that we, you know, ask for the authorization of EHR up front at this initial step. 05:09.800 --> 05:14.200 Right now we're only recruiting adults who are 18 years and older. 05:14.200 --> 05:19.040 We do have plans to expand to children, but right now we're only recruiting 18 years and older. 05:19.960 --> 05:26.560 It's also important to note that at this initial enrollment, we have in-person enrollment and we have digital enrollment capabilities. 05:26.560 --> 05:29.740 Of course, right now the in-person enrollments have all been paused. 05:30.200 --> 05:37.900 So, in that interim, we've still been able to accept participants joining the program, but only through the digital method. 05:38.180 --> 05:39.940 Then you can answer surveys. 05:39.940 --> 05:49.360 There are many different surveys that are already available, including the basics, overall health, lifestyle, health care access and utilization, family medical history, personal health history. 05:49.440 --> 06:03.380 And we also released a survey around the coronavirus pandemic called the COPE survey, which focuses not only on signs and symptoms and exposure to coronavirus, but also, you know, the social and economic aspects of that. 06:04.080 --> 06:08.040 The next step, the next two steps actually kind of usually go hand in hand. 06:08.040 --> 06:15.040 And so usually if you're coming to us through one of our health care provider organizations, you'll provide physical measurements, which are all listed here. 06:15.040 --> 06:18.780 Blood pressure, heart rate, height, weight, BMI, hip circumference, and waist circumference. 06:18.780 --> 06:21.220 At the same time, you'll provide a bio sample. 06:21.640 --> 06:24.800 Usually blood, that's the bio sample of choice. 06:24.800 --> 06:34.180 However, if the blood draw is unsuccessful, or if you're coming to us as a direct volunteer digitally, we can send you a kit to give a saliva sample. 06:34.780 --> 06:37.140 Very few participants give your assessments. 06:37.140 --> 06:38.780 We are collecting some of those. 06:38.920 --> 06:42.760 And then finally, digital health technologies, which is what I'm going to be talking about today. 06:42.760 --> 06:49.400 We have a bring your own device program already set up with Fitbit and Apple HealthKit connections. 06:49.780 --> 06:57.140 We're also developing integrated apps to track mood and cardiorespiratory fitness, among other things to come. 06:57.140 --> 07:05.640 So now shifting from the participant to the researcher, and how are we going to make these data available widely to the research community in a usable way? 07:05.640 --> 07:07.900 That sounds very easy, but it's not. 07:08.200 --> 07:16.800 So on the left is kind of the traditional model where a researcher would come to the program, apply for the data, and then they would be basically shipped out. 07:16.800 --> 07:19.200 It would be basically shipped out to them. 07:19.220 --> 07:23.880 But with this model, it certainly discourages sharing of research. 07:23.940 --> 07:27.340 It also is a very weak link in terms of security. 07:27.340 --> 07:31.240 You have no control over those data once they leave your facility. 07:31.380 --> 07:37.740 You also need a high infrastructure to be able to support that to pay for multiple copies and all of that. 07:37.740 --> 07:43.640 So what we've decided to do is to create a cloud eccentric approach where we bring researchers to the data. 07:43.640 --> 07:50.960 All of the data are hosted in the cloud, the tools are being built in the cloud, and then researchers are able to get access to it. 07:51.080 --> 07:53.080 And in this way, it facilitates collaboration. 07:53.080 --> 07:55.320 There are centralized security controls. 07:55.320 --> 08:04.900 We can make it accessible to all researchers, and we can even think of ways of making different types of data accessible to different types of researchers depending on the need. 08:05.040 --> 08:10.620 So with this cloud-centric model, you can also think of providing the data in tiers. 08:10.880 --> 08:18.920 We certainly have a public level of data sharing that's happening, publicly accessible, no login is needed, shared on the website. 08:18.920 --> 08:21.560 I showed you the blurbs of the enrollment over time. 08:21.560 --> 08:23.340 That's one of the snapshots. 08:23.440 --> 08:33.960 We've also created a registered tier, which requires identity verification, signing a data use agreement, and completing research ethics training. 08:34.100 --> 08:43.040 There's also the idea of potentially developing a controlled tier of data, which would give people even more access to the data. 08:43.040 --> 08:55.000 So at each step, where the security goes higher, and we are instituting more and more controls, because you're getting more and more access to data. 08:55.000 --> 09:04.300 So with the registered tier, there are some specific privacy controls that are in place there that would not be in place in the controlled tier. 09:04.300 --> 09:08.480 So we would need an additional trust with the users of that data. 09:09.380 --> 09:11.380 So here's the Researcher Hub website. 09:11.380 --> 09:13.140 You can go to researchallofus.org. 09:13.140 --> 09:14.720 I encourage everybody to do that. 09:14.720 --> 09:19.980 From there, you can see the data snapshots, the data browser, and the survey explorer. 09:20.140 --> 09:25.820 The data browser that's available publicly does not let you drill down to individual level data. 09:25.820 --> 09:30.960 To do that, you have to move into what's on the right here, the Researcher Workbench. 09:30.960 --> 09:35.080 So this is just how the Researcher Hub components fit together. 09:35.120 --> 09:38.640 On the left, you have the data browser, data snapshot, survey explorer. 09:38.820 --> 09:44.740 And then in the middle is this kind of data passport model, which is how we are making the data available to researchers. 09:44.740 --> 09:49.400 We don't require you to ask or tell us what specific project you're going to be working on. 09:49.600 --> 10:00.340 We just verify your identity and, you know, kind of build a level of trust with you as a researcher, and then give you access to the data. 10:00.400 --> 10:07.160 In the Researcher Workbench, it includes the data dictionary, concept sets, cohort builder, notebooks, and help desk. 10:07.340 --> 10:13.640 And the one exciting piece of news to share is that the All of Us Research Program Researcher Workbench is open for beta testing. 10:13.640 --> 10:15.680 So it's a very important moment for us. 10:15.680 --> 10:26.920 We want to build this platform together with the research community, but we absolutely need the input of researchers to make it more robust to learn how well the data tools and policies are working. 10:27.200 --> 10:33.560 So the current components in terms of data include physical measurements, survey data, some EHR data. 10:33.560 --> 10:38.000 Much more is to come here, including genomics and digital health technologies data. 10:38.400 --> 10:53.060 The tools that are included include a dataset builder, a cohort builder, Jupyter notebooks with R and Python supported, and the help resources are wide and varied, including FAQs, sample notebooks and workspaces, including code snippets, documentation, 10:53.060 --> 10:54.660 and help desk. 10:54.660 --> 10:57.440 So access and security, just a moment here. 10:57.560 --> 11:08.120 Again, All of Us employs a data passport model for access to this researcher tier to grant researchers broad permission to explore the data for a wide range of studies. 11:08.320 --> 11:12.020 Data testing is available to researchers who have an eRA Commons account. 11:12.020 --> 11:14.440 This is what we're using for identity verification. 11:14.440 --> 11:16.380 So you have to have an eRA Commons account. 11:16.380 --> 11:18.000 Most researchers do. 11:18.180 --> 11:20.900 And if not, you can look into the process for creating that. 11:20.900 --> 11:25.460 And then you also have to have a signed data use agreement with your institution. 11:25.460 --> 11:36.060 So the institution that houses the data resource center finds an institution with your institution, and then we verify your identity through the eRA Commons account, and then that's how you're able to get access. 11:36.960 --> 11:42.040 So again, to facilitate collaboration, the researcher workbench is hosted in a cloud-based system. 11:42.340 --> 11:45.920 Data downloads onto your local machine are not at all allowed. 11:46.000 --> 11:52.880 And during the beta testing phase, another really nice perk is that researchers will receive free credits to cover computing storage costs. 11:53.800 --> 11:59.060 So here's just a peak look at what the workspace looks like when you come to the researcher workbench. 11:59.060 --> 12:03.700 First, you create a workspace, and this is where everything kind of will be all collected. 12:03.920 --> 12:10.440 And then you can create cohorts, concept sets, cohort reviews, and notebooks to run your analyses. 12:10.700 --> 12:15.940 This is just a look at what the tools look like in terms of the cohorts, data sets, and concept sets. 12:15.940 --> 12:29.940 And then just one more summary about, you And I have to say, the support features are really extensive, so anyone who is not a coder coming to this should be able to even get up and running. 12:29.940 --> 12:40.760 So there's the user support hub, researcher workbench, featured workspaces, reusable code snippets, demonstration projects, and then a help desk form that can give you kind of one-on-one help. 12:41.380 --> 12:58.080 So just to summarize this section, we are providing a robust data ecosystem with survey data, physical measurements, structured EHR data, with plans to add additional surveys, digital health technologies data, unstructured EHR data, linkages to other data sets, 12:58.080 --> 13:01.820 additional assays and biospecimens, and genomics. 13:02.020 --> 13:07.680 So moving on to the topic of the day, digital health technologies in the All of Us Research Program. 13:07.960 --> 13:10.840 So what is the vision really for digital health technologies? 13:10.840 --> 13:28.220 Well, digital health technologies data from things like fitness trackers, smartwatches, apps, other types of devices, can capture health-relevant data from individuals outside the hospital or clinic, which can really supplement current medical information. 13:28.520 --> 13:45.280 Another feature, especially for fitness trackers and smartwatches, is that the data are longitudinal in nature, and it can really give researchers an unprecedented ability to better understand the impact of lifestyle, environment, and other factors on health outcomes, 13:45.280 --> 13:51.500 and eventually and ultimately develop strategies for keeping people healthy in a very precise and individualized way. 13:51.620 --> 14:01.380 The program is currently collecting data from participants in a bring-your-own-device model, so we are not distributing devices to participants, although that is something that we are planning for. 14:01.380 --> 14:04.420 And we currently have connections to Fitbit and Apple HealthKit. 14:04.420 --> 14:07.220 So again, just bring your own device. 14:07.220 --> 14:12.480 We are also considering developing pilots of smartphone-based apps in the future. 14:13.080 --> 14:20.280 So what is the vision or impact specifically for the Fitbit bring-your-own-device data in the Researcher Workbench? 14:20.280 --> 14:28.400 So I believe that all of us really truly have a unique opportunity to release one of the largest and most diverse digital health technology data sets. 14:28.400 --> 14:33.920 We currently have data from over 9,000 participants, some beginning in 2009. 14:33.920 --> 14:41.220 In addition to current survey, EHR, and physical measurements, future data releases will also include genomics and other data sets. 14:41.220 --> 14:46.640 So when you think about the power of combining all of these different data types together, it's truly amazing. 14:46.800 --> 14:57.000 And then this can promote research to develop new and improved disease prevention capabilities, remote patient monitoring for connected care, and other opportunities for high-cost diseases. 14:57.680 --> 15:07.160 So the approach that we are using for the Fitbit data in the All of Us Research Program is this Minimum Viable Product, or MVP, approach. 15:07.160 --> 15:19.400 So first step of this is to obtain the data, and we have all of the data currently through May 2020 that's going to all be curated and released in the next data release. 15:19.400 --> 15:28.700 We've also developed scientific research use cases around the Fitbit data just to make sure that the way that we're curating the data is going to make it useful to the research community. 15:28.820 --> 15:35.960 We are now assessing the data set for the longitudinal nature, missingness, and trends, or basically doing exploratory data analysis. 15:36.240 --> 15:45.980 And then based on two and three, based on scientific research use cases and the EDA of the data, defining the priority data elements for inclusion in the Researcher Workbench. 15:45.980 --> 15:56.780 And again, the timeline for that is to release it in the next data release, which is slated for late fall 2020, currently anyway. 15:56.900 --> 16:03.920 And then to develop and implement a Fitbit curation schema coordinated with the security privacy team and other approvals as needed. 16:03.920 --> 16:07.860 So this is what the currently what the Fitbit MVP looks like. 16:07.960 --> 16:11.180 You can see the data elements table over on the right. 16:11.460 --> 16:24.200 The data elements that we are currently pulling from Fitbit include heart rate, minute level heart rate, heart rate summary data, intraday steps, which is also the minute level, daily activity summary, daily weight, daily fat, food, and water. 16:24.200 --> 16:30.620 And it's important to note that daily weight, daily fat, food, and water are all user supplied elements. 16:30.700 --> 16:35.300 So just by nature of that, there are, there is much less of those data. 16:35.300 --> 16:39.640 The others are derived from parameters that are derived from the device. 16:39.740 --> 16:55.780 So what we've decided to do is to provide a series of data tables using BigQuery of parsed data tables, including heart rate, summary by zone, heart rate, minute level, activity, daily summary, and activity intraday steps in that level. 16:56.440 --> 17:07.160 So to dig into the data a little bit more, as well as some of the participant demographics, which are important for a number of reasons I'll go into. 17:07.500 --> 17:11.840 So these are characteristics of the current Fitbit data set in terms of the data elements. 17:11.840 --> 17:17.280 So we have 9,255 Fitbit bring your own device participants. 17:17.280 --> 17:21.240 And again, this is until March, March, May 2020 timeframe. 17:21.700 --> 17:34.660 8,535 of these have primary and electronic health record consent with activity measures as early as 2012 and heart rate as early as 2015. 17:34.720 --> 17:37.660 So that is quite a lot of data, if you think about it. 17:37.660 --> 17:48.580 The data elements that we are pulling from Fitbit API are shown on the right with the device derived parameters in green, daily activity, intraday activity, heart rate, and sleep. 17:48.580 --> 17:52.580 And the user supplied are in blue, daily fat, daily weight, food, and water. 17:52.940 --> 18:04.620 So when you look at the data corpus as a whole, and you look just by data element to see which elements have the most data in them, intraday steps and activity summary are the data elements with the most data. 18:04.620 --> 18:15.400 Heart rate and sleep summary have about 40% less, and the user supplied daily weight, daily fat, food, and water intake have greater than 90% less data than the intraday steps. 18:15.400 --> 18:29.240 So in terms of what is going to be most relevant or useful, you know, for giving to the research community, definitely the, in my mind, the intraday steps activity summary, as well as heart rate. 18:31.440 --> 18:37.260 So then we wanted to try to assess the completeness of each participant's data. 18:37.360 --> 18:44.380 So percent completeness was assessed within each of the 8,535 participants. 18:45.000 --> 18:55.540 And the way that we did this was to do a theoretical maximum kind of comparison, where if you have to, that's based on the start and end time of donation. 18:55.540 --> 19:09.560 So if you have a user that comes to you, and they started donating data, and then they ended donating data, and it was about 100 days, then your theoretical maximum number of data elements for the daily elements would be 100. 19:09.760 --> 19:16.000 For the minute level, it would be 100 times the number of minutes in a day, which is 1,440. 19:16.200 --> 19:24.720 So then by comparing the number of data elements that you have for each of those data types within a person, by the theoretical maximum, you can get this percent completeness. 19:24.720 --> 19:31.520 And we were looking at two thresholds, greater than 60% complete, and greater than 80% complete. 19:31.520 --> 19:43.140 So because intraday steps, daily activity, and heart rate are all, they're all device-derived data types, you would assume that they're all relatively similar, right? 19:43.140 --> 19:49.700 Because if the person is wearing the device versus not, you wouldn't get steps and not heart rate, right? 19:49.700 --> 19:52.020 Unless the device just didn't have that capability. 19:52.020 --> 19:55.620 So it's actually kind of a good control to see that these numbers are very similar. 19:55.620 --> 20:09.580 So for intraday steps, daily activity, and heart rate, we have about 77 to 79% of the participants that are donating data have more than 60% of the data compared to the theoretical maximum amount, which is pretty good. 20:09.580 --> 20:12.460 This is a pretty complete set, which is what that means. 20:12.460 --> 20:16.060 When you look at 80%, it's slightly lower, but it's still very good. 20:16.220 --> 20:24.580 Around 62 to 65% of the participants have more than 80% of the maximum possible data available. 20:25.600 --> 20:46.420 So an interesting thing to note, though, is that when we looked at how complete a participant who is only donating data for one year or less versus a participant that's donating several years of data, we noticed that participants with data available for only one year or less had the lowest percentage of completeness, 20:46.420 --> 20:52.500 and completeness of participant data increased as duration of BYOD data donation increases. 20:52.500 --> 21:06.180 So this just goes to show that, I think, something that we've all noticed that, you know, if you've had a device for more than one year, if you wear it regularly, you know, it kind of becomes integrated into your life, and so the chance of having a more complete data set goes up. 21:06.560 --> 21:26.540 So before I move into the next section where I talk a little bit about the demographics of the participants in the Fitbit BYOD program versus all of us participants, I want to first just give you a summary of how we think about diversity in the All of Us research program. 21:26.540 --> 21:43.720 Diversity is critical to doing precision medicine research, because when you think about, you know, if we want to be able to deliver healthcare that's specifically for, you know, a specific type of person, we need to have research on all of those various types of people to be able to find those differences. 21:43.720 --> 21:50.560 So this is how we think about these groups that we call underrepresented in biomedical research and their definitions. 21:50.660 --> 22:00.000 So race and ethnicity, participants are identified as Hispanic, Latino, Spanish, race, or race identified other than white. 22:00.000 --> 22:02.500 Age, participant is under 18. 22:02.500 --> 22:04.740 Again, we're not doing that in this program yet. 22:04.740 --> 22:07.260 So over 65 is really what we focus on. 22:07.260 --> 22:12.620 Sexual and gender minorities, participant sex reported is different from that assigned at birth. 22:12.620 --> 22:15.880 Or orientation, sexual orientation is something other than straight. 22:15.880 --> 22:21.560 Income, the participant lives below the federal poverty level, which is less than 25,000 a year. 22:21.580 --> 22:24.580 Education, the participant has less than a high school degree. 22:24.580 --> 22:27.040 Geography, you live in a rural zip code. 22:27.040 --> 22:31.440 And then access to care, you can't readily access the healthcare system or can't pay for care. 22:31.440 --> 22:35.780 And then disability, some type of physical and or mental disability. 22:36.900 --> 22:48.020 So just first to start, I wanted to show you that the breakdown of age group in the FYPIT, versus all of us core participants. 22:48.020 --> 22:54.540 So the FYPIT BYOD are in the dark bars, and all of us are in the lighter bars. 22:54.540 --> 22:59.000 So what you can see here is that the distribution of age is fairly similar. 22:59.000 --> 23:04.880 You do lose a little bit of the 75 to 85, and this does fall into that EBR category. 23:04.880 --> 23:09.160 You do gain a little bit in the mid-20s to mid-30s. 23:09.160 --> 23:12.620 But overall, this is pretty comparable. 23:13.120 --> 23:18.620 So in the next chart, in this chart, I tried to highlight some of where we're seeing some of the gaps. 23:18.620 --> 23:26.620 And we're not able to capture EBRs in the same percentages with FYPIT participants versus all of us participants. 23:26.620 --> 23:39.280 So you can see that the vast majority of FYPIT participants, over 78% are white, whereas with all of us, we only have about 47, 48% are white. 23:39.280 --> 23:53.280 And then you have 20% Black or African American, 16% Hispanic or Latino, whereas in FYPIT BYOD, you only have 4 to 5% of those two EBR categories. 23:54.060 --> 24:00.100 Another feature of the FYPIT participants is that over 36% have an advanced degree. 24:00.240 --> 24:04.780 In all of us, only 18% have an advanced degree. 24:04.780 --> 24:10.920 In terms of high school only, we've got about 21% in all of us who have high school only degrees. 24:11.060 --> 24:14.080 And in FYPIT participants, it's only 6%. 24:14.080 --> 24:25.980 In terms of living below the poverty line, not surprisingly, we only have 8% of people donating data from that group, whereas all of us participants, we have 31%. 24:25.980 --> 24:42.820 So I wanted to really drive this point home that even though we're trying so hard, this program is so focused on diversity, we're trying really hard to capture the diversity, we can't do it in this kind of bring your own device model because this is probably reflecting people that have the devices, 24:42.820 --> 24:43.380 right? 24:43.400 --> 24:48.580 So we're going to have to be a little bit creative in order to try to capture the data from other groups. 24:48.580 --> 24:56.120 So just to summarize, there are specific diversity gaps in the FYPIT BYOD participants when compared with all of us core participants. 24:56.120 --> 25:07.800 And skipping down to the bottom and the two bulleted points that are in bold, there's a definite bias towards white versus other race categories, and that FYPIT participants are skewed to higher education and income levels. 25:08.100 --> 25:09.720 So why is diversity important? 25:09.720 --> 25:11.580 Just one more piece of evidence. 25:11.580 --> 25:25.380 This is a very popular paper in Nature from 2016, which surveyed the data that has been used to do GWAS or genome-wide association studies from 2009 to 2016. 25:25.380 --> 25:35.420 And what it shows is that in 2009, 96% of GWAS studies were done using data from or using samples from people with European ancestry. 25:35.420 --> 25:38.720 That is a tremendous bias, right? 25:38.720 --> 25:41.680 I mean, this is something to be really, really careful about. 25:41.840 --> 25:47.360 Since then, since this paper was published in 2016, it went down to 81%. 25:48.120 --> 25:59.660 And it looks like, you know, in particular, people with Asian ancestry are becoming more represented, but we're doing a terrible job at getting diversity in this data set. 25:59.660 --> 26:09.680 And my worry, my concern is that if we're not careful in how we do research, you know, using these digital health technologies data, we're going to be in the same situation. 26:09.680 --> 26:17.280 And that's certainly not something that we want, because these types of gaps are what lead to disparities in healthcare. 26:17.900 --> 26:23.380 So what is the opportunity and some considerations for DHT in the All of Us Research Program? 26:23.380 --> 26:32.420 So the All of Us Research Program truly has a unique opportunity to address some of the current challenges in DHT research due to the following core values of the program. 26:32.420 --> 26:35.740 First, the size and diversity of the cohort. 26:35.880 --> 26:38.460 Again, we're trying to get to a million. 26:38.460 --> 26:41.140 We've currently got more than 350,000. 26:41.340 --> 26:54.780 And the focus on diversity means that we really do have a need to focus on diversity, not just at the level of the participants, but also at the level of the data type. 26:55.840 --> 27:09.140 It's also important to the program to capture data from those new VR groups, and the goal to develop an open access, rich, and complete biomedical data set that's not biased, and make it available to the research community. 27:09.140 --> 27:10.360 So that's really key. 27:10.920 --> 27:29.140 So because diversity of participants and of data is of critical importance to the All of Us Program, this information that we've learned from the PYOD analysis will be used to plan future digital health technology efforts, potentially piloting, giving out devices to people, 27:29.800 --> 27:35.880 to try to mitigate gaps in the traditionally underrepresented in biomedical research participants. 27:36.480 --> 27:51.460 So really the question that I want to get to, and that I want people to think about, and to hopefully spark some discussion, is how can the All of Us Program leverage DHT to provide unique opportunities for our participants and for the research community? 27:52.040 --> 27:57.020 So just some other key considerations for All of Us and digital health technologies. 27:57.020 --> 27:58.320 Again, diversity is key. 27:58.320 --> 27:59.460 I can't say it enough. 27:59.460 --> 28:08.080 Even in the All of Us Research Program, where there's a huge focus on diversity, it's a challenge to get that diversity in the digital health technology data. 28:08.660 --> 28:12.500 But it's critical and key for eliminating bias from results. 28:13.360 --> 28:24.740 You know, the All of Us Research Program, because we have the opportunity to release one of the richest data sets and largest data sets with the Fitbit release, it's an opportunity to democratize access to this data type. 28:24.740 --> 28:34.260 You know, there are a lot of research studies that have just a very few number of participants, or for the larger studies with lots more data, they're mostly in the private sector. 28:34.260 --> 28:44.940 So how can we, you know, in this safely enable a larger number of researchers to get familiar with these data types and start to do innovative things with them? 28:45.660 --> 28:50.260 The other thing is that, you know, a lot in a lot of cases, these are novel data types. 28:50.260 --> 28:54.740 So a lot of what has been built for privacy and security is unknown until proven, right? 28:54.740 --> 29:01.680 And so certainly data egress is something that the program guards against and is critical. 29:01.680 --> 29:11.800 And then with privacy, re-identification of participants is also another consideration that we need to make sure that we are protecting for. 29:12.100 --> 29:16.940 So I'm just going to show you slides to let you know that it really does take all of us. 29:17.020 --> 29:21.820 These are all of the All of Us consortium members beyond the community partners. 29:22.120 --> 29:26.080 You may or may not be able to read all of these organizations. 29:26.240 --> 29:27.600 It's truly massive. 29:27.600 --> 29:30.380 These are the community partner and provider networks. 29:31.400 --> 29:35.680 And here's a picture of All of Us in there somewhere. 29:36.000 --> 29:38.540 And it really is truly a monumental effort. 29:38.540 --> 29:46.140 So please feel free to find out more information, shoot me a note, ask a question, discuss, and thank you so much for your time.