1
00:00:00,000 --> 00:00:10,680
First of all, thank you so much to the organizers for having me here today live from home to talk to you today about some of my work with the All of Us Research Program.

2
00:00:10,960 --> 00:00:12,260
My name is Michelle Holko.

3
00:00:12,260 --> 00:00:19,600
I'm currently a Presidential Innovation Fellow, detailed to the NIH, to work with the program, specifically with digital health technologies.

4
00:00:19,600 --> 00:00:25,240
I'm going to talk to you about one of the projects I've been working on there since about December last year.

5
00:00:25,800 --> 00:00:29,120
So I'm going to give you first an introduction to the All of Us Research Program.

6
00:00:29,120 --> 00:00:36,480
I'm going to run pretty fast through those slides so that we can get to the digital health technologies portfolio, but more broadly in the program.

7
00:00:36,580 --> 00:00:45,520
And then I'm going to spend the most time talking about the Fitbit, bring your own device data analysis and participant demographics, which is the area that I'm going to be talking about most today.

8
00:00:45,860 --> 00:00:52,360
And then we'll close with some opportunity and considerations for digital health technologies in the All of Us Research Program.

9
00:00:52,540 --> 00:01:11,500
So just again, to give you a kind of an introduction to the program, in case you're not familiar with it, the All of Us Research Program grew out of the Precision Medicine Initiative, which was founded several years ago in the previous administration and has $1.5 billion of congressional dollars to get the program started and to continue running.

10
00:01:11,640 --> 00:01:21,320
And the mission really for the program is to accelerate health research and medical breakthroughs to promote individualized prevention, treatment, and care for all of us.

11
00:01:21,320 --> 00:01:26,060
There are three core values to the program that are really important to us.

12
00:01:26,060 --> 00:01:32,220
The first is to nurture relationships with one million or more participant partners from all walks of life for decades.

13
00:01:32,220 --> 00:01:36,340
And again, diversity is really critical to the program in that.

14
00:01:36,340 --> 00:01:47,060
To deliver the largest, richest biomedical data set ever, as well as making it easy to use to a wide range of researchers and to catalyze a robust ecosystem of researchers.

15
00:01:48,700 --> 00:01:55,060
So the All of Us Research Program truly is an innovative and ambitious research effort.

16
00:01:55,480 --> 00:02:04,640
And here are some reasons why really the diversity at the scale of a million people or more and focusing on, you know, capturing the diversity that makes America, America.

17
00:02:04,860 --> 00:02:06,880
Focusing on participants as partners.

18
00:02:06,880 --> 00:02:14,860
So not just, you know, taking data from them, but also giving back to them and continuing to engage with them throughout the length of the program.

19
00:02:14,920 --> 00:02:22,400
The longitudinal design of the program means that we have the ability to recontact participants, which we've already done in some cases around COVID.

20
00:02:22,920 --> 00:02:25,280
And multiple data types.

21
00:02:25,280 --> 00:02:39,600
So diversity in participants, but also diversity in data, including electronic health records data, survey data, baseline physical measurements, biospecimens, genomics, and digital health technologies, which is what I'm going to be talking about today.

22
00:02:40,160 --> 00:02:46,960
Creating a national open resource for all, which is broadly accessible to all researchers with open source software and tools.

23
00:02:46,960 --> 00:02:50,960
And building in security and privacy safeguards for all participants.

24
00:02:51,120 --> 00:02:52,720
So some current progress.

25
00:02:52,720 --> 00:02:56,000
We opened the doors nationally on May 6, 2018.

26
00:02:56,080 --> 00:03:07,220
So far, we've enrolled over 271,000 participants that have not only completed the first initial enrollment step, but have also completed what we call the full protocol.

27
00:03:07,280 --> 00:03:10,720
And this represents individuals from all 50 states.

28
00:03:10,860 --> 00:03:25,820
Furthermore, greater than 80% of these participants represent one or more of the underrepresented in biomedical research categories, which includes racial and ethnic minorities, you know, living in a rural region, and there are many more.

29
00:03:25,820 --> 00:03:28,640
I'm going to go through those in more detail a little bit later.

30
00:03:28,800 --> 00:03:32,800
And greater than 50% of the participants are racial and ethnic minorities.

31
00:03:33,000 --> 00:03:47,200
We built a significant infrastructure to support the program with over 100 plus academic groups working with us, the VA, the FQHCs, which are the federally qualifying healthcare centers, technology and community partners.

32
00:03:47,200 --> 00:03:52,340
We have over 320 clinics enrolling participants, and that number is still expanding.

33
00:03:52,420 --> 00:03:54,300
We also have a bilingual website.

34
00:03:54,300 --> 00:03:59,740
So the website is available in English and in Spanish, as well as the participant portal app and call center.

35
00:03:59,760 --> 00:04:07,900
The biobank at the Mayo Clinic supports 24-hour shipping and processing with capacity for 335 million plus files.

36
00:04:07,900 --> 00:04:11,000
And we also have an interactive mobile exhibit that travels the country.

37
00:04:11,000 --> 00:04:13,060
The picture is on the bottom right of the screen.

38
00:04:13,060 --> 00:04:23,860
Certainly not traveling during right now, but usually it is, and it's actually wonderful because it can reach, you know, more rural areas where we don't have partners.

39
00:04:24,460 --> 00:04:29,140
So here is just a snapshot of enrollment to date.

40
00:04:29,240 --> 00:04:34,560
So you can see enrollment over time in the graph on the right, and then the bubbles kind of tell you where we are right now.

41
00:04:34,560 --> 00:04:48,440
Like I said, we have 271,000 that have completed the initial steps of the program, which includes consenting, agreeing to share your EHR data, completing the first three surveys, providing physical measurements, and donating at least one bio assessment.

42
00:04:48,680 --> 00:04:56,640
We have many more than that, 353,000 who have completed some part of the program, but they haven't completed all of those kind of initial steps.

43
00:04:57,520 --> 00:04:59,220
This is what the current protocol looks like.

44
00:04:59,220 --> 00:05:04,400
The first step is to enroll, to provide your informed consent, and to authorize your EHR.

45
00:05:04,400 --> 00:05:09,660
It's really important that we, you know, ask for the authorization of EHR up front at this initial step.

46
00:05:09,800 --> 00:05:14,200
Right now we're only recruiting adults who are 18 years and older.

47
00:05:14,200 --> 00:05:19,040
We do have plans to expand to children, but right now we're only recruiting 18 years and older.

48
00:05:19,960 --> 00:05:26,560
It's also important to note that at this initial enrollment, we have in-person enrollment and we have digital enrollment capabilities.

49
00:05:26,560 --> 00:05:29,740
Of course, right now the in-person enrollments have all been paused.

50
00:05:30,200 --> 00:05:37,900
So, in that interim, we've still been able to accept participants joining the program, but only through the digital method.

51
00:05:38,180 --> 00:05:39,940
Then you can answer surveys.

52
00:05:39,940 --> 00:05:49,360
There are many different surveys that are already available, including the basics, overall health, lifestyle, health care access and utilization, family medical history, personal health history.

53
00:05:49,440 --> 00:06:03,380
And we also released a survey around the coronavirus pandemic called the COPE survey, which focuses not only on signs and symptoms and exposure to coronavirus, but also, you know, the social and economic aspects of that.

54
00:06:04,080 --> 00:06:08,040
The next step, the next two steps actually kind of usually go hand in hand.

55
00:06:08,040 --> 00:06:15,040
And so usually if you're coming to us through one of our health care provider organizations, you'll provide physical measurements, which are all listed here.

56
00:06:15,040 --> 00:06:18,780
Blood pressure, heart rate, height, weight, BMI, hip circumference, and waist circumference.

57
00:06:18,780 --> 00:06:21,220
At the same time, you'll provide a bio sample.

58
00:06:21,640 --> 00:06:24,800
Usually blood, that's the bio sample of choice.

59
00:06:24,800 --> 00:06:34,180
However, if the blood draw is unsuccessful, or if you're coming to us as a direct volunteer digitally, we can send you a kit to give a saliva sample.

60
00:06:34,780 --> 00:06:37,140
Very few participants give your assessments.

61
00:06:37,140 --> 00:06:38,780
We are collecting some of those.

62
00:06:38,920 --> 00:06:42,760
And then finally, digital health technologies, which is what I'm going to be talking about today.

63
00:06:42,760 --> 00:06:49,400
We have a bring your own device program already set up with Fitbit and Apple HealthKit connections.

64
00:06:49,780 --> 00:06:57,140
We're also developing integrated apps to track mood and cardiorespiratory fitness, among other things to come.

65
00:06:57,140 --> 00:07:05,640
So now shifting from the participant to the researcher, and how are we going to make these data available widely to the research community in a usable way?

66
00:07:05,640 --> 00:07:07,900
That sounds very easy, but it's not.

67
00:07:08,200 --> 00:07:16,800
So on the left is kind of the traditional model where a researcher would come to the program, apply for the data, and then they would be basically shipped out.

68
00:07:16,800 --> 00:07:19,200
It would be basically shipped out to them.

69
00:07:19,220 --> 00:07:23,880
But with this model, it certainly discourages sharing of research.

70
00:07:23,940 --> 00:07:27,340
It also is a very weak link in terms of security.

71
00:07:27,340 --> 00:07:31,240
You have no control over those data once they leave your facility.

72
00:07:31,380 --> 00:07:37,740
You also need a high infrastructure to be able to support that to pay for multiple copies and all of that.

73
00:07:37,740 --> 00:07:43,640
So what we've decided to do is to create a cloud eccentric approach where we bring researchers to the data.

74
00:07:43,640 --> 00:07:50,960
All of the data are hosted in the cloud, the tools are being built in the cloud, and then researchers are able to get access to it.

75
00:07:51,080 --> 00:07:53,080
And in this way, it facilitates collaboration.

76
00:07:53,080 --> 00:07:55,320
There are centralized security controls.

77
00:07:55,320 --> 00:08:04,900
We can make it accessible to all researchers, and we can even think of ways of making different types of data accessible to different types of researchers depending on the need.

78
00:08:05,040 --> 00:08:10,620
So with this cloud-centric model, you can also think of providing the data in tiers.

79
00:08:10,880 --> 00:08:18,920
We certainly have a public level of data sharing that's happening, publicly accessible, no login is needed, shared on the website.

80
00:08:18,920 --> 00:08:21,560
I showed you the blurbs of the enrollment over time.

81
00:08:21,560 --> 00:08:23,340
That's one of the snapshots.

82
00:08:23,440 --> 00:08:33,960
We've also created a registered tier, which requires identity verification, signing a data use agreement, and completing research ethics training.

83
00:08:34,100 --> 00:08:43,040
There's also the idea of potentially developing a controlled tier of data, which would give people even more access to the data.

84
00:08:43,040 --> 00:08:55,000
So at each step, where the security goes higher, and we are instituting more and more controls, because you're getting more and more access to data.

85
00:08:55,000 --> 00:09:04,300
So with the registered tier, there are some specific privacy controls that are in place there that would not be in place in the controlled tier.

86
00:09:04,300 --> 00:09:08,480
So we would need an additional trust with the users of that data.

87
00:09:09,380 --> 00:09:11,380
So here's the Researcher Hub website.

88
00:09:11,380 --> 00:09:13,140
You can go to researchallofus.org.

89
00:09:13,140 --> 00:09:14,720
I encourage everybody to do that.

90
00:09:14,720 --> 00:09:19,980
From there, you can see the data snapshots, the data browser, and the survey explorer.

91
00:09:20,140 --> 00:09:25,820
The data browser that's available publicly does not let you drill down to individual level data.

92
00:09:25,820 --> 00:09:30,960
To do that, you have to move into what's on the right here, the Researcher Workbench.

93
00:09:30,960 --> 00:09:35,080
So this is just how the Researcher Hub components fit together.

94
00:09:35,120 --> 00:09:38,640
On the left, you have the data browser, data snapshot, survey explorer.

95
00:09:38,820 --> 00:09:44,740
And then in the middle is this kind of data passport model, which is how we are making the data available to researchers.

96
00:09:44,740 --> 00:09:49,400
We don't require you to ask or tell us what specific project you're going to be working on.

97
00:09:49,600 --> 00:10:00,340
We just verify your identity and, you know, kind of build a level of trust with you as a researcher, and then give you access to the data.

98
00:10:00,400 --> 00:10:07,160
In the Researcher Workbench, it includes the data dictionary, concept sets, cohort builder, notebooks, and help desk.

99
00:10:07,340 --> 00:10:13,640
And the one exciting piece of news to share is that the All of Us Research Program Researcher Workbench is open for beta testing.

100
00:10:13,640 --> 00:10:15,680
So it's a very important moment for us.

101
00:10:15,680 --> 00:10:26,920
We want to build this platform together with the research community, but we absolutely need the input of researchers to make it more robust to learn how well the data tools and policies are working.

102
00:10:27,200 --> 00:10:33,560
So the current components in terms of data include physical measurements, survey data, some EHR data.

103
00:10:33,560 --> 00:10:38,000
Much more is to come here, including genomics and digital health technologies data.

104
00:10:38,400 --> 00:10:53,060
The tools that are included include a dataset builder, a cohort builder, Jupyter notebooks with R and Python supported, and the help resources are wide and varied, including FAQs, sample notebooks and workspaces, including code snippets, documentation,

105
00:10:53,060 --> 00:10:54,660
and help desk.

106
00:10:54,660 --> 00:10:57,440
So access and security, just a moment here.

107
00:10:57,560 --> 00:11:08,120
Again, All of Us employs a data passport model for access to this researcher tier to grant researchers broad permission to explore the data for a wide range of studies.

108
00:11:08,320 --> 00:11:12,020
Data testing is available to researchers who have an eRA Commons account.

109
00:11:12,020 --> 00:11:14,440
This is what we're using for identity verification.

110
00:11:14,440 --> 00:11:16,380
So you have to have an eRA Commons account.

111
00:11:16,380 --> 00:11:18,000
Most researchers do.

112
00:11:18,180 --> 00:11:20,900
And if not, you can look into the process for creating that.

113
00:11:20,900 --> 00:11:25,460
And then you also have to have a signed data use agreement with your institution.

114
00:11:25,460 --> 00:11:36,060
So the institution that houses the data resource center finds an institution with your institution, and then we verify your identity through the eRA Commons account, and then that's how you're able to get access.

115
00:11:36,960 --> 00:11:42,040
So again, to facilitate collaboration, the researcher workbench is hosted in a cloud-based system.

116
00:11:42,340 --> 00:11:45,920
Data downloads onto your local machine are not at all allowed.

117
00:11:46,000 --> 00:11:52,880
And during the beta testing phase, another really nice perk is that researchers will receive free credits to cover computing storage costs.

118
00:11:53,800 --> 00:11:59,060
So here's just a peak look at what the workspace looks like when you come to the researcher workbench.

119
00:11:59,060 --> 00:12:03,700
First, you create a workspace, and this is where everything kind of will be all collected.

120
00:12:03,920 --> 00:12:10,440
And then you can create cohorts, concept sets, cohort reviews, and notebooks to run your analyses.

121
00:12:10,700 --> 00:12:15,940
This is just a look at what the tools look like in terms of the cohorts, data sets, and concept sets.

122
00:12:15,940 --> 00:12:29,940
And then just one more summary about, you And I have to say, the support features are really extensive, so anyone who is not a coder coming to this should be able to even get up and running.

123
00:12:29,940 --> 00:12:40,760
So there's the user support hub, researcher workbench, featured workspaces, reusable code snippets, demonstration projects, and then a help desk form that can give you kind of one-on-one help.

124
00:12:41,380 --> 00:12:58,080
So just to summarize this section, we are providing a robust data ecosystem with survey data, physical measurements, structured EHR data, with plans to add additional surveys, digital health technologies data, unstructured EHR data, linkages to other data sets,

125
00:12:58,080 --> 00:13:01,820
additional assays and biospecimens, and genomics.

126
00:13:02,020 --> 00:13:07,680
So moving on to the topic of the day, digital health technologies in the All of Us Research Program.

127
00:13:07,960 --> 00:13:10,840
So what is the vision really for digital health technologies?

128
00:13:10,840 --> 00:13:28,220
Well, digital health technologies data from things like fitness trackers, smartwatches, apps, other types of devices, can capture health-relevant data from individuals outside the hospital or clinic, which can really supplement current medical information.

129
00:13:28,520 --> 00:13:45,280
Another feature, especially for fitness trackers and smartwatches, is that the data are longitudinal in nature, and it can really give researchers an unprecedented ability to better understand the impact of lifestyle, environment, and other factors on health outcomes,

130
00:13:45,280 --> 00:13:51,500
and eventually and ultimately develop strategies for keeping people healthy in a very precise and individualized way.

131
00:13:51,620 --> 00:14:01,380
The program is currently collecting data from participants in a bring-your-own-device model, so we are not distributing devices to participants, although that is something that we are planning for.

132
00:14:01,380 --> 00:14:04,420
And we currently have connections to Fitbit and Apple HealthKit.

133
00:14:04,420 --> 00:14:07,220
So again, just bring your own device.

134
00:14:07,220 --> 00:14:12,480
We are also considering developing pilots of smartphone-based apps in the future.

135
00:14:13,080 --> 00:14:20,280
So what is the vision or impact specifically for the Fitbit bring-your-own-device data in the Researcher Workbench?

136
00:14:20,280 --> 00:14:28,400
So I believe that all of us really truly have a unique opportunity to release one of the largest and most diverse digital health technology data sets.

137
00:14:28,400 --> 00:14:33,920
We currently have data from over 9,000 participants, some beginning in 2009.

138
00:14:33,920 --> 00:14:41,220
In addition to current survey, EHR, and physical measurements, future data releases will also include genomics and other data sets.

139
00:14:41,220 --> 00:14:46,640
So when you think about the power of combining all of these different data types together, it's truly amazing.

140
00:14:46,800 --> 00:14:57,000
And then this can promote research to develop new and improved disease prevention capabilities, remote patient monitoring for connected care, and other opportunities for high-cost diseases.

141
00:14:57,680 --> 00:15:07,160
So the approach that we are using for the Fitbit data in the All of Us Research Program is this Minimum Viable Product, or MVP, approach.

142
00:15:07,160 --> 00:15:19,400
So first step of this is to obtain the data, and we have all of the data currently through May 2020 that's going to all be curated and released in the next data release.

143
00:15:19,400 --> 00:15:28,700
We've also developed scientific research use cases around the Fitbit data just to make sure that the way that we're curating the data is going to make it useful to the research community.

144
00:15:28,820 --> 00:15:35,960
We are now assessing the data set for the longitudinal nature, missingness, and trends, or basically doing exploratory data analysis.

145
00:15:36,240 --> 00:15:45,980
And then based on two and three, based on scientific research use cases and the EDA of the data, defining the priority data elements for inclusion in the Researcher Workbench.

146
00:15:45,980 --> 00:15:56,780
And again, the timeline for that is to release it in the next data release, which is slated for late fall 2020, currently anyway.

147
00:15:56,900 --> 00:16:03,920
And then to develop and implement a Fitbit curation schema coordinated with the security privacy team and other approvals as needed.

148
00:16:03,920 --> 00:16:07,860
So this is what the currently what the Fitbit MVP looks like.

149
00:16:07,960 --> 00:16:11,180
You can see the data elements table over on the right.

150
00:16:11,460 --> 00:16:24,200
The data elements that we are currently pulling from Fitbit include heart rate, minute level heart rate, heart rate summary data, intraday steps, which is also the minute level, daily activity summary, daily weight, daily fat, food, and water.

151
00:16:24,200 --> 00:16:30,620
And it's important to note that daily weight, daily fat, food, and water are all user supplied elements.

152
00:16:30,700 --> 00:16:35,300
So just by nature of that, there are, there is much less of those data.

153
00:16:35,300 --> 00:16:39,640
The others are derived from parameters that are derived from the device.

154
00:16:39,740 --> 00:16:55,780
So what we've decided to do is to provide a series of data tables using BigQuery of parsed data tables, including heart rate, summary by zone, heart rate, minute level, activity, daily summary, and activity intraday steps in that level.

155
00:16:56,440 --> 00:17:07,160
So to dig into the data a little bit more, as well as some of the participant demographics, which are important for a number of reasons I'll go into.

156
00:17:07,500 --> 00:17:11,840
So these are characteristics of the current Fitbit data set in terms of the data elements.

157
00:17:11,840 --> 00:17:17,280
So we have 9,255 Fitbit bring your own device participants.

158
00:17:17,280 --> 00:17:21,240
And again, this is until March, March, May 2020 timeframe.

159
00:17:21,700 --> 00:17:34,660
8,535 of these have primary and electronic health record consent with activity measures as early as 2012 and heart rate as early as 2015.

160
00:17:34,720 --> 00:17:37,660
So that is quite a lot of data, if you think about it.

161
00:17:37,660 --> 00:17:48,580
The data elements that we are pulling from Fitbit API are shown on the right with the device derived parameters in green, daily activity, intraday activity, heart rate, and sleep.

162
00:17:48,580 --> 00:17:52,580
And the user supplied are in blue, daily fat, daily weight, food, and water.

163
00:17:52,940 --> 00:18:04,620
So when you look at the data corpus as a whole, and you look just by data element to see which elements have the most data in them, intraday steps and activity summary are the data elements with the most data.

164
00:18:04,620 --> 00:18:15,400
Heart rate and sleep summary have about 40% less, and the user supplied daily weight, daily fat, food, and water intake have greater than 90% less data than the intraday steps.

165
00:18:15,400 --> 00:18:29,240
So in terms of what is going to be most relevant or useful, you know, for giving to the research community, definitely the, in my mind, the intraday steps activity summary, as well as heart rate.

166
00:18:31,440 --> 00:18:37,260
So then we wanted to try to assess the completeness of each participant's data.

167
00:18:37,360 --> 00:18:44,380
So percent completeness was assessed within each of the 8,535 participants.

168
00:18:45,000 --> 00:18:55,540
And the way that we did this was to do a theoretical maximum kind of comparison, where if you have to, that's based on the start and end time of donation.

169
00:18:55,540 --> 00:19:09,560
So if you have a user that comes to you, and they started donating data, and then they ended donating data, and it was about 100 days, then your theoretical maximum number of data elements for the daily elements would be 100.

170
00:19:09,760 --> 00:19:16,000
For the minute level, it would be 100 times the number of minutes in a day, which is 1,440.

171
00:19:16,200 --> 00:19:24,720
So then by comparing the number of data elements that you have for each of those data types within a person, by the theoretical maximum, you can get this percent completeness.

172
00:19:24,720 --> 00:19:31,520
And we were looking at two thresholds, greater than 60% complete, and greater than 80% complete.

173
00:19:31,520 --> 00:19:43,140
So because intraday steps, daily activity, and heart rate are all, they're all device-derived data types, you would assume that they're all relatively similar, right?

174
00:19:43,140 --> 00:19:49,700
Because if the person is wearing the device versus not, you wouldn't get steps and not heart rate, right?

175
00:19:49,700 --> 00:19:52,020
Unless the device just didn't have that capability.

176
00:19:52,020 --> 00:19:55,620
So it's actually kind of a good control to see that these numbers are very similar.

177
00:19:55,620 --> 00:20:09,580
So for intraday steps, daily activity, and heart rate, we have about 77 to 79% of the participants that are donating data have more than 60% of the data compared to the theoretical maximum amount, which is pretty good.

178
00:20:09,580 --> 00:20:12,460
This is a pretty complete set, which is what that means.

179
00:20:12,460 --> 00:20:16,060
When you look at 80%, it's slightly lower, but it's still very good.

180
00:20:16,220 --> 00:20:24,580
Around 62 to 65% of the participants have more than 80% of the maximum possible data available.

181
00:20:25,600 --> 00:20:46,420
So an interesting thing to note, though, is that when we looked at how complete a participant who is only donating data for one year or less versus a participant that's donating several years of data, we noticed that participants with data available for only one year or less had the lowest percentage of completeness,

182
00:20:46,420 --> 00:20:52,500
and completeness of participant data increased as duration of BYOD data donation increases.

183
00:20:52,500 --> 00:21:06,180
So this just goes to show that, I think, something that we've all noticed that, you know, if you've had a device for more than one year, if you wear it regularly, you know, it kind of becomes integrated into your life, and so the chance of having a more complete data set goes up.

184
00:21:06,560 --> 00:21:26,540
So before I move into the next section where I talk a little bit about the demographics of the participants in the Fitbit BYOD program versus all of us participants, I want to first just give you a summary of how we think about diversity in the All of Us research program.

185
00:21:26,540 --> 00:21:43,720
Diversity is critical to doing precision medicine research, because when you think about, you know, if we want to be able to deliver healthcare that's specifically for, you know, a specific type of person, we need to have research on all of those various types of people to be able to find those differences.

186
00:21:43,720 --> 00:21:50,560
So this is how we think about these groups that we call underrepresented in biomedical research and their definitions.

187
00:21:50,660 --> 00:22:00,000
So race and ethnicity, participants are identified as Hispanic, Latino, Spanish, race, or race identified other than white.

188
00:22:00,000 --> 00:22:02,500
Age, participant is under 18.

189
00:22:02,500 --> 00:22:04,740
Again, we're not doing that in this program yet.

190
00:22:04,740 --> 00:22:07,260
So over 65 is really what we focus on.

191
00:22:07,260 --> 00:22:12,620
Sexual and gender minorities, participant sex reported is different from that assigned at birth.

192
00:22:12,620 --> 00:22:15,880
Or orientation, sexual orientation is something other than straight.

193
00:22:15,880 --> 00:22:21,560
Income, the participant lives below the federal poverty level, which is less than 25,000 a year.

194
00:22:21,580 --> 00:22:24,580
Education, the participant has less than a high school degree.

195
00:22:24,580 --> 00:22:27,040
Geography, you live in a rural zip code.

196
00:22:27,040 --> 00:22:31,440
And then access to care, you can't readily access the healthcare system or can't pay for care.

197
00:22:31,440 --> 00:22:35,780
And then disability, some type of physical and or mental disability.

198
00:22:36,900 --> 00:22:48,020
So just first to start, I wanted to show you that the breakdown of age group in the FYPIT, versus all of us core participants.

199
00:22:48,020 --> 00:22:54,540
So the FYPIT BYOD are in the dark bars, and all of us are in the lighter bars.

200
00:22:54,540 --> 00:22:59,000
So what you can see here is that the distribution of age is fairly similar.

201
00:22:59,000 --> 00:23:04,880
You do lose a little bit of the 75 to 85, and this does fall into that EBR category.

202
00:23:04,880 --> 00:23:09,160
You do gain a little bit in the mid-20s to mid-30s.

203
00:23:09,160 --> 00:23:12,620
But overall, this is pretty comparable.

204
00:23:13,120 --> 00:23:18,620
So in the next chart, in this chart, I tried to highlight some of where we're seeing some of the gaps.

205
00:23:18,620 --> 00:23:26,620
And we're not able to capture EBRs in the same percentages with FYPIT participants versus all of us participants.

206
00:23:26,620 --> 00:23:39,280
So you can see that the vast majority of FYPIT participants, over 78% are white, whereas with all of us, we only have about 47, 48% are white.

207
00:23:39,280 --> 00:23:53,280
And then you have 20% Black or African American, 16% Hispanic or Latino, whereas in FYPIT BYOD, you only have 4 to 5% of those two EBR categories.

208
00:23:54,060 --> 00:24:00,100
Another feature of the FYPIT participants is that over 36% have an advanced degree.

209
00:24:00,240 --> 00:24:04,780
In all of us, only 18% have an advanced degree.

210
00:24:04,780 --> 00:24:10,920
In terms of high school only, we've got about 21% in all of us who have high school only degrees.

211
00:24:11,060 --> 00:24:14,080
And in FYPIT participants, it's only 6%.

212
00:24:14,080 --> 00:24:25,980
In terms of living below the poverty line, not surprisingly, we only have 8% of people donating data from that group, whereas all of us participants, we have 31%.

213
00:24:25,980 --> 00:24:42,820
So I wanted to really drive this point home that even though we're trying so hard, this program is so focused on diversity, we're trying really hard to capture the diversity, we can't do it in this kind of bring your own device model because this is probably reflecting people that have the devices,

214
00:24:42,820 --> 00:24:43,380
right?

215
00:24:43,400 --> 00:24:48,580
So we're going to have to be a little bit creative in order to try to capture the data from other groups.

216
00:24:48,580 --> 00:24:56,120
So just to summarize, there are specific diversity gaps in the FYPIT BYOD participants when compared with all of us core participants.

217
00:24:56,120 --> 00:25:07,800
And skipping down to the bottom and the two bulleted points that are in bold, there's a definite bias towards white versus other race categories, and that FYPIT participants are skewed to higher education and income levels.

218
00:25:08,100 --> 00:25:09,720
So why is diversity important?

219
00:25:09,720 --> 00:25:11,580
Just one more piece of evidence.

220
00:25:11,580 --> 00:25:25,380
This is a very popular paper in Nature from 2016, which surveyed the data that has been used to do GWAS or genome-wide association studies from 2009 to 2016.

221
00:25:25,380 --> 00:25:35,420
And what it shows is that in 2009, 96% of GWAS studies were done using data from or using samples from people with European ancestry.

222
00:25:35,420 --> 00:25:38,720
That is a tremendous bias, right?

223
00:25:38,720 --> 00:25:41,680
I mean, this is something to be really, really careful about.

224
00:25:41,840 --> 00:25:47,360
Since then, since this paper was published in 2016, it went down to 81%.

225
00:25:48,120 --> 00:25:59,660
And it looks like, you know, in particular, people with Asian ancestry are becoming more represented, but we're doing a terrible job at getting diversity in this data set.

226
00:25:59,660 --> 00:26:09,680
And my worry, my concern is that if we're not careful in how we do research, you know, using these digital health technologies data, we're going to be in the same situation.

227
00:26:09,680 --> 00:26:17,280
And that's certainly not something that we want, because these types of gaps are what lead to disparities in healthcare.

228
00:26:17,900 --> 00:26:23,380
So what is the opportunity and some considerations for DHT in the All of Us Research Program?

229
00:26:23,380 --> 00:26:32,420
So the All of Us Research Program truly has a unique opportunity to address some of the current challenges in DHT research due to the following core values of the program.

230
00:26:32,420 --> 00:26:35,740
First, the size and diversity of the cohort.

231
00:26:35,880 --> 00:26:38,460
Again, we're trying to get to a million.

232
00:26:38,460 --> 00:26:41,140
We've currently got more than 350,000.

233
00:26:41,340 --> 00:26:54,780
And the focus on diversity means that we really do have a need to focus on diversity, not just at the level of the participants, but also at the level of the data type.

234
00:26:55,840 --> 00:27:09,140
It's also important to the program to capture data from those new VR groups, and the goal to develop an open access, rich, and complete biomedical data set that's not biased, and make it available to the research community.

235
00:27:09,140 --> 00:27:10,360
So that's really key.

236
00:27:10,920 --> 00:27:29,140
So because diversity of participants and of data is of critical importance to the All of Us Program, this information that we've learned from the PYOD analysis will be used to plan future digital health technology efforts, potentially piloting, giving out devices to people,

237
00:27:29,800 --> 00:27:35,880
to try to mitigate gaps in the traditionally underrepresented in biomedical research participants.

238
00:27:36,480 --> 00:27:51,460
So really the question that I want to get to, and that I want people to think about, and to hopefully spark some discussion, is how can the All of Us Program leverage DHT to provide unique opportunities for our participants and for the research community?

239
00:27:52,040 --> 00:27:57,020
So just some other key considerations for All of Us and digital health technologies.

240
00:27:57,020 --> 00:27:58,320
Again, diversity is key.

241
00:27:58,320 --> 00:27:59,460
I can't say it enough.

242
00:27:59,460 --> 00:28:08,080
Even in the All of Us Research Program, where there's a huge focus on diversity, it's a challenge to get that diversity in the digital health technology data.

243
00:28:08,660 --> 00:28:12,500
But it's critical and key for eliminating bias from results.

244
00:28:13,360 --> 00:28:24,740
You know, the All of Us Research Program, because we have the opportunity to release one of the richest data sets and largest data sets with the Fitbit release, it's an opportunity to democratize access to this data type.

245
00:28:24,740 --> 00:28:34,260
You know, there are a lot of research studies that have just a very few number of participants, or for the larger studies with lots more data, they're mostly in the private sector.

246
00:28:34,260 --> 00:28:44,940
So how can we, you know, in this safely enable a larger number of researchers to get familiar with these data types and start to do innovative things with them?

247
00:28:45,660 --> 00:28:50,260
The other thing is that, you know, a lot in a lot of cases, these are novel data types.

248
00:28:50,260 --> 00:28:54,740
So a lot of what has been built for privacy and security is unknown until proven, right?

249
00:28:54,740 --> 00:29:01,680
And so certainly data egress is something that the program guards against and is critical.

250
00:29:01,680 --> 00:29:11,800
And then with privacy, re-identification of participants is also another consideration that we need to make sure that we are protecting for.

251
00:29:12,100 --> 00:29:16,940
So I'm just going to show you slides to let you know that it really does take all of us.

252
00:29:17,020 --> 00:29:21,820
These are all of the All of Us consortium members beyond the community partners.

253
00:29:22,120 --> 00:29:26,080
You may or may not be able to read all of these organizations.

254
00:29:26,240 --> 00:29:27,600
It's truly massive.

255
00:29:27,600 --> 00:29:30,380
These are the community partner and provider networks.

256
00:29:31,400 --> 00:29:35,680
And here's a picture of All of Us in there somewhere.

257
00:29:36,000 --> 00:29:38,540
And it really is truly a monumental effort.

258
00:29:38,540 --> 00:29:46,140
So please feel free to find out more information, shoot me a note, ask a question, discuss, and thank you so much for your time.