WEBVTT 00:02.060 --> 00:14.560 So welcome everyone to our conversation about consumer privacy, usable privacy, how do we actually build privacy for consumers in a way that they can actually exercise their data rights. 00:14.560 --> 00:26.980 Unfortunately, one of the trends that we're seeing as laws and regulations become more complex, it's increasingly difficult for companies to provide the ability for consumers to actually exercise those rights. 00:26.980 --> 00:35.600 And so one of the things that we are concerned about as privacy professionals is making sure that consumers actually have the ability to exercise those rights. 00:35.600 --> 00:42.680 Otherwise, they really don't get the full protections that are granted to them under the law, depending on where they live. 00:42.680 --> 00:46.160 By way of introduction, my name is Melanie Ensign. 00:46.160 --> 00:51.100 I am the founder and CEO of a new company called Discernible. 00:51.100 --> 00:56.640 It's a communications firm dedicated specifically to security, privacy, and risk organizations. 00:56.640 --> 01:18.260 And I am very excited to be joined by a number of esteemed colleagues today who have a lot of experience, not just in security and privacy, but in building tools and in being part of the process for building and thinking about how do we make privacy a user-centric experience. 01:18.580 --> 01:22.960 So a couple of expectations setting things right off the front. 01:22.960 --> 01:25.240 None of us here are lawyers. 01:25.320 --> 01:28.000 Nothing we say is going to be legal advice. 01:28.000 --> 01:30.360 We're also not legal experts. 01:30.360 --> 01:36.700 And so we're not going to debate the merit of the law or try to interpret the law. 01:36.700 --> 01:41.340 But we are on the receiving end of these legal obligations. 01:41.340 --> 01:52.180 And we do have the professional responsibility to advise or lead the building of their subsequent products and tools for consumers so that they can exercise those rights. 01:52.180 --> 01:56.440 So just a disclaimer up front that none of us are lawyers. 01:57.240 --> 01:58.800 Troll us if you like. 01:59.200 --> 02:05.300 There'll be plenty of lawyers in the comment section who can actually provide the legal guidance that you're looking for. 02:05.540 --> 02:21.660 So with that said, I'm going to ask each of the panelists to introduce themselves and tell us a little bit about their experience in privacy and specifically their perspective and worldview in terms of how they approach this particular challenge given their function and expertise. 02:21.820 --> 02:23.860 So Megan, let's start with you. 02:24.160 --> 02:25.040 Sure. 02:25.040 --> 02:26.380 No trolling, please. 02:26.380 --> 02:27.900 I beg you. 02:27.900 --> 02:29.660 My name is Megan DeBlois. 02:29.660 --> 02:31.460 Super excited to be on this panel. 02:31.460 --> 02:35.420 I'm an information security advisor and technologist over at Internews. 02:36.480 --> 02:39.780 Internews itself is an international non-profit. 02:40.220 --> 02:50.720 We work on media development really focused on empowering local information sources worldwide while connecting and giving people voice. 02:50.760 --> 02:53.400 So that's kind of the area that I tend to focus in. 02:53.400 --> 02:55.260 But at the organization, I wear two hats. 02:55.260 --> 03:01.720 So internally, I help improve our information security processes, procedures, and practices org-wide. 03:01.720 --> 03:19.880 And externally, I work with a pretty diverse group of civil society folks to help improve them on their information security needs from organizational security assessments to risk reduction strategy design and development to threat analysis and sharing. 03:20.340 --> 03:45.280 A big part of that and a big part of our approach at Internews when it comes to privacy and security is really trying to keep our users at the center, whether that's going to be an organization or civil society group that we're working with or the developer communities in the open source world who were trying to build capacity around their abilities in creating these really important security and privacy products for really amazing communities, 03:45.280 --> 03:47.040 but often who are targeted. 03:48.080 --> 03:49.740 Yeah, and that's me. 03:51.580 --> 03:52.500 All right. 03:52.500 --> 03:53.200 Hi, everybody. 03:53.200 --> 03:56.700 I'm Ben Brook, the co-founder and CEO of Transcend. 03:56.720 --> 04:00.360 So Transcend is a data privacy infrastructure company. 04:00.400 --> 04:05.240 And what we do is we enable other companies to honor their users' data rights. 04:05.300 --> 04:10.340 So users can't really exercise their data rights today, even though it is their legal right. 04:10.340 --> 04:15.100 And you can go to any major company's privacy policy and see that for yourself. 04:15.100 --> 04:25.900 So we've made a series of breakthroughs in tech that enables any company to give their users data rights in a way that's completely on demand and you should have it in seconds. 04:26.060 --> 04:38.700 So what we're doing is we're replacing a lot of broken internal processes that are both resulting in noncompliance and and that have been an issue in terms of user experience. 04:38.700 --> 04:46.160 So if we're successful, the hope is that we're going to make it easier for everybody to have the ability to control their personal data. 04:47.940 --> 04:48.920 Hi, everyone. 04:48.920 --> 04:49.980 I'm Marietza Johnson. 04:49.980 --> 04:52.920 I'm a UX researcher at Google Research. 04:53.080 --> 05:07.380 And for about the past 10-ish years, I've been really focused on this question of how do we make it easy for people to manage their data and the security and privacy decisions that they're faced with in their daily lives with the technology that they use. 05:07.780 --> 05:15.460 This focus has taken me to a bunch of different kind of places where I can exercise those interests. 05:15.480 --> 05:22.660 Right now I'm at Google Research working on a tool for what we're calling meaningful control over data in a distributed system. 05:22.880 --> 05:23.820 I tweet a lot. 05:23.820 --> 05:28.820 You can see a project description on Twitter if you're interested in that. 05:28.820 --> 05:30.260 I think it's super fun. 05:30.600 --> 05:46.460 Also, I used to be on Facebook as a member of their privacy product review process and have also done some academic work just doing some research on looking at how do people understand this stuff, attitudes, behaviors, comprehensibility. 05:46.540 --> 05:48.360 So glad to be here. 05:50.120 --> 05:51.460 I'm Zach. 05:51.460 --> 05:54.400 I lead the global privacy product team at Uber. 05:54.400 --> 06:01.240 I focus on building user-facing features that build trust with our users. 06:01.240 --> 06:12.660 I also work on developing systems and services in response to new and emerging laws to allow users to exercise their rights over those laws. 06:12.680 --> 06:30.400 And the last thing that I'll describe is I guess a little bit different than your traditional product role because I'm also charged with consulting and reviewing the product launches of other teams across Uber that pertain to privacy. 06:30.400 --> 06:43.400 And that can take on many different forms from early in the idea phase to just before launch to post-launch and remediation and anything under that sun. 06:43.400 --> 06:45.320 So that's me. 06:45.860 --> 06:46.700 Awesome. 06:46.700 --> 06:51.480 Thank you so much for being here and lending us your expertise today. 06:51.480 --> 07:02.120 So let's start with operationalizing privacy and in the steps that actually come after these protections are codified in law. 07:02.120 --> 07:20.860 So once a law or new regulation is passed, particularly those that are then going to require some additional engineering or product development, there's a different type of skill set that's needed to work alongside our legal colleagues in order to actually bring that to life. 07:20.860 --> 07:25.160 So policy and regulations is one piece of it. 07:25.160 --> 07:37.120 But especially now with the new types of regulations we're seeing, there are very real technical requirements and changes that have to happen in order for companies to comply with those new obligations. 07:37.160 --> 07:45.920 And I want us to dive a little bit more into what that actually looks like in practice in terms of these different functions working together. 07:45.920 --> 07:49.300 So Ben and Zach, let's start with you. 07:49.300 --> 08:00.400 Based on your experience, and Ben, I know that you're working externally and helping these companies kind of repair or build some of these broken systems. 08:00.400 --> 08:03.200 And Zach, you're obviously in-house at Uber. 08:03.200 --> 08:16.040 What does the typical process look like from the perspective of those internal privacy product and engineering teams once a protection has been codified in law or in a new regulatory standard? 08:17.540 --> 08:18.520 Sure. 08:18.520 --> 08:28.840 So when a new privacy law has been encoded, there's usually a period before it comes into an effect, which means there's a huge surge of preparatory work inside a business. 08:28.840 --> 08:36.580 And that can include everything from updating contracts and policies, doing risk assessments, and building frameworks for the company to operate under. 08:36.940 --> 08:41.140 And then after the laws come into effect, there's a few new processes. 08:41.140 --> 08:46.800 So of course, as Zach alluded to, new product lines, you have to make sure that you're applying those frameworks. 08:46.880 --> 08:56.220 But what's really unique about these modern privacy laws like GDPR and CCPA is that there's also this new set of indefinitely recurring processes around user data rights. 08:56.220 --> 09:02.920 And so these are things like the right of access to your data and the right to be forgotten. 09:03.180 --> 09:10.540 So that means companies now have a consistent stream of requests coming in, typically via support channels. 09:10.600 --> 09:20.400 And so that means the support and the legal teams are now receiving these requests and working to manually authenticate users over an email conversation. 09:20.400 --> 09:32.880 You can imagine how that would be at high risk to social engineering, where users are making claims to accounts, asking to erase that account or to get access to all of the data in that account. 09:33.640 --> 09:51.760 And then after that's been received, there's also an incredible manual process that happens inside many companies today, where it's basically the entire org is going across dozens, if not hundreds, of their data systems and their vendors to go and scrub that data. 09:51.760 --> 09:55.000 And so that ends up being almost every part of the company. 09:55.000 --> 10:05.900 And whether it's sales and marketing or support or analytics, people are logging into their respective tools and scrubbing data or exporting and emailing it over to legal. 10:06.900 --> 10:13.260 And so in short, there's an incredible amount of very cumbersome and manual processes happening today. 10:13.300 --> 10:16.520 And it's sort of like this temporary solution. 10:16.520 --> 10:19.420 It seems like we're in a lot of businesses today. 10:19.460 --> 10:24.070 And again, these processes are still at high risk of social engineering. 10:25.380 --> 10:27.340 Yeah, so I'll jump in there. 10:27.340 --> 10:41.300 I think, as has been described, when you're just starting out and you perhaps don't have that much of an established privacy team, these efforts can be pretty unstructured and ad hoc. 10:41.300 --> 10:48.520 And there's a lot of manual efforts that take place throughout your company to try to comply with these new laws. 10:48.660 --> 10:58.040 At Uber, we've been working on this for five plus years, if not more, at least as long as I've been there. 10:58.260 --> 11:09.100 And what I would say is to get to the best outcome is definitely a collaborative process across legal product and engineering. 11:09.500 --> 11:18.780 So the way it works for us when a new law comes into effect, as Ben said, our legal team takes some time to digest that law. 11:18.780 --> 11:33.340 And depending on how complex it is, it could be something as large as GDPR, or one article that's something very small in a single state or even region that we need to respond to. 11:33.340 --> 11:38.820 So that can take anywhere from a couple of weeks to several months. 11:38.840 --> 11:45.900 But eventually, the legal team will get back to us with a set of legal requirements. 11:46.100 --> 11:56.190 My team then, on the product side, reviews those requirements to convert them into a PRD, or a product requirements document. 11:56.540 --> 12:12.760 In order to do that, we assess the requirements that we receive in kind of three areas to make sure that we're going to implement and launch something that complies with the law, but that is also practical. 12:13.240 --> 12:22.440 So one of the things we consider is the implications for the user experience of the requirements that have been shared over with us. 12:22.800 --> 12:29.420 Another thing is the feasibility from an engineering perspective of what's been shared over. 12:29.560 --> 12:32.820 And the other thing is the impact to the business. 12:33.220 --> 12:44.860 You know, a lot of times, new and emerging privacy laws might be at odds with what your team wants to do or your company wants to do on the marketing side, for instance. 12:45.000 --> 12:52.060 If you're providing users with more controls to opt out of ad targeting or something like that. 12:52.060 --> 13:01.720 And so, you know, we are trying to do these things in the right way in compliance with the law, but also optimize to fit our business, as I said. 13:01.840 --> 13:17.540 So once we've kind of rationalized the requirements we've received and we then have a follow-up conversation with our legal team to make sure that they are paying with the direction that we're recommending and the timeline we've set forth for which we could deliver on, 13:17.540 --> 13:22.560 we will define a set of metrics or key performance indicators. 13:22.600 --> 13:31.260 And these usually serve the purpose of mitigating the risk of being in non-compliance. 13:31.260 --> 13:40.560 We will design our metrics in that sense, in particular, if it's something to do with complying with a new law. 13:40.700 --> 13:53.800 And then finally, we will have a design review with stakeholders across Uber that could be impacted by the changes that we're going to make. 13:53.800 --> 13:58.440 And those stakeholders will be from legal, from policy, from other engineering and product teams. 13:58.440 --> 14:07.220 And we'll walk through exactly what's going to change or be introduced into the experience, how it's going to work end-to-end, and we'll get some feedback. 14:07.760 --> 14:11.920 And then we'll be all aligned across the company on moving forward. 14:12.700 --> 14:15.900 And then from there, we just start building. 14:16.240 --> 14:22.560 So that's kind of how it works in-house at a little bit more of a mature privacy operation. 14:23.760 --> 14:24.520 Awesome. 14:24.520 --> 14:25.200 Thanks. 14:25.200 --> 14:36.740 Megan and Marta, I want to ask you, does the process look any different from the perspective of end-users or even developers in the field that you're working with? 14:36.980 --> 14:45.140 We've heard from Ben and Zach a little bit, like the chaos-to-order evolution that's happening inside companies. 14:45.140 --> 14:58.720 What does it look like externally to those who are either the users of these services or perhaps other developers who aren't in the position of having these more mature privacy organizations? 15:00.160 --> 15:00.980 Sure. 15:00.980 --> 15:03.640 So if I can go first. 15:04.480 --> 15:10.120 Again, I think it's important to kind of frame up my experiences and observations here. 15:10.120 --> 15:18.040 It's really more tailored to leading and working on open-source projects and working with at-risk communities. 15:19.480 --> 15:29.160 In the past life, I led a project that was really, really aiming at building the capacity of developers and creating feedback loops among their users, which tended to be more at-risk communities. 15:29.160 --> 15:34.400 So that's kind of where I'm coming from with my input and observation. 15:34.400 --> 15:46.240 From the developer side of things, again, coming from open-source landia, where you have very few resources, including human capacity, this looks dramatically different. 15:47.000 --> 15:53.680 The vast majority of security and privacy tool developers who I've worked with are super concerned about privacy. 15:53.680 --> 15:58.660 Naturally, they're creating security and privacy tools specifically to give users more privacy online. 15:59.040 --> 16:07.420 And because of this, we see that a lot of them try really hard to capture the least amount of data as possible. 16:07.420 --> 16:09.360 So most of them have no trackers. 16:09.360 --> 16:18.140 Most of them intentionally do not collect user data at a very, very basic level. 16:18.400 --> 16:33.100 I think a lot of smaller businesses, if you're operating with European users, I think that put definitely a lot of strain on those communities who have very little capacity to begin with. 16:33.280 --> 16:42.400 But at the end of the day, I think the general thought process around privacy in those communities is sort of it's an all or nothing. 16:42.500 --> 16:51.420 I'm either going to... in the open-source community, in my observation is that they just really don't want to collect anything. 16:51.580 --> 17:05.060 And that impacts them, not just from a UX perspective, because they're not able to make really important insights to improve their tools and roadmap to actually make insightful decisions on the usability front. 17:05.140 --> 17:07.940 And I don't think it's an all or nothing game. 17:08.080 --> 17:28.600 One of the projects that we've been supporting is a project by the Guardian Project who has created an amazing and is continuing to create an amazing tool for Android applications called Clean Insights to be more plug-and-playable with these open-source tools or other products that want to be more privacy-respecting in the analytics that they're capturing. 17:29.260 --> 17:38.200 I think more things like this will be really important as more legislation and more regulation starts to happen around the globe. 17:38.300 --> 17:49.500 From an end-user perspective, I think that for me, and I don't think this is really specific to civil society, I think there's just still a really large gap in user understanding of their digital rights. 17:49.680 --> 18:00.120 So GDPR happened, I would say, you know, the vast majority of folks around the globe probably didn't know exactly what that means for them as a user. 18:00.680 --> 18:07.960 And the before and after that they really saw was they got a lot of emails in their inboxes saying privacy policies were changing for a lot of different companies. 18:08.040 --> 18:19.260 And that's sort of like a very end-user perspective of, you know, not truly knowing exactly how to engage with or activate these new rights that they might have. 18:19.880 --> 18:34.440 I think the other really important thing to note is that for a lot of at-risk communities or targeted groups that are using products, their calculus isn't necessarily privacy from, you know, companies is at the top. 18:34.440 --> 18:40.500 They're really threat modeling and trying to understand, you know, what it is I'm trying to protect here. 18:40.500 --> 19:05.980 And, you know, they're building on this trade-off between sort of, do I really need this really important security encryption protocol being used in this particular product versus a more privacy preserving application that, you know, isn't going to be used as much in my network or it's difficult to use. 19:05.980 --> 19:12.040 So it's more of a threat modeling challenge rather than a sort of privacy calculation. 19:13.020 --> 19:25.400 And our approach at Internet News has been really pragmatic in trying to, you know, get the individual, get the community, get the organization to be more secure today than they were tomorrow. 19:25.620 --> 19:29.960 And that's the ultimate priority from an end-user's perspective. 19:32.040 --> 19:46.940 I love that as a build into how I would be thinking about it because I feel like product teams need to really be thinking about the threat model and how the different groups they aim to serve may have different needs and basically taking care of, you know, 19:46.940 --> 19:48.280 as much as they can. 19:48.280 --> 19:58.540 But then inevitably we do get into situations where individuals are making their own decisions about, you know, which setting to use, which control do I consent to this? 19:58.840 --> 20:03.000 And to me, I always think about it, I want to start from the human-centered perspective. 20:03.360 --> 20:06.740 Like, we can't assume that people are reading privacy policies. 20:06.740 --> 20:13.860 We can't assume that people are seeing all the great material that we put out when we're promoting a new privacy feature or a new security feature. 20:13.860 --> 20:21.140 So then if you're starting from the point of view of the thing that they see is the UI and the user experience, what is that? 20:21.140 --> 20:22.800 How are they using the tool? 20:22.880 --> 20:25.160 So, of course, we're always mindful. 20:25.160 --> 20:31.160 You're diligent in implementing controls that respect the regulations and meet the regulations. 20:31.200 --> 20:33.460 I feel like that's one way to look at it. 20:33.460 --> 20:39.260 But then if jump over to the human-centered point of view, you start to ask different questions. 20:39.260 --> 20:44.820 So then for the context that you're building a product or a service, how is it being used? 20:44.820 --> 20:45.820 Like, who's using it? 20:45.820 --> 20:47.220 How are they using it? 20:47.220 --> 20:48.860 What are their expectations? 20:48.880 --> 20:50.800 What's their attitude toward the data use? 20:50.800 --> 20:54.640 And how do you design an experience that matches that as best as you can? 20:55.120 --> 21:04.240 I love the way, Megan, that you alluded to kind of the difficulties that we have right now where in some situations, it's quite a complicated system. 21:04.580 --> 21:10.560 How do you communicate succinctly and comprehensively to somebody about all the ways in which data is collected and used? 21:11.520 --> 21:16.700 Of course, people, I feel like we constantly hear things like, you know, oh, people don't care. 21:16.700 --> 21:17.780 If they cared, they wouldn't do this. 21:17.780 --> 21:19.300 If they cared, they wouldn't do that. 21:19.300 --> 21:22.780 But privacy and security aren't the top level concern. 21:22.780 --> 21:25.960 Just because they're not your top priority doesn't mean you don't care. 21:25.960 --> 21:31.780 So maybe it's you care more about something else right now, or maybe you'll care more about that at a later point in time. 21:31.820 --> 21:36.560 So then how do you kind of take all these different perspectives and put it into a UX that makes sense? 21:37.120 --> 21:39.280 So I'd say that's where my role comes in. 21:39.280 --> 21:42.900 So I'm always interested in, you know, how can we take a human-centered approach? 21:42.900 --> 21:52.560 How can we ask these questions about what do people need, and then design a user research program around evaluating whether or not we've met the goals there. 21:53.200 --> 22:03.120 So it's building on top of regulation, hopefully always going well beyond, but definitely something that, you know, is a constant work in progress. 22:04.380 --> 22:29.280 Yeah, one of the things that I've often advised companies that I've been working with is you don't get credit for hitting the legal minimum, you know, and truthfully, you're, you know, Marta, I think you said it best, like the people that you're serving are actually expecting more from you because they're expecting that you know better than even the lawmakers in terms of how to build better products and better experiences. 22:29.680 --> 22:47.500 So I want to shift gears just briefly to take a moment to talk about privacy policies and kind of why they're so unpopular and seem to be a pretty focused target for a lot of public scrutiny, which we could debate all day whether or not that is deserved or not. 22:48.080 --> 23:01.980 But I think the problematic aspects of privacy policies are often interpreted publicly as, you know, a company not caring about privacy or not wanting to inform consumers. 23:02.100 --> 23:05.240 And, you know, we've all worked on this challenge for years. 23:05.240 --> 23:06.720 We know that that's not true. 23:06.720 --> 23:16.360 We know that the problems that exist in privacy policies are not reflective of a company's actual commitment and investment level in privacy. 23:16.600 --> 23:27.560 And even companies with strong privacy commitments and protections have difficulty creating policies that consumers can easily access and understand. 23:28.260 --> 23:37.880 And so I want to ask each of you, and Marta, you know, I'd love to start with your thoughts just in terms of what is the actual intent of these policies? 23:37.880 --> 23:50.140 And should we still be relying on this type of documentation to inform and educate consumers about the data practices of the different businesses and services that they use? 23:50.140 --> 24:06.040 I've even heard from a couple of folks in the EU that, you know, terms of service and privacy policies are no longer going to be considered sufficient in terms of informed consent because of how complicated they are for consumers to understand. 24:06.040 --> 24:19.860 I think it is a valid argument that, you know, this very long legal document cannot... nobody realistically expects that consumers are reading that and understanding it and processing what that means for them. 24:19.860 --> 24:40.820 So obviously, again, since none of us are lawyers, without getting into all the legal definitions of consent and informed consent and all of that, what are your thoughts in terms of how we could, you know, either improve policies or perhaps look at other channels and avenues through which to educate consumers about data practices as well as their rights? 24:42.180 --> 24:48.540 Okay, so from my previous answer, you won't be surprised to hear that I think you could do a lot more than a privacy policy. 24:48.540 --> 24:56.000 And there's a big opportunity to do a better job of both crafting, living, and communicating your privacy narrative. 24:56.000 --> 24:59.880 And not just a privacy narrative, but like, how data is used. 24:59.880 --> 25:01.260 Like, how is data collected? 25:01.260 --> 25:02.380 How is it used? 25:02.380 --> 25:03.620 How is it stored? 25:03.940 --> 25:05.500 And how are you deleting it? 25:05.500 --> 25:09.480 I think that we're seeing a lot of progress in this area. 25:10.220 --> 25:21.720 At Google, I think it was about a year ago, we had this new feature, so private set intersection, which for the cryptography community is an amazing, cool, sexy result, and they want to be talking about it. 25:21.720 --> 25:29.180 But usually you'd see that in an academic paper, and it would go to the academic community, folks would talk about it, and it'd be a neat talking point. 25:29.480 --> 25:36.780 I was super excited to see that this one innovation had basically like a full-com strategy. 25:37.040 --> 25:46.720 So it was an academic paper, it was a blog post, it was a kind of infographic diagram to explain what it meant, plus a YouTube video. 25:46.720 --> 25:49.100 And that to me was great. 25:49.100 --> 25:51.440 It's basically, who's the audience? 25:51.440 --> 25:55.400 Like, who are the audiences who need to see this information and understand it? 25:55.500 --> 25:57.840 And how do we cater it to them? 25:58.300 --> 26:01.560 So I would love to see a lot more like that. 26:02.480 --> 26:11.440 I mean, like I said, I've worked in privacy and use human-centered privacy for a long time, and I don't read privacy policies. 26:12.020 --> 26:14.840 Like, I have better things to do than read a privacy policy. 26:14.840 --> 26:16.260 I never read them. 26:16.860 --> 26:22.180 And like, I've read a few because I'm supposed to, because I should know what they look like. 26:22.460 --> 26:23.840 But I have better things to do. 26:23.840 --> 26:27.040 So I'm all for better ways of communicating. 26:28.200 --> 26:33.500 I feel like I have like a habit of also wanting to figure out how to teach these concepts. 26:33.500 --> 26:45.540 And I think taking more of an education approach, taking the burden of explaining complicated topics is more of... it's what we should be doing as the creators of these systems. 26:47.020 --> 26:49.510 So I think there's a lot of room for improvement. 26:51.640 --> 26:54.460 Yeah, I agree with that. 26:54.460 --> 27:07.340 I think policies are... because they're legally required and they're highly regulated, companies are incentivized to be as broad and non-committal as possible in those privacy policies. 27:07.340 --> 27:11.220 So what you end up getting is these like long walls of legal text. 27:12.840 --> 27:17.060 And effectively, you need a law degree to like decipher it properly. 27:17.800 --> 27:31.540 So if you're thinking about compliance, and I think like one theme of this panel is that compliance is like the bare minimum, but companies now should be thinking about trust as well and like how to go past that bar. 27:31.760 --> 27:42.580 And policies just don't cut it because they are regulated like documents and it's very hard to find a compromise within the privacy policy. 27:42.780 --> 27:49.420 You may be able to have a more accessible policy, but at the end of the day, companies are indeed like incentivized to be non-committal there. 27:50.220 --> 27:56.980 So I think, and this is something we think about a lot, is like how can we help educate and inform users? 27:57.100 --> 28:07.020 How can we help our customers actually create a place where users can go to understand the data practices of this company? 28:07.220 --> 28:10.500 What type of data specifically are they collecting? 28:10.500 --> 28:12.380 How are they using that information? 28:12.460 --> 28:14.640 Who are they sharing it with, if anyone? 28:15.940 --> 28:23.140 And then also, there's this shift now because historically, a lot of privacy law is around disclosure. 28:23.320 --> 28:26.380 Modern privacy law, a lot of it is about choice as well. 28:26.420 --> 28:31.720 And so the choices should be in a more interactive control panel. 28:31.720 --> 28:33.060 We are in the modern world. 28:33.060 --> 28:34.900 We don't need to use documents only. 28:34.900 --> 28:38.520 We can also use things that are like actual user interfaces. 28:38.860 --> 28:53.480 And so when we're talking about choice, we're often talking about control panels and ways to actually show users that you had these choices and there are buttons you can click that will change how we operate our platform and will have implications on your privacy. 28:53.880 --> 28:56.900 So there's a lot of exciting stuff that we see there. 28:57.040 --> 29:02.880 And just generally sort of tailoring disclosures to the consumers. 29:02.880 --> 29:10.060 Some people want to drill all the way deep and get a really granular idea of what the company has. 29:10.060 --> 29:16.760 So being able to sort of layer information and let people sort of zoom in is really exciting there. 29:17.160 --> 29:40.040 And then the other thing that we've been working on in this space is actually contextualizing a disclosure by leveraging the right of access as well where when you say you collect this type of information, you can actually show them a copy of that information because the user can click a button that's going to let them download it anyway. 29:40.080 --> 29:49.600 So if you can be proactive and say here's a specific example of like an analytics event that we would track about you, there's really exciting like new ways of doing that. 29:49.600 --> 29:54.240 So yeah, I think companies can go really above and beyond here. 29:54.240 --> 30:06.100 There's a lot of opportunity in this new sort of modern privacy world and a lot of opportunity to start moving from just disclosure and compliance to increased choice and trust building. 30:08.270 --> 30:15.130 Yeah, I'll just, you know, add to a different aspect of the challenge with the privacy policies. 30:15.130 --> 30:30.430 As someone who used to own our sort of custom website that our privacy policy used to sit in, I can say that like a lot of different folks contribute to writing that policy on the legal side. 30:30.430 --> 30:43.270 So think of think of it like this lengthy novel that has many different authors who are using words that nobody understands because they have to, you know, based on, you know, laws that are like all over the world. 30:43.350 --> 30:59.990 And then communicating to, you know, engineers and product folks like myself, the fact that like, we've got to deploy this in, you know, 30 different languages on different dates because of different laws in these different markets. 30:59.990 --> 31:08.810 And we need specific appendages for these five countries that are unique to certain regulations that have emerged there. 31:08.810 --> 31:19.830 And by the way, in three months, there's going to be these two launches that are coming up that we need to make an update to the policy for, but we're not ready to do that right now. 31:19.830 --> 31:31.310 And we've got to work with, you know, you've got to work with your comms team on your strategy for letting your customers know about all these changes in a way that isn't overwhelming. 31:31.910 --> 31:43.730 Because if you if you get an in-app notification, an email, you know, like five times a year about a company's privacy notice or privacy policy update, you're going to be scared. 31:43.730 --> 31:49.410 You're going to have a lot of questions and be wondering, like, what that company is doing with your data. 31:49.730 --> 31:58.410 So I just want to describe for folks, like, that art of, like, getting this right and appreciate that aspect of it. 31:58.730 --> 32:02.410 But I also agree with what Maritza and Ben were saying. 32:02.450 --> 32:16.290 And, you know, what I add there is, you know, the reality around these privacy policies is they're required, they're here to stay, and they're going to get increasingly complex because of emerging laws around the world. 32:16.290 --> 32:18.930 We already saw what happened with GDPR and CCPA. 32:18.930 --> 32:21.430 We have LGBT coming up. 32:22.170 --> 32:38.410 And that means you do need, again, you do need a user-focused experience that's written in easy-to-understand language that explains to users their rights and the controls and choices that are available to them to exercise those rights. 32:38.410 --> 32:41.890 And I think that's a different place than your privacy policy. 32:42.390 --> 32:44.070 It could be another website. 32:44.070 --> 32:46.250 It could be things that you do in-app. 32:46.250 --> 32:56.650 It could be, you know, norms or best practices that you follow in all of your consent models whenever you're looking to collect or make use of data in a different way. 32:56.650 --> 33:14.430 And it could also be thought about deeply with your UX research team as part of the sign-up flow and unique things you might do there in terms of even leveraging animations or video to explain, like, what's going to happen with your information and what your choices are. 33:14.450 --> 33:24.390 So I think there's a lot of actually cool things you can do to explain to users their rights and their roles available to them that go beyond the notice. 33:25.130 --> 33:39.010 And the last thing I would add is you should definitely be doing research to make sure that you're focused on, you know, what matters to your users the most as it pertains to the data that your company collects and uses. 33:39.010 --> 33:41.490 It's certainly not a one-size-fits-all. 33:42.510 --> 33:44.330 Huge plus one to that. 33:44.970 --> 33:49.510 Alice is so contextual and depends on your business, your customers. 33:51.890 --> 34:10.990 Just to add on a little bit to the conversation, even though, again, I'm not building product right now, I'm working on an organization, but from, again, looking at the end users, I think it's truly unfair to ever expect end users to understand or comprehend a privacy policy at scale. 34:10.990 --> 34:15.790 And I think we're kind of all in agreement there that it's not enough to have a privacy policy. 34:16.070 --> 34:29.450 Sort of using the same metaphors that we use in training, you know, we say in security education, if you have to provide a training for your tool, you're not going to scale that tool like you would if you don't need a training. 34:29.450 --> 34:31.250 And that's the same thing in this instance. 34:31.250 --> 34:41.910 You're not going to scale people engaging or activating these new rights that they have unless it's truly like embedded in your product or get 34:51.340 --> 34:59.620 interested and actually take a look at what data that is being collected and stored and how it's being stored. 34:59.620 --> 35:02.600 I think the other piece of this for me is also transparency. 35:03.140 --> 35:06.480 Huge bend on what you were saying about building trust. 35:06.480 --> 35:11.080 I think transparency is one way to build this trust, and we need a lot more of it. 35:11.720 --> 35:21.820 Understanding data practices and the governance of data by company is a super important part of that, but shouldn't be buried, I would argue, in a privacy policy. 35:21.820 --> 35:30.880 That should be more, you know, bubble that up at a user level that a user would come across in their engagement with your product. 35:33.100 --> 35:52.580 So, I want to move the discussion now into some of the specific usability challenges that we see in the technical and in UX blockers that actually can prevent us from offering better accessibility and usability for privacy enabling tools or even processes, 35:52.580 --> 35:52.960 right? 35:52.960 --> 36:12.400 So, I want to dig a little bit deeper into some examples of difficult either compromises or tradeoffs, you know, that we've made in our careers to either meet these legal obligations at the expense of user experience accessibility or usability, as well as, 36:12.400 --> 36:32.420 you know, the limitations of UX research and testing and where we might be able to grow in that area as well to make sure that we're not excluding, you know, specific populations and groups of people so that everybody really can have access to their data rights. 36:32.620 --> 36:50.240 So, Zach, let's start with you just in terms of what are some of those, you know, specific usability challenges or some of the tradeoffs that you've seen in trying to build a product that's both compliant with these regulations but also easy to use so people can exercise their rights? 36:50.880 --> 36:59.820 Yeah, what that immediately makes me think of is when we what we went through as a company with managing and navigating GDPR. 36:59.820 --> 37:25.460 So, when that law came into effect, it was so large and so comprehensive and, you know, something we hadn't seen before that we spent that entire year just heavily focused on complying with erasure and different export kind of laws that were or articles that were part of that law and updating our consent UI across all apps and services. 37:25.580 --> 37:34.860 And so, this meant there wasn't a whole lot of time left after that for our team to spend on user-facing experiences. 37:35.120 --> 37:38.320 And I don't think that was the intention at all. 37:38.980 --> 37:46.880 And the interesting thing is that as a team, we felt that we had made a huge amount of progress for user privacy. 37:46.880 --> 37:53.560 And this is 2018 and according to the regulations, but that our users couldn't necessarily see it. 37:53.560 --> 38:04.500 And we didn't necessarily feel comfortable, you know, bragging about it or doing a whole lot of comms around it because we were required to make a lot of these changes by law. 38:04.500 --> 38:09.220 So, it wasn't us, you know, going beyond expectations. 38:09.220 --> 38:25.340 But we had this realization that ultimately if our users can't see the changes we're making or if there are products that users won't take advantage of very often, it may not move the needle on trust, right? 38:25.340 --> 38:28.800 Which has come up many times in this conversation. 38:28.800 --> 38:37.840 It may not move the needle as much on trust as new user-facing controls or products that are really designed to drive that trust home. 38:37.840 --> 38:41.180 So, for me, that's really the trade-off. 38:41.820 --> 38:46.620 And, you know, in any given year, it's something that we actively think about. 38:46.700 --> 38:54.260 We want to make sure to comply and do our best to comply with every new law that's coming out around the world. 38:54.260 --> 39:05.300 You know, we're a global company and we'd like to think that we have, you know, good and always improving relationships with different regulatory bodies around the world. 39:05.300 --> 39:24.560 But we also need to make sure that our roadmap consists of enough things that are going to raise transparency, improve choice, and communicate to our users how much that we value their privacy and want them to really understand it. 39:24.560 --> 39:27.060 So, that's what I'd say the trade-off is. 39:29.390 --> 39:32.610 Megan, anything that you would want to add from your perspective? 39:33.750 --> 39:34.690 Sure. 39:34.690 --> 39:50.690 So, very, again, not coming from a private sector background, I can't help but wonder what types of incentives there are for companies to refine that user experience to activate, for users to activate certain privacy rights. 39:50.690 --> 40:02.430 And so, I'm kind of super interested to hear about, from other panelists, actually, to understand how companies are working through that whole flow and that whole user experience. 40:02.430 --> 40:15.090 Because if you make that a blocker, you're not going to get, you know, people activating in trying to, you know, put forward these particular rights that are now granted to them, even if they're aware of it. 40:16.130 --> 40:23.310 I think, you know, understanding how much user testing is actually going on there, it would be super interesting to me. 40:23.310 --> 40:49.730 I think from, again, from a sort of trade-off point of view here, I think really understanding, again, knowing your audience, knowing your users, and having real recipes to give to the people building products on how to operationalize this sort of UX experience when it comes to privacy, 40:49.730 --> 41:05.330 be it due to compliance or regulation reasons, or data protection reasons, or, you know, other security privacy reasons that you might be trying to initiate at your particular company. 41:05.590 --> 41:09.230 So, I super want to listen to some of the other perspectives. 41:12.340 --> 41:32.120 So, from my perspective, one of, you know, I'm just like, I'm running through some examples where I'm thinking about the process that I've gone through when being brought on to convince a team basically, you know, like, I'm always selling my stakeholders on this is the research that we should do. 41:32.680 --> 41:33.880 At some point, I'm tapped. 41:33.880 --> 41:35.520 It's like, here's the team that's getting together. 41:35.520 --> 41:38.100 We're going to be delivering this thing. 41:38.320 --> 41:40.160 I hear what we're supposed to do. 41:40.160 --> 41:42.320 You know, we have UX involved, of course. 41:42.400 --> 41:43.860 Merit's a privacy person. 41:43.860 --> 41:45.740 Let's, you know, get the UXR. 41:45.740 --> 41:46.580 What do we do? 41:47.560 --> 41:50.460 And it's changed a lot over the last couple years. 41:50.460 --> 42:14.840 I'm thinking back to stuff that happened maybe back in, um, where at first, one of the kind of blockers that I saw to doing better usability work for privacy controls was to open people's minds to actually think about making this a part of their actual objectives. 42:14.840 --> 42:28.880 Like, explicitly acknowledging that designing a good experience for consent or for disclosure or for a data-intensive tool is a thing that requires going beyond the privacy policy. 42:29.500 --> 42:37.380 And that's a bit of a, I mean, that's a conversation that you need to, you're changing, you're kind of, you're working on your persuasion skills, right? 42:37.380 --> 42:41.040 Like, it's not something that's intuitive to everybody that this is a thing that should happen. 42:41.120 --> 42:47.980 So, I'm actually really thrilled to be on a panel where everybody already agrees that this is a thing, because sometimes that can be some work. 42:48.220 --> 42:49.780 But, um, I think that's the first part. 42:49.780 --> 43:02.180 It's basically, like, convincing stakeholders that we can and should be setting objectives and metrics and things to measure in user testing that are related to comprehension, understandability. 43:02.740 --> 43:13.740 At various points in my career, I'm remembering one time where I basically convinced a team to take on a metric of people aren't upset. 43:14.760 --> 43:18.380 So, as a usability person, that is an incredibly low bar. 43:18.380 --> 43:26.780 Like, ISO has a definition of usability, and it is people can effectively, efficiently, with satisfaction, complete a task. 43:28.260 --> 43:34.440 To set your bar for satisfaction at they're not pissed off, kind of low. 43:34.820 --> 43:35.980 But it's a start. 43:35.980 --> 43:40.460 So, it's basically, like, setting at, like, maybe, you know, it's a neutral response. 43:40.460 --> 43:44.520 Maybe it's, maybe, sure, they're a little bit, like, grumpy about it. 43:44.520 --> 43:49.100 But through the flow, they kind of, like, they get why you've got to ask about it. 43:49.640 --> 43:50.900 So, that's one step. 43:50.900 --> 43:54.740 Basically, like, convincing a team that these are things that we can and should measure. 43:55.300 --> 44:04.440 And then from there, I'd say recently, like, that was years ago when that was kind of a battle that I needed to, not a battle, but those years ago when that was kind of a conversation. 44:04.440 --> 44:09.580 And now I think it's more having and setting high expectations. 44:10.720 --> 44:14.040 So, when I hear folks that are, like, oh, people don't care. 44:14.360 --> 44:16.560 I'm, like, no, you're telling me you don't care. 44:16.660 --> 44:19.160 I've never seen any evidence that people don't care. 44:19.280 --> 44:23.420 So, then how do we turn this into, you know, again, we're working for a specific group. 44:23.420 --> 44:25.780 We have users, they have expectations. 44:25.780 --> 44:34.080 If they don't have expectations, and they do, so that's kind of an easy thing, then, you know, what's the user research we can pull into to support that point? 44:34.200 --> 44:42.180 And it's just kind of, I guess, forcing the conversation, which can be difficult. 44:42.180 --> 44:58.500 I've learned over time that there's a lot of work that the privacy community can do to establish a better baseline for having productive conversations about how data is used to every single audience across the board. 44:59.040 --> 45:12.880 And so, even when I go into a room and I start talking with new stakeholders on a new project, I kind of put myself in a position of, like, let's just assume that nobody really, like, let's pretend people kind of don't know that much about privacy. 45:13.100 --> 45:19.680 So, let's start the conversation from building up our shared vocabulary and building up our shared understanding of what's needed here. 45:20.380 --> 45:27.680 And that's been an interesting kind of, I guess that's, like, my continual beginner's mindset as I work through these things. 45:29.560 --> 45:37.380 Yeah, maybe just to follow on to the definition of usability and whether users are upset. 45:37.480 --> 45:48.700 I've submitted so many data rights requests in my lifetime, like, trying to download my data, erase my data from various companies, and it certainly does not meet the satisfaction bar. 45:48.820 --> 45:56.600 It can feel like you're going to the DMV, where you are, you know, calling support, emailing them, going back and forth. 45:56.600 --> 45:59.080 I have to, like, take selfies with my government ID. 45:59.080 --> 46:02.580 I have to, like, email all this, like, documents back and forth. 46:02.580 --> 46:06.720 And it can take, like, over a month to get access to your data. 46:07.100 --> 46:12.120 And frankly, that just, like, builds a world where data rights are not exercisable, even if you have them on paper. 46:12.800 --> 46:19.080 It's just... it's a terrible user experience right now, and I think people are starting to wake up to that. 46:19.360 --> 46:26.880 And as these laws just roll through new regions, that volume is going to keep going up for companies, and the burden on both sides is going to go up. 46:27.300 --> 46:30.220 So I think that's a huge area for UX. 46:30.780 --> 46:32.840 I'll talk about maybe two other things here. 46:32.840 --> 46:53.320 I think one really interesting, like, sort of paradox in this UX is there's a trade-off between being concise and being complete in your disclosures, where everybody wants to have a very simple, distilled disclosure around privacy. 46:53.320 --> 46:56.200 But more often than not, there's more to the story. 46:56.200 --> 47:03.420 And actually, like, making that concise is a very hard task. 47:04.660 --> 47:07.320 And yeah, so that's one thing we see a lot. 47:07.320 --> 47:27.020 And then the other is just that disclosure of both the choices that you have, as well as how the company is using data, kind of has to happen both at the right time when it's, like, contextually happening, as well as being... and it also has to be available to rediscover later. 47:27.020 --> 47:29.520 So that means that companies have to do two things. 47:29.520 --> 47:39.620 It means that when users are using a new feature, you have to disclose the impact of using that feature and typically offer a consent preference there. 47:39.920 --> 47:46.280 But then also, you know, after that user has passed that point, they have to be able to rediscover it. 47:46.280 --> 48:00.360 So you should also offer a privacy center as a company to your customers, where they can go back and see what your data practices are in plain terms, as well as any prior consents you've set. 48:00.360 --> 48:12.140 And hopefully, in this privacy center is also where you have a much easier experience to exercise those data rights and not have to sit on the phone with support for hours. 48:14.120 --> 48:16.940 Oh, and cookie banners should definitely go away. 48:16.940 --> 48:19.700 Those are more useless than privacy policies. 48:20.880 --> 48:24.560 Actually, Ben, that brings us to our closing topic. 48:24.560 --> 48:27.160 And we have about five minutes left. 48:27.240 --> 48:41.620 So with the last remaining minutes that we have, I do want to ask just for your closing advice on some pragmatic steps that we can take to actually up-level some of our industry practices. 48:41.760 --> 48:47.820 So, you know, what are some of those like immediate pain points that we can address? 48:47.900 --> 48:54.900 And perhaps, you know, like things that we could actually do right now, you know, while things are still on fire. 48:55.620 --> 48:57.260 Maritza, let's start with you. 48:58.960 --> 48:59.840 Yeah. 49:00.000 --> 49:06.720 Some of the stuff that I would love to see, I would love if everybody across the board in general was just like, be curious. 49:06.780 --> 49:08.200 Be curious about data. 49:08.200 --> 49:10.280 Be curious about what your users' expectations are. 49:10.280 --> 49:11.620 Be curious about their attitudes. 49:11.620 --> 49:13.380 Be curious about how you could do better. 49:13.420 --> 49:20.800 Be curious about, you know, how do you reveal like the things that you thought were kind of unsaid or didn't need to be said? 49:20.860 --> 49:22.600 And maybe consider saying them again. 49:23.660 --> 49:34.900 And then kind of my big hope and dream for all of data use and data conversations is if we could get to a point where we had simple answers for simple questions. 49:37.720 --> 49:39.200 Plus one to that. 49:39.360 --> 49:40.580 Ben, what about you? 49:42.780 --> 49:43.700 Sure. 49:43.700 --> 49:48.500 I would say first, realize that consumers care deeply about privacy. 49:48.520 --> 49:51.040 They may not have four years ago, but they do now. 49:51.040 --> 49:53.260 And this is a huge part of your trust equation. 49:53.840 --> 49:57.800 And then use that as a way to go beyond compliance. 49:57.800 --> 50:11.280 We've talked about this, but there are some simple things that you can do, whether that's getting a privacy center to be more upfront with your users about how you're using that data. 50:11.400 --> 50:16.320 Offer a lot of consent preferences and be very clear and concise there. 50:17.180 --> 50:22.000 And make data subject requests easier to exercise for your users. 50:23.960 --> 50:24.780 Great. 50:24.780 --> 50:26.720 Megan, any thoughts that you'd want to add? 50:27.400 --> 50:31.060 Just echoing that people absolutely care about privacy. 50:31.060 --> 50:37.860 I've never met, you know, a community member who I've worked with who's like, nah, I don't really care about privacy. 50:37.860 --> 50:40.360 They've just got a lot of other things that they care about too. 50:40.360 --> 50:47.920 And I think you put it really well in talking about prioritization of needs that different users have. 50:47.920 --> 50:59.120 Echoing again, figuring out how we can incentivize companies to test their UX on their exercising their data rights workflows. 50:59.120 --> 51:03.920 I think I'm not convinced right now that they're properly incentivized to do this. 51:03.920 --> 51:06.120 And it's only like a self-reinforcing thing. 51:06.120 --> 51:10.080 If you don't have people exercising this right, you kind of reinforce what people don't care. 51:10.080 --> 51:16.200 So I think this is sort of an interesting problem that where a lot of work can be done. 51:16.200 --> 51:29.260 I think specifically looking at, you know, who's doing it well in industry right now and getting those case studies and those stories shared and told more publicly, I think will help drive that conversation. 51:30.380 --> 51:31.140 Great. 51:31.140 --> 51:34.400 Zach, can you bring us home with some final thoughts? 51:35.280 --> 51:36.700 I can, I can. 51:36.700 --> 51:46.940 So I think, you know, sticking with this point of, you know, privacy is certainly important for our users, but it may be more important at certain moments, right? 51:46.940 --> 51:51.160 When you're using Google, what's most important to you is how well search works, right? 51:51.160 --> 51:54.720 When you're using Uber, it's getting from A to B. 51:54.720 --> 52:02.800 When it's Instagram, it's, you know, how well can I, you know, you know, share photos and view others' photos. 52:02.800 --> 52:07.020 But there will be moments when privacy becomes the most important thing. 52:07.020 --> 52:15.460 And in those moments, it has to be there and it has to work for you, or else you're scared, you're frustrated, you're mad. 52:15.460 --> 52:19.580 I read, you know, those customer complaints from our users. 52:19.720 --> 52:23.540 And trust me, privacy is definitely important to them. 52:23.540 --> 52:32.600 So what I would say is, you know, one longer term solution I'm thinking about is contextual and personalized privacy experiences. 52:32.600 --> 52:36.000 So that privacy shows up when you need it. 52:36.000 --> 52:56.100 Maybe it's when you take certain actions on our platform, or you write into customer support with certain kinds of questions, we can then surface experience to you, experiences to you, based on your activity on the platform, and not only rely on you to go to our privacy policy or even a privacy center, 52:56.100 --> 53:03.040 because as we know, those are not going to be placed on the very front of an app or the front of an experience. 53:03.040 --> 53:06.000 It's going to take a few clicks to get there. 53:06.600 --> 53:08.340 So that's one thing I'd say. 53:08.340 --> 53:23.900 The other thing that I think is going to be important as we move forward is a point of view or best practices for what privacy will look like in this world where it is governed a lot more by machine learning and AI, right? 53:23.900 --> 53:31.380 Like how do we offer choice and transparency in this world that we're starting to move into? 53:31.380 --> 53:51.820 Because it's not going to be, you know, the same as it is today with just like a bunch of toggles on a page, like trying to deliver that in a much more complex environment where we're actually probably going to collect more data from you and to offer you richer experiences. 53:51.820 --> 53:58.480 You know, you can just look at what the Googles and Amazons of the world are doing with, you know, Alexa, right? 53:58.480 --> 54:06.300 And with products like that, and they're amazing and people are going to want them, but how do you offer privacy in those contexts? 54:06.300 --> 54:08.960 And this is a bit of a head-scratcher for me, right now. 54:08.960 --> 54:13.360 So I think we need to be forward-looking and in that regard. 54:13.360 --> 54:15.260 So that's what I'd leave us with. 54:17.020 --> 54:17.740 Great. 54:17.740 --> 54:18.600 Thank you so much. 54:18.600 --> 54:30.400 And thank you to each of you for sharing your expertise and being part of this conversation today to share with the Crypto and Privacy Village during DEF CON's safe mode. 54:30.400 --> 54:34.540 I hope you're enjoying the rest of the virtual event. 54:34.540 --> 54:44.980 And for those that are watching, I encourage you to jump into the Discord server if you want to have a conversation with our speakers and with the broader privacy community about some of these issues. 54:44.980 --> 54:51.080 So thank you everyone for watching and thank you again to our panelists for sharing their expertise with us today. 54:51.200 --> 54:52.040 Take care.