[00:02.060 --> 00:07.980] So welcome everyone to our conversation about consumer privacy, usable privacy, [00:07.980 --> 00:13.860] how do we actually build privacy for consumers in a way that they can actually exercise their [00:13.860 --> 00:18.680] data rights. Unfortunately, one of the trends that we're seeing as laws and regulations become [00:18.680 --> 00:25.220] more complex, it's increasingly difficult for companies to provide the ability for consumers [00:25.220 --> 00:29.820] to actually exercise those rights. And so one of the things that we are concerned about as [00:29.820 --> 00:34.800] privacy professionals is making sure that consumers actually have the ability to exercise [00:34.800 --> 00:39.960] those rights. Otherwise, they really don't get the full protections that are granted to them [00:39.960 --> 00:46.160] under the law, depending on where they live. By way of introduction, my name is Melanie Ensign. [00:46.160 --> 00:52.680] I am the founder and CEO of a new company called Discernible. It's a communications firm dedicated [00:52.680 --> 00:59.140] specifically to security, privacy, and risk organizations. And I am very excited to be joined [00:59.140 --> 01:06.360] by a number of esteemed colleagues today who have a lot of experience, not just in security and [01:06.360 --> 01:14.080] privacy, but in building tools and in being part of the process for building and thinking about [01:14.080 --> 01:21.920] how do we make privacy a user-centric experience. So a couple of expectations setting things right [01:21.920 --> 01:28.000] off the front. None of us here are lawyers. Nothing we say is going to be legal advice. [01:28.000 --> 01:35.300] We're also not legal experts. And so we're not going to debate the merit of the law or try to [01:35.300 --> 01:42.580] interpret the law. But we are on the receiving end of these legal obligations. And we do have [01:42.580 --> 01:49.220] the professional responsibility to advise or lead the building of their subsequent products and tools [01:49.220 --> 01:55.080] for consumers so that they can exercise those rights. So just a disclaimer up front that none [01:55.080 --> 02:02.060] of us are lawyers. Troll us if you like. There'll be plenty of lawyers in the comment section who [02:02.060 --> 02:07.560] can actually provide the legal guidance that you're looking for. So with that said, I'm going [02:07.560 --> 02:11.940] to ask each of the panelists to introduce themselves and tell us a little bit about [02:11.940 --> 02:17.580] their experience in privacy and specifically their perspective and worldview in terms of how [02:17.580 --> 02:22.740] they approach this particular challenge given their function and expertise. So Megan, let's [02:22.740 --> 02:30.240] start with you. Sure. No trolling, please. I beg you. My name is Megan DeBlois. Super excited [02:30.240 --> 02:34.340] to be on this panel. I'm an information security advisor and technologist over at [02:34.340 --> 02:41.800] Internews. Internews itself is an international non-profit. We work on media development [02:41.800 --> 02:48.160] really focused on empowering local information sources worldwide while connecting and giving [02:48.160 --> 02:54.020] people voice. So that's kind of the area that I tend to focus in. But at the organization, [02:54.020 --> 03:00.000] I wear two hats. So internally, I help improve our information security processes, procedures, [03:00.000 --> 03:07.040] and practices org-wide. And externally, I work with a pretty diverse group of civil society [03:07.040 --> 03:13.860] folks to help improve them on their information security needs from organizational security [03:13.860 --> 03:19.880] assessments to risk reduction strategy design and development to threat analysis and sharing. [03:20.340 --> 03:26.000] A big part of that and a big part of our approach at Internews when it comes to privacy and security [03:26.420 --> 03:30.820] is really trying to keep our users at the center, whether that's going to be an organization or [03:30.820 --> 03:35.960] civil society group that we're working with or the developer communities in the open source world who [03:35.960 --> 03:40.620] were trying to build capacity around their abilities in creating these really important [03:40.620 --> 03:47.040] security and privacy products for really amazing communities, but often who are targeted. [03:48.080 --> 03:49.740] Yeah, and that's me. [03:51.580 --> 03:57.800] All right. Hi, everybody. I'm Ben Brook, the co-founder and CEO of Transcend. So Transcend [03:57.800 --> 04:03.560] is a data privacy infrastructure company. And what we do is we enable other companies to honor their [04:03.560 --> 04:09.320] users' data rights. So users can't really exercise their data rights today, even though it is their [04:09.320 --> 04:15.600] legal right. And you can go to any major company's privacy policy and see that for yourself. So we've [04:15.600 --> 04:22.400] made a series of breakthroughs in tech that enables any company to give their users data rights in a [04:22.400 --> 04:27.500] way that's completely on demand and you should have it in seconds. So what we're doing is we're [04:27.720 --> 04:33.640] replacing a lot of broken internal processes that are both resulting in noncompliance and [04:33.640 --> 04:40.740] and that have been an issue in terms of user experience. So if we're successful, the hope is [04:40.740 --> 04:46.160] that we're going to make it easier for everybody to have the ability to control their personal data. [04:47.940 --> 04:54.760] Hi, everyone. I'm Marietza Johnson. I'm a UX researcher at Google Research. And for about [04:54.760 --> 04:59.360] the past 10-ish years, I've been really focused on this question of how do we make it easy for [04:59.360 --> 05:04.160] people to manage their data and the security and privacy decisions that they're faced with [05:04.160 --> 05:10.460] in their daily lives with the technology that they use. This focus has taken me to a bunch of [05:10.460 --> 05:16.880] different kind of places where I can exercise those interests. Right now I'm at Google Research [05:16.880 --> 05:22.660] working on a tool for what we're calling meaningful control over data in a distributed system. [05:22.880 --> 05:29.120] I tweet a lot. You can see a project description on Twitter if you're interested in that. I think [05:29.120 --> 05:35.860] it's super fun. Also, I used to be on Facebook as a member of their privacy product review process [05:35.860 --> 05:42.020] and have also done some academic work just doing some research on looking at how do people [05:42.020 --> 05:48.360] understand this stuff, attitudes, behaviors, comprehensibility. So glad to be here. [05:50.120 --> 05:57.560] I'm Zach. I lead the global privacy product team at Uber. I focus on building user-facing features [05:58.280 --> 06:05.980] that build trust with our users. I also work on developing systems and services in response to [06:05.980 --> 06:12.660] new and emerging laws to allow users to exercise their rights over those laws. [06:12.680 --> 06:20.340] And the last thing that I'll describe is I guess a little bit different than your traditional [06:20.340 --> 06:26.200] product role because I'm also charged with consulting and reviewing the product launches [06:26.200 --> 06:32.380] of other teams across Uber that pertain to privacy. And that can take on many different [06:32.380 --> 06:40.400] forms from early in the idea phase to just before launch to post-launch and remediation [06:40.400 --> 06:48.680] and anything under that sun. So that's me. Awesome. Thank you so much for being here and [06:48.680 --> 06:57.740] lending us your expertise today. So let's start with operationalizing privacy and in the steps [06:57.740 --> 07:04.580] that actually come after these protections are codified in law. So once a law or new regulation [07:04.580 --> 07:11.160] is passed, particularly those that are then going to require some additional engineering or product [07:11.160 --> 07:17.960] development, there's a different type of skill set that's needed to work alongside our legal [07:17.960 --> 07:24.560] colleagues in order to actually bring that to life. So policy and regulations is one piece of [07:24.560 --> 07:29.320] it. But especially now with the new types of regulations we're seeing, there are very real [07:29.320 --> 07:35.920] technical requirements and changes that have to happen in order for companies to comply with those [07:35.920 --> 07:41.620] new obligations. And I want us to dive a little bit more into what that actually looks like in [07:41.620 --> 07:48.420] practice in terms of these different functions working together. So Ben and Zach, let's start [07:48.420 --> 07:54.980] with you. Based on your experience, and Ben, I know that you're working externally and helping [07:54.980 --> 08:01.440] these companies kind of repair or build some of these broken systems. And Zach, you're obviously [08:01.440 --> 08:07.320] in-house at Uber. What does the typical process look like from the perspective of those internal [08:07.320 --> 08:14.200] privacy product and engineering teams once a protection has been codified in law or in a new [08:14.200 --> 08:23.220] regulatory standard? Sure. So when a new privacy law has been encoded, there's usually a period [08:23.220 --> 08:27.960] before it comes into an effect, which means there's a huge surge of preparatory work inside [08:28.100 --> 08:32.800] a business. And that can include everything from updating contracts and policies, doing risk [08:32.800 --> 08:38.680] assessments, and building frameworks for the company to operate under. And then after the [08:38.680 --> 08:44.480] laws come into effect, there's a few new processes. So of course, as Zach alluded to, new product lines, [08:44.480 --> 08:48.460] you have to make sure that you're applying those frameworks. But what's really unique about these [08:48.460 --> 08:54.120] modern privacy laws like GDPR and CCPA is that there's also this new set of indefinitely recurring [08:54.120 --> 09:00.680] processes around user data rights. And so these are things like the right of access to your data [09:00.680 --> 09:06.960] and the right to be forgotten. So that means companies now have a consistent stream of [09:06.960 --> 09:12.520] requests coming in, typically via support channels. And so that means the support and the legal teams [09:12.520 --> 09:19.260] are now receiving these requests and working to manually authenticate users over an email [09:19.260 --> 09:24.760] conversation. You can imagine how that would be at high risk to social engineering, where users [09:24.760 --> 09:31.540] are making claims to accounts, asking to erase that account or to get access to all of the data [09:31.540 --> 09:38.220] in that account. And then after that's been received, there's also an incredible manual [09:38.220 --> 09:43.680] process that happens inside many companies today, where it's basically the entire org [09:44.380 --> 09:51.020] is going across dozens, if not hundreds, of their data systems and their vendors to go and scrub [09:51.020 --> 09:55.700] that data. And so that ends up being almost every part of the company. And whether it's sales and [09:55.700 --> 10:01.860] marketing or support or analytics, people are logging into their respective tools and scrubbing [10:01.860 --> 10:09.320] data or exporting and emailing it over to legal. And so in short, there's an incredible [10:09.320 --> 10:15.100] amount of very cumbersome and manual processes happening today. And it's sort of like [10:15.100 --> 10:20.560] this temporary solution. It seems like we're in a lot of businesses today. And again, [10:20.560 --> 10:24.070] these processes are still at high risk of social engineering. [10:25.380 --> 10:31.640] Yeah, so I'll jump in there. I think, as has been described, when you're just [10:31.640 --> 10:39.000] starting out and you perhaps don't have that much of an established privacy team, these efforts can [10:39.000 --> 10:44.840] be pretty unstructured and ad hoc. And there's a lot of manual efforts that take place [10:44.840 --> 10:48.520] throughout your company to try to comply with these new laws. [10:48.660 --> 10:56.780] At Uber, we've been working on this for five plus years, if not more, at least as long as [10:56.780 --> 11:03.960] I've been there. And what I would say is to get to the best outcome is definitely a collaborative [11:03.960 --> 11:12.840] process across legal product and engineering. So the way it works for us when a new law comes [11:12.840 --> 11:20.020] into effect, as Ben said, our legal team takes some time to digest that law. And depending on [11:20.020 --> 11:26.160] how complex it is, it could be something as large as GDPR, or one article that's something very [11:26.160 --> 11:34.480] small in a single state or even region that we need to respond to. So that can take anywhere [11:34.480 --> 11:42.680] from a couple of weeks to several months. But eventually, the legal team will get back to us [11:43.350 --> 11:49.980] with a set of legal requirements. My team then, on the product side, reviews those [11:50.600 --> 11:56.190] requirements to convert them into a PRD, or a product requirements document. [11:56.540 --> 12:03.580] In order to do that, we assess the requirements that we receive in kind of three areas [12:04.440 --> 12:10.980] to make sure that we're going to implement and launch something that complies with the law, [12:10.980 --> 12:18.840] but that is also practical. So one of the things we consider is the implications for the user [12:18.840 --> 12:25.040] experience of the requirements that have been shared over with us. Another thing is the feasibility [12:25.040 --> 12:31.400] from an engineering perspective of what's been shared over. And the other thing is the impact [12:31.400 --> 12:38.620] to the business. You know, a lot of times, new and emerging privacy laws might be at odds [12:38.620 --> 12:44.860] with what your team wants to do or your company wants to do on the marketing side, for instance. [12:45.000 --> 12:52.060] If you're providing users with more controls to opt out of ad targeting or something like that. [12:52.060 --> 12:57.460] And so, you know, we are trying to do these things in the right way in compliance with the law, [12:57.460 --> 13:05.300] but also optimize to fit our business, as I said. So once we've kind of rationalized the requirements [13:05.300 --> 13:10.340] we've received and we then have a follow-up conversation with our legal team to make sure [13:10.340 --> 13:15.560] that they are paying with the direction that we're recommending and the timeline we've set forth [13:15.560 --> 13:22.560] for which we could deliver on, we will define a set of metrics or key performance indicators. [13:22.600 --> 13:31.260] And these usually serve the purpose of mitigating the risk of being in non-compliance. [13:31.260 --> 13:38.780] We will design our metrics in that sense, in particular, if it's something to do with [13:38.780 --> 13:46.320] complying with a new law. And then finally, we will have a design review with stakeholders across [13:46.320 --> 13:54.700] Uber that could be impacted by the changes that we're going to make. And those stakeholders will [13:54.700 --> 14:00.340] be from legal, from policy, from other engineering and product teams. And we'll walk through exactly [14:00.340 --> 14:05.800] what's going to change or be introduced into the experience, how it's going to work end-to-end, [14:05.800 --> 14:11.920] and we'll get some feedback. And then we'll be all aligned across the company on moving forward. [14:12.700 --> 14:15.900] And then from there, we just start building. [14:16.240 --> 14:22.560] So that's kind of how it works in-house at a little bit more of a mature privacy operation. [14:23.760 --> 14:30.500] Awesome. Thanks. Megan and Marta, I want to ask you, does the process look any different [14:30.500 --> 14:36.320] from the perspective of end-users or even developers in the field that you're working [14:36.320 --> 14:43.280] with? We've heard from Ben and Zach a little bit, like the chaos-to-order evolution that's [14:43.280 --> 14:49.480] happening inside companies. What does it look like externally to those who are either the [14:49.480 --> 14:55.500] users of these services or perhaps other developers who aren't in the position of [14:55.500 --> 15:05.540] having these more mature privacy organizations? Sure. So if I can go first. Again, I think it's [15:05.540 --> 15:12.000] important to kind of frame up my experiences and observations here. It's really more tailored to [15:12.680 --> 15:18.040] leading and working on open-source projects and working with at-risk communities. [15:19.480 --> 15:23.460] In the past life, I led a project that was really, really aiming at building the capacity [15:23.460 --> 15:27.600] of developers and creating feedback loops among their users, which tended to be more [15:27.600 --> 15:34.400] at-risk communities. So that's kind of where I'm coming from with my input and observation. [15:34.400 --> 15:39.080] From the developer side of things, again, coming from open-source landia, [15:39.080 --> 15:46.240] where you have very few resources, including human capacity, this looks dramatically different. [15:47.000 --> 15:52.380] The vast majority of security and privacy tool developers who I've worked with are super [15:52.380 --> 15:56.280] concerned about privacy. Naturally, they're creating security and privacy tools specifically [15:56.280 --> 16:04.380] to give users more privacy online. And because of this, we see that a lot of them try really hard [16:04.380 --> 16:09.780] to capture the least amount of data as possible. So most of them have no trackers. Most of them [16:09.780 --> 16:21.280] intentionally do not collect user data at a very, very basic level. I think a lot of smaller [16:22.100 --> 16:28.300] businesses, if you're operating with European users, I think that put definitely a lot of strain [16:28.300 --> 16:35.300] on those communities who have very little capacity to begin with. But at the end of the day, [16:35.300 --> 16:41.600] I think the general thought process around privacy in those communities is sort of it's an all or [16:41.600 --> 16:49.680] nothing. I'm either going to... in the open-source community, in my observation is that they just [16:49.680 --> 16:55.420] really don't want to collect anything. And that impacts them, not just from a UX perspective, [16:55.420 --> 17:00.160] because they're not able to make really important insights to improve their tools and roadmap to [17:01.260 --> 17:06.320] actually make insightful decisions on the usability front. And I don't think it's an [17:06.320 --> 17:12.080] all or nothing game. One of the projects that we've been supporting is a project by the Guardian [17:12.080 --> 17:16.940] Project who has created an amazing and is continuing to create an amazing tool for [17:16.940 --> 17:23.540] Android applications called Clean Insights to be more plug-and-playable with these open-source [17:23.540 --> 17:27.640] tools or other products that want to be more privacy-respecting in the analytics that they're [17:27.640 --> 17:34.140] capturing. I think more things like this will be really important as more legislation and more [17:34.140 --> 17:41.320] regulation starts to happen around the globe. From an end-user perspective, I think that [17:41.900 --> 17:45.580] for me, and I don't think this is really specific to civil society, I think there's [17:45.580 --> 17:49.500] just still a really large gap in user understanding of their digital rights. [17:49.680 --> 17:55.460] So GDPR happened, I would say, you know, the vast majority of folks around the globe probably [17:55.460 --> 18:02.080] didn't know exactly what that means for them as a user. And the before and after that they [18:02.080 --> 18:06.380] really saw was they got a lot of emails in their inboxes saying privacy policies were changing for [18:06.540 --> 18:11.800] a lot of different companies. And that's sort of like a very end-user perspective of, you know, [18:11.800 --> 18:18.120] not truly knowing exactly how to engage with or activate these new rights that they [18:18.120 --> 18:24.820] might have. I think the other really important thing to note is that for a lot of at-risk communities [18:24.820 --> 18:31.240] or targeted groups that are using products, their calculus isn't necessarily privacy [18:32.000 --> 18:37.060] from, you know, companies is at the top. They're really threat modeling and trying to understand, [18:37.060 --> 18:43.140] you know, what it is I'm trying to protect here. And, you know, they're building [18:43.140 --> 18:48.020] on this trade-off between sort of, do I really need this really important security [18:48.740 --> 18:54.060] encryption protocol being used in this particular product versus a more privacy [18:55.000 --> 19:03.140] preserving application that, you know, isn't going to be used as much in my network or it's difficult [19:04.720 --> 19:12.040] to use. So it's more of a threat modeling challenge rather than a sort of privacy calculation. [19:13.020 --> 19:18.840] And our approach at Internet News has been really pragmatic in trying to, you know, [19:18.840 --> 19:24.220] get the individual, get the community, get the organization to be more secure today than they [19:24.220 --> 19:29.960] were tomorrow. And that's the ultimate priority from an end-user's perspective. [19:32.040 --> 19:35.800] I love that as a build into how I would be thinking about it because I feel like [19:35.800 --> 19:40.840] product teams need to really be thinking about the threat model and how the different groups [19:40.840 --> 19:46.940] they aim to serve may have different needs and basically taking care of, you know, [19:46.940 --> 19:52.280] as much as they can. But then inevitably we do get into situations where individuals are making [19:52.280 --> 19:58.540] their own decisions about, you know, which setting to use, which control do I consent to this? [19:58.840 --> 20:03.000] And to me, I always think about it, I want to start from the human-centered perspective. [20:03.360 --> 20:07.820] Like, we can't assume that people are reading privacy policies. We can't assume that people [20:07.820 --> 20:12.100] are seeing all the great material that we put out when we're promoting a new privacy feature [20:12.100 --> 20:16.400] or a new security feature. So then if you're starting from the point of view of the thing [20:16.400 --> 20:22.800] that they see is the UI and the user experience, what is that? How are they using the tool? [20:22.880 --> 20:28.700] So, of course, we're always mindful. You're diligent in implementing controls that respect [20:28.700 --> 20:33.840] the regulations and meet the regulations. I feel like that's one way to look at it. But then if [20:33.840 --> 20:39.900] jump over to the human-centered point of view, you start to ask different questions. So then for [20:39.900 --> 20:45.520] the context that you're building a product or a service, how is it being used? Like, who's using [20:45.520 --> 20:49.900] it? How are they using it? What are their expectations? What's their attitude toward the [20:49.900 --> 20:56.040] data use? And how do you design an experience that matches that as best as you can? I love the way, [20:56.040 --> 20:59.980] Megan, that you alluded to kind of the difficulties that we have right now where [21:00.540 --> 21:06.340] in some situations, it's quite a complicated system. How do you communicate succinctly and [21:06.780 --> 21:10.560] comprehensively to somebody about all the ways in which data is collected and used? [21:11.520 --> 21:16.760] Of course, people, I feel like we constantly hear things like, you know, oh, people don't care. If [21:16.760 --> 21:20.680] they cared, they wouldn't do this. If they cared, they wouldn't do that. But privacy and security [21:20.680 --> 21:25.960] aren't the top level concern. Just because they're not your top priority doesn't mean you don't care. [21:25.960 --> 21:30.540] So maybe it's you care more about something else right now, or maybe you'll care more about that [21:30.540 --> 21:34.920] at a later point in time. So then how do you kind of take all these different perspectives and put [21:34.920 --> 21:40.100] it into a UX that makes sense? So I'd say that's where my role comes in. So I'm always interested [21:40.100 --> 21:44.540] in, you know, how can we take a human-centered approach? How can we ask these questions about [21:44.540 --> 21:50.540] what do people need, and then design a user research program around evaluating whether or [21:50.540 --> 21:56.980] not we've met the goals there. So it's building on top of regulation, hopefully always going well [21:56.980 --> 22:03.120] beyond, but definitely something that, you know, is a constant work in progress. [22:04.380 --> 22:08.420] Yeah, one of the things that I've often advised companies that I've been working with is [22:08.420 --> 22:16.540] you don't get credit for hitting the legal minimum, you know, and truthfully, you're, [22:16.540 --> 22:20.620] you know, Marta, I think you said it best, like the people that you're serving are actually [22:20.620 --> 22:25.760] expecting more from you because they're expecting that you know better than even the lawmakers in [22:25.760 --> 22:31.160] terms of how to build better products and better experiences. So I want to shift gears [22:31.820 --> 22:37.100] just briefly to take a moment to talk about privacy policies and kind of why they're so [22:37.100 --> 22:43.420] unpopular and seem to be a pretty focused target for a lot of public scrutiny, which we could [22:43.420 --> 22:52.080] debate all day whether or not that is deserved or not. But I think the problematic aspects of [22:52.080 --> 22:58.720] privacy policies are often interpreted publicly as, you know, a company not caring about privacy [22:58.720 --> 23:05.240] or not wanting to inform consumers. And, you know, we've all worked on this challenge for years. [23:05.240 --> 23:10.580] We know that that's not true. We know that the problems that exist in privacy policies are not [23:10.580 --> 23:17.300] reflective of a company's actual commitment and investment level in privacy. And even [23:17.300 --> 23:22.840] companies with strong privacy commitments and protections have difficulty creating policies [23:23.510 --> 23:31.760] that consumers can easily access and understand. And so I want to ask each of you, and Marta, [23:31.760 --> 23:36.940] you know, I'd love to start with your thoughts just in terms of what is the actual intent of [23:36.940 --> 23:42.720] these policies? And should we still be relying on this type of documentation to inform and educate [23:42.720 --> 23:50.680] consumers about the data practices of the different businesses and services that they use? I've even [23:50.680 --> 23:57.180] heard from a couple of folks in the EU that, you know, terms of service and privacy policies [23:57.640 --> 24:03.640] are no longer going to be considered sufficient in terms of informed consent because of how [24:03.640 --> 24:10.320] complicated they are for consumers to understand. I think it is a valid argument that, you know, [24:10.320 --> 24:16.300] this very long legal document cannot... nobody realistically expects that consumers are reading [24:16.300 --> 24:21.700] that and understanding it and processing what that means for them. So obviously, again, since [24:21.700 --> 24:26.340] none of us are lawyers, without getting into all the legal definitions of consent and informed [24:26.340 --> 24:32.400] consent and all of that, what are your thoughts in terms of how we could, you know, either improve [24:32.400 --> 24:38.380] policies or perhaps look at other channels and avenues through which to educate consumers about [24:38.380 --> 24:44.720] data practices as well as their rights? Okay, so from my previous answer, you won't be surprised [24:44.720 --> 24:50.200] to hear that I think you could do a lot more than a privacy policy. And there's a big opportunity to [24:50.780 --> 24:56.280] do a better job of both crafting, living, and communicating your privacy narrative. And not [24:56.280 --> 25:02.380] just a privacy narrative, but like, how data is used. Like, how is data collected? How is it used? [25:02.380 --> 25:08.760] How is it stored? And how are you deleting it? I think that we're seeing a lot of progress in this [25:08.760 --> 25:15.260] area. At Google, I think it was about a year ago, we had this new feature, so private set [25:15.260 --> 25:20.660] intersection, which for the cryptography community is an amazing, cool, sexy result, and they want to [25:20.660 --> 25:25.380] be talking about it. But usually you'd see that in an academic paper, and it would go to the academic [25:25.380 --> 25:31.200] community, folks would talk about it, and it'd be a neat talking point. I was super excited to see [25:31.200 --> 25:39.020] that this one innovation had basically like a full-com strategy. So it was an academic paper, [25:39.020 --> 25:45.760] it was a blog post, it was a kind of infographic diagram to explain what it meant, plus a YouTube [25:45.760 --> 25:52.500] video. And that to me was great. It's basically, who's the audience? Like, who are the audiences [25:52.500 --> 25:59.060] who need to see this information and understand it? And how do we cater it to them? So I would [25:59.060 --> 26:06.380] love to see a lot more like that. I mean, like I said, I've worked in privacy and use human-centered [26:06.380 --> 26:13.780] privacy for a long time, and I don't read privacy policies. Like, I have better things to do than [26:13.780 --> 26:20.300] read a privacy policy. I never read them. And like, I've read a few because I'm supposed to, [26:20.300 --> 26:25.460] because I should know what they look like. But I have better things to do. So I'm all for [26:25.460 --> 26:32.100] better ways of communicating. I feel like I have like a habit of also wanting to figure out how [26:32.100 --> 26:36.840] to teach these concepts. And I think taking more of an education approach, taking the burden of [26:37.700 --> 26:44.560] explaining complicated topics is more of... it's what we should be doing as the creators of these [26:44.560 --> 26:49.510] systems. So I think there's a lot of room for improvement. [26:51.640 --> 26:59.320] Yeah, I agree with that. I think policies are... because they're legally required and [26:59.320 --> 27:04.720] they're highly regulated, companies are incentivized to be as broad and non-committal [27:04.720 --> 27:10.100] as possible in those privacy policies. So what you end up getting is these like long walls of [27:10.100 --> 27:19.880] legal text. And effectively, you need a law degree to like decipher it properly. So if you're [27:19.880 --> 27:24.240] thinking about compliance, and I think like one theme of this panel is that compliance is like [27:24.240 --> 27:29.740] the bare minimum, but companies now should be thinking about trust as well and like how to [27:29.740 --> 27:38.080] go past that bar. And policies just don't cut it because they are regulated like documents and [27:38.640 --> 27:44.000] it's very hard to find a compromise within the privacy policy. You may be able to have a more [27:44.000 --> 27:48.300] accessible policy, but at the end of the day, companies are indeed like incentivized to be [27:48.300 --> 27:54.440] non-committal there. So I think, and this is something we think about a lot, is like how can [27:54.440 --> 28:03.440] we help educate and inform users? How can we help our customers actually create a place where users [28:03.440 --> 28:09.680] can go to understand the data practices of this company? What type of data specifically are they [28:09.680 --> 28:14.640] collecting? How are they using that information? Who are they sharing it with, if anyone? [28:15.940 --> 28:22.140] And then also, there's this shift now because historically, a lot of privacy law is around [28:22.140 --> 28:29.380] disclosure. Modern privacy law, a lot of it is about choice as well. And so the choices should [28:29.380 --> 28:34.000] be in a more interactive control panel. We are in the modern world. We don't need to use [28:34.000 --> 28:39.660] documents only. We can also use things that are like actual user interfaces. And so when we're [28:39.660 --> 28:43.980] talking about choice, we're often talking about control panels and ways to actually show users [28:45.220 --> 28:50.260] that you had these choices and there are buttons you can click that will change how we operate [28:50.260 --> 28:55.700] our platform and will have implications on your privacy. So there's a lot of exciting stuff that [28:55.700 --> 29:02.880] we see there. And just generally sort of tailoring disclosures to the consumers. [29:02.880 --> 29:09.640] Some people want to drill all the way deep and get a really granular idea of what the company [29:09.640 --> 29:15.940] has. So being able to sort of layer information and let people sort of zoom in is really exciting [29:15.940 --> 29:23.980] there. And then the other thing that we've been working on in this space is actually contextualizing [29:24.120 --> 29:32.640] a disclosure by leveraging the right of access as well where when you say you collect this type [29:32.640 --> 29:38.000] of information, you can actually show them a copy of that information because the user can click a [29:38.000 --> 29:42.140] button that's going to let them download it anyway. So if you can be proactive and say here's [29:42.140 --> 29:48.400] a specific example of like an analytics event that we would track about you, there's really [29:48.400 --> 29:53.640] exciting like new ways of doing that. So yeah, I think companies can go really above and beyond [29:53.640 --> 29:59.520] here. There's a lot of opportunity in this new sort of modern privacy world and a lot of [29:59.520 --> 30:06.100] opportunity to start moving from just disclosure and compliance to increased choice and trust building. [30:08.270 --> 30:15.130] Yeah, I'll just, you know, add to a different aspect of the challenge with the privacy policies. [30:15.130 --> 30:23.350] As someone who used to own our sort of custom website that our privacy policy used to sit in, [30:23.350 --> 30:29.710] I can say that like a lot of different folks contribute to writing that policy on the legal [30:29.710 --> 30:36.150] side. So think of think of it like this lengthy novel that has many different authors who are [30:36.150 --> 30:41.690] using words that nobody understands because they have to, you know, based on, you know, laws that [30:41.690 --> 30:47.770] are like all over the world. And then communicating to, you know, engineers and product folks like [30:47.770 --> 30:55.990] myself, the fact that like, we've got to deploy this in, you know, 30 different languages on [30:55.990 --> 31:02.790] different dates because of different laws in these different markets. And we need specific appendages [31:02.790 --> 31:08.810] for these five countries that are unique to certain regulations that have emerged there. [31:08.810 --> 31:15.830] And by the way, in three months, there's going to be these two launches that are coming up that we [31:15.830 --> 31:20.410] need to make an update to the policy for, but we're not ready to do that right now. And we've [31:20.410 --> 31:27.130] got to work with, you know, you've got to work with your comms team on your strategy for letting [31:27.130 --> 31:32.690] your customers know about all these changes in a way that isn't overwhelming. Because if you [31:32.690 --> 31:39.690] if you get an in-app notification, an email, you know, like five times a year about a company's [31:39.690 --> 31:44.370] privacy notice or privacy policy update, you're going to be scared. You're going to have a lot [31:44.370 --> 31:51.110] of questions and be wondering, like, what that company is doing with your data. So I just want [31:51.110 --> 31:57.670] to describe for folks, like, that art of, like, getting this right and appreciate that aspect of [31:57.670 --> 32:04.170] it. But I also agree with what Maritza and Ben were saying. And, you know, what I add there is, [32:04.170 --> 32:09.790] you know, the reality around these privacy policies is they're required, they're here to [32:09.790 --> 32:15.970] stay, and they're going to get increasingly complex because of emerging laws around the [32:15.970 --> 32:23.410] world. We already saw what happened with GDPR and CCPA. We have LGBT coming up. And that means [32:24.010 --> 32:30.530] you do need, again, you do need a user-focused experience that's written in easy-to-understand [32:30.530 --> 32:35.730] language that explains to users their rights and the controls and choices that are available [32:35.730 --> 32:41.890] to them to exercise those rights. And I think that's a different place than your privacy policy. [32:42.390 --> 32:48.790] It could be another website. It could be things that you do in-app. It could be, you know, norms [32:48.790 --> 32:54.250] or best practices that you follow in all of your consent models whenever you're looking to collect [32:54.250 --> 33:02.050] or make use of data in a different way. And it could also be thought about deeply with your UX [33:02.050 --> 33:07.050] research team as part of the sign-up flow and unique things you might do there in terms of [33:07.050 --> 33:12.770] even leveraging animations or video to explain, like, what's going to happen with your information [33:12.770 --> 33:18.750] and what your choices are. So I think there's a lot of actually cool things you can do to explain [33:18.750 --> 33:25.750] to users their rights and their roles available to them that go beyond the notice. And the last [33:25.750 --> 33:30.270] thing I would add is you should definitely be doing research to make sure that you're focused [33:30.270 --> 33:37.190] on, you know, what matters to your users the most as it pertains to the data that your company [33:37.850 --> 33:41.490] collects and uses. It's certainly not a one-size-fits-all. [33:42.510 --> 33:49.510] Huge plus one to that. Alice is so contextual and depends on your business, your customers. [33:51.890 --> 33:57.110] Just to add on a little bit to the conversation, even though, again, I'm not building product [33:57.110 --> 34:02.710] right now, I'm working on an organization, but from, again, looking at the end users, I think [34:02.710 --> 34:10.310] it's truly unfair to ever expect end users to understand or comprehend a privacy policy at [34:10.310 --> 34:14.670] scale. And I think we're kind of all in agreement there that it's not enough to have a privacy [34:14.670 --> 34:22.270] policy. Sort of using the same metaphors that we use in training, you know, we say in security [34:22.270 --> 34:27.690] education, if you have to provide a training for your tool, you're not going to scale that tool [34:27.690 --> 34:31.350] like you would if you don't need a training. And that's the same thing in this instance. You're [34:31.350 --> 34:37.890] not going to scale people engaging or activating these new rights that they have unless it's truly [34:37.890 --> 34:56.940] like embedded in your product or get interested and actually take a look at what data that is [34:56.940 --> 35:01.080] being collected and stored and how it's being stored. I think the other piece of this for me [35:01.080 --> 35:06.640] is also transparency. Huge bend on what you were saying about building trust. I think [35:06.640 --> 35:12.700] transparency is one way to build this trust, and we need a lot more of it. Understanding data [35:12.700 --> 35:18.840] practices and the governance of data by company is a super important part of that, but shouldn't [35:18.840 --> 35:25.100] be buried, I would argue, in a privacy policy. That should be more, you know, bubble that up [35:25.560 --> 35:30.880] at a user level that a user would come across in their engagement with your product. [35:33.100 --> 35:39.060] So, I want to move the discussion now into some of the specific usability challenges that we see [35:39.060 --> 35:47.120] in the technical and in UX blockers that actually can prevent us from offering better accessibility [35:47.120 --> 35:56.300] and usability for privacy enabling tools or even processes, right? So, I want to dig a little bit [35:56.300 --> 36:02.880] deeper into some examples of difficult either compromises or tradeoffs, you know, that we've [36:02.880 --> 36:08.180] made in our careers to either meet these legal obligations at the expense of user experience [36:08.180 --> 36:16.660] accessibility or usability, as well as, you know, the limitations of UX research and testing and [36:16.660 --> 36:22.580] where we might be able to grow in that area as well to make sure that we're not excluding, [36:23.080 --> 36:29.660] you know, specific populations and groups of people so that everybody really can have access [36:30.180 --> 36:37.180] to their data rights. So, Zach, let's start with you just in terms of what are some of those, [36:37.180 --> 36:43.240] you know, specific usability challenges or some of the tradeoffs that you've seen in trying to [36:43.240 --> 36:46.960] build a product that's both compliant with these regulations but also [36:46.960 --> 36:53.900] easy to use so people can exercise their rights? Yeah, what that immediately makes me think of is [36:54.340 --> 37:00.540] when we what we went through as a company with managing and navigating GDPR. So, when that [37:00.540 --> 37:08.320] law came into effect, it was so large and so comprehensive and, you know, something we hadn't [37:08.320 --> 37:15.620] seen before that we spent that entire year just heavily focused on complying with erasure and [37:15.620 --> 37:21.700] different export kind of laws that were or articles that were part of that law and updating [37:21.700 --> 37:28.280] our consent UI across all apps and services. And so, this meant there wasn't a whole lot of time [37:28.280 --> 37:36.240] left after that for our team to spend on user-facing experiences. And I don't think that [37:36.240 --> 37:43.580] was the intention at all. And the interesting thing is that as a team, we felt that we had [37:43.580 --> 37:50.180] made a huge amount of progress for user privacy. And this is 2018 and according to the regulations, [37:50.180 --> 37:55.600] but that our users couldn't necessarily see it. And we didn't necessarily feel comfortable, [37:56.280 --> 38:01.820] you know, bragging about it or doing a whole lot of comms around it because we were required to [38:01.820 --> 38:09.220] make a lot of these changes by law. So, it wasn't us, you know, going beyond expectations. [38:09.220 --> 38:15.560] But we had this realization that ultimately if our users can't see the changes we're making [38:16.260 --> 38:24.060] or if there are products that users won't take advantage of very often, it may not move the [38:24.060 --> 38:29.660] needle on trust, right? Which has come up many times in this conversation. It may not move the [38:29.660 --> 38:36.480] needle as much on trust as new user-facing controls or products that are really designed [38:36.480 --> 38:43.240] to drive that trust home. So, for me, that's really the trade-off. And, you know, in any given [38:43.760 --> 38:51.160] year, it's something that we actively think about. We want to make sure to comply and do our best to [38:51.160 --> 38:56.780] comply with every new law that's coming out around the world. You know, we're a global company and [38:56.780 --> 39:03.240] we'd like to think that we have, you know, good and always improving relationships with different [39:03.240 --> 39:09.320] regulatory bodies around the world. But we also need to make sure that our roadmap consists of [39:09.320 --> 39:17.920] enough things that are going to raise transparency, improve choice, and communicate to our users [39:18.600 --> 39:25.240] how much that we value their privacy and want them to really understand it. So, that's what [39:25.240 --> 39:32.610] I'd say the trade-off is. Megan, anything that you would want to add from your perspective? [39:33.750 --> 39:41.350] Sure. So, very, again, not coming from a private sector background, I can't help but wonder what [39:41.350 --> 39:47.430] types of incentives there are for companies to refine that user experience to activate, for users [39:47.430 --> 39:53.310] to activate certain privacy rights. And so, I'm kind of super interested to hear about, from other [39:53.310 --> 40:01.230] panelists, actually, to understand how companies are working through that whole flow and that [40:01.230 --> 40:06.490] whole user experience. Because if you make that a blocker, you're not going to get, you know, people [40:06.490 --> 40:12.890] activating in trying to, you know, put forward these particular rights that are now granted to [40:12.890 --> 40:20.630] them, even if they're aware of it. I think, you know, understanding how much user testing is [40:20.630 --> 40:26.790] actually going on there, it would be super interesting to me. I think from, again, from a [40:27.520 --> 40:34.810] sort of trade-off point of view here, I think really understanding, [40:34.810 --> 40:42.230] again, knowing your audience, knowing your users, and having real recipes to give to [40:42.230 --> 40:48.150] the people building products on how to operationalize this sort of UX experience [40:48.150 --> 40:54.970] when it comes to privacy, be it due to compliance or regulation reasons, or data protection reasons, [40:55.770 --> 41:03.770] or, you know, other security privacy reasons that you might be trying to initiate at [41:03.770 --> 41:09.230] your particular company. So, I super want to listen to some of the other perspectives. [41:12.340 --> 41:19.060] So, from my perspective, one of, you know, I'm just like, I'm running through some examples where I'm [41:19.060 --> 41:25.940] thinking about the process that I've gone through when being brought on to convince a team [41:27.180 --> 41:31.340] basically, you know, like, I'm always selling my stakeholders on this is the research that we [41:31.340 --> 41:35.820] should do. At some point, I'm tapped. It's like, here's the team that's getting together. We're [41:35.820 --> 41:41.380] going to be delivering this thing. I hear what we're supposed to do. You know, we have UX involved, [41:41.380 --> 41:48.600] of course. Merit's a privacy person. Let's, you know, get the UXR. What do we do? And it's [41:48.600 --> 41:55.060] changed a lot over the last couple years. I'm thinking back to stuff that happened maybe back in, [41:55.060 --> 42:04.600] um, where at first, one of the kind of blockers that I saw to doing better usability work [42:05.140 --> 42:12.920] for privacy controls was to open people's minds to actually think about making this a part of their [42:12.920 --> 42:21.560] actual objectives. Like, explicitly acknowledging that designing a good experience for consent or [42:21.560 --> 42:27.980] for disclosure or for a data-intensive tool is a thing that requires going beyond the privacy [42:27.980 --> 42:34.580] policy. And that's a bit of a, I mean, that's a conversation that you need to, you're changing, [42:34.580 --> 42:38.180] you're kind of, you're working on your persuasion skills, right? Like, it's not something that's [42:38.180 --> 42:42.460] intuitive to everybody that this is a thing that should happen. So, I'm actually really thrilled [42:42.460 --> 42:46.860] to be on a panel where everybody already agrees that this is a thing, because sometimes that can [42:46.860 --> 42:51.720] be some work. But, um, I think that's the first part. It's basically, like, convincing stakeholders [42:51.720 --> 42:58.800] that we can and should be setting objectives and metrics and things to measure in user testing that [42:58.800 --> 43:05.620] are related to comprehension, understandability. At various points in my career, I'm remembering [43:05.620 --> 43:13.740] one time where I basically convinced a team to take on a metric of people aren't upset. [43:14.760 --> 43:20.860] So, as a usability person, that is an incredibly low bar. Like, ISO has a definition of usability, [43:20.860 --> 43:26.780] and it is people can effectively, efficiently, with satisfaction, complete a task. [43:28.260 --> 43:36.140] To set your bar for satisfaction at they're not pissed off, kind of low. But it's a start. So, [43:36.140 --> 43:41.040] it's basically, like, setting at, like, maybe, you know, it's a neutral response. Maybe it's, [43:41.040 --> 43:44.780] maybe, sure, they're a little bit, like, grumpy about it. But [43:46.060 --> 43:50.540] through the flow, they kind of, like, they get why you've got to ask about it. So, that's one [43:50.540 --> 43:54.740] step. Basically, like, convincing a team that these are things that we can and should measure. [43:55.300 --> 44:00.100] And then from there, I'd say recently, like, that was years ago when that was kind of a battle that [44:00.100 --> 44:04.440] I needed to, not a battle, but those years ago when that was kind of a conversation. [44:04.440 --> 44:12.680] And now I think it's more having and setting high expectations. So, when I hear folks that are, [44:12.680 --> 44:17.180] like, oh, people don't care. I'm, like, no, you're telling me you don't care. I've never [44:17.180 --> 44:21.820] seen any evidence that people don't care. So, then how do we turn this into, you know, again, [44:21.820 --> 44:26.400] we're working for a specific group. We have users, they have expectations. If they don't have [44:26.400 --> 44:30.640] expectations, and they do, so that's kind of an easy thing, then, you know, what's the user [44:30.640 --> 44:37.960] research we can pull into to support that point? And it's just kind of, I guess, forcing the [44:37.960 --> 44:47.880] conversation, which can be difficult. I've learned over time that there's a lot of work that the [44:47.880 --> 44:53.240] privacy community can do to establish a better baseline for having productive conversations [44:53.240 --> 45:00.540] about how data is used to every single audience across the board. And so, even when I go into a [45:00.540 --> 45:05.860] room and I start talking with new stakeholders on a new project, I kind of put myself in a position [45:05.860 --> 45:11.820] of, like, let's just assume that nobody really, like, let's pretend people kind of don't know that [45:11.820 --> 45:17.020] much about privacy. So, let's start the conversation from building up our shared vocabulary and [45:17.020 --> 45:22.860] building up our shared understanding of what's needed here. And that's been an interesting kind [45:22.860 --> 45:27.680] of, I guess that's, like, my continual beginner's mindset as I work through these things. [45:29.560 --> 45:37.380] Yeah, maybe just to follow on to the definition of usability and whether users are upset. [45:37.480 --> 45:43.440] I've submitted so many data rights requests in my lifetime, like, trying to download my data, [45:43.440 --> 45:48.700] erase my data from various companies, and it certainly does not meet the satisfaction bar. [45:48.820 --> 45:55.280] It can feel like you're going to the DMV, where you are, you know, calling support, emailing them, [45:55.280 --> 46:00.120] going back and forth. I have to, like, take selfies with my government ID. I have to, like, [46:00.120 --> 46:05.760] email all this, like, documents back and forth. And it can take, like, over a month to get access [46:05.760 --> 46:10.460] to your data. And frankly, that just, like, builds a world where data rights are not exercisable, [46:10.460 --> 46:16.660] even if you have them on paper. It's just... it's a terrible user experience right now, [46:16.660 --> 46:21.460] and I think people are starting to wake up to that. And as these laws just roll through new [46:21.460 --> 46:25.820] regions, that volume is going to keep going up for companies, and the burden on both sides is [46:25.820 --> 46:32.840] going to go up. So I think that's a huge area for UX. I'll talk about maybe two other things here. [46:32.840 --> 46:42.240] I think one really interesting, like, sort of paradox in this UX is there's a trade-off between [46:42.240 --> 46:48.980] being concise and being complete in your disclosures, where everybody wants to have a [46:48.980 --> 46:55.620] very simple, distilled disclosure around privacy. But more often than not, there's more to the [46:55.620 --> 47:06.820] story. And actually, like, making that concise is a very hard task. And yeah, so that's one thing we [47:06.820 --> 47:15.980] see a lot. And then the other is just that disclosure of both the choices that you have, [47:15.980 --> 47:21.460] as well as how the company is using data, kind of has to happen both at the right time when it's, [47:21.460 --> 47:26.320] like, contextually happening, as well as being... and it also has to be available to rediscover [47:26.320 --> 47:30.840] later. So that means that companies have to do two things. It means that when users are [47:30.840 --> 47:38.060] using a new feature, you have to disclose the impact of using that feature and typically offer [47:38.240 --> 47:44.120] a consent preference there. But then also, you know, after that user has passed that point, [47:44.120 --> 47:49.800] they have to be able to rediscover it. So you should also offer a privacy center as a company [47:49.800 --> 47:57.620] to your customers, where they can go back and see what your data practices are in plain terms, [47:57.620 --> 48:03.060] as well as any prior consents you've set. And hopefully, in this privacy center is also where [48:03.600 --> 48:07.440] you have a much easier experience to exercise those data rights and [48:07.440 --> 48:12.140] not have to sit on the phone with support for hours. [48:14.120 --> 48:19.700] Oh, and cookie banners should definitely go away. Those are more useless than privacy policies. [48:20.880 --> 48:27.160] Actually, Ben, that brings us to our closing topic. And we have about five minutes left. [48:27.240 --> 48:34.680] So with the last remaining minutes that we have, I do want to ask just for your closing advice on [48:34.680 --> 48:42.040] some pragmatic steps that we can take to actually up-level some of our industry practices. So, [48:42.660 --> 48:48.980] you know, what are some of those like immediate pain points that we can address? And perhaps, [48:48.980 --> 48:54.140] you know, like things that we could actually do right now, you know, while things are still on [48:54.140 --> 49:01.740] fire. Maritza, let's start with you. Yeah. Some of the stuff that I would love to see, [49:01.740 --> 49:07.400] I would love if everybody across the board in general was just like, be curious. Be curious [49:07.400 --> 49:11.040] about data. Be curious about what your users' expectations are. Be curious about their [49:11.040 --> 49:15.240] attitudes. Be curious about how you could do better. Be curious about, you know, how do you [49:16.980 --> 49:20.800] reveal like the things that you thought were kind of unsaid or didn't need to be said? [49:20.860 --> 49:27.800] And maybe consider saying them again. And then kind of my big hope and dream for all of [49:27.800 --> 49:33.740] data use and data conversations is if we could get to a point where we had simple answers for [49:33.740 --> 49:40.580] simple questions. Plus one to that. Ben, what about you? [49:42.780 --> 49:49.720] Sure. I would say first, realize that consumers care deeply about privacy. They may not have four [49:49.720 --> 49:55.340] years ago, but they do now. And this is a huge part of your trust equation. And then use that [49:55.340 --> 50:01.100] as a way to go beyond compliance. We've talked about this, but there are some simple things that [50:01.100 --> 50:08.140] you can do, whether that's getting a privacy center to be more upfront with your users about [50:08.140 --> 50:15.160] how you're using that data. Offer a lot of consent preferences and be very clear and [50:15.160 --> 50:22.000] concise there. And make data subject requests easier to exercise for your users. [50:23.960 --> 50:26.720] Great. Megan, any thoughts that you'd want to add? [50:27.400 --> 50:33.100] Just echoing that people absolutely care about privacy. I've never met, you know, [50:33.340 --> 50:37.860] a community member who I've worked with who's like, nah, I don't really care about privacy. [50:37.860 --> 50:42.200] They've just got a lot of other things that they care about too. And I think you put it really well [50:42.200 --> 50:49.540] in talking about prioritization of needs that different users have. Echoing again, [50:49.540 --> 50:58.060] figuring out how we can incentivize companies to test their UX on their exercising their data [50:58.060 --> 51:03.920] rights workflows. I think I'm not convinced right now that they're properly incentivized to do this. [51:03.920 --> 51:08.380] And it's only like a self-reinforcing thing. If you don't have people exercising this right, [51:08.380 --> 51:13.340] you kind of reinforce what people don't care. So I think this is sort of an interesting problem [51:13.340 --> 51:19.260] that where a lot of work can be done. I think specifically looking at, you know, who's doing it [51:19.260 --> 51:25.720] well in industry right now and getting those case studies and those stories shared and told [51:25.720 --> 51:29.260] more publicly, I think will help drive that conversation. [51:30.380 --> 51:34.400] Great. Zach, can you bring us home with some final thoughts? [51:35.280 --> 51:40.900] I can, I can. So I think, you know, sticking with this point of, you know, privacy is certainly [51:40.900 --> 51:47.600] important for our users, but it may be more important at certain moments, right? When you're [51:47.600 --> 51:52.460] using Google, what's most important to you is how well search works, right? When you're using Uber, [51:52.460 --> 51:58.940] it's getting from A to B. When it's Instagram, it's, you know, how well can I, you know, [51:59.650 --> 52:05.280] you know, share photos and view others' photos. But there will be moments when privacy becomes [52:05.280 --> 52:11.280] the most important thing. And in those moments, it has to be there and it has to work for you, [52:11.280 --> 52:18.240] or else you're scared, you're frustrated, you're mad. I read, you know, those customer complaints [52:18.240 --> 52:24.480] from our users. And trust me, privacy is definitely important to them. So what I would say [52:24.480 --> 52:30.800] is, you know, one longer term solution I'm thinking about is contextual and personalized [52:30.800 --> 52:37.080] privacy experiences. So that privacy shows up when you need it. Maybe it's when you take certain [52:37.080 --> 52:43.320] actions on our platform, or you write into customer support with certain kinds of questions, [52:43.320 --> 52:50.560] we can then surface experience to you, experiences to you, based on your activity on the platform, [52:50.560 --> 52:57.000] and not only rely on you to go to our privacy policy or even a privacy center, because as we [52:57.000 --> 53:03.040] know, those are not going to be placed on the very front of an app or the front of an experience. [53:03.040 --> 53:09.460] It's going to take a few clicks to get there. So that's one thing I'd say. The other thing that I [53:09.460 --> 53:16.160] think is going to be important as we move forward is a point of view or best practices for what [53:16.160 --> 53:23.660] privacy will look like in this world where it is governed a lot more by machine learning and AI, [53:23.660 --> 53:31.380] right? Like how do we offer choice and transparency in this world that we're starting to move into? [53:31.380 --> 53:38.360] Because it's not going to be, you know, the same as it is today with just like a bunch of toggles [53:38.360 --> 53:45.400] on a page, like trying to deliver that in a much more complex environment where we're actually [53:45.400 --> 53:51.920] probably going to collect more data from you and to offer you richer experiences. You know, [53:51.920 --> 53:58.260] you can just look at what the Googles and Amazons of the world are doing with, you know, Alexa, [53:58.260 --> 54:01.960] right? And with products like that, and they're amazing and people are going to want them, [54:01.960 --> 54:07.960] but how do you offer privacy in those contexts? And this is a bit of a head-scratcher for me, [54:08.360 --> 54:14.520] right now. So I think we need to be forward-looking and in that regard. So that's what I'd leave us [54:14.520 --> 54:23.180] with. Great. Thank you so much. And thank you to each of you for sharing your expertise and [54:23.180 --> 54:27.540] being part of this conversation today to share with the Crypto and Privacy Village [54:28.120 --> 54:34.540] during DEF CON's safe mode. I hope you're enjoying the rest of the virtual event. [54:34.540 --> 54:40.600] And for those that are watching, I encourage you to jump into the Discord server if you want to [54:40.600 --> 54:44.440] have a conversation with our speakers and with the broader privacy community about some of these [54:44.440 --> 54:49.260] issues. So thank you everyone for watching and thank you again to our panelists for sharing [54:49.260 --> 54:52.040] their expertise with us today. Take care.