So welcome everyone to our conversation about consumer privacy, usable privacy, how do we actually build privacy for consumers in a way that they can actually exercise their data rights. Unfortunately, one of the trends that we're seeing as laws and regulations become more complex, it's increasingly difficult for companies to provide the ability for consumers to actually exercise those rights. And so one of the things that we are concerned about as privacy professionals is making sure that consumers actually have the ability to exercise those rights. Otherwise, they really don't get the full protections that are granted to them under the law, depending on where they live. By way of introduction, my name is Melanie Ensign. I am the founder and CEO of a new company called Discernible. It's a communications firm dedicated specifically to security, privacy, and risk organizations. And I am very excited to be joined by a number of esteemed colleagues today who have a lot of experience, not just in security and privacy, but in building tools and in being part of the process for building and thinking about how do we make privacy a user-centric experience. So a couple of expectations setting things right off the front. None of us here are lawyers. Nothing we say is going to be legal advice. We're also not legal experts. And so we're not going to debate the merit of the law or try to interpret the law. But we are on the receiving end of these legal obligations. And we do have the professional responsibility to advise or lead the building of their subsequent products and tools for consumers so that they can exercise those rights. So just a disclaimer up front that none of us are lawyers. Troll us if you like. There'll be plenty of lawyers in the comment section who can actually provide the legal guidance that you're looking for. So with that said, I'm going to ask each of the panelists to introduce themselves and tell us a little bit about their experience in privacy and specifically their perspective and worldview in terms of how they approach this particular challenge given their function and expertise. So Megan, let's start with you. Sure. No trolling, please. I beg you. My name is Megan DeBlois. Super excited to be on this panel. I'm an information security advisor and technologist over at Internews. Internews itself is an international non-profit. We work on media development really focused on empowering local information sources worldwide while connecting and giving people voice. So that's kind of the area that I tend to focus in. But at the organization, I wear two hats. So internally, I help improve our information security processes, procedures, and practices org-wide. And externally, I work with a pretty diverse group of civil society folks to help improve them on their information security needs from organizational security assessments to risk reduction strategy design and development to threat analysis and sharing. A big part of that and a big part of our approach at Internews when it comes to privacy and security is really trying to keep our users at the center, whether that's going to be an organization or civil society group that we're working with or the developer communities in the open source world who were trying to build capacity around their abilities in creating these really important security and privacy products for really amazing communities, but often who are targeted. Yeah, and that's me. All right. Hi, everybody. I'm Ben Brook, the co-founder and CEO of Transcend. So Transcend is a data privacy infrastructure company. And what we do is we enable other companies to honor their users' data rights. So users can't really exercise their data rights today, even though it is their legal right. And you can go to any major company's privacy policy and see that for yourself. So we've made a series of breakthroughs in tech that enables any company to give their users data rights in a way that's completely on demand and you should have it in seconds. So what we're doing is we're replacing a lot of broken internal processes that are both resulting in noncompliance and and that have been an issue in terms of user experience. So if we're successful, the hope is that we're going to make it easier for everybody to have the ability to control their personal data. Hi, everyone. I'm Marietza Johnson. I'm a UX researcher at Google Research. And for about the past 10-ish years, I've been really focused on this question of how do we make it easy for people to manage their data and the security and privacy decisions that they're faced with in their daily lives with the technology that they use. This focus has taken me to a bunch of different kind of places where I can exercise those interests. Right now I'm at Google Research working on a tool for what we're calling meaningful control over data in a distributed system. I tweet a lot. You can see a project description on Twitter if you're interested in that. I think it's super fun. Also, I used to be on Facebook as a member of their privacy product review process and have also done some academic work just doing some research on looking at how do people understand this stuff, attitudes, behaviors, comprehensibility. So glad to be here. I'm Zach. I lead the global privacy product team at Uber. I focus on building user-facing features that build trust with our users. I also work on developing systems and services in response to new and emerging laws to allow users to exercise their rights over those laws. And the last thing that I'll describe is I guess a little bit different than your traditional product role because I'm also charged with consulting and reviewing the product launches of other teams across Uber that pertain to privacy. And that can take on many different forms from early in the idea phase to just before launch to post-launch and remediation and anything under that sun. So that's me. Awesome. Thank you so much for being here and lending us your expertise today. So let's start with operationalizing privacy and in the steps that actually come after these protections are codified in law. So once a law or new regulation is passed, particularly those that are then going to require some additional engineering or product development, there's a different type of skill set that's needed to work alongside our legal colleagues in order to actually bring that to life. So policy and regulations is one piece of it. But especially now with the new types of regulations we're seeing, there are very real technical requirements and changes that have to happen in order for companies to comply with those new obligations. And I want us to dive a little bit more into what that actually looks like in practice in terms of these different functions working together. So Ben and Zach, let's start with you. Based on your experience, and Ben, I know that you're working externally and helping these companies kind of repair or build some of these broken systems. And Zach, you're obviously in-house at Uber. What does the typical process look like from the perspective of those internal privacy product and engineering teams once a protection has been codified in law or in a new regulatory standard? Sure. So when a new privacy law has been encoded, there's usually a period before it comes into an effect, which means there's a huge surge of preparatory work inside a business. And that can include everything from updating contracts and policies, doing risk assessments, and building frameworks for the company to operate under. And then after the laws come into effect, there's a few new processes. So of course, as Zach alluded to, new product lines, you have to make sure that you're applying those frameworks. But what's really unique about these modern privacy laws like GDPR and CCPA is that there's also this new set of indefinitely recurring processes around user data rights. And so these are things like the right of access to your data and the right to be forgotten. So that means companies now have a consistent stream of requests coming in, typically via support channels. And so that means the support and the legal teams are now receiving these requests and working to manually authenticate users over an email conversation. You can imagine how that would be at high risk to social engineering, where users are making claims to accounts, asking to erase that account or to get access to all of the data in that account. And then after that's been received, there's also an incredible manual process that happens inside many companies today, where it's basically the entire org is going across dozens, if not hundreds, of their data systems and their vendors to go and scrub that data. And so that ends up being almost every part of the company. And whether it's sales and marketing or support or analytics, people are logging into their respective tools and scrubbing data or exporting and emailing it over to legal. And so in short, there's an incredible amount of very cumbersome and manual processes happening today. And it's sort of like this temporary solution. It seems like we're in a lot of businesses today. And again, these processes are still at high risk of social engineering. Yeah, so I'll jump in there. I think, as has been described, when you're just starting out and you perhaps don't have that much of an established privacy team, these efforts can be pretty unstructured and ad hoc. And there's a lot of manual efforts that take place throughout your company to try to comply with these new laws. At Uber, we've been working on this for five plus years, if not more, at least as long as I've been there. And what I would say is to get to the best outcome is definitely a collaborative process across legal product and engineering. So the way it works for us when a new law comes into effect, as Ben said, our legal team takes some time to digest that law. And depending on how complex it is, it could be something as large as GDPR, or one article that's something very small in a single state or even region that we need to respond to. So that can take anywhere from a couple of weeks to several months. But eventually, the legal team will get back to us with a set of legal requirements. My team then, on the product side, reviews those requirements to convert them into a PRD, or a product requirements document. In order to do that, we assess the requirements that we receive in kind of three areas to make sure that we're going to implement and launch something that complies with the law, but that is also practical. So one of the things we consider is the implications for the user experience of the requirements that have been shared over with us. Another thing is the feasibility from an engineering perspective of what's been shared over. And the other thing is the impact to the business. You know, a lot of times, new and emerging privacy laws might be at odds with what your team wants to do or your company wants to do on the marketing side, for instance. If you're providing users with more controls to opt out of ad targeting or something like that. And so, you know, we are trying to do these things in the right way in compliance with the law, but also optimize to fit our business, as I said. So once we've kind of rationalized the requirements we've received and we then have a follow-up conversation with our legal team to make sure that they are paying with the direction that we're recommending and the timeline we've set forth for which we could deliver on, we will define a set of metrics or key performance indicators. And these usually serve the purpose of mitigating the risk of being in non-compliance. We will design our metrics in that sense, in particular, if it's something to do with complying with a new law. And then finally, we will have a design review with stakeholders across Uber that could be impacted by the changes that we're going to make. And those stakeholders will be from legal, from policy, from other engineering and product teams. And we'll walk through exactly what's going to change or be introduced into the experience, how it's going to work end-to-end, and we'll get some feedback. And then we'll be all aligned across the company on moving forward. And then from there, we just start building. So that's kind of how it works in-house at a little bit more of a mature privacy operation. Awesome. Thanks. Megan and Marta, I want to ask you, does the process look any different from the perspective of end-users or even developers in the field that you're working with? We've heard from Ben and Zach a little bit, like the chaos-to-order evolution that's happening inside companies. What does it look like externally to those who are either the users of these services or perhaps other developers who aren't in the position of having these more mature privacy organizations? Sure. So if I can go first. Again, I think it's important to kind of frame up my experiences and observations here. It's really more tailored to leading and working on open-source projects and working with at-risk communities. In the past life, I led a project that was really, really aiming at building the capacity of developers and creating feedback loops among their users, which tended to be more at-risk communities. So that's kind of where I'm coming from with my input and observation. From the developer side of things, again, coming from open-source landia, where you have very few resources, including human capacity, this looks dramatically different. The vast majority of security and privacy tool developers who I've worked with are super concerned about privacy. Naturally, they're creating security and privacy tools specifically to give users more privacy online. And because of this, we see that a lot of them try really hard to capture the least amount of data as possible. So most of them have no trackers. Most of them intentionally do not collect user data at a very, very basic level. I think a lot of smaller businesses, if you're operating with European users, I think that put definitely a lot of strain on those communities who have very little capacity to begin with. But at the end of the day, I think the general thought process around privacy in those communities is sort of it's an all or nothing. I'm either going to... in the open-source community, in my observation is that they just really don't want to collect anything. And that impacts them, not just from a UX perspective, because they're not able to make really important insights to improve their tools and roadmap to actually make insightful decisions on the usability front. And I don't think it's an all or nothing game. One of the projects that we've been supporting is a project by the Guardian Project who has created an amazing and is continuing to create an amazing tool for Android applications called Clean Insights to be more plug-and-playable with these open-source tools or other products that want to be more privacy-respecting in the analytics that they're capturing. I think more things like this will be really important as more legislation and more regulation starts to happen around the globe. From an end-user perspective, I think that for me, and I don't think this is really specific to civil society, I think there's just still a really large gap in user understanding of their digital rights. So GDPR happened, I would say, you know, the vast majority of folks around the globe probably didn't know exactly what that means for them as a user. And the before and after that they really saw was they got a lot of emails in their inboxes saying privacy policies were changing for a lot of different companies. And that's sort of like a very end-user perspective of, you know, not truly knowing exactly how to engage with or activate these new rights that they might have. I think the other really important thing to note is that for a lot of at-risk communities or targeted groups that are using products, their calculus isn't necessarily privacy from, you know, companies is at the top. They're really threat modeling and trying to understand, you know, what it is I'm trying to protect here. And, you know, they're building on this trade-off between sort of, do I really need this really important security encryption protocol being used in this particular product versus a more privacy preserving application that, you know, isn't going to be used as much in my network or it's difficult to use. So it's more of a threat modeling challenge rather than a sort of privacy calculation. And our approach at Internet News has been really pragmatic in trying to, you know, get the individual, get the community, get the organization to be more secure today than they were tomorrow. And that's the ultimate priority from an end-user's perspective. I love that as a build into how I would be thinking about it because I feel like product teams need to really be thinking about the threat model and how the different groups they aim to serve may have different needs and basically taking care of, you know, as much as they can. But then inevitably we do get into situations where individuals are making their own decisions about, you know, which setting to use, which control do I consent to this? And to me, I always think about it, I want to start from the human-centered perspective. Like, we can't assume that people are reading privacy policies. We can't assume that people are seeing all the great material that we put out when we're promoting a new privacy feature or a new security feature. So then if you're starting from the point of view of the thing that they see is the UI and the user experience, what is that? How are they using the tool? So, of course, we're always mindful. You're diligent in implementing controls that respect the regulations and meet the regulations. I feel like that's one way to look at it. But then if jump over to the human-centered point of view, you start to ask different questions. So then for the context that you're building a product or a service, how is it being used? Like, who's using it? How are they using it? What are their expectations? What's their attitude toward the data use? And how do you design an experience that matches that as best as you can? I love the way, Megan, that you alluded to kind of the difficulties that we have right now where in some situations, it's quite a complicated system. How do you communicate succinctly and comprehensively to somebody about all the ways in which data is collected and used? Of course, people, I feel like we constantly hear things like, you know, oh, people don't care. If they cared, they wouldn't do this. If they cared, they wouldn't do that. But privacy and security aren't the top level concern. Just because they're not your top priority doesn't mean you don't care. So maybe it's you care more about something else right now, or maybe you'll care more about that at a later point in time. So then how do you kind of take all these different perspectives and put it into a UX that makes sense? So I'd say that's where my role comes in. So I'm always interested in, you know, how can we take a human-centered approach? How can we ask these questions about what do people need, and then design a user research program around evaluating whether or not we've met the goals there. So it's building on top of regulation, hopefully always going well beyond, but definitely something that, you know, is a constant work in progress. Yeah, one of the things that I've often advised companies that I've been working with is you don't get credit for hitting the legal minimum, you know, and truthfully, you're, you know, Marta, I think you said it best, like the people that you're serving are actually expecting more from you because they're expecting that you know better than even the lawmakers in terms of how to build better products and better experiences. So I want to shift gears just briefly to take a moment to talk about privacy policies and kind of why they're so unpopular and seem to be a pretty focused target for a lot of public scrutiny, which we could debate all day whether or not that is deserved or not. But I think the problematic aspects of privacy policies are often interpreted publicly as, you know, a company not caring about privacy or not wanting to inform consumers. And, you know, we've all worked on this challenge for years. We know that that's not true. We know that the problems that exist in privacy policies are not reflective of a company's actual commitment and investment level in privacy. And even companies with strong privacy commitments and protections have difficulty creating policies that consumers can easily access and understand. And so I want to ask each of you, and Marta, you know, I'd love to start with your thoughts just in terms of what is the actual intent of these policies? And should we still be relying on this type of documentation to inform and educate consumers about the data practices of the different businesses and services that they use? I've even heard from a couple of folks in the EU that, you know, terms of service and privacy policies are no longer going to be considered sufficient in terms of informed consent because of how complicated they are for consumers to understand. I think it is a valid argument that, you know, this very long legal document cannot... nobody realistically expects that consumers are reading that and understanding it and processing what that means for them. So obviously, again, since none of us are lawyers, without getting into all the legal definitions of consent and informed consent and all of that, what are your thoughts in terms of how we could, you know, either improve policies or perhaps look at other channels and avenues through which to educate consumers about data practices as well as their rights? Okay, so from my previous answer, you won't be surprised to hear that I think you could do a lot more than a privacy policy. And there's a big opportunity to do a better job of both crafting, living, and communicating your privacy narrative. And not just a privacy narrative, but like, how data is used. Like, how is data collected? How is it used? How is it stored? And how are you deleting it? I think that we're seeing a lot of progress in this area. At Google, I think it was about a year ago, we had this new feature, so private set intersection, which for the cryptography community is an amazing, cool, sexy result, and they want to be talking about it. But usually you'd see that in an academic paper, and it would go to the academic community, folks would talk about it, and it'd be a neat talking point. I was super excited to see that this one innovation had basically like a full-com strategy. So it was an academic paper, it was a blog post, it was a kind of infographic diagram to explain what it meant, plus a YouTube video. And that to me was great. It's basically, who's the audience? Like, who are the audiences who need to see this information and understand it? And how do we cater it to them? So I would love to see a lot more like that. I mean, like I said, I've worked in privacy and use human-centered privacy for a long time, and I don't read privacy policies. Like, I have better things to do than read a privacy policy. I never read them. And like, I've read a few because I'm supposed to, because I should know what they look like. But I have better things to do. So I'm all for better ways of communicating. I feel like I have like a habit of also wanting to figure out how to teach these concepts. And I think taking more of an education approach, taking the burden of explaining complicated topics is more of... it's what we should be doing as the creators of these systems. So I think there's a lot of room for improvement. Yeah, I agree with that. I think policies are... because they're legally required and they're highly regulated, companies are incentivized to be as broad and non-committal as possible in those privacy policies. So what you end up getting is these like long walls of legal text. And effectively, you need a law degree to like decipher it properly. So if you're thinking about compliance, and I think like one theme of this panel is that compliance is like the bare minimum, but companies now should be thinking about trust as well and like how to go past that bar. And policies just don't cut it because they are regulated like documents and it's very hard to find a compromise within the privacy policy. You may be able to have a more accessible policy, but at the end of the day, companies are indeed like incentivized to be non-committal there. So I think, and this is something we think about a lot, is like how can we help educate and inform users? How can we help our customers actually create a place where users can go to understand the data practices of this company? What type of data specifically are they collecting? How are they using that information? Who are they sharing it with, if anyone? And then also, there's this shift now because historically, a lot of privacy law is around disclosure. Modern privacy law, a lot of it is about choice as well. And so the choices should be in a more interactive control panel. We are in the modern world. We don't need to use documents only. We can also use things that are like actual user interfaces. And so when we're talking about choice, we're often talking about control panels and ways to actually show users that you had these choices and there are buttons you can click that will change how we operate our platform and will have implications on your privacy. So there's a lot of exciting stuff that we see there. And just generally sort of tailoring disclosures to the consumers. Some people want to drill all the way deep and get a really granular idea of what the company has. So being able to sort of layer information and let people sort of zoom in is really exciting there. And then the other thing that we've been working on in this space is actually contextualizing a disclosure by leveraging the right of access as well where when you say you collect this type of information, you can actually show them a copy of that information because the user can click a button that's going to let them download it anyway. So if you can be proactive and say here's a specific example of like an analytics event that we would track about you, there's really exciting like new ways of doing that. So yeah, I think companies can go really above and beyond here. There's a lot of opportunity in this new sort of modern privacy world and a lot of opportunity to start moving from just disclosure and compliance to increased choice and trust building. Yeah, I'll just, you know, add to a different aspect of the challenge with the privacy policies. As someone who used to own our sort of custom website that our privacy policy used to sit in, I can say that like a lot of different folks contribute to writing that policy on the legal side. So think of think of it like this lengthy novel that has many different authors who are using words that nobody understands because they have to, you know, based on, you know, laws that are like all over the world. And then communicating to, you know, engineers and product folks like myself, the fact that like, we've got to deploy this in, you know, 30 different languages on different dates because of different laws in these different markets. And we need specific appendages for these five countries that are unique to certain regulations that have emerged there. And by the way, in three months, there's going to be these two launches that are coming up that we need to make an update to the policy for, but we're not ready to do that right now. And we've got to work with, you know, you've got to work with your comms team on your strategy for letting your customers know about all these changes in a way that isn't overwhelming. Because if you if you get an in-app notification, an email, you know, like five times a year about a company's privacy notice or privacy policy update, you're going to be scared. You're going to have a lot of questions and be wondering, like, what that company is doing with your data. So I just want to describe for folks, like, that art of, like, getting this right and appreciate that aspect of it. But I also agree with what Maritza and Ben were saying. And, you know, what I add there is, you know, the reality around these privacy policies is they're required, they're here to stay, and they're going to get increasingly complex because of emerging laws around the world. We already saw what happened with GDPR and CCPA. We have LGBT coming up. And that means you do need, again, you do need a user-focused experience that's written in easy-to-understand language that explains to users their rights and the controls and choices that are available to them to exercise those rights. And I think that's a different place than your privacy policy. It could be another website. It could be things that you do in-app. It could be, you know, norms or best practices that you follow in all of your consent models whenever you're looking to collect or make use of data in a different way. And it could also be thought about deeply with your UX research team as part of the sign-up flow and unique things you might do there in terms of even leveraging animations or video to explain, like, what's going to happen with your information and what your choices are. So I think there's a lot of actually cool things you can do to explain to users their rights and their roles available to them that go beyond the notice. And the last thing I would add is you should definitely be doing research to make sure that you're focused on, you know, what matters to your users the most as it pertains to the data that your company collects and uses. It's certainly not a one-size-fits-all. Huge plus one to that. Alice is so contextual and depends on your business, your customers. Just to add on a little bit to the conversation, even though, again, I'm not building product right now, I'm working on an organization, but from, again, looking at the end users, I think it's truly unfair to ever expect end users to understand or comprehend a privacy policy at scale. And I think we're kind of all in agreement there that it's not enough to have a privacy policy. Sort of using the same metaphors that we use in training, you know, we say in security education, if you have to provide a training for your tool, you're not going to scale that tool like you would if you don't need a training. And that's the same thing in this instance. You're not going to scale people engaging or activating these new rights that they have unless it's truly like embedded in your product or get interested and actually take a look at what data that is being collected and stored and how it's being stored. I think the other piece of this for me is also transparency. Huge bend on what you were saying about building trust. I think transparency is one way to build this trust, and we need a lot more of it. Understanding data practices and the governance of data by company is a super important part of that, but shouldn't be buried, I would argue, in a privacy policy. That should be more, you know, bubble that up at a user level that a user would come across in their engagement with your product. So, I want to move the discussion now into some of the specific usability challenges that we see in the technical and in UX blockers that actually can prevent us from offering better accessibility and usability for privacy enabling tools or even processes, right? So, I want to dig a little bit deeper into some examples of difficult either compromises or tradeoffs, you know, that we've made in our careers to either meet these legal obligations at the expense of user experience accessibility or usability, as well as, you know, the limitations of UX research and testing and where we might be able to grow in that area as well to make sure that we're not excluding, you know, specific populations and groups of people so that everybody really can have access to their data rights. So, Zach, let's start with you just in terms of what are some of those, you know, specific usability challenges or some of the tradeoffs that you've seen in trying to build a product that's both compliant with these regulations but also easy to use so people can exercise their rights? Yeah, what that immediately makes me think of is when we what we went through as a company with managing and navigating GDPR. So, when that law came into effect, it was so large and so comprehensive and, you know, something we hadn't seen before that we spent that entire year just heavily focused on complying with erasure and different export kind of laws that were or articles that were part of that law and updating our consent UI across all apps and services. And so, this meant there wasn't a whole lot of time left after that for our team to spend on user-facing experiences. And I don't think that was the intention at all. And the interesting thing is that as a team, we felt that we had made a huge amount of progress for user privacy. And this is 2018 and according to the regulations, but that our users couldn't necessarily see it. And we didn't necessarily feel comfortable, you know, bragging about it or doing a whole lot of comms around it because we were required to make a lot of these changes by law. So, it wasn't us, you know, going beyond expectations. But we had this realization that ultimately if our users can't see the changes we're making or if there are products that users won't take advantage of very often, it may not move the needle on trust, right? Which has come up many times in this conversation. It may not move the needle as much on trust as new user-facing controls or products that are really designed to drive that trust home. So, for me, that's really the trade-off. And, you know, in any given year, it's something that we actively think about. We want to make sure to comply and do our best to comply with every new law that's coming out around the world. You know, we're a global company and we'd like to think that we have, you know, good and always improving relationships with different regulatory bodies around the world. But we also need to make sure that our roadmap consists of enough things that are going to raise transparency, improve choice, and communicate to our users how much that we value their privacy and want them to really understand it. So, that's what I'd say the trade-off is. Megan, anything that you would want to add from your perspective? Sure. So, very, again, not coming from a private sector background, I can't help but wonder what types of incentives there are for companies to refine that user experience to activate, for users to activate certain privacy rights. And so, I'm kind of super interested to hear about, from other panelists, actually, to understand how companies are working through that whole flow and that whole user experience. Because if you make that a blocker, you're not going to get, you know, people activating in trying to, you know, put forward these particular rights that are now granted to them, even if they're aware of it. I think, you know, understanding how much user testing is actually going on there, it would be super interesting to me. I think from, again, from a sort of trade-off point of view here, I think really understanding, again, knowing your audience, knowing your users, and having real recipes to give to the people building products on how to operationalize this sort of UX experience when it comes to privacy, be it due to compliance or regulation reasons, or data protection reasons, or, you know, other security privacy reasons that you might be trying to initiate at your particular company. So, I super want to listen to some of the other perspectives. So, from my perspective, one of, you know, I'm just like, I'm running through some examples where I'm thinking about the process that I've gone through when being brought on to convince a team basically, you know, like, I'm always selling my stakeholders on this is the research that we should do. At some point, I'm tapped. It's like, here's the team that's getting together. We're going to be delivering this thing. I hear what we're supposed to do. You know, we have UX involved, of course. Merit's a privacy person. Let's, you know, get the UXR. What do we do? And it's changed a lot over the last couple years. I'm thinking back to stuff that happened maybe back in, um, where at first, one of the kind of blockers that I saw to doing better usability work for privacy controls was to open people's minds to actually think about making this a part of their actual objectives. Like, explicitly acknowledging that designing a good experience for consent or for disclosure or for a data-intensive tool is a thing that requires going beyond the privacy policy. And that's a bit of a, I mean, that's a conversation that you need to, you're changing, you're kind of, you're working on your persuasion skills, right? Like, it's not something that's intuitive to everybody that this is a thing that should happen. So, I'm actually really thrilled to be on a panel where everybody already agrees that this is a thing, because sometimes that can be some work. But, um, I think that's the first part. It's basically, like, convincing stakeholders that we can and should be setting objectives and metrics and things to measure in user testing that are related to comprehension, understandability. At various points in my career, I'm remembering one time where I basically convinced a team to take on a metric of people aren't upset. So, as a usability person, that is an incredibly low bar. Like, ISO has a definition of usability, and it is people can effectively, efficiently, with satisfaction, complete a task. To set your bar for satisfaction at they're not pissed off, kind of low. But it's a start. So, it's basically, like, setting at, like, maybe, you know, it's a neutral response. Maybe it's, maybe, sure, they're a little bit, like, grumpy about it. But through the flow, they kind of, like, they get why you've got to ask about it. So, that's one step. Basically, like, convincing a team that these are things that we can and should measure. And then from there, I'd say recently, like, that was years ago when that was kind of a battle that I needed to, not a battle, but those years ago when that was kind of a conversation. And now I think it's more having and setting high expectations. So, when I hear folks that are, like, oh, people don't care. I'm, like, no, you're telling me you don't care. I've never seen any evidence that people don't care. So, then how do we turn this into, you know, again, we're working for a specific group. We have users, they have expectations. If they don't have expectations, and they do, so that's kind of an easy thing, then, you know, what's the user research we can pull into to support that point? And it's just kind of, I guess, forcing the conversation, which can be difficult. I've learned over time that there's a lot of work that the privacy community can do to establish a better baseline for having productive conversations about how data is used to every single audience across the board. And so, even when I go into a room and I start talking with new stakeholders on a new project, I kind of put myself in a position of, like, let's just assume that nobody really, like, let's pretend people kind of don't know that much about privacy. So, let's start the conversation from building up our shared vocabulary and building up our shared understanding of what's needed here. And that's been an interesting kind of, I guess that's, like, my continual beginner's mindset as I work through these things. Yeah, maybe just to follow on to the definition of usability and whether users are upset. I've submitted so many data rights requests in my lifetime, like, trying to download my data, erase my data from various companies, and it certainly does not meet the satisfaction bar. It can feel like you're going to the DMV, where you are, you know, calling support, emailing them, going back and forth. I have to, like, take selfies with my government ID. I have to, like, email all this, like, documents back and forth. And it can take, like, over a month to get access to your data. And frankly, that just, like, builds a world where data rights are not exercisable, even if you have them on paper. It's just... it's a terrible user experience right now, and I think people are starting to wake up to that. And as these laws just roll through new regions, that volume is going to keep going up for companies, and the burden on both sides is going to go up. So I think that's a huge area for UX. I'll talk about maybe two other things here. I think one really interesting, like, sort of paradox in this UX is there's a trade-off between being concise and being complete in your disclosures, where everybody wants to have a very simple, distilled disclosure around privacy. But more often than not, there's more to the story. And actually, like, making that concise is a very hard task. And yeah, so that's one thing we see a lot. And then the other is just that disclosure of both the choices that you have, as well as how the company is using data, kind of has to happen both at the right time when it's, like, contextually happening, as well as being... and it also has to be available to rediscover later. So that means that companies have to do two things. It means that when users are using a new feature, you have to disclose the impact of using that feature and typically offer a consent preference there. But then also, you know, after that user has passed that point, they have to be able to rediscover it. So you should also offer a privacy center as a company to your customers, where they can go back and see what your data practices are in plain terms, as well as any prior consents you've set. And hopefully, in this privacy center is also where you have a much easier experience to exercise those data rights and not have to sit on the phone with support for hours. Oh, and cookie banners should definitely go away. Those are more useless than privacy policies. Actually, Ben, that brings us to our closing topic. And we have about five minutes left. So with the last remaining minutes that we have, I do want to ask just for your closing advice on some pragmatic steps that we can take to actually up-level some of our industry practices. So, you know, what are some of those like immediate pain points that we can address? And perhaps, you know, like things that we could actually do right now, you know, while things are still on fire. Maritza, let's start with you. Yeah. Some of the stuff that I would love to see, I would love if everybody across the board in general was just like, be curious. Be curious about data. Be curious about what your users' expectations are. Be curious about their attitudes. Be curious about how you could do better. Be curious about, you know, how do you reveal like the things that you thought were kind of unsaid or didn't need to be said? And maybe consider saying them again. And then kind of my big hope and dream for all of data use and data conversations is if we could get to a point where we had simple answers for simple questions. Plus one to that. Ben, what about you? Sure. I would say first, realize that consumers care deeply about privacy. They may not have four years ago, but they do now. And this is a huge part of your trust equation. And then use that as a way to go beyond compliance. We've talked about this, but there are some simple things that you can do, whether that's getting a privacy center to be more upfront with your users about how you're using that data. Offer a lot of consent preferences and be very clear and concise there. And make data subject requests easier to exercise for your users. Great. Megan, any thoughts that you'd want to add? Just echoing that people absolutely care about privacy. I've never met, you know, a community member who I've worked with who's like, nah, I don't really care about privacy. They've just got a lot of other things that they care about too. And I think you put it really well in talking about prioritization of needs that different users have. Echoing again, figuring out how we can incentivize companies to test their UX on their exercising their data rights workflows. I think I'm not convinced right now that they're properly incentivized to do this. And it's only like a self-reinforcing thing. If you don't have people exercising this right, you kind of reinforce what people don't care. So I think this is sort of an interesting problem that where a lot of work can be done. I think specifically looking at, you know, who's doing it well in industry right now and getting those case studies and those stories shared and told more publicly, I think will help drive that conversation. Great. Zach, can you bring us home with some final thoughts? I can, I can. So I think, you know, sticking with this point of, you know, privacy is certainly important for our users, but it may be more important at certain moments, right? When you're using Google, what's most important to you is how well search works, right? When you're using Uber, it's getting from A to B. When it's Instagram, it's, you know, how well can I, you know, you know, share photos and view others' photos. But there will be moments when privacy becomes the most important thing. And in those moments, it has to be there and it has to work for you, or else you're scared, you're frustrated, you're mad. I read, you know, those customer complaints from our users. And trust me, privacy is definitely important to them. So what I would say is, you know, one longer term solution I'm thinking about is contextual and personalized privacy experiences. So that privacy shows up when you need it. Maybe it's when you take certain actions on our platform, or you write into customer support with certain kinds of questions, we can then surface experience to you, experiences to you, based on your activity on the platform, and not only rely on you to go to our privacy policy or even a privacy center, because as we know, those are not going to be placed on the very front of an app or the front of an experience. It's going to take a few clicks to get there. So that's one thing I'd say. The other thing that I think is going to be important as we move forward is a point of view or best practices for what privacy will look like in this world where it is governed a lot more by machine learning and AI, right? Like how do we offer choice and transparency in this world that we're starting to move into? Because it's not going to be, you know, the same as it is today with just like a bunch of toggles on a page, like trying to deliver that in a much more complex environment where we're actually probably going to collect more data from you and to offer you richer experiences. You know, you can just look at what the Googles and Amazons of the world are doing with, you know, Alexa, right? And with products like that, and they're amazing and people are going to want them, but how do you offer privacy in those contexts? And this is a bit of a head-scratcher for me, right now. So I think we need to be forward-looking and in that regard. So that's what I'd leave us with. Great. Thank you so much. And thank you to each of you for sharing your expertise and being part of this conversation today to share with the Crypto and Privacy Village during DEF CON's safe mode. I hope you're enjoying the rest of the virtual event. And for those that are watching, I encourage you to jump into the Discord server if you want to have a conversation with our speakers and with the broader privacy community about some of these issues. So thank you everyone for watching and thank you again to our panelists for sharing their expertise with us today. Take care.