00:00:00.567-->00:00:05.873 >>Um, so yeah I'm going to talk about psychographic targeting in politic- political campaigns. 00:00:05.873-->00:00:10.878 Um, psychographic meaning using personality and behavior, uh to target uh ad specific groups of 00:00:13.013-->00:00:17.351 people. Uh, I'm going to speak a little bit more about the title of the talk in a couple of 00:00:17.351-->00:00:22.356 slides. Um, but let me explain how we got into this uh research and this particular talk. So, in 00:00:25.926-->00:00:30.931 2016 we did some research into the EU Referendum, some of you might know as Brexit. Uh, we did 00:00:33.066-->00:00:37.704 six studies consisting of about eleven thousand participants and we looked at personality 00:00:37.704-->00:00:44.544 differences. Differences in numerous skills uh, thinking styles and cognitive biases. And 00:00:44.544-->00:00:49.549 in three of those studies we added a question around if, can, can you all hear me? I guess. 00:00:54.221-->00:00:57.190 [off-mice audience comments] Closer? Wow. I'll probably be eating it but, uh ok. So, in, 00:00:57.190-->00:01:03.730 um, in three of those studies with about five thousand people we added an additional question 00:01:03.730-->00:01:09.169 that's, uh, asked people's opinions to how much they agreed on the nothing uh, nothing to 00:01:09.169-->00:01:13.740 hide nothing to fear argument. So, if you've got nothing to hide you've got nothing to fear. 00:01:13.740-->00:01:20.580 What we saw in that research was that people who voted to leave the European Union were much 00:01:20.580-->00:01:24.551 more likely to support the nothing to hide nothing to fear argument than those that voted 00:01:24.551-->00:01:28.755 to remain. So, we wanted to study that in greater depth to understand kind of what, what 00:01:28.755-->00:01:34.261 was going on there. Uh and understand a little bit more about the drivers. Then in 00:01:34.261-->00:01:40.500 around January of uh this year, some of my friends started pointing out articles like this 00:01:40.500-->00:01:45.739 which appeared in Scout AI and this is where we got the title of the talk from really. They 00:01:45.739-->00:01:51.144 introduced it as the rise of the weaponized AI propaganda machine. So, using psychographic 00:01:51.144-->00:01:56.116 information in our political campaigns. So, my friends weren't the only people that 00:01:56.116-->00:02:00.320 were beginning to sort of question the efficacy of ya know, how well does this work 00:02:00.320-->00:02:04.891 essentially. Uh, other journalists started looking, ya know looking at that and saying 00:02:04.891-->00:02:10.130 well does this really hold water or, or not? So, we've got a little bit of background here in 00:02:10.130-->00:02:17.070 this topic. In 2013 we presented at Def Con on predicting the susceptibility to social bots on 00:02:17.070-->00:02:22.409 Twitter. So, seeing which users would interact with a social bot based on their Twitter activity. 00:02:22.409-->00:02:28.815 Uh and we also looked at personality in, in that. In 2011 and 2012 we looked at 00:02:28.815-->00:02:34.688 determining personality through Facebook activity, uh and through Twitter. Both of those 00:02:34.688-->00:02:39.693 were pretend at Def Con also. So, what we decided to do was use the, uh, using Attitudes 00:02:42.162-->00:02:47.434 Online Surveillance with digital rights. Use that as like the central thesis for exploring the 00:02:47.434-->00:02:52.572 efficacy of psychographic marketing in political campaigns. So, the rest of the 00:02:52.572-->00:02:59.046 talk is structured into five main sections. First of all, we look at why people are divided. 00:02:59.046-->00:03:04.818 Then we move onto what influences their views and nudges them one way or another. 00:03:04.818-->00:03:08.588 Then we look at, once we, once we understand how people are divided, assuming they are, 00:03:08.588-->00:03:13.894 which spoiler alert, they are, how can you effectively target groups with different attitudes 00:03:13.894-->00:03:20.300 in relation to online communication surveillance? Uh, once we know how we can target 00:03:20.300-->00:03:26.273 different users, or different people essentially, we look at um, how persuasive are targeted 00:03:26.273-->00:03:31.278 ads. And finally, we wrap up with looking at, okay once the message is out there, how 00:03:33.313-->00:03:39.086 effective it, how, how able are you to sort of debunk misleading information or misinterpreted 00:03:39.086-->00:03:45.125 information once it gets out of the gate? So, onto the first question. Looking at why people 00:03:45.125-->00:03:49.629 are divided. Well we chose the nothing to fear, nothing to hide argument. But we could have 00:03:49.629-->00:03:55.268 chosen the, the death penalty. We could have chose the topic of same sex marriage and we could 00:03:55.268-->00:04:00.207 have chosen the topic of immigration. So, what, what seems to be at the center or at 00:04:04.444-->00:04:09.449 least a major factor in all of these differing world views is uh, something called 00:04:11.718-->00:04:17.524 Authoritarianism. Which, uh began as a study in the, I guess following the second world war 00:04:17.524-->00:04:22.729 but lost some favor, some credibility in those early years and has regained credibility in 00:04:22.729-->00:04:27.734 more recent years, in the last few decades essentially. Um, so Authoritarianism, as you look at 00:04:29.836-->00:04:35.642 it you might be forgiven for thinking it's a bimodal distribution with people either 00:04:35.642-->00:04:39.980 low or high and not a great deal in the middle. But what we've actually, ya know what we 00:04:39.980-->00:04:45.819 actually find is that it, it's a much more skewed distribution than that. At least in uh the 00:04:45.819-->00:04:50.824 Facebook samples that we see. So, what, what characterizes Authoritarianism is that people 00:04:53.627-->00:04:59.499 high in Authoritarianism tend to see the world in, uh in so, black and white. Whereas people 00:04:59.499-->00:05:04.604 lower in Authoritarianism see it tend to see in sort of shades, shades of grey really and it's a 00:05:04.604-->00:05:09.776 sliding scale. Ya know, so people are, it, it's very rare to have people at the certain 00:05:09.776-->00:05:14.781 extremes essentially. People who are higher in Authoritari-, Authoritarianism um, have, are 00:05:18.552-->00:05:24.191 more concerned about the upkeep of societal norms and traditions whereas people lower in 00:05:24.191-->00:05:29.196 Authoritarianism, uh are more embracing of new cultures. Um, new ideas. So, uh, the, the 00:05:32.899-->00:05:38.738 footloose image there. Uh, additionally what we see is that those that are higher in 00:05:38.738-->00:05:43.743 Authoritarianism have lower tolerance for out-groups. Uh, whereas people who are lower in 00:05:45.912-->00:05:50.417 Authoritarianism uh are much more embracing of uh different cultures or ya know difference 00:05:50.417-->00:05:55.422 races and what have you. So, the scales that we use throughout, our studies stem from some work 00:05:59.960-->00:06:05.365 by uh, researchers called Feldman and Stenner in 1997 where they developed uh a very 00:06:05.365-->00:06:10.370 simple way of looking at Authoritarianism. They developed a, a four question, uh if you 00:06:12.872-->00:06:17.877 like child rearing questionnaire. So, if you're um, if you select answers to these 00:06:21.047-->00:06:25.685 questions which, ya know you have to select one of the other for each of the line items. If 00:06:25.685-->00:06:30.724 you're predominately choosing answers on the right-hand side than you're more, you're higher 00:06:30.724-->00:06:36.496 in Authoritarianism than people um choosing answers on the left-hand side. Ya know 00:06:36.496-->00:06:41.868 generally, there's some exceptions to that. So, moving onto our study. Uh and our 00:06:41.868-->00:06:47.140 results for the first thing and looking at why people, uh why people differ. Ya know we chose 00:06:47.140-->00:06:53.947 Authoritarianism to uh, to, to, to look at in greater depth. We conducted uh four different 00:06:53.947-->00:06:59.953 studies, uh over, well from February to, uh to May with differing sample sizes. You'll 00:06:59.953-->00:07:02.656 note that there's studies uh, A and studies B there's a slight difference in, in, in the uh 00:07:02.656-->00:07:05.625 studies. All of the questions, all of the studies uh took um, basic demographic information, 00:07:05.625-->00:07:10.630 age and sex and they asked, uh a number of questions about people's attitudes to online 00:07:17.437-->00:07:22.442 surveillance. Which we essentially replicated form prior studies or prior surveys, 00:07:26.579-->00:07:29.683 so that we could actually go back and reference and see if our numbers were sort of 00:07:29.683-->00:07:35.388 consistent. The difference between studies B and A was uh, uh the, the B studies also 00:07:35.388-->00:07:38.792 contained a numerical question which we'll get to a little bit later in the, in the 00:07:38.792-->00:07:43.897 presentation. Because the studies differed we treat them kind of individually as well. We 00:07:43.897-->00:07:49.736 don't group them, uh with, uh, with one exception. So, we had about two thousand four hundred 00:07:49.736-->00:07:54.908 people took part in those, those surveys. Which we ran through Facebook. So, the questions we 00:07:54.908-->00:08:01.748 asked, were uh, your, how much do, ya know do you agree? And it was a binary choice, the binary 00:08:01.748-->00:08:06.853 choice in the uh, question stemmed from work from Professor Hetherington and Elizabeth Suhay 00:08:06.853-->00:08:13.193 who had looked at uh, previously looked at uh, Authoritarianism in American politics in 2011. 00:08:13.193-->00:08:17.063 So, we wanted to try and replicate some of their work, so. Do you have nothing to fear? 00:08:17.063-->00:08:21.034 Do you agree or disagree with you have nothing to fear if you have nothing, you have nothing 00:08:21.034-->00:08:26.039 to fear if you have nothing to hide? So, it was, the questions replicate our EU referendum 00:08:28.808-->00:08:34.781 work. Then the next question was, uh it is acceptable that law enforcement agencies have 00:08:34.781-->00:08:40.253 the right to access the content of citizen's online communication without a warrant 00:08:40.253-->00:08:44.958 in order to investigate terrorism. That question was from the, the Hetherington 00:08:44.958-->00:08:51.131 study. Then the next question we ask is, it is acceptable that immigrants and visitors from 00:08:51.131-->00:08:55.935 potentially dangerous countries should have to reveal their social media account passwords 00:08:55.935-->00:09:00.874 to border agents. So, we, we used the UK samples, but you can do this on a US sample set, too. 00:09:04.978-->00:09:10.950 The next question, which came from a survey on attitudes to privacy in 2016, were simply the 00:09:10.950-->00:09:15.955 darknet should be shut down. Now quite sure how you do that, but um, but it comes from a survey. 00:09:19.059-->00:09:24.464 So, we wanted to, you know it gave me a chuckle so we added that in. And the final question 00:09:24.464-->00:09:29.135 we asked was companies should not be allowed to develop technologies that prevent law 00:09:29.135-->00:09:34.274 enforcement from accessing your online conversations. And that came from that same uh, privacy 00:09:34.274-->00:09:40.980 survey. So, going back to our surveys I mentioned we treat them differently because, uh 00:09:40.980-->00:09:47.487 there were different um, B, studies B and A have, were, were slightly different so we treated 00:09:47.487-->00:09:53.426 them different statistically. Uh the other thing that happened was that in March uh, late March 00:09:53.426-->00:09:59.866 there was a terrorist attack in London. Um, so what we wanted to do was, well see what the effect 00:09:59.866-->00:10:05.171 was of the terrorist attack attitudes to online surveillance. And there's been 00:10:05.171-->00:10:10.777 previous work in that space, uh researchers have looked at the differing attitudes before and 00:10:10.777-->00:10:16.749 after the Charlie Hebdo attacks in, in Paris. So, what we see from the results of uh surveys 00:10:16.749-->00:10:22.755 A1 and A2, the ones without the numeric question is that uh, for most of the questions you get 00:10:22.755-->00:10:28.328 around a sort of a thirty, thir- ya know, thirty percentage agree with the statement. Uh only one 00:10:28.328-->00:10:33.333 of those was statistically um, different before and after the attacks. And that was the 00:10:36.002-->00:10:40.773 banning the anti-surveillance technology question. Uh the rest while it looks like there is a 00:10:40.773-->00:10:45.078 visual difference, well there is a visual difference, um we didn't see a statistically 00:10:45.078-->00:10:50.884 significant different between opinions. Um the thing that interested me or fascinated me 00:10:50.884-->00:10:55.688 the most about the results is that there was uh ya know more support for shutting down the 00:10:55.688-->00:11:00.627 dark net, people just want to shut that circuit down essentially. Um, so looking at 00:11:02.896-->00:11:09.702 studies B1 and B2, uh which we can, the, the B2 study was conducted forty-one days after 00:11:09.702-->00:11:15.542 the, uh, a- attacks. We see that there's almost no difference in opinions uh from before and 00:11:15.542-->00:11:20.280 after. So, the opinion sort of drop off after the, uh, after the terrorist attacks in those 00:11:20.280-->00:11:25.285 surveys. So, we see broadly the, the level of support there is hovers, hovers around th- thirty 00:11:27.987-->00:11:32.425 percent and we'll get a little, we'll get to that a little more in the presentation. So, looking 00:11:32.425-->00:11:38.498 at the role of Authoritarianism in, uh, in these attitudes when we look at the nothing to hide, 00:11:38.498-->00:11:44.037 nothing to fear question, uh in particular. We see, um you know somewhere between a one and a 00:11:44.037-->00:11:48.808 half to a two and a half uh, times inc- increase between those who agree with the 00:11:48.808-->00:11:55.515 statement being more Authoritarian uh versus those who disagree with the statement. 00:11:55.515-->00:12:01.387 Uh, we also performed like, um, uh, uh the logistic regression essentially to look at plotting 00:12:01.387-->00:12:05.959 the odds and some other statistical methods and we find the same difference. You know, 00:12:05.959-->00:12:10.697 broadly through all of them. In terms of, the, the other questions we see sort of broadly 00:12:10.697-->00:12:14.567 similar results as well around wiretapping, shutting down the dark net, banning 00:12:14.567-->00:12:19.105 anti-surveillance technology, and extreme vetting. So, we see consistently, not just across 00:12:19.105-->00:12:24.110 the questions but also between all of the different age groups and sexes uh that the, these 00:12:26.412-->00:12:29.682 opinions tend to, tend to be fairly consistent. So, assuming that Authoritarianism is more a, 00:12:29.682-->00:12:31.684 a constant, what is it that can influence people's uh, differing views? So, it's said, uh by a 00:12:31.684-->00:12:33.686 gentleman called Bob, uh, Altemeyer who developed the uh, what was called the Right Wing 00:12:33.686-->00:12:38.691 Authoritarian Scale in 1996 that uh Authoritarians are characterized as being like ten 00:12:43.997-->00:12:49.002 steps closer to the panic button. So, we saw a plot there on, on this graph and show the 00:13:02.882-->00:13:09.656 blue line is essentially the, the baseline of support for, uh for surveillance. Then there 00:13:09.656-->00:13:14.661 are, uh two streams of research that have looked at this previously. The first stream of 00:13:17.330-->00:13:22.335 research uh, by Feldman and Stenner, had uh, it essentially argued that a threat inc-, the, 00:13:24.737-->00:13:28.808 the perception of threat increases the support among those who are higher in 00:13:28.808-->00:13:35.615 Authoritarianism. So, uh the, the folks at the higher end of the scale are going to support 00:13:35.615-->00:13:40.953 it more if they are more concerned about the threats of terrorism. Another stream of 00:13:40.953-->00:13:46.893 research by Hetherington and Suhay in two thousand and- 2011, found that the threat increased 00:13:46.893-->00:13:53.132 su-, support among those lower in uh, Authoritarianism so um, people at the higher end didn't 00:13:53.132-->00:13:58.337 really change their opinion if something, um ya know if they perceived a threat to be high 00:13:58.337-->00:14:03.810 but those much lower more likely to adopt an Authoritarianism position, uh, if they perceived 00:14:03.810-->00:14:10.616 the threat to be high. Uh, the researchers that looked at the difference in uh, before and 00:14:10.616-->00:14:15.488 after the Charlie Hebdo attack which are mentioned a little bit early looked at this and suggest 00:14:15.488-->00:14:21.561 that both are partly right. And that it's anxiety that, um is responsible for causing the 00:14:21.561-->00:14:25.865 uplift in those that are lower in Authoritarianism if they are concerned about the threat of 00:14:25.865-->00:14:31.871 terror versus uh, anger in uh, people higher in Authoritarianism, if they 00:14:31.871-->00:14:36.876 perceived the threat of terror to be high. So, in the study that we conducted uh what we've 00:14:38.878-->00:14:45.184 done in all the studies we had uh, uh a question simply asked, uh, how worried are you that you 00:14:45.184-->00:14:50.156 might personally become a victim uh, of a terrorist attack? And that was coded. People could 00:14:50.156-->00:14:55.428 answer in like four ways from like I'm not worried at all to, yea I'm very concerned. And that 00:14:55.428-->00:15:01.234 replicates the work from Hetherington and Suhay. So, what we did, is we used a logistic 00:15:01.234-->00:15:06.773 regression. Essentially uh, replicating Hetherington's work. Um plotting the probability for 00:15:06.773-->00:15:11.043 support for each of those questions against Authoritarianism and then 00:15:11.043-->00:15:17.784 looking at the effect of people's perceptions on, uh, on threats. So, Hetherington and 00:15:17.784-->00:15:23.523 Suhay's work, I figured out that I- I- I am just so sorry about the, the bad quality photo from 00:15:23.523-->00:15:27.093 uh, from Hetherington's book. But here you can see is the effect that he was talking 00:15:27.093-->00:15:32.999 about. Essentially people lower in Authoritarianism, you see that the support, uh really 00:15:32.999-->00:15:38.905 increases from about twenty-five percent of support on their wiretapping question to about uh 00:15:38.905-->00:15:43.609 seventy-five, eighty percent probability of support. And you'll see at the higher end of 00:15:43.609-->00:15:50.316 Authoritarianism there's really not a great deal of difference. So, in uh, our study, in terms 00:15:50.316-->00:15:56.722 of the, uh nothing to hide nothing to fear question, we see essentially a similar sort of 00:15:56.722-->00:16:01.861 effect happening to what Hetherington and Suhay found. Um, so going from about an 00:16:01.861-->00:16:07.133 eighteen percent probability of support for those who are not concerned um and are lower in 00:16:07.133-->00:16:12.972 Authoritarianism. That jumped to about a sixty percent um level of support. But there's not many 00:16:12.972-->00:16:18.644 in that category. Uh and at the higher end there's almost no difference. Looking at all of 00:16:18.644-->00:16:23.516 the other questions uh, at least for wiretapping and banning anti-surveillance tech we see a 00:16:23.516-->00:16:28.421 similar sort of trend going on. Um, I'll talk about extreme vetting and shutting down the 00:16:28.421-->00:16:34.627 dark net shortly. So, extreme vetting is uh, is, is an anomaly but Hetherington had seen in uh, 00:16:34.627-->00:16:40.099 or, had seen a similar sort of uh interaction in one of his other questions. So, that needs 00:16:40.099-->00:16:45.304 more, more analysis. But I didn't want to shirk away from calling that one out. Uh the 00:16:45.304-->00:16:51.611 other thing, which interested me was that you, here you see reflected the high level of 00:16:51.611-->00:16:56.816 support for shutting down the dark net and you see how that jumps up even from like a medium 00:16:56.816-->00:17:01.254 high, very high level. You see the interactions so that if people are concerned of the 00:17:01.254-->00:17:06.259 threat of terror, you know, there's broad support for uh, tackling the, the dark net. So, 00:17:08.461-->00:17:15.434 okay, now we've looked at how people differ around the, the uh disposition of Authoritarianism. 00:17:15.434-->00:17:22.375 Uh we can see the role, uh perception of threat has uh on people's attitudes. Now we look 00:17:22.375-->00:17:28.114 at, okay how can we target those different groups of people? How can we target people who are low 00:17:28.114-->00:17:33.119 in Authoritarianism versus people who are high in Authoritarianism? So essentially 00:17:35.187-->00:17:40.559 a lot of research is looked at, um the relationship between age, sex, geography, Facebook 00:17:40.559-->00:17:46.432 interest, Facebook activity and such like and personality. What we want to be able to do here is 00:17:46.432-->00:17:50.836 almost get uh, personality, in our case we use Authoritarianism, put it through 00:17:50.836-->00:17:56.409 a function and get the age, sex, geography and Facebook interests that are going to be most 00:17:56.409-->00:18:01.314 closely associated with people that are either low or high in Authoritarianism. So, if you 00:18:01.314-->00:18:07.720 like, we create two book-it, one book-it for low Authoritarian and one book-it for uh, high 00:18:07.720-->00:18:12.725 Authoritarian. So, onto the study on our, our findings. Uh first of all, tackling age and 00:18:15.728-->00:18:20.733 sex. What we saw in the EU Referendum studies was that the difference between female voters 00:18:24.337-->00:18:29.342 who voted to leave or remain in the EU had much closer level of Authoritarianism. There wasn't a 00:18:32.078-->00:18:36.415 great deal to separate them and indeed looking at the results there are very low percentages 00:18:36.415-->00:18:41.420 of females in eighteen to thirty-five age group who wanted to uh leave, the, the EU. We 00:18:43.789-->00:18:48.794 also saw that reflected in uh the studies that we, we'd look at there. It, with regards to 00:18:50.997-->00:18:55.901 uh, the, um the nothing to fear nothing to hide argument, online um, communication and 00:18:55.901-->00:19:01.107 surveillance. You'll see that as females get older the difference extends so there is a starker 00:19:01.107-->00:19:06.912 difference. And that gets like more consistent with males. Of males, there was, uh there was 00:19:06.912-->00:19:12.985 differences at every age. So, in terms of the book-its we created and we created these book-its so 00:19:12.985-->00:19:18.491 we could add them into the Facebook advertising audiences. Uh, we have the low 00:19:18.491-->00:19:23.896 Authoritarian groups, of males of any age but females only if they're under thirty-five and 00:19:23.896-->00:19:28.634 for the high Authoritarian category we used males of any age and females who are 00:19:28.634-->00:19:33.639 thirty-five and over. So, that's a simple uh, like age and sex classification. And, uh it, it 00:19:36.142-->00:19:42.481 is, you know rather typically simple but as we see it becomes quite effective. So, for 00:19:42.481-->00:19:47.486 geography, uh Rentfrow and uh, and other researchers have looked at regional differences 00:19:49.688-->00:19:55.261 in personality. These are maps of the United Kingdom and the letters at the bottom reflect 00:19:55.261-->00:20:00.132 what's called the Big Five personalities. So, you've got openness to experience, uh which 00:20:00.132-->00:20:05.771 is you know strongly correlated with creativity. Conscientiousness, which is more 00:20:05.771-->00:20:10.509 concerned about some of the, the traditions and what have you or relate, or correlates with that. 00:20:10.509-->00:20:17.016 You've got extroversion, agreeableness and neuroticism. So, he's looked at different 00:20:17.016-->00:20:23.923 areas with a, a very large uh samples that across the UK and found different regional, uh 00:20:23.923-->00:20:27.927 regional differences. He's also done that with the United States as well though if you're 00:20:27.927-->00:20:33.032 interested in those regional differences you can check that out. So, he's presented uh, a 00:20:33.032-->00:20:37.870 lot of results and findings for each of about three hundred different local area districts 00:20:37.870-->00:20:43.476 in the United Kingdom. So, what we did is we mapped Authoritarianism to the big five 00:20:43.476-->00:20:49.782 using uh some matter analysis from uh researchers Sibley and Duckett. And in terms of 00:20:49.782-->00:20:55.855 Authoritarianism it's classified uh or high Authoritarianism is classified by high degrees of 00:20:55.855-->00:21:01.427 conscientious and lower degrees of openness mostly. There's a slight difference in neuroticism 00:21:01.427-->00:21:07.266 but we can you know, pretty safely ignore that. So, for our research what we did, we plotted 00:21:07.266-->00:21:13.606 openness on the uh, openness here, but we reverse it. So, we get low openness essentially. 00:21:13.606-->00:21:18.611 Um, against conscientiousness. And then we take the results from Rentfrow's study of uh over 00:21:21.814-->00:21:26.152 three hundred areas in the UK and we plot them on a graph that's a little bit like this. 00:21:26.152-->00:21:30.823 So, you get the high Authoritarians at the top right hand corner and you get the low 00:21:30.823-->00:21:37.663 Authoritarians towards the uh bottom left. And um, since it's a hacker conference it's quite 00:21:37.663-->00:21:43.469 interesting that the least Authoritarian place in the United Kingdom is called Hackney 00:21:43.469-->00:21:48.474 in London. And I didn't make that up for the, for the talk. But um, so we chose some towns, 00:21:51.644-->00:21:57.950 um that were most likely to be in our opinion high Authoritarian and some towns and 00:21:57.950-->00:22:03.088 cities that were most likely to be low in Authoritarianism. We also looked at like how many 00:22:03.088-->00:22:07.693 people were there for reach and stuff like that. And so, we have some other characteristics that 00:22:07.693-->00:22:12.231 we, we looked at. The towns that we chose for low Authoritarian were towns that most of you have 00:22:12.231-->00:22:19.104 probably heard of, Cambridge, Liverpool, Edinburgh and now you have all heard of Hackney. Um, 00:22:19.104-->00:22:23.242 and in the high Authoritarian town we've got high Authoritarian districts we've 00:22:23.242-->00:22:27.913 got places like Basildon, Thurrock and Mansfield. Uh and Swindon, you might have heard of 00:22:27.913-->00:22:34.086 Swindon if you like the game, like my son does, The Amazon Frog and Pungence's YouTube 00:22:34.086-->00:22:38.490 channel. Other than that, I'm not sure that any of those towns are that famous outside of the 00:22:38.490-->00:22:44.830 United Kingdom. So now we've got a book-it for age and sex and we've got the regions that we're 00:22:44.830-->00:22:50.536 going to use in our Facebook advertising. And incidentally the results that we found using 00:22:50.536-->00:22:55.741 just a simple logistic regression were actually really good at correctly classifying 00:22:55.741-->00:23:00.813 the EU Referendum vote. And we did the same thing when we looked at the uh regional 00:23:00.813-->00:23:06.952 differences in the US, um predicting the, uh recent election that you guys had. So, 00:23:06.952-->00:23:10.956 onto Facebook interests. There's a tool here called the PreferenceTool, uh that's 00:23:10.956-->00:23:16.262 available to researchers where you use like slider bars if you like to put in uh difference 00:23:16.262-->00:23:22.034 effects of personality and that shows you some of the Facebook interests that those groups use. 00:23:22.034-->00:23:27.406 Now, we did not use the preference tool because it's available only to uh, to 00:23:27.406-->00:23:33.412 researchers. Um, and we did, we didn't need to do that really because we looked at some of the 00:23:33.412-->00:23:38.684 Facebooks interests that we already knew were highly correlated with Authoritarianism 00:23:38.684-->00:23:44.423 and low Authoritarians. So, the low Authoritarian category we chose just simply the interest 00:23:44.423-->00:23:50.896 of liberalism, uh which is a Facebook interests. And um, the Guardian newspaper which I guess 00:23:50.896-->00:23:56.168 is like your Washington Post, I guess. Uh, for the high Authoritarian category we chose 00:23:56.168-->00:24:01.106 conservatism and, and the Mail Online. I'm not sure what the equivalent US newspaper is. 00:24:07.179-->00:24:10.149 [laughter] Somebody shouted something out but I didn't hear it. But um, so the, the baseline 00:24:10.149-->00:24:12.151 we've got to, to, to beat if you like is uh the nothing to hide, nothing to fear argument we had 00:24:12.151-->00:24:15.988 uh sort of a baseline of thirty-eight percent support for that particular argument in the 00:24:15.988-->00:24:21.193 studies that we use in this sample set. For the targeted audience just based on uh, age, 00:24:21.193-->00:24:26.198 sex and geography we saw a drop from thirty-eight percent to twenty-two, uh thirty-two 00:24:29.935-->00:24:34.940 percent but it wasn't statistically significant. We saw an uplift in um, support for 00:24:40.279-->00:24:45.284 the high Authoritarian towns from thirty-eight to forty-six. Um when we added the Facebook 00:24:47.820-->00:24:54.626 interests in that we looked at for the correlation to uh to, to Authoritarianism we saw that we 00:24:54.626-->00:24:59.098 started to get significant results although for the first group it was relatively low. 00:24:59.098-->00:25:03.402 Possibly due to the overall skew to look towards low Authoritarianism anyway. So, 00:25:03.402-->00:25:09.441 from thirty-eight agreement with the argument to twenty-five is not bad. Uh, for a very simple 00:25:09.441-->00:25:14.780 experiment. And in the high Authoritarian category adding in personality we see the jump go 00:25:14.780-->00:25:19.785 from thirty-eight percent to sixty-one percent, um which was highly statically significant. A 00:25:22.154-->00:25:28.093 quick note on the significance, um if, if I'm doing this in the wild I might not be so 00:25:28.093-->00:25:33.265 interested or worried about P levels and stuff like that. If I consistently see the results 00:25:33.265-->00:25:37.936 like that maybe I just take a gamble on it anyway and it might give me sort of a higher return 00:25:37.936-->00:25:43.642 on my advertising investment. So, okay, we've seen the differences in personality and 00:25:43.642-->00:25:48.647 we've seen how we can target different groups based on just some um, simple factors that uh, 00:25:50.849-->00:25:55.854 uh enable us to reach audiences with uh significantly different views on online communication 00:25:58.590-->00:26:04.963 surveillance. But, now we want to turn to, well what about persuasive ads and uh, targeting 00:26:04.963-->00:26:10.135 of ads? So, this is uh Alexander Nix from Cambridge Analytica and he states and this video is, is 00:26:10.135-->00:26:15.941 online, if you knew the personality of the people you're targeting you can nuance your 00:26:15.941-->00:26:20.913 messaging to resonate more effectively with those key audience groups. So, there's 00:26:20.913-->00:26:26.118 been some research in this space. The study by Hirsh, Kang and Bodenhausen. They looked at, 00:26:26.118-->00:26:31.423 they created five separate telephone uh ads, that would design to appeal to different 00:26:31.423-->00:26:36.528 levels of the Big Five of personality. So, this is the app that they used or at least this 00:26:36.528-->00:26:41.533 is the text that they used and uh, in uh app that was aimed at uh, extroversion. The XPhone, 00:26:43.569-->00:26:49.808 with Xphone you'll always be where the excitement is. And then for neuroticism, they have 00:26:49.808-->00:26:54.580 Stay safe and secure with the Xphone. And they always saw statistically significant 00:26:54.580-->00:27:01.153 differences. Uh, Matz, Popov, Kosinski and Stillwell did a similar thing looking at aiming 00:27:01.153-->00:27:06.158 at introverts and extroverts, uh for a beauty product and they saw that indeed uh, targeting 00:27:08.727-->00:27:15.133 those ads based on, um, on the personality had uh, significant uh effect on their click through 00:27:15.133-->00:27:20.639 rates so their return on their advertising investment. So, moving onto our study and 00:27:20.639-->00:27:25.444 results, what we did was we had ads that were aimed at um, people in the high Authoritarian 00:27:25.444-->00:27:31.683 book-it and ads that were targeted at people who were in the low Authoritarian book-its. 00:27:31.683-->00:27:34.953 And we split that into, uh to ads that are designed to be pro-surveillance and ads that 00:27:34.953-->00:27:39.958 are uh, designed to be anti-surveillance. So, the first ad for pro was really appealing 00:27:42.828-->00:27:49.401 to uh, to the high Authoritarian categories. Terrorists, don't let them hide online. Um, so 00:27:49.401-->00:27:56.108 hopefully invoking some anger there. For the low Authoritarian group, we tried to highlight 00:27:56.108-->00:28:01.413 that uh there was more than just terrorism uh, online. We had the tagline, Crime doesn't stop 00:28:01.413-->00:28:06.218 where the internet starts. And I know it's cheesy but um, the, the thinking was that people 00:28:06.218-->00:28:10.289 would look at uh, uh broader crimes like human traffic-, trafficking and child 00:28:10.289-->00:28:15.294 exploitation. So, for the uh low authoritarian group with the anti-surveillance message we 00:28:19.464-->00:28:23.302 used the image of uh, Anne Frank and said you know, do you really have nothing to fear if you've 00:28:23.302-->00:28:28.307 got nothing to hide? And for the high Authoritarian group we used an image of the D-Day landings 00:28:30.943-->00:28:37.416 um appealing to the Authoritarian characteristics of um, affiliation with military 00:28:37.416-->00:28:42.521 and respect for elders. So, those were the ads we categorized and then in looking 00:28:42.521-->00:28:47.359 at the results of those ads we, uh references them here and these are the self-reported 00:28:47.359-->00:28:52.664 levels of Authoritarianism and people that took part of our surveys or uh, samples to uh, 00:28:52.664-->00:28:57.302 to, to rate the ads. So, they were asked a bunch of questions, like how much do you like that 00:28:57.302-->00:29:04.076 ad? How much does the ad resonate with my beliefs, uh how persuasive is the ad, how likely 00:29:04.076-->00:29:08.814 am I to click like, or share on the app? That, that kind of thing. And we split that then in 00:29:08.814-->00:29:14.753 people's self-reported um, self-reported groups. So, this was, this was actually where 00:29:14.753-->00:29:21.193 the, where the uh persuasive of the ad is rated. And that is, that is what people's level of 00:29:21.193-->00:29:27.666 Authoritarianism were classed as. So, high, medium and low. Okay, so the performance of the 00:29:27.666-->00:29:32.304 high Authoritarian ad. We see that people who are high in Authoritarianism like the ad 00:29:32.304-->00:29:37.743 much more than people who were like medium and low. When we moved to the ad for the low 00:29:37.743-->00:29:41.713 Authoritarian group we see people who were high still like it. It's kind of bought in 00:29:41.713-->00:29:46.618 because they're, they're, they're more pre, more pro uh surveillance anyway. But we see 00:29:46.618-->00:29:52.190 then an uplift in uh, uh well an expected uplift in the low and medium. We see an uplift in the 00:29:52.190-->00:29:58.563 low and medium Authoritarian categories. So, the ad seems to be having the sort of desired 00:29:58.563-->00:30:04.369 effect if you'd like. For the low Authoritarian category in anti-surveillance we see kind of 00:30:04.369-->00:30:09.941 a reversal of the higher auth-, uh pro add where people who are low in Authoritarianism like it 00:30:09.941-->00:30:16.748 and people who are high don't like it so much. Um, when we target the ad towards or when we 00:30:16.748-->00:30:22.120 created the ads for um, the high Authoritarian group we see a similar sort of uplift to the 00:30:22.120-->00:30:27.826 one we've got in the top right hand corner. Um, and there's a broad level of uh increase of 00:30:27.826-->00:30:33.799 support from those medium and high. So, so that's people's responses to how much they like 00:30:33.799-->00:30:38.837 the ad. Then we took it a step further and tested it in the wild with, with um Facebook ads 00:30:38.837-->00:30:43.408 designed or actually, there were Facebook posts and we used the boosting in the Facebook 00:30:43.408-->00:30:49.681 advertising, um to appeal to those different uh, those different audiences. So, here we 00:30:49.681-->00:30:55.053 see the Facebook target audiences of high and low Authoritarian. There's only tow, 00:30:55.053-->00:30:59.991 tow group from our target audience. And then um, the, the ads in those, um boxes uh re- 00:31:03.762-->00:31:09.968 relate to the ads we just saw. So, what we do for this, using the, the Facebook advertising 00:31:09.968-->00:31:15.173 essentially is then look at the click through rates or actually, this is actually the ratios of 00:31:15.173-->00:31:20.545 likes and shares by, by the level of reach for each ad. Um so the numbers look pretty low 00:31:20.545-->00:31:26.852 but I actually um you know those are fairly consistent with click through rates that you would 00:31:26.852-->00:31:32.924 expect. So, for the ad aimed at the high auth group we see as expected people who are in high 00:31:32.924-->00:31:38.230 in Authoritarian like the ad, people who are low think it sucks. Uh, in the low 00:31:38.230-->00:31:44.703 Authoritarian ad uh there was a, a lot less traction altogether and fakebook gives you 00:31:44.703-->00:31:49.975 forewarning. It basically says that you know, ads that got a lot of text uh, do less well. 00:31:49.975-->00:31:55.180 You might want to consider changing your ad. But what I draw your attention to, at least 00:31:55.180-->00:32:01.486 is that there was a, an expected difference in the, the correlation coefficient if you 00:32:01.486-->00:32:06.491 like. So, the, the gap uh the gap reduced between the, the low and high. So, if we move onto 00:32:10.195-->00:32:15.834 the low Authoritarian ads, uh for anti-surveillance we see the expected results that low, 00:32:15.834-->00:32:20.839 people in the low Authoritarian target like the ad or interact with the ad much more than those 00:32:20.839-->00:32:26.678 in the high Authoritarian or more than those in the high. But then what we see for the ad 00:32:26.678-->00:32:33.185 that's targeted at people in the high Authoritarian, uh, uh, designed to appeal to people who 00:32:33.185-->00:32:36.888 are high in Authoritarianism and anti. We still get a lot of people who are low in 00:32:36.888-->00:32:41.193 Authoritarian doesn't like it. Because they like the message of I don't, you know, they don't, 00:32:41.193-->00:32:47.999 um, on, online communication surveillance or the idea of it. But people higher end support it 00:32:47.999-->00:32:51.903 a lot more. That was kind of surprising to us. We, we thought we'd actually get more of a 00:32:51.903-->00:32:58.143 backlash from those, uh those groups. Uh rather than, rather than the, the interaction so 00:32:58.143-->00:33:04.583 again we see the correlation coefficient reduces as we would have expected from the ads. But 00:33:04.583-->00:33:07.485 uh, some of you might be thinking, ok well, that's well and good, we're seeing how much 00:33:07.485-->00:33:11.389 people are liking click on ads, but does persuasion actually work. You haven't actually told 00:33:11.389-->00:33:17.329 us that, fair, fair comment, I haven't. Uh and there is very little research in, into that. 00:33:17.329-->00:33:22.400 But this research by, um, researchers at the University of Mannheim in, in Germany, 00:33:22.400-->00:33:27.472 Jungherr, Wuttke, and Co. What they did is say, uh took people's attitudes to 00:33:27.472-->00:33:32.911 transatlantic trade and the sample set was a large, it was about 8-->000 people. Then they 00:33:32.911-->00:33:38.650 sent some of them some ads, um explaining or some literature about the benefits of 00:33:38.650-->00:33:44.689 transatlantic trade, uh and they found that over, the, the baseline which is the zero line, 00:33:44.689-->00:33:49.361 the, the control of people who didn't receive the literature, they found that people became 00:33:49.361-->00:33:55.500 more, much more supportive after they, of transatlantic trade after they received the uh, the 00:33:55.500-->00:34:00.772 literature. And they look at um, the difference in opinions shortly after receiving the 00:34:00.772-->00:34:05.977 literature and then some time after and they did not- they did notice a drop off in uh, the 00:34:05.977-->00:34:11.149 time. But uh you know it suggests that if you're wanting to be, get persuasive material 00:34:11.149-->00:34:16.988 out there, the time is of the essence. So, okay, now we've seen, you know you can get the 00:34:16.988-->00:34:22.160 message out there but what if the message is erroneous? SO how would you go about sort of 00:34:22.160-->00:34:27.332 debunking propaganda. Well to, to this, well it's not really how we care about debunking it. 00:34:27.332-->00:34:31.202 TI's what are the challenges of debunking propaganda. So, for this we turn to some research 00:34:31.202-->00:34:38.209 from Professor uh, Dan Kahan at Yale University. He divided participants in his experiment 00:34:38.209-->00:34:45.216 into 4 categories, A, B, C, and D. For categories A and B, he gave them a numerical question 00:34:45.216-->00:34:50.221 about how well uh, skin cream does at treating a rash. Um, and he gave them a number grid which 00:34:52.324-->00:34:58.830 like this. So, group A received this number grid and were asked to say, okay did skin cream 00:34:58.830-->00:35:04.869 essentially make the rash better or did it make it worse? And people had to go figure that 00:35:04.869-->00:35:10.842 out. Whereas group B, the text was uh, slightly changed. Um but the numbers remained the same, 00:35:10.842-->00:35:14.446 so the rash gets better or the rash gets worse. Well they found that people did roughly the, uh 00:35:14.446-->00:35:19.451 had roughly the same um, ability to answer the, the, the question. But when he changed 00:35:22.988-->00:35:29.494 the question to one about uh, does gun control reduce or increase crime, he found that 00:35:29.494-->00:35:34.699 people's ideological beliefs really hampered their ability to answer the question. And I'll 00:35:34.699-->00:35:39.304 demonstrate that because we replicated uh Kahan's study in our EU Referendum work. But we 00:35:39.304-->00:35:43.508 changed the question from one about gun control to one about immigration increasing or 00:35:43.508-->00:35:48.513 decreasing crime. So here we see how um, how people do at the skin cream ad if you'd like so 00:35:52.217-->00:35:57.222 uh, broadly consistent. Uh just under 60% getting it right in the referendum result, leave vs. 00:35:59.724-->00:36:04.663 remain. But when we ask about does immigration uh, decrease crime people who voted to, to 00:36:07.365-->00:36:12.370 leave would just not have any of that or were unable to answer the question or their 00:36:12.370-->00:36:17.809 performance tanked essentially. Whereas people who voted to remain their performance at 00:36:17.809-->00:36:24.015 answering the question actually increased. Then when we switch it to immigration increases 00:36:24.015-->00:36:29.621 crime then the leave voters start doing better again. But the remain voters performance 00:36:29.621-->00:36:34.626 tanks. Uh, and uh, the, the, the research that he's conducted is, is termed 'Ideologically 00:36:36.761-->00:36:42.600 Motivated Reasoning' and it was dubbed, uh in one article the most depressing brain finding 00:36:42.600-->00:36:48.206 ever, which I liked. His study was from a little while ago but it's pretty exciting. He's got 00:36:48.206-->00:36:53.645 uh, a web page you should go check out called the cultural cognition uh project and it's 00:36:53.645-->00:37:00.518 well worth a read. So, in our study, um we sort of replicated that research again but used 00:37:00.518-->00:37:06.458 the, the, you know the attitudes to online surveillance. So in the skin cream ads we see 00:37:06.458-->00:37:10.962 similar sorts of trends between those that agree and disagree with the nothing to fear nothing 00:37:10.962-->00:37:17.068 to hide argument. But then when we suggest that surveillance actually increases the threat of 00:37:17.068-->00:37:22.574 a terrorist attack than those who agree with the statement, their performance drops quite 00:37:22.574-->00:37:26.678 significantly. Whereas those who disagree, their performance drops a bit. Um and I would 00:37:26.678-->00:37:31.249 suggest that the performance drops a bit because the numerical question is actually, 00:37:31.249-->00:37:36.187 the, the text in it is actually harder to read than it was for the immigration and gun control. 00:37:38.223-->00:37:42.761 When we switched to a surveillance actually does decrease the threat then we see 00:37:42.761-->00:37:49.734 the interaction just not as starkly. Uh however the interaction does seem to 00:37:49.734-->00:37:54.739 increase as people get older. Um, so you see a more significant result as people get 00:38:04.582-->00:38:09.587 over 35 and here we just show 44 and over. So, in terms of uh, wrapping the, wrapping the talk 00:38:12.757-->00:38:17.762 up. What we've seen is that Authoritarianism plays a part in why, why people differ or why 00:38:20.365-->00:38:26.804 their opinions differ. We've found that the perception of, of threat plays a part in actually 00:38:26.804-->00:38:33.478 influencing or nudging people from a baseline position to a different position. Uh we found 00:38:33.478-->00:38:39.284 that we are able to actually target people with different attitudes to online uh, 00:38:39.284-->00:38:45.757 surveillance. Uh just by using uh some simple knowledge about the, the psychological traits 00:38:45.757-->00:38:50.828 associated with Authoritarianism. Uh we find that the persuasiveness of, of 00:38:50.828-->00:38:55.767 ads does indeed um seem to work and that challenging or tackling misleading information is going 00:38:55.767-->00:39:00.238 to be a, a very challenging problem because people's ability to, to uh interpret uh evidence 00:39:00.238-->00:39:05.243 essentially becomes a lot harder if it goes against their existing beliefs. So question we 00:39:08.980-->00:39:13.985 received uh recently was ok, well if both sides are using this do the effects equal each 00:39:18.890-->00:39:24.495 other out? Uh and I thought about that and though oh yeah in theory they sort of do, uh or 00:39:24.495-->00:39:31.236 could do. But this, and, and another potential theory is that it's possible that one side has 00:39:31.236-->00:39:36.808 a home team advantage essentially. Um, so one of the ways they'd have a home team 00:39:36.808-->00:39:42.814 advantage is, well if you'd got momentum on your side um then you can go with that and just 00:39:42.814-->00:39:48.286 like double down on it. So here we see the shuttin down the dark net question um and a level of 00:39:48.286-->00:39:53.858 support for it. Well if you just go with that and just increase the, the already existing fears 00:39:53.858-->00:40:00.131 that people have. Um, you, it's going to be a lot harder to be in the defense, to, to defend 00:40:00.131-->00:40:06.938 that ar- that argument, than it is to attack the argument if you like. Uh there's a lot uh the 00:40:06.938-->00:40:13.444 role of uh, of fear. So, uh if you've not see it there's a great video on line by Dee 00:40:13.444-->00:40:18.449 Madigan on propaganda. And Dee Madigan is a Australian uh, campaign advertiser. And does a, 00:40:23.021-->00:40:28.960 a really fantastic talk. And she talks about the uh effect of fear based advertising. And she, 00:40:28.960-->00:40:33.731 she says yeah, people have said that, have said to her that negative campaigns really such 00:40:33.731-->00:40:38.736 and they don't' work on my. And her comment was essentially that, yes negative campaigns do 00:40:41.472-->00:40:46.477 work and that's why we do them. Um, something like that. Anyway, paraphrasing. But uh, the fear 00:40:48.546-->00:40:54.385 actually, or increasing fear works. We've seen some studies on that as well from Brader in 00:40:54.385-->00:41:00.692 2005. That shows how political ads motivate and persuade voters uh based on emotion. People, uh, 00:41:00.692-->00:41:05.697 people have higher attention for uh ads that are, uh create more, more fear if you like. And you 00:41:09.000-->00:41:14.005 saw grainy black and white images versus ads that have a, a lower um, versus ads that you 00:41:16.975-->00:41:22.080 use pictures of smiling children if you like in front of flags and you know all that kind of 00:41:22.080-->00:41:28.619 good stuff. So people's attention is drawn to the more negative ads in the first place. 00:41:28.619-->00:41:33.791 So ok, some thoughts on how, um how folks might or how society might begin tackling the 00:41:33.791-->00:41:39.797 problem. One of the things we saw that the effective cognitive biases in the EU referendum uh 00:41:39.797-->00:41:45.370 research that we've, we've done. And that research we're going to be talking about at the 00:41:45.370-->00:41:49.574 international conference on political psychology in October. And we've teamed up with some 00:41:49.574-->00:41:56.147 researchers at Missouri State, to, to help us tackle the challenge of uh, uh extremely 00:41:56.147-->00:42:01.085 non-normal data. Uh, but one of the ways we can tackle this is, uh or tackle effective cognitive 00:42:03.421-->00:42:09.727 biases is perhaps adopt a method that, uh, this lady here Jane Elliott had used. And if you're 00:42:09.727-->00:42:15.400 not familiar with Jane Elliott's work, she basically splits um, participants, it was with 00:42:15.400-->00:42:19.003 children in a classroom but she's done this with older group, into people with blue or 00:42:19.003-->00:42:24.942 brown eyes, uh and uses that as the basis for helping educate about racism and the effects of 00:42:24.942-->00:42:29.313 racism and kind of how that works. So, it's possible that you could do something similar 00:42:29.313-->00:42:34.185 with, in relation to, to cognitive biases to make people aware of that. Um, which would 00:42:34.185-->00:42:38.723 have benefits outside the political campaigns to prevent people getting hustled at uh, 00:42:38.723-->00:42:44.562 car dealerships and stuff like that. The other thing we noticed is that levels of numeracy were 00:42:44.562-->00:42:50.001 right, really shockingly poor in the UK. And I don't think the UK, uh the US is a great deal 00:42:50.001-->00:42:55.006 better. Um, so increasing people's numerical literacy skills is being able to 00:42:57.175-->00:43:02.113 determine that 1-->000 % APR is a bad deal on a credit card. Um, but it also has effects quite, 00:43:05.049-->00:43:10.087 um you, uh, it also has effects in people choosing medical options and what not as, as 00:43:10.087-->00:43:14.992 well. There's a lot of research actually in that area liter-, risk literacy and like making 00:43:14.992-->00:43:20.965 medical based uh choices. So, that needs increasing I think uh all around as well in addition 00:43:20.965-->00:43:25.369 to the biases. And then the next thing which I think is going to be a lot harder to achieve 00:43:25.369-->00:43:29.307 because literally anyone can do this, so if you've got big pockets you're going to get ya 00:43:29.307-->00:43:35.413 know, a lot more attraction. Is really start enforcing or going after people who are uh abusing 00:43:35.413-->00:43:40.418 this sort of, uh this sort of political advertising. So, that wraps up the uh, the 00:43:42.753-->00:43:48.192 presentation. I'm sure folks will have uh, questions. You can reach me at uh, chris at 00:43:48.192-->00:43:54.031 onlineprivacyfoundation dot org. Um, I'm happy to address questions afterwards but we'll 00:43:54.031-->00:43:59.403 have to go outside somewhere I think. And I'll leave some uh, some cards on the table if folks 00:43:59.403-->00:44:04.342 have questions. Aright? Thanks very much. [applause]