Highlights & transcript from Zuckerberg’s 20K-word ethics talk
Mark Zuckerberg says it might be right for Facebook to let people pay to not see ads, but that it would feel wrong to charge users for extra privacy controls. That's just one of the fascinating philosophical views the CEO shared during the first of his public talks he's promised as part of his 2019 personal challenge.
Talking to Harvard Law and computer science professor Jonathan Zittrain on the campus of the university he dropped out of, Zuckerberg managed to escape the 100-minute conversation with just a few gaffes. At one point he said "we definitely don't want a society where there's a camera in everyone's living room watching the content of those conversations". Zittrain swiftly reminded him that's exactly what Facebook Portal is, and Zuckerberg tried to deflect by saying Portal's recordings would be encrypted.
Later Zuckerberg mentioned "the ads, in a lot of places are not even that different from the organic content in terms of the quality of what people are being able to see" which is pretty sad and derisive assessment of the personal photos and status updates people share. And when he suggested crowdsourced fact-checking, Zittrain chimed in that this could become an avenue for "astroturfing" where mobs of users provide purposefully biased information to promote their interests, like a political group's supporting voting that their opponents' facts are lies. While sometimes avoiding hard stances on questions, Zuckerberg was otherwise relatively logical and coherent.
The CEO touched on his borderline content policy that quietly demotes posts that come close to breaking its policy against nudity, hate speech etc that otherwise are the most sensational and get the most distribution but don't make people feel good. Zuckerberg noted some progress here, saying "a lot of the things that we've done in the last year were focused on that problem and it really improves the quality of the service and people appreciate that."
This aligns with Zuckerberg contemplating Facebook's role as a "data fiduciary" where rather than necessarily giving in to users' urges or prioritizing its short-term share price, the company tries to do what's in the best long-term interest of its communities. "There's a hard balance here which is - I mean if you're talking about what people want to want versus what they want- you know, often people's revealed preferences of what they actually do shows a deeper sense of what they want than what they think they want to want" he said. Essentially, people might tap on clickbait even if it doesn't make them feel good.
On working with governments, Zuckerberg explained how incentives weren't always aligned, like when law enforcement is monitoring someone accidentally dropping clues about their crimes and collaborators. The government and society might benefit from that continued surveillance but Facebook might want to immediately suspend the account if it found out. "But as you build up the relationships and trust, you can get to that kind of a relationship where they can also flag for you, 'Hey, this is where we're at'", implying Facebook might purposefully allow that person to keep incriminating themselves to assist the authorities.
But disagreements between governments can flare up, Zuckerberg notes that "we've had employees thrown in jail because we have gotten court orders that we have to turnover data that we wouldn't probably anyway, but we can't because it's encrypted." That's likely a reference to the 2016 arrest of Facebook's VP for Latin Amercia Diego Dzodan over WhatsApp's encryption preventing the company from providing evidence for a drug case.
Decentralizing FacebookThe tradeoffs of encryption and decentralization were a central theme. He discussed how while many people fear how encryption could mask illegal or offensive activity, Facebook doesn't have to peek at someone's actual content to determine they're violating policy. "One of the - I guess, somewhat surprising to me - findings of the last couple of years of working on content governance and enforcement is that it often is much more effective to identify fake accounts and bad actors upstream of them doing something bad by patterns of activity rather than looking at the content" Zuckerberg said.
With Facebook rapidly building out a blockchain team to potentially launch a cryptocurrency for fee-less payments or an identity layer for decentralized applications, Zittrain asked about the potential for letting users control which other apps they give their profile information to without Facebook as an intermediary.
SAN JOSE, CA - MAY 01: Facebook CEO Mark Zuckerberg (Photo by Justin Sullivan/Getty Images)
Zuckerberg stressed that at Facebook's scale, moving to a less efficient distributed architecture would be extremely "computationally intense" though it might eventually be possible. Instead, he said "One of the things that I've been thinking about a lot is a use of blockchain that I am potentially interesting in- although I haven't figured out a way to make this work out, is around authentication and bringing- and basically granting access to your information and to different services. So, basically, replacing the notion of what we have with Facebook Connect with something that's fully distributed." This might be attractive to developers who would know Facebook couldn't cut them off from the users.
The problem is that if a developer was abusing users, Zuckerberg fears that "in a fully distributed system there would be no one who could cut off the developers' access. So, the question is if you have a fully distributed system, it dramatically empowers individuals on the one hand, but it really raises the stakes and it gets to your questions around, well, what are the boundaries on consent and how people can really actually effectively know that they're giving consent to an institution?"
No "Pay For Privacy"But perhaps most novel and urgent were Zuckerberg's comments on the secondary questions raised by where Facebook should let people pay to remove ads. "You start getting into a principle question which is 'are we going to let people pay to have different controls on data use than other people?' And my answer to that is a hard no." Facebook has promised to always operate free version so everyone can have a voice. Yet some including myself have suggested that a premium ad-free subscription to Facebook could help ween it off maximizing data collection and engagement, though it might break Facebook's revenue machine by pulling the most affluent and desired users out of the ad targeting pool.
"What I'm saying is on the data use, I don't believe that that's something that people should buy. I think the data principles that we have need to be uniformly available to everyone. That to me is a really important principle" Zuckerberg expands. "It's, like, maybe you could have a conversation about whether you should be able to pay and not see ads. That doesn't feel like a moral question to me. But the question of whether you can pay to have different privacy controls feels wrong."
Back in May, Zuckerberg announced Facebook would build a Clear History button in 2018 that deletes all the web browsing data the social network has collected about you, but that data's deep integration into the company's systems has delayed the launch. Research suggests users don't want the inconvenience of getting logged out of all their Facebook Connected services, though, they'd like to hide certain data from the company.
"Clear history is a prerequisite, I think, for being able to do anything like subscriptions. Because, like, partially what someone would want to do if they were going to really actually pay for a not ad supported version where their data wasn't being used in a system like that, you would want to have a control so that Facebook didn't have access or wasn't using that data or associating it with your account. And as a principled matter, we are not going to just offer a control like that to people who pay."
Of all the apologies, promises, and predictions Zuckerberg has made recently, this pledge might instill the most confidence. While some might think of Zuckerberg as a data tyrant out to absorb and exploit as much of our personal info as possible, there are at least lines he's not willing to cross. Facebook could try to charge you for privacy, but it won't. And given Facebook's dominance in social networking and messaging plus Zuckerberg's voting control of the company, a greedier man could make the internet much worse.
-TRANSCRIPT - MARK ZUCKERBERG AT HARVARD / FIRST PERSONAL CHALLENGE 2019Jonathan Zittrain: Very good. So, thank you, Mark, for coming to talk to me and to our students from the Techtopia program and from my "Internet and Society" course at Harvard Law School. We're really pleased to have a chance to talk about any number of issues and we should just dive right in. So, privacy, autonomy, and information fiduciaries.
Mark Zuckerberg: All right!
Jonathan Zittrain: Love to talk about that.
Mark Zuckerberg: Yeah! I read your piece in The New York Times.
Jonathan Zittrain: The one with the headline that said, "Mark Zuckerberg can fix this mess"?
Mark Zuckerberg: Yeah.
Jonathan Zittrain: Yeah.
Mark Zuckerberg: Although that was last year.
<laughter>
Jonathan Zittrain: That's true! Are you suggesting it's all fixed?
<laughter>
Mark Zuckerberg: No. No.
<laughter>
Jonathan Zittrain: Okay, good. So-
Jonathan Zittrain: I'm suggesting that I'm curious whether you still think that we can fix this mess?
Jonathan Zittrain: Ah! <laughter>
Jonathan Zittrain: I hope- <laughter>
Jonathan Zittrain: "Hope springs eternal"-
Mark Zuckerberg: Yeah, there you go.
Jonathan Zittrain: -is my motto. So, all right, let me give a quick characterization of this idea that the coinage and the scaffolding for it is from my colleague, Jack Balkin, at Yale. And the two of us have been developing it out further. There are a standard number of privacy questions with which you might have some familiarity, having to do with people conveying information that they know they're conveying or they're not so sure they are, but "mouse droppings" as we used to call them when they run in the rafters of the Internet and leave traces. And then the standard way of talking about that is you want to make sure that that stuff doesn't go where you don't want it to go. And we call that "informational privacy". We don't want people to know stuff that we want maybe our friends only to know. And on a place like Facebook, you're supposed to be able to tweak your settings and say, "Give them to this and not to that." But there's also ways in which stuff that we share with consent could still sort of be used against us and it feels like, "Well, you consented," may not end the discussion. And the analogy that my colleague Jack brought to bear was one of a doctor and a patient or a lawyer and a client or- sometimes in America, but not always- a financial advisor and a client that says that those professionals have certain expertise, they get trusted with all sorts of sensitive information from their clients and patients and, so, they have an extra duty to act in the interests of those clients even if their own interests conflict. And, so, maybe just one quick hypo to get us started. I wrote a piece in 2014, that maybe you read, that was a hypothetical about elections in which it said, "Just hypothetically, imagine that Facebook had a view about which candidate should win and they reminded people likely to vote for the favored candidate that it was Election Day," and to others they simply sent a cat photo. Would that be wrong? And I find- I have no idea if it's illegal; it does seem wrong to me and it might be that the fiduciary approach captures what makes it wrong.
Mark Zuckerberg: All right. So, I think we could probably spend the whole next hour just talking about that! <laughter>
Mark Zuckerberg: So, I read your op-ed and I also read Balkin's blogpost on information fiduciaries. And I've had a conversation with him, too.
Jonathan Zittrain: Great.
Mark Zuckerberg: And the- at first blush, kind of reading through this, my reaction is there's a lot here that makes sense. Right? The idea of us having a fiduciary relationship with the people who use our services is kind of intuitively- it's how we think about how we're building what we're building. So, reading through this, it's like, all right, you know, a lot of people seem to have this mistaken notion that when we're putting together news feed and doing ranking that we have a team of people who are focused on maximizing the time that people spend, but that's not the goal that we give them. We tell people on the team, "Produce the service-" that we think is going to be the highest quality that- we try to ground it in kind of getting people to come in and tell us, right, of the content that we could potentially show what is going to be- they tell us what they want to see, then we build models that kind of- that can predict that, and build that service.
Jonathan Zittrain: And, by the way, was that always the case or-
Mark Zuckerberg: No.
Jonathan Zittrain: -was that a place you got to through some course adjustments?
Mark Zuckerberg: Through course adjustments. I mean, you start off using simpler signals like what people are clicking on in feed, but then you pretty quickly learn, "Hey, that gets you to local optimum," right? Where if you're focusing on what people click on and predicting what people click on, then you select for click bait. Right? So, pretty quickly you realize from real feedback, from real people, that's not actually what people want. You're not going to build the best service by doing that. So, you bring in people and actually have these panels of- we call it "getting to ground truth"- of you show people all the candidates for what can be shown to them and you have people say, "What's the most meaningful thing that I wish that this system were showing us? So, all this is kind of a way of saying that our own self image of ourselves and what we're doing is that we're acting as fiduciaries and trying to build the best services for people. Where I think that this ends up getting interesting is then the question of who gets to decide in the legal sense or the policy sense of what's in people's best interest? Right? So, we come in every day and think, "Hey, we're building a service where we're ranking newsfeed trying to show people the most relevant content with an assumption that's backed by data; that, in general, people want us to show them the most relevant content. But, at some level, you could ask the question which is "Who gets to decide that ranking newsfeed or showing relevant ads?" or any of the other things that we choose to work on are actually in people's interest. And we're doing the best that we can to try to build the services [ph?] that we think are the best. At the end of the day, a lot of this is grounded in "People choose to use it." Right? Because, clearly, they're getting some value from it. But then there are all these questions like you say about, you have- about where people can effectively give consent and not.
Jonathan Zittrain: Yes.
Mark Zuckerberg: So, I think that there's a lot of interesting questions in this to unpack about how you'd implement a model like that. But, at a high level I think, you know, one of the things that I think about in terms of we're running this big company; it's important in society that people trust the institutions of society. Clearly, I think we're in a position now where people rightly have a lot of questions about big internet companies, Facebook in particular, and I do think getting to a point there there's the right regulation and rules in place just provides a kind of societal guardrail framework where people can have confidence that, okay, these companies are operating within a framework that we've all agreed. That's better than them just doing whatever they want. And I think that that would give people confidence. So, figuring out what that framework is, I think, is a really important thing. And I'm sure we'll talk about that as it relates-
Jonathan Zittrain: Yes.
Mark Zuckerberg: -to a lot of the content areas today. But getting to that question of how do you- "Who determines what's in people's best interest, if not people themselves?"Jonathan Zittrain: Yes.
Mark Zuckerberg: -is a really interesting question.
Jonathan Zittrain: Yes, so, we should surely talk about that. So, on our agenda is the "Who decides?" question.
Mark Zuckerberg: All right.
Jonathan Zittrain: Other agenda items include- just as you say, the fiduciary framework sounds nice to you- doctors, patients, Facebook users. And I hear you saying that's pretty much where you're wanting to end up anyway. There are some interesting questions about what people want, versus what they want to want.
Mark Zuckerberg: Yeah.
Jonathan Zittrain: People will say "On January 1st, what I want-" New Year's resolution- "is a gym membership." And then on January 2nd, they don't want to go to the gym. They want to want to go to the gym, but they never quite make it. And then, of course, a business model of pay for the whole year ahead of time and they know you'll never turn up develops around that. And I guess a specific area to delve into for a moment on that might be on the advertising side of things, maybe the dichotomy between personalization and does it ever going into exploitation? Now, there might be stuff- I know Facebook, for example, bans payday loans as best it can.
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: That's just a substantive area that it's like, "All right, we don't want to do that."
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: But when we think about good personalization so that Facebook knows I have a dog and not a cat, and a targeter can then offer me dog food and not cat food. How about, if not now, a future day in which an advertising platform can offer to an ad targeter some sense of "I just lost my pet, I'm really upset, I'm ready to make some snap decisions that I might regret later, but when I make them-"
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: "-I'm going to make them." So, this is the perfect time to tee up
Mark Zuckerberg: Yeah.
Jonathan Zittrain: -a Cubic Zirconia or whatever the thing is that- .
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: That seems to me a fiduciary approach would say, ideally- how we get there I don't know, but ideally we wouldn't permit that kind of approach to somebody using the information we've gleaned from them to know they're in a tough spot-
Mark Zuckerberg: Yeah.
Jonathan Zittrain: -and then to exploit them. But I don't know. I don't know how you would think about something like that. Could you write an algorithm to detect something like that?
Mark Zuckerberg: Well, I think one of the key principles is that we're trying to run this company for the long term. And I think that people think that a lot of things that- if you were just trying to optimize the profits for next quarter or something like that, you might want to do things that people might like in the near term, but over the long term will come to resent. But if you actually care about building a community and achieving this mission and building the company for the long term, I think you're just much more aligned than people often think companies are. And it gets back to the idea before, where I think our self image is largely acting as- in this kind of fiduciary relationship as you're saying- and across- we could probably go through a lot of different examples. I mean, we don't want to show people content that they're going to click on and engage with, but then feel like they wasted their time afterwards. Where we don't want to show them things that they're going to make a decision based off of that and then regret later. I mean, there's a hard balance here which is- I mean if you're talking about what people want to want versus what they want- you know, often people's revealed preferences of what they actually do shows a deeper sense of what they want than what they think they want to want. So, I think there's a question between when something is exploitative versus when something is real, but isn't what you would say that you want.
Jonathan Zittrain: Yes.
Mark Zuckerberg: And that's a really hard thing to get at.
Jonathan Zittrain: Yes.
Mark Zuckerberg: But on a lot of these cases my experience of running the company is that you start off building a system, you have relatively unsophisticated signals to start, and you build up increasingly complex models over time that try to take into account more of what people care about. And there are all these examples that we can go through. I think probably newsfeed and ads are probably the two most complex ranking examples-
Jonathan Zittrain: Yes.
Mark Zuckerberg: -that we have. But it's- like we were talking about a second ago, when we started off with the systems, I mean, just start with newsfeeds- but you could do this on ads, too- you know, the most naive signals, right, are what people click on or what people "Like". But then you just very quickly realize that that doesn't- it approximates something, but it's a very crude approximation of the ground truth of what people actually care about. So, what you really want to get to is as much as possible getting real people to look at the real candidates for content and tell you in a multi-dimensional way what matters to them and try to build systems that model that. And then you want to be kind of conservative on preventing downside. So, your example of the payday loans- and when we've talked about this in the past, your- you've put the question to me of "How do you know when a payday loan is going to be exploitative?" right? "If you're targeting someone who is in a bad situation?" And our answer is, "Well, we don't really know when it's going to be exploitative, but we think that the whole category potentially has a massive risk of that, so we just ban it-
Jonathan Zittrain: Right. Which makes it an easy case.
Mark Zuckerberg: Yes. And I think that the harder cases are when there's significant upside and significant downside and you want to weigh both of them. So, I mean, for example, once we started putting together a really big effort on preventing election interference, one of the initial ideas that came up was "Why don't we just ban all ads that relate to anything that is political?" And they you pretty quickly get into, all right, well, what's a political ad? The classic legal definition is things that are around elections and candidates, but that's not actually what Russia and other folks were primarily doing. Right? It's- you know, a lot of the issues that we've seen are around issue ads, right, and basically sewing division on what are social issues. So, all right, I don't think you're going to get in the way of people's speech and ability to promote and do advocacy on issues that they care about. So, then the question is "All right, well, so, then what's the right balance?" of how do you make sure that you're providing the right level of controls, that people who aren't supposed to be participating in these debates aren't or that at least you're providing the right transparency. But I think we've veered a little bit from the original questionJonathan Zittrain: Yes.
Mark Zuckerberg: -but the- but, yeah. So, let's get back to where you were
Jonathan Zittrain: Well, here's- and this is a way of maybe moving it forward, which is: A platform as complete as Facebook is these days offers lots of opportunities to shape what people see and possibly to help them with those nudges, that it's time to go to the gym or to avoid them from falling into the depredations of the payday loan. And it is a question of so long as the platform to do it, does it now have an ethical obligation to do it, to help people achieve the good life?
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: And I worry that it is too great a burden for any company to bear to have to figure out, say, if not the perfect, the most reasonable newsfeed for every one of the- how many? Two and a half billion active users? Something like that.
Mark Zuckerberg: Yeah. On that order.
Jonathan Zittrain: All the time and there might be some ways that start a little bit to get into the engineering of the thing that would say, "Okay, with all hindsight, are there ways to architect this so that the stakes aren't as high, aren't as focused on just, "Gosh, is Facebook doing this right?" It's as if there was only one newspaper in the whole world or one or two, and it's like, "Well, then what The New York Times chooses to put on it's home page, if it were the only newspaper, would have outsize importance."
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: So, just as a technical matter, a number of the students in this room had a chance to hear from Tim Berners-Lee, inventor of the World Wide Web, and he has a new idea for something called "Solid". I don't know if you've heard of Solid. It's a protocol more than it is a product. So, there's no car to move off the lot today. But its idea is allowing people to have the data that they generate as they motor around the web end up in their own kind of data locker. Now, for somebody like Tim, it might mean literally in a locker under his desk and he could wake up in the middle of the night and see where his data is. For others, it might mean Iraq somewhere, guarded perhaps by a fiduciary who's looking out for them, the way that we put money in a bank and then we can sleep at night knowing the bankers are- this is maybe not the best analogy in 2019, but watching.
<laughter>
Mark Zuckerberg: We'll get there.
Jonathan Zittrain: We'll get there. But Solid says if you did that, people would then- or their helpful proxies- be able to say, "All right, Facebook is coming along. It wants the following data from me and including that data that it has generated about me as I use it, but stored back in my locker and it kind of has to come back to my well to draw water each time. And that way if I want to switch to Schmacebook or something, it's still in my well and I can just immediately grant permission to Schmacebook to see it and I don't have to do a kind of data slurp and then re-upload it. It's a fully distributed way of thinking about data. And I'm curious from an engineering perspective does this seem doable with something of the size and the number of spinning wheels that Facebook has and does it seem like a
Mark Zuckerberg: Yeah-
Jonathan Zittrain: -and I'm curious your reaction to an idea like that.
Mark Zuckerberg: So, I think it's quite interesting. Certainly, the level of computation that Facebook is doing and all the services that we're building is really intense to do in a distributed way. I mean, I think as a basic model I think we're building out the data center capacity over the next five years and our plan for what we think we need to do that we think is on the order of all of what AWS and Google Cloud are doing for supporting all of their customers. So, okay, so, this is like a relatively computationally intense thing.
Over time you assume you'll get more compute. So, decentralized things which are less efficient computationally will be harder- sorry, they're harder to do computation on, but eventually maybe you have the compute resources to do that. I think the more interesting questions there are not feasibility in the near term, but are the philosophical questions of the goodness of a system like that.
So, one question if you want to- so, we can get into decentralization, one of the things that I've been thinking about a lot is a use of blockchain that I am potentially interesting in- although I haven't figured out a way to make this work out, is around authentication and bringing- and basically granting access to your information and to different services. So, basically, replacing the notion of what we have with Facebook Connect with something that's fully distributed.
Jonathan Zittrain: "Do you want to login with your Facebook account?" is the status quo
Mark Zuckerberg: Basically, you take your information, you store it on some decentralized system and you have the choice of whether to login to different places and you're not going through an intermediary, which is kind of like what you're suggesting here-
Jonathan Zittrain: Yes.
Mark Zuckerberg: -in a sense. Okay, now, there's a lot of things that I think would be quite attractive about that. You know, for developers one of the things that is really troubling about working with our system, or Google's system for that matter, or having your services through Apple's app store, is that you don't want to have an intermediary between serving your- the people who are using your service and you, right, where someone can just say, "Hey, we as a developer have to follow your policy and if we don't, then you can cut off access to the people we're serving." That's kind of a difficult and troubling position to be in. I think developers-
<overlapping conversation>
Jonathan Zittrain: -you're referring to a recent incident.
Mark Zuckerberg: No, well, I was- well, sure<laughter>
Mark Zuckerberg: But I think it underscores the- I think every developer probably feels this: People are using any app store but also login with Facebook, with Google; any of these services, you want a direct relationship with the people you serve.
Jonathan Zittrain: Yes.
Mark Zuckerberg: Now, okay, but let's look at the flip side. So, what we saw in the last couple of years with Cambridge Analytica, was basically an example where people chose to take data that they- some of it was their data, some of it was data that they had seen from their friends, right? Because if you want to do things like making it so alternative services can build a competing newsfeed, then you need to be able to make it so that people can bring the data that they see you [ph?] within the system. Okay, theybasically, people chose to give their data to a developer who's affiliated with Cambridge University, which is a really respected institution, and then that developer turned around and sold the data to the firm Cambridge Analytica, which is in violation of our policies. So, we cut off the developers' access. And, of course, in a fully distributed system there would be no one who could cut off the developers' access. So, the question is if you have a fully distributed system, it dramatically empowers individuals on the one hand, but it really raises the stakes and it gets to your questions around, well, what are the boundaries on consent and how people can really actually effectively know that they're giving consent to an institution?
In some ways it's a lot easier to regulate and hold accountable large companies like Facebook or Google, because they're more visible, they're more transparent than the long tail of services that people would chose to then go interact with directly. So, I think that this is a really interesting social question. To some degree I think this idea of going in the direction of blockchain authentication is less gated on the technology and capacity to do that. I think if you were doing fully decentralized Facebook, that would take massive computation, but I'm sure we could do fully decentralized authentication if we wanted to. I think the real question is do you really want that?
Jonathan Zittrain: Yes.
Mark Zuckerberg: Right? And I think you'd have more cases where, yes, people would be able to not have an intermediary, but you'd also have more cases of abuse and the recourse would be much harder.
Jonathan Zittrain: Yes. What I hear you saying is people as they go about their business online are generating data about themselves that's quite valuable, if not to themselves, to others who might interact with them. And the more they are empowered, possibly through a distributed system, to decide where that data goes, with whom they want to share it, the more they could be exposed to exploitation. this is a genuine dilemma-
Mark Zuckerberg: Yeah, yeah.
Jonathan Zittrain: -because I'm a huge fan of decentralization.
Mark Zuckerberg: Yeah, yeah.
Jonathan Zittrain: But I also see the problem. And maybe one answer is there's some data that's just so toxic there's no vessel we should put it in; it might eat a whole through it or something, metaphorically speaking. But, then again, innocuous data can so quickly be assembled into something scary. So, I don't know if the next election-
<overlapping conversation>
Mark Zuckerberg: Yeah. [ph?] I mean, I think in general we're talking about the large-scale of data being assembled into meaning something different from what the individual data points mean.
Jonathan Zittrain: Yes.
Mark Zuckerberg: And I think that's the whole challenge here. But I philosophically agree with you thatI mean, I want to think about the- like, I do think about the work that we're doing as a decentralizing force in the world, right? A lot of the reason why I think people of my generation got into technology is because we believe that technology gives individuals power and isn't massively centralizing. Now you've built a bunch of big companies in the process, but I think what has largely happened is that individuals today have more voice, more ability to affiliate with who they want, and stay connected with people, ability to form communities in ways that they couldn't before, and I think that's massively empowering to individuals and that's philosophically kind of the side that I tend to be on. So, that's why I'm thinking about going back to decentralized or blockchain authentication. That's why I'm kind of bouncing around how could you potentially make this work, because from my orientation is to try to go in that direction.
Jonathan Zittrain: Yes.
Mark Zuckerberg: An example where I think we're generally a lot closer to going in that direction is encryption. I mean, this is, like, one of the really big debates today is basically what are the boundaries on where you would want a messaging service to be encrypted. And there are all these benefits from a privacy and security perspective, but, on the other hand, if what we're trying to do- one of the big issues that we're grappling with content governance and where is the line between free expression and, I suppose, privacy on one side, but safety on the other as people do really bad things, right, some of the time. And I think people rightfully have an expectation of us that we're going to do everything we can to stop terrorists from recruiting people or people from exploiting children or doing different things. And moving in the direction of making these systems more encrypted certainly reduces some of the signals that we would have access to be able to do some of that really important work.
But here we are, right, we're sitting in this position where we're running WhatsApp, which is the largest end-to-end encrypting service in the world; we're running messenger, which is another one of the largest messaging systems in the world where encryption is an option, but it isn't the default. I don't think long term it really makes sense to be running different systems with very different policies on this. I think this is sort of a philosophical question where you want to figure out where you want to be on it. And, so, my question for you- now,
I'll talk about how I'm thinking about this- is all right, if you were in my position and you got to flip a switch is probably too glib, because there's a lot of work that goes into this, and go in one direction for both of those services, who would you think about that?
Jonathan Zittrain: Well, the question you're putting on the table, which is a hard one is "Is it okay," and let's just take the simple case, "for two people to communicate with each other in a way that makes it difficult for any third party to casually listen in?" Is that okay? And I think that the way we normally answer that question is kind of a form of what you might call status quo-ism, which is not satisfying. It's whatever has been the case is-
Mark Zuckerberg: Yeah, yeah.
Jonathan Zittrain: -whatever has been the case is what should stay the case.
Mark Zuckerberg: Yeah.
Jonathan Zittrain: And, so, for WhatsApp, it's like right now WhatsApp, as I understand it, you could correct me if I'm wrong, is pretty hard to get into if-
Mark Zuckerberg: It's fully end-to-end encrypted.
Jonathan Zittrain: Right. So, if Facebook gets handed a subpoena or a warrant or something from name-your-favorite-country-
Mark Zuckerberg: Yeah.
Jonathan Zittrain: -and you're just like, "Thank you for playing. We have nothing to-" <overlapping conversation>
Mark Zuckerberg: Oh, yeah, we've had employees thrown in jail because we have gotten court orders that we have to turnover data that we wouldn't probably anyway, but we can't because it's encrypted.
Jonathan Zittrain: Yes. And then, on the other hand, and this is not as clean as it could be in theory, but Messenger is sometimes encrypted, sometimes not. If it doesn't happen to have been encrypted by the users, then that subpoena could work and, more than that, there could start to be some automated systems either on Facebook's own initiative or under pressure from governments in the general case, not a specific warrant, to say, "Hey, if the following phrases appear, if there's some telltale that says, "This is somebody going after a kid for exploitation," it should be forwarded up. If that's already happening and we can produce x-number of people who have been identified and a number of crimes averted that way, who wants to be the person to be like, "Lock it down!" Like, "We don't want any more of that!" But I guess, to put myself now to your question, when I look out over years rather than just weeks or months, the ability to casually peek at any conversation going on between two people or among a small group of people or even to have a machine do it for you, so, you can just set your alert list, you know, crudely speaking, and get stuff back, that- it's always trite to call something Orwellian, but it makes Orwell look like a piker. I mean, it seems like a classic case where you- the next sentence would be "What could possible go wrong?"
Jonathan Zittrain: And we can fill that in! And it does mean, though, I think that we have to confront the fact that if we choose to allow that kind of communication, then there's going to be crimes unsolved that could've been solved. There's going to be crimes not prevented that could have been prevented. And the only thing that kind of blunts it a little is it is not really all or nothing. The modern surveillance states of note in the world, have a lot of arrows in their quivers. And just being able to darken you door and demand surveillance of a certain kind, that might be a first thing they would go to, but they've got a Plan B, and Plan C, and a Plan D. And I guess it really gets to what's your threat model? If you think everybody is kind of a threat, think about the battles of copyright 15 years ago. Everybody is a potential infringer. All they have to do is fire up Napster, then you're wanting some massive technical infrastructure to prevent the bad thing. If what you're thinking is instead, they are a few really bad apples and they tend to- when they congregate online or otherwise with one another- tend to identify themselves and then we might have to send somebody near their house to listen with a cup at the window, metaphorically speaking. That's a different threat model and [sic] might not need it.
Mark Zuckerberg: Yeah.
Jonathan Zittrain: Is that getting to an answer to your question?
Mark Zuckerberg: Yeah, and I think I generally agree. I mean, I've already said publically that my inclination is to move these services in the direction of being all encrypted, at least the private communication version. I basically think if you want to kind of talk in metaphors, messaging is like people's living room, right? And I think we- you know, we definitely don't want a society where there's a camera in everyone's living room watching the content of those conversations.
Jonathan Zittrain: Even as we're now- I mean, it is 2019, people are happily are putting cameras in their living rooms.
Mark Zuckerberg: That's their choice, but I guess they're putting cameras in their living rooms, well, for a number of reasons, but-
Jonathan Zittrain: And Facebook has a camera that you can go into your living room- <laughter> Mark Zuckerberg: That is, I guess-
Jonathan Zittrain: I just want to be clear.
<laughter>
Mark Zuckerberg: Yeah, although that would be encrypted in this world.
Jonathan Zittrain: Encrypted between you and Facebook!
Mark Zuckerberg: No, no, no. I think- but it also-
<overlapping conversation>
Jonathan Zittrain: Doesn't it have like a little Alexa functionality, too?
Mark Zuckerberg: Well, Portal works over Messenger. So, if we go towards encryption on Messenger, then that'll be fully encrypted, which I think, frankly, is probably what people want.
Jonathan Zittrain: Yeah.
Mark Zuckerberg: The other model, beside the living room is the town square and that, I think, just has different social norms and different policies and norms that should be at play around that. But I do think that these things are very different. Right? You're not going to- you may end up in a world where the town square is a fully decentralized or fully encrypted thing, but it's not clear what value there is in encrypting something that's public content anyway, or very broad.
Jonathan Zittrain: But, now, you were put to it pretty hard in that as I understand it there's now a change to how WhatsApp works, that there's only five forwards permitted.
Mark Zuckerberg: Yeah, so, this is a really interesting point, right? So, when people talk about how encryption will darken some of the signals that we'll be able to use, you know, both for potentially providing better services and for preventing harm. One of the- I guess, somewhat surprising to me, findings of the last couple of years of working on content governance and enforcement is that it often is much more effective to identify fake accounts and bad actors upstream of them doing something bad by patterns of activity rather than looking at the content.
Jonathan Zittrain: So-called meta data.
Mark Zuckerberg: Sure.
Jonathan Zittrain: "I don't know what they're saying, but here's who's they're calling" kind of thing.
Mark Zuckerberg: Yeah, or just like they- this account doesn't seem to really act like a person, right?
And I guess as AI gets more advanced and you build these adversarial networks or generalized adversarial networks, you'll get to a place where you have Ai that can probably more effectively
Jonathan Zittrain: Go under mimic [ph?] cover. Mimic act like another person- <overlapping conversation>
Mark Zuckerberg: -for a while.
Mark Zuckerberg: Yeah. But, at the same time, you'll be building up contrary AI on the other side, but is better at identifying AIs that are doing that. But this has certainly been the most effective tactic across a lot of the areas where we've needed to focus to preventing harm. You know, the ability to identify fake accounts, which, like, a huge amount of the- under any category of issue that you're talking about, a lot of the issues downstream come from fake accounts or people who are clearly acting in some malicious or not normal way. You can identify a lot of that without necessarily even looking at the content itself. And if you have to look at a piece of content, then in some cases, you're already late, because the content exists and the activity has already happened. So, that's one of the things that makes me feel like encryption for these messaging services is really the right direction to go, because you're- it's a very proprivacy and per security move to give people that control and assurance and I'm relatively confident that even though you are losing some tools to- on the finding harmful content side of the ledger, I don't think at the end of the day that those are going to end up being the most important tools
Jonathan Zittrain: Yes.
Mark Zuckerberg: -for finding the most of the-
<overlapping conversation>
Jonathan Zittrain: But now connect it up quickly to the five forwards thing.
Mark Zuckerberg: Oh, yeah, sure. So, that gets down to if you're not operating on a piece of content directly, you need to operate on patterns of behavior in the network. And what we, basically found was there weren't that many good uses for people forwarding things more than five times except to basically spam or blast stuff off. It was being disproportionately abused. So, you end up thinking about different tactics when you're not operating on content specifically; you end up thinking about patterns of usage more.
Jonathan Zittrain: Well, spam, I get and that- I'm always in favor of things that reduce spam. However, you could also say the second category was just to spread content. You could have the classic, I don't know, like Les Mis, or Paul Revere's ride, or Arab Spring-esque in the romanticized vision of it: "Gosh, this is a way for people to do a tree," and pass along a message that "you can't stop the signal," to use a Joss Whedon reference. You really want to get the word out. This would obviously stop that, too.
Mark Zuckerberg: Yeah, and then I think the question is you're just weighing whether you want this private communication tool where the vast majority of the use and the reason why it was designed was the vast majority of just one-on-one; there's a large amount of groups that people communicate into, but it's a pretty small edge case of people operating this with, like- you have a lot of different groups and you're trying to organize something and almost hack public content-type or public sharing- type utility into an encrypted space and, again, there I think you start getting into "Is this the living room or is this the town square?" And when people start trying to use tools that are designed for one thing to get around what I think the social norms are for the town square, that's when I think you probably start to have some issues. This is not- we're not done addressing these issues. There's a lot more to think through on this
Jonathan Zittrain: Yeah.
Mark Zuckerberg: -but that's the general shape of the problem that at least I perceive from the work that we're doing.
Jonathan Zittrain: Well, without any particular segue, let's talk about fake news.
<laughter>
Jonathan Zittrain: So, insert your favorite segue here. There's some choice or at least some decision that gets made to figure out what's going to be next in my newsfeed when I scroll up a little more.
Mark Zuckerberg: Mm-hm.
Jonathan Zittrain: And in the last conversation bit, we were talking about how much we're looking at content versus telltales and metadata, things that surround the content.
Mark Zuckerberg: Yeah.
Jonathan Zittrain: For knowing about what that next thing in the newsfeed should be, is it a valid desirable material consideration, do you think, for a platform like Facebook to say is the thing we are about to present true, whatever true means?
Mark Zuckerberg: Well, yes, because, again, getting at trying to serve people, people tell us that they don't want fake content. Right. I mean, I don't know anyone who wants fake content. I think the whole issue is, again, who gets to decide. Right. So broadly speaking, I don't know any individual who would sit there and say, "Yes, please show me things that you know are false and that are fake." People want good quality content and information. That said, I don't really think that people want us to be deciding what is true for them and people disagree on what is true. And, like, truth is, I mean, there are different levels of when someone is telling a story, maybe the meta arc is talking about something that is true but the facts that were used in it are wrong in some nuanced way but, like, it speaks to some deeper experience. Well, was that true or not? And do people want that disqualified from to them? I think different people are going to come to different places on this.
Now, so I've been very sensitive, which, on, like, we really want to make sure that we're showing people high quality content and information. We know that people don't want false information. So we're building quite advanced systems to be able to- to make sure that we're emphasizing and showing stuff that is going to be high quality. But the big question is where do you get the signal on what the quality is? So the kind of initial v.1 of this was working with third party fact checkers.
Right, I believe very strongly that people do not want Facebook and that we should not be the arbiters of truth in deciding what is correct for everyone in the society. I think people already generally think that we have too much power in deciding what content is good. I tend to also be concerned about that and we should talk about some of the governance stuff that we're working on separately to try to make it so that we can bring more independent oversight into that.
Jonathan Zittrain: Yes.
Mark Zuckerberg: But let's put that in a box for now and just say that with those concerns in mind, I'm definitely not looking to try to take on a lot more in terms of also deciding in addition to enforcing all the content policies, also deciding what is true for everyone in the world. Okay, so v.1 of that is we're going to work with-
Jonathan Zittrain: Truth experts.
Mark Zuckerberg: We're working with fact checkers.
Jonathan Zittrain: Yeah.
Mark Zuckerberg: And, and they're experts and basically, there's like a whole field of how you go and assess certain content. They're accredited. People can disagree with the leaning of some of these organizations.
Jonathan Zittrain: <laughter> Who does accredited fact checkers?
Mark Zuckerberg: <laughs> The Poynter Institute for Journalism.
Jonathan Zittrain: I should apply for my certification.
Mark Zuckerberg: You may.
Jonathan Zittrain: Okay, good.
Mark Zuckerberg: You'd probably get it, but you have to- You'd have to go through the process.
Mark Zuckerberg: The issue there is there aren't enough of them, right. So there's a large content. There's obviously a lot of information is shared every day and there just aren't a lot of fact checkers. So then the question is okay, that is probably
Jonathan Zittrain: But the portion- You're saying the food is good, it's just the portions are small. But the food is good.
Mark Zuckerberg: I think in general, but so you build systems, which is what we've done especially leading up to elections where I think are some of the most fraught times around this where people really are aggressively trying to spread misinformation.
Jonathan Zittrain: Yes.
Mark Zuckerberg: You build systems that prioritize content that seems like it's going viral because you want to reduce the prevalence of how widespread the stuff gets, so that way the fact checkers have tools to be able to, like, prioritize what they need to go- what they need to go look at. But it's still getting to a relatively small percent of the content. So I think the real thing that we want to try to get to over time is more of a crowd sourced model where people, it's not that people are trusting some sort, some basic set of experts who are accredited but are in some kid of lofty institution somewhere else. It's like do you trust, yeah, like, if you get enough data points from within the community of people reasonably looking at something and assessing it over time, then the question is can you compound that together into something that is a strong enough signal that we can then use that?
Jonathan Zittrain: Kind of in the old school like a slash-dot moderating system
Mark Zuckerberg: Yeah.
Jonathan Zittrain: With only the worry that if the stakes get high enough, somebody wants to Astroturf that.
Mark Zuckerberg: Yes.
Jonathan Zittrain: I'd be-
Mark Zuckerberg: There are a lot of questions here, which is why I'm not sitting here and announcing a new program.
<laughter>
Mark Zuckerberg: But what I'm saying is this is, like,-
Jonathan Zittrain: Yeah,
Mark Zuckerberg: This is the general direction that I think we should be thinking about when we haveand I think that there's a lot of questions and-
Jonathan Zittrain: Yes.
Mark Zuckerberg: And we'd like to run some tests in this area to see whether this can help out. Which would be upholding the principles which are that we want to stop-
Jonathan Zittrain: Yes.
Mark Zuckerberg: The spread of misinformation.
Jonathan Zittrain: Yes.
Mark Zuckerberg: Knowing that no one wants misinformation. And the other principle, which is that we do not want to be arbiters of truth.
Jonathan Zittrain: Want to be the decider, yes.
Mark Zuckerberg: And I think that that's the basic- those are the basic contours I think of that, of that problem.
Jonathan Zittrain: So let me run an idea by you that you can process in real time and tell me the eight reasons I have not thought of why this is a terrible idea. And that would be people see something in their Facebook feed. They're about to share it out because it's got a kind of outrage factor to it. I think of the classic story from two years ago in The Denver Guardian about "FBI agent suspected in Hilary Clinton email leak implicated in murder-suicide." I have just uttered fake news.
None of that was true if you clicked through The Denver Guardian. There was just that article. There is Denver Guardian. If you live in Denver, you cannot subscribe. Like, it is unambiguously fake. And it was shared more times than the most shared story during the election season of The Boston Globe. And so
Mark Zuckerberg: So, and this is actually an example, by the way, of where trying to figure out fake accounts is a much simpler solution.
Jonathan Zittrain: Yes.
Mark Zuckerberg: Than trying to down-
Jonathan Zittrain: So if newspaper has one article-
Mark Zuckerberg: Yeah.
Jonathan Zittrain: Wait for ten more before you decide they're a newspaper.
Mark Zuckerberg: Yeah. Or, you know, I mean, it's there are any number of systems that you could build to basically detect, "Hey, this is-"
Jonathan Zittrain: A Potemkin.
Mark Zuckerberg: This is a fraudulent thing.
Jonathan Zittrain: Yes.
Mark Zuckerberg: And then you can take that down. And again, that ends up being a much less controversial decision because you're doing it upstream based on the basis of inauthenticity.
Jonathan Zittrain: Yes.
Mark Zuckerberg: In a system where people are supposed to be their real and represent that they're their real selves than downstream, trying to say, "Hey, is this true or false?"
Jonathan Zittrain: I made a mistake in giving you the easy case.
Mark Zuckerberg: Okay.
<laughter>
Jonathan Zittrain: So I should have not used that example.
Mark Zuckerberg: Too simple.
Jonathan Zittrain: You're right and you knocked that one out of the park and, like, Denver Guardian, come up with more articles and be real and then come back and talk to us.
<laughter>
Jonathan Zittrain: So, here's the harder case which is something that might be in an outlet that is, you know, viewed as legitimate, has a number of users, et cetera. So you can't use the metadata as easily.
Imagine if somebody as they shared it out could say, "By the way, I want to follow this. I want to learn a little bit more about this." They click a button that says that. And I also realized when I talked earlier to somebody at Facebook on this that adding a new button to the homepage is, like, everybody's first idea
Mark Zuckerberg: Oh, yeah.
Jonathan Zittrain: And it's-
Mark Zuckerberg: But it's a reasonable thought experiment, even though it would lead to a very bad UI.
Jonathan Zittrain: Fair enough. I understand this is already-
Mark Zuckerberg: Yeah. <laughs>
Jonathan Zittrain: In the land of fantasy. So they add the button. They say, "I want to follow up on this."
If enough people are clicking comparatively on the same thing to say, "I want to learn more about this. If anything else develops, let me know, Facebook," that, then, if I have my pneumatic tube, it then goes to a convened virtually panel of three librarians. We go to the librarians of the nation and the world at public and private libraries across the land who agree to participate in this program. Maybe we set up a little foundation for it that's endowed permanently and no long connected to whoever endowed it. And those librarians together discuss the piece and they come back with what they would tell a patron if somebody came up to them and said, "I'm about to cite this in my social studies paper. What do you think?" And librarians, like, live for questions like that.
Mark Zuckerberg: Mm-hmm, yeah.
Jonathan Zittrain: They're like, "Wow. Let us tell you." And they have a huge fiduciary notion of patron duty that says, "I may disapprove of you even studying this, whatever, but I'm here to serve you, the user."
Mark Zuckerberg: Yeah.
Jonathan Zittrain: "And I just think you should know, this is why maybe it's not such a good source." And when they come up with that they can send it back and it gets pushed out to everybody who asks for follow-up-
Mark Zuckerberg: Yeah.
Jonathan Zittrain: And they can do with it as they will. And last piece of the puzzle, we have high school students who apprentice as librarian number three for credit.
<laughter>
Jonathan Zittrain: And then they can get graded on how well they participated in this exercise which helps generate a new generation of librarian-themed people who are better off at reading things, so.
Mark Zuckerberg: All right, well, I think you have a side goal here which I haven't been thinking about on the librarian thing.
<laughter>
Mark Zuckerberg: Which is the evil goal of promoting libraries.
Jonathan Zittrain: Well, it's
Mark Zuckerberg: No, but I mean, look, I think solving- preventing misinformation or spreading misinformation is hard enough without also trying to develop high school students in a direction.
Jonathan Zittrain: Ah. My colleague