Electronic Freedom Foundation

Hearing Friday: Plaintiffs Challenging FOSTA Ask Court to Reinstate Lawsuit Seeking To Block Its Enforcement

EFF - 49 min 5 sec ago
Risk of Prosecution Has Caused Groups to Self-Censor, Platforms to Shut Out Legal Services

Washington D.C.—On Friday, Sept. 20, at 9:30 am, attorneys for five plaintiffs suing the government to block enforcement of FOSTA will ask a federal appeals court to reverse a judge’s decision to dismiss the case.

The plaintiffs—Woodhull Freedom Foundation, the Internet Archive, Human Rights Watch, and individuals Alex Andrews and Eric Koszyk—contend that FOSTA, a federal law passed in 2018 that expansively criminalizes online speech related to sex work and removes important protections for online intermediaries, violates their First Amendment rights.

Electronic Frontier Foundation (EFF) is counsel for the plaintiffs along with co-counsel Davis Wright Tremaine LLP, Walters Law Group, and Daphne Keller.

FOSTA, or the Allow States and Victims to Fight Online Sex Trafficking Act, makes it a felony to use or operate an online service with the intent to “promote or facilitate the prostitution of another person,” vague terms with wide-ranging meanings that can include speech that makes sex work easier in any way. FOSTA also expanded the scope of other federal laws on sex trafficking to include online speech, and reduced statutory immunities previously provided under Section 230 of the Communications Decency Act. The plaintiffs sued to block enforcement of the law because its overbroad language sweeps up Internet speech about sex, sex workers, and sexual freedom, including harm reduction information and speech advocating decriminalization of prostitution.

A federal judge dismissed the case, ruling that the plaintiffs lacked “standing” because they failed to prove a credible threat that they would be prosecuted for violating FOSTA. Because the court dismissed the case on procedural grounds, it did not rule on whether FOSTA is constitutional.

Attorney Robert Corn-Revere, counsel for the plaintiffs, will argue at a hearing on Sept. 20 that the plaintiffs don’t have to wait until they face prosecution before challenging a law regulating speech when, as here, the vague and overbroad prohibitions of the law are causing numerous speakers to censor themselves and their users. FOSTA specifically authorized enforcement by state prosecutors and private litigants, vastly increasing the risk of being sued under the statute and greatly exacerbating the speech-chilling effects of the law. FOSTA has also reportedly generated increased risks for sex workers and frustrated law enforcement efforts to investigate trafficking.

WHAT:
Oral argument in Woodhull Freedom Foundation v. U.S.

WHO:
Robert Corn-Revere of Davis Wright  Tremaine LLP

WHEN:
Friday, Sept. 20, at 9:30 am

WHERE:
E. Barrett Prettyman U.S. Courthouse and William B. Bryant Annex
Courtroom 31
333 Constitution Avenue, NW
Washington, DC 20001

For more on this case:
https://www.eff.org/cases/woodhull-freedom-foundation-et-al-v-united-states
https://www.woodhullfoundation.org/our-work/fosta/

For more on FOSTA:
https://www.eff.org/deeplinks/2018/03/how-congress-censored-internet

 

 

Contact:  DavidGreeneCivil Liberties Directordavidg@eff.org

Thanks For Helping Us Defend the California Consumer Privacy Act

EFF - 2 hours 11 min ago

The California Consumer Privacy Act will go into effect on January 1, 2020—having fended off a year of targeted efforts by technology giants who wanted to gut the bill. Most recently, industry tried to weaken its important privacy protections in the last days of the legislative session.

Californians made history last year when, after 600,000 people signed petitions in support of a ballot initiative, the California State Legislature answered their constituents’ call for a new data privacy law. It’s been a long fight to defend the CCPA against a raft of amendments that would have weakened this law and the protections it enshrines for Californians. Big technology companies backed a number of bills that each would have weakened the CCPA’s protections. Taken together, this package would have significantly undermined this historic law.

Fortunately, the worst provisions of these bills did not make it through the legislature—though it wasn’t for lack of trying. Lawmakers proposed bills that would have opened up loopholes in the law and made it easier for businesses to skirt privacy protections if they shared information with governments, changed definitions in the bill to broaden its exemptions, and made it easier for businesses to require customers to pay for their privacy rights.

These bills sailed through the Assembly but were stopped in July by the Senate Judiciary Committee, chaired by Senator Hannah-Beth Jackson. The final amendments to the CCPA that passed through the legislature last week make small changes to the law, and do not weaken its important protections.

We want to thank everyone who called or wrote to their lawmakers to protect the CCPA this year and amplified how important data privacy is to the people of California. Your voices are invaluable to our advocacy.

We also appreciate the time that lawmakers, our coalition partners, and other stakeholders devoted to discussions about these amendments. As a result of this hard work, the California State Legislature stood up for the privacy law that they passed last year.

Still, while the CCPA is important for Californians’ consumer data privacy, it needs to be stronger. EFF and other privacy organizations earlier this year advanced two bills to strengthen the CCPA, which met significant opposition from technology industry trade association groups. Most importantly, these bills would have improved enforcement by allowing consumers to bring their own privacy claims to court. We particularly thank Assemblymember Buffy Wicks. Sen. Jackson, and the California Attorney General’s Office for leading the charge to improve the CCPA in the legislature.

More than anything, this year’s CCPA fight shows that when voters speak up for their privacy, it makes a big difference with legislators. We look forward to continuing to work with legislators and our coalition partners to advance measures that improve everyone’s privacy. We also look forward to offering input on the Attorney General’s regulations for the CCPA, expected this fall. And as technology trade groups redouble their efforts to weaken state privacy laws or override them with a national law, we encourage everyone to keep pushing for strong consumer data privacy laws across the country.

Big Tech’s Disingenuous Push for a Federal Privacy Law

EFF - 3 hours 18 min ago

This week, the Internet Association launched a campaign asking the federal government to pass a new privacy law.

The Internet Association (IA) is a trade group funded by some of the largest tech companies in the world, including Google, Microsoft, Facebook, Amazon, and Uber. Many of its members keep their lights on by tracking users and monetizing their personal data. So why do they want a federal consumer privacy law?

Surprise! It’s not to protect your privacy. Rather, this campaign is a disingenuous ploy to undermine real progress on privacy being made around the country at the state level. IA member companies want to establish a national “privacy law” that undoes stronger state laws and lets them continue business as usual. Lawyers call this “preemption.” IA calls this “a unified, national standard” to avoid “a patchwork of state laws.” We call this a big step backwards for all of our privacy.

The question we should be asking is, “What are they afraid of?”

Stronger state laws

After years of privacy scandals, Americans across the political spectrum want better consumer privacy protections. So far, Congress has failed to act, but states have taken matters into their own hands. The Illinois Biometric Information Privacy Act (BIPA), passed in 2008, makes it illegal to collect biometric data from Illinois citizens without their express, informed, opt-in consent. Vermont requires data brokers to register with the state and report on their activities. And the California Consumer Privacy Act (CCPA), passed in 2018, gives users the right to access their personal data and opt out of its sale. In state legislatures across the country, consumer privacy bills are gaining momentum.

This terrifies big tech companies. Last quarter alone, the IA spent nearly $176,000 lobbying the California legislature, largely to weaken CCPA before it takes effect in January 2021. Thanks to the efforts of a coalition of privacy advocates, including EFF, it failed. The IA and its allies are losing the fight against state privacy laws. So, after years of fighting any kind of privacy legislation, they’re now looking to the federal government to save them from the states. The IA has joined Technet, a group of tech CEOs, and Business Roundtable, another industry lobbying organization, in calls for a weak national “privacy” law that will preempt stronger state laws. In other words, they want to roll back all the progress states like California have made, and prevent other states from protecting consumers in the future. We must not allow them to succeed.

A private right of action

Laws with a private right of action allow ordinary people to sue companies when they break the law. This is essential to make sure the law is properly enforced. Without a private right of action, it’s up to regulators like the Federal Trade Commission or the U.S. Department of Justice to go after misbehaving companies. Even in the best of times, regulatory bodies often don’t have the resources needed to police a multi-trillion dollar industry. And regulators can fall prey to regulatory capture. If all the power of enforcement is left in the hands of a single group, an industry can lobby the government to fill that group with its own people. Federal Communications Commission chair Ajit Pai is a former Verizon lawyer, and he’s overseen massive deregulation of the telecom industry his office is supposed to keep in check.

The strongest state privacy laws include private rights of action. Illinois BIPA allows users whose biometric data is illegally collected or handled to sue the companies responsible. And CCPA lets users sue when a company’s negligence results in a breach of personal information. The IA wants to erase these laws and reduce the penalties its member companies can face for their misconduct in legal proceedings brought by ordinary consumers.

Real changes to the surveillance business model

We don’t know what the IA’s final legislative proposal will say, but its campaign website is thick with weasel words and equivocation. For example, the section on “Controls” says:

Individuals should have meaningful controls over how personal information they provide to companies is collected, used, and shared, except where that information is necessary for the basic operation of the business[.]

The “basic operation” of data brokers involves collecting and selling personal data without your consent. Does that mean you shouldn’t be able to stop them?

The rest of IA’s proposals follow the same pattern. The section on “transparency” says that users should be able to know the “categories of entities” that their data is shared with, but not the names of actual companies or people that receive it. This will make it unnecessarily difficult for people to trace how their personal information is bought and sold. The section on “access” says that users’ ability to access their data should not “unreasonably interfere with a company’s business operations.” Again, if a business depends on gathering data about people without their knowledge, will users ever be able to access their information? Sometimes, exercising your privacy rights will mean “interfering” with a company’s business.

The bottom line is that tech companies are happy for Congress to enact a privacy law—as long as it doesn’t affect their “business operations” in any way. In other words, they’d like a privacy law that doesn’t change anything at all.

The Internet Association knows which way the wind is blowing. Across the country, people are fed up with Big Tech’s empty promises and serial mishandling of personal data. They want real change, and state legislatures are listening. We must allow states to continue passing innovative new privacy laws. Any federal privacy legislation needs to build a floor, not a ceiling.

Facebook Must Better Limit Its Face Surveillance

EFF - Fri, 09/13/2019 - 7:05pm

Last week, Facebook started sending a small portion of its users a new notification about its face surveillance program, which concludes with two important buttons: “keep off” and “turn on.” This is a step in the right direction: for these users, the default will be no face surveillance, unless the user gives their affirmative opt-in consent.

But as EFF recently explained, Facebook will not provide this privacy-protective default to billions of its current users, and it is unclear whether the company will provide it to its new users. Facebook should not subject any of its current or new users to face surveillance, absent their informed opt-in consent.

We have two additional objections. First, Facebook’s announcement of this new program fails to mention that the company is acting under FTC compulsion. Second, the notice Facebook is sending to some of its users lacks critical information about the privacy hazards of face surveillance, so people who opt-in will not be fully informed.

The FTC Required Facebook to Change Its Face Surveillance Settings

On July 24, 2019, the Federal Trade Commission (FTC) filed a complaint in court against Facebook for violating a 2012 privacy order by the FTC against Facebook. Much of this FTC complaint concerns Facebook’s role in the Cambridge Analytica scandal. But the FTC also alleges that, in 2018, Facebook misled 60 million of its users by telling them that the company would not subject them to face surveillance unless they chose to “turn on” the feature. In fact, the feature was on by default. According to the FTC, Facebook made this misleading statement to only some of its users: those the company had not yet moved from its original face surveillance program (which Facebook calls “tag suggestions”) to its current face surveillance program (which the company calls “face recognition”).

Also on July 24, the FTC and Facebook filed a proposed order to settle the issues raised by the FTC’s complaint. (EFF at that time objected that this settlement does not solve the problems that led to the Cambridge Analytica scandal.) Part of this FTC settlement requires Facebook, as to its users still using “tag suggestions” at the time of the settlement, to obtain consent before subjecting them to further face surveillance.

Thus, the new Facebook program is required by the FTC settlement, though the new Facebook announcement does not mention this.

Facebook’s Incomplete Description of Face Surveillance

The FTC settlement requires Facebook to provide notice, to its remaining “tag suggestions” users, of how Facebook will use and share the “facial recognition templates” of these users. The new notice from Facebook does provide such information.

Unfortunately, the FTC did not require Facebook to notify its users of the inherent privacy hazards posed by face surveillance, and Facebook did not do so on its own. As with any kind of personal information, the hazards of corporate collection include theft by outside hackers, misuse by company employees, and seizure by government officials. There also is the risk of “mission creep”—when company leaders seek new ways to profit from old data. Ominously, Facebook has applied to patent face surveillance systems that would link its users’ online profiles to their physical-world activities.

Moreover, face templates are a uniquely hazardous form of personal information: most of us cannot hide or change our faces, and the technology that tracks our faces is rapidly improving and proliferating.

In light of this gap in Facebook’s notice, users who opt-in to face surveillance might not be doing so on the basis of all the relevant information.

Conclusion

We are pleased that the FTC required Facebook to individually notify some of its users about how the company uses and shares face recognition templates, and forbade the company from applying face surveillance to these users unless they affirmatively opt-in. As we explained in our last post, however, we are disappointed that the FTC did not require Facebook to obtain consent before subjecting any of its users to face surveillance. And as we explain in this post, we are also disappointed that Facebook’s notice fails to identify the privacy hazards of face surveillance. This failure is all the more reason to enact strong consumer data privacy laws.

Don't Let Congress Hand Patent Abusers Their Ultimate Wishlist

EFF - Fri, 09/13/2019 - 5:46pm


Congress is considering a bill that would throw out the best defenses against bad patents. The Senate IP Subcommittee recently had a hearing about the Stronger Patents Act, a batch of recurring terrible ideas that has been introduced by Sen. Chris Coons (D-Del.) for the third time in three years.

The Stronger Patents Act would tear apart inter partes review (IPR), an critical tool for challenging bad patents. People who are charged with patent violations shouldn’t have to have millions of dollars in the bank to defend themselves. IPR provides a more cost-effective way of evaluating patents than expensive federal court litigation.

TAKE ACTION

PRESERVE OUR DEFENSES AGAINST PATENT ABUSE

Patent trolls, drug companies, and IP lawyer groups have been attacking IPR for years now, and they’re all big supporters of this bill. Big patent owners have grown so used to gaming the patent system that they’re willing to throw out IPRs, despite the fact that these reviews are clearly in the public interest.

IPR allows companies to fight back against patent accusations for a fraction of the cost of district court. It also allows organizations like EFF to challenge bogus patents like we did when we busted the podcasting patent. If the Stronger Patents Act passes, EFF and our supporters won’t be allowed to file challenges anymore.

Taking a second look at patents is in the public interest. In the seven years IPRs have been active, the specialized judges at the Patent Office have thrown out more than 1,500 patents that never should have been issued in the first place. Many of those are, unsurprisingly, software patents.

The U.S. Patent Office often issues patents it shouldn’t have, particularly in areas like software, where examiners don’t always have access to the most relevant prior art. The office is funded by the fees paid by patent applicants. PTO examiners spend an average of about 18 hours per application, and that leads to wrongly issued patents.

Too often, weak patents get used to threaten small businesses—patents that claim things like picture menus, or crowdfunding, or online contests. The IPR process is the best process, so far, for dealing with those improperly issued patents.

When IPR was challenged in court, the Supreme Court upheld the process. The public has an important interest in ensuring that patents stay within their proper bounds.

The Stronger Patents Act has another bad provision that will give huge amounts of leverage directly to patent trolls. Under rules laid out by the Supreme Court in 2007, it’s very hard for patent trolls to get court-ordered injunctions that can knock products off the market. The Stronger Patents Act would undo that rule, giving patent trolls leverage to scare massive cash settlements out of companies. In 2006, Blackberry (then called RIM) paid out a $612 million settlement to a patent-assertion entity when it was threatened with an injunction. That money went straight into the hands of some bad actors in the patent world, who used the capital to invest in—what else—more lawsuits against tech firms.

The Stronger Patents Act will wreak havoc on a system that’s already balanced in favor of patent holders. Tell Congress to reject this proposal.

TAKE ACTION

PRESERVE OUR DEFENSES AGAINST PATENT ABUSE

Related Cases: Abstract Patent Litigation

EFF's 2019 Pioneer Awards Winner Remarks and Speeches

EFF - Fri, 09/13/2019 - 4:19pm

EFF’s annual Pioneer Awards ceremony celebrates individuals and groups who have made outstanding contributions to freedom and innovation on the electronic frontier. On Sept. 12, EFF welcomed keynote speaker Adam Savage, who spoke on the importance of storytelling, scientific exploration, and personal discovery. And each of our honorees had important messages to share with us: legendary science fiction author William Gibson reminded us how early science fiction shaped the world we live in now; the inspiring anti-surveillance group Oakland Privacy showed how we can stand together to make lasting differences in how technology is used in our communities today; and trailblazing tech scholar danah boyd challenged everyone in the tech world to shape a better future. 

Opening the ceremony was EFF Executive Director Cindy Cohn, who framed the evening by reminding us that we must articulate what that better future looks like and work to make it happen—because "honestly, we don’t have any other choice." Additionally, she underscored how important it is to recognize our past and move toward a better future. "Even now, especially now, we need hope," she said. "In the end, we cannot build a better world unless we envision it and talk about it."

Below are transcripts or prepared remarks of the keynote and award winners' speeches. Audio of the entire ceremony is available here, and individual audio recordings of each speech are below.

Opening Remarks by Cindy Cohn

Audio

Thank you so much, Aaron.  I am just delighted to see everyone here tonight and to honor these amazing people.  Tonight we take a moment to celebrate our community. 

But as we begin I want to send a moment out for our friend Chelsea Manning, who is again incarcerated by a vindictive government. Our hearts go out to her and we wish she could be with us here tonight.

On to our awardees.  Each of them will have an individual introduction, but I think tonight’s awardees represent a great cross-section of the work that is being done to make our digital world better. 

Executive Director Cindy Cohn delivers the opening remarks

First, there’s Dr. danah boyd, who has spent her professional life trying to figure out and reflect back to us the ways in which people, especially young people, are interacting with technologies.  That would be enough, but danah has now gone far beyond that to both support and inspire other researchers and build a community thinking about how Data and Society do and should interact.

Second, there’s Oakland Privacy, who represent what a supporting, inspiring, grassroots community can accomplish – putting the city of Oakland far ahead of the national conversation on these issues.

And finally William Gibson, whose imagination and storytelling have framed our digital world, with both its benefits and its perils. William pioneered the vision that we needed, and he did so before EFF and these awards even existed

We gather tonight in a time of reckoning and change for our community. It’s one where we desperately need to articulate and push for a better technical world because so many people have lost hope: unable to think of the future as anything but a dystopian hellscape, even as they feel trapped behind their phones or their keyboards.

Outside our world, the blush of tech-excitement has given way to a tech-lash that is needed. If not conducted thoughtfully, however, this moment threatens those who most need digital tools to keep themselves safe. It threatens those who have used and are using the Net to find community, support, and solidarity, and join together to find and implement solutions to many, many problems we see pressing against us all.  Politicians of all stripes are angry at those big, brand name tech companies, powerful and unaccountable, but for very different and often sharply contradictory reasons.  But as they shoot at Big Tech, we know that the public interest Internet, the marginal voices it has empowered and the innovators that could challenge and reform the current status quo, all sit nearby and stand a great risk of becoming collateral damage. We must not let that happen.

So far, we’ve seen that many of the efforts to combat the problems of big tech actually threaten to empower and ossify it.  I shed no tears for the big companies, who join John Perry’s weary giants of Flesh and Steel as the unwelcome would-be governors of cyberspace. But if we want to move toward an Internet that works for us, where power is shifted to the users and builders and away from the Wall Street financiers and surveillance capitalists who would turn us into insecure, surveilled rats in a maze, we must step up now more than ever. 

But there’s a reckoning inside our world too.  Recent events have demonstrated the need to take a hard look the shift from technology being a niche issue led by quirky geeks and outcasts to one of big business, with the attendant money and power and corruption. We also need to look at the frankly horrible treatment that some in tech have wrought: from young girls to aspiring women scientists and technologists to contract and gig workers to people of color both in the U.S. and around the world. We must address our roles and own blind spots in letting this happen to so many. We must address the ways in which our embrace of the hero-narrative, and a hunger for the fruits of innovation, allowed a world in which being a genius made it OK to be an asshole, or much worse. Those days must be over now, and I say good riddance.

But this shift requires work by all of us who believe that technology can be a force for good in the world. It won’t happen automatically and the decisions along the way are not simple. We must do it together. We must stand with the survivors and ensure that, as we do so, we work to bring people of good will and good intentions along with us.   

Barlow said, echoing Alan Kay, that the way to make a better future is to invent it.  And it’s true.  But as recent events have unfolded, I think that even he would likely have had to reconsider some of his own role in creating some parts of this world.  But I also know that Barlow would have wanted the unvarnished truth, and was always hopeful we would find ways to discover it, and that ultimately that truth would help bring us to a better place.

Even now when the tools we built to help us see have given us the clarity to uncover the very worst. When we’ve built systems that let everyone speak, we must accept that those new channels will be filled with the voices of those who have long been silenced, who speak their truth and make us confront their pain. We also know that they are filled with those who want to keep them silenced.

Even now, especially now, we  need hope.  In the end, we cannot build a better world unless we envision it and talk about it.  Being here with all of you tonight renews my faith that there are so many good, smart, thoughtful and kind people in this community. And we know that there are many more of us out there, outside our community, waiting to come in. We must revel in each other and not let the awful things we’ve heard and seen make us turn away from the truth, or each other.  

So that’s my challenge to all of you tonight. Even as we’re unflinching in talking about and addressing the problems and harms that our current world has created or encouraged or even just rides alongside, we must also articulate what a better future looks like and work to make it happen.  Honestly, we don’t have any other choice.

Now, on to the celebration part of the evening.

Keynote Speech by Adam Savage

Audio

I want to start by thanking EFF for asking me to be here and deliver this keynote. I've been a supporter and true believer in your mission since its inception. I was lucky enough to be at your 20th birthday party and party with John Perry Barlow, whose long-distance vision of the promise and perils of the Internet was prescient, to say the least.

I'm humbled to be in the room with tonight's award winners, each heroes in their own right. Specifically, Mr. Gibson, if you knew how much your books meant to my early days in San Francisco, they equate to me at 24 first coming here in 1990 and the city that I found when I moved here. And so I want to thank you personally for all the time I've spent and the realities that you have weaved.

I wanted to talk tonight about facts and stories. I've had a lot of different jobs and even careers in my life so far. Even in hosting MythBusters for 14 years on Discovery Channel, I spent a lot of time trying to figure out what that job actually was.

Adam Savage delivers the keynote to the 2019 Pioneer Awards



In the first season, newly divorced and going through the particular insanity that befalls all of the recently divorced, three months into filming, I stopped dating entirely just to hunker down and figure out what this new endeavor of hosting a TV show was asking from me, what I had to contribute to it. And the answer would take me more than a decade.

At first, I thought I was there to build stuff and talk about it. And then I realized maybe my job is to concoct entertaining scientific methodologies and execute them and talk about them. And then I thought it was to make something explode in every episode. That may have come from a note from the network.

In 2006, I met Neil deGrasse Tyson for the first time and did his podcast, and I was sitting across from him, watching him go, and thinking, "Look at this guy. He is like an arrow pointed towards a goal of illuminating science for people." To use a phrase from Mr. Gibson, "He is vat-grown for this job." He is a science communicator. What a great mission. Wait a minute. I'm a science communicator. What a cool mission. Albeit, I'm a science communicator with only a high school diploma.

In 2008, we filmed an episode called Lead Balloon in which we made a 14-foot diameter balloon out of 28 pounds of rolled lead. No explosions. No fire. And when we talked to editorial about this episode, they expected that the cut for the lead balloon portion of the episode would maybe be 15 minutes. The first rough cut of Lead Balloon was 55 minutes long. The final cut was so thrilling and rated really well.

And I realized that one of the key things that made this episode great was Jamie's and my enthusiasm. If we were engaged, it turns out, so was the audience. And that's when I started wearing more costumes on the show, and it's when Jamie started asking questions that had no myth at all attached to them, like, "Well, if you could put square wheels on a car, how fast would you have to go to get a smooth ride?"

It took us two tries. The first try, all four of the brakes fell off the car at the same time, an injury I would have trouble doing if you asked me to do it on purpose. On the second try, the answer was 38 miles an hour.

It wasn't until season 11 that I realized the simplicity of my job. Storytelling. We were there to tell a story about the search for a hidden truth, to quote Raymond Chandler. Often, a hidden truth in something absurd. That, in fact, it turns out, was all I had ever done for a living.

When I spent several years as a graphic designer, and every designer will tell you this, the final design works not because it has the proper information, but because that information tells a story to the person who's looking at it. Your eye is guided to the right parts of the design at the right time. Instead of using time to tell a story like in a movie, a graphic designer uses space to parcel out the information so our brains can process it.

When I was working as a model maker in commercials and films, making spaceships, attaching little details to a ship, we called them greebles. Every single greeble has to have a story attached to it, and that story has to be known by the model maker gluing that greeble to that ship. Otherwise, it won't work aesthetically, because the surface details on the Millennium Falcon tell a very different story than the surface details on the Enterprise. The model maker is required to know that story. Otherwise, the story won't scan.

And on MythBusters, the story was one of scientific discovery but of also personal discovery. It was about watching Jamie and Kari, Tory, Grant, and I, and Jessi, and the entire team confront new ideas and new materials, and collaborating and learning what they can do, and seeing what we can learn from them.

Stories are what make us human. I think that we invented language in order to tell stories. I think the story is the first mover. We don't prioritize stories enough culturally, in my opinion. Every one of us has been annoyed by the self-proclaimed science geek who simply spits out facts they found on Reddit that day. It is an easy mistake to make, because we are trained in school to think like this. Fields like math and science and geography are most often taught in public schools as monolithic groups of facts to memorize by the test next Tuesday.

And when you make people memorize endless math tables or state capitals or the freezing point of elements, you lead them to believe a terrible thing, that facts equal knowledge. But they don't. Knowledge comes from taking facts and putting them in a context with each other. That context is narrative.

I have a great example. My high school freshman earth science teacher, Dan Frare, was telling us about glaciers, and he was trying to explain the features you saw in glaciers as they were moving. And he was trying to explain how slowly they moved. And he said to us, "The best way to picture a glacier is it's a river on Quaaludes." It was the '80s.

In fact, it was so long ago, I would go to Dan Frare's class at lunchtime, because I didn't have any friends. And I would pepper him with questions about science, and he would sit there and chain smoke in school while grading papers. This is a different time. Wait a second. Where was I? Quaaludes. Yes.

This is a beautiful way to talk about glaciers because it actually gave me a deep understanding of the physics of a glacier in one sentence. He took facts, and he put them in a story and gave my brain that story for the rest of my life.

Having told stories in the service of both art and science, I feel uniquely qualified—and you should know, I feel uniquely qualified for very few things—I feel uniquely qualified to tell you that I've come to understand that far from being at either end of a spectrum of human experience, people often say, "Oh, it's both an art and a science." And what we do when we say that is we place those things in opposition to each other and at a distance from each other.

And what I have come to understand is that science and art are simply both ways of telling stories, and for the same reason. We use these stories to figure out the shape of the universe around us.

I'm telling you all of this to talk about what I see as the two important missions that the EFF has been fulfilling throughout its tenure. One is, of course, the legal and logistical aspect of their job. Fighting in court, writing amicus briefs, and tirelessly using the tools available to them to help all of us enjoy a safer Internet with proper privacy, autonomy, and genuine dignity.

But in addition, in order to wake up the public to the realities of the problem, it's not enough to recount just the facts, ma'am. We have to make compelling arguments for why we need privacy and safe spaces as well as free speech and openness. And in addition to the legal vanguard it occupies, EFF is also always working to help people understand what they are fighting for and how the issues affect them.

In order to understand the thing, we need to see our place in and adjacent to it. And this is arguably the most difficult part of their job. Tonight's award winners are here for the fight, and just as much, they are here for the stories, because it is a universal human truth that when we share and listen to each other's stories, the world moves forward in a positive way.

We are living through a difficult and critical time. I now truly understand the meaning of the famous curse, "May you live in interesting times." And I am genuinely not sure that we're going to make it out of this. It is the central fact of my current and probably all of our current existence.

But if we make it out, and I believe this with my whole heart, if we do make it out, it'll be because we have listened to each other's stories and connected with realities different than ours, than the ones we might occupy, and we have worked hard to let all of those stories be told. I hope we do. Thank you so much to EFF, and thank you for your time.

Acceptance Speech by danah boyd — "Facing the Great Reckoning Head-On"

Audio

I cannot begin to express how honored I am to receive this award. My awe of the Electronic Frontier Foundation dates back to my teenage years. EFF has always inspired me to think deeply about what values should shape the internet. And so I want to talk about values tonight, and what happens when those values are lost, or violated, as we have seen recently in our industry and institutions.

But before I begin, I would like to ask you to join me in a moment of silence out of respect to all of those who have been raped, trafficked, harassed, and abused. For those of you who have been there, take this moment to breathe. For those who haven’t, take a moment to reflect on how the work that you do has enabled the harm of others, even when you never meant to.

<silence>

The story of how I got to be standing here is rife with pain and I need to expose part of my story in order to make visible why we need to have a Great Reckoning in the tech industry. This award may be about me, but it’s also not. It should be about all of the women and other minorities who have been excluded from tech by people who thought they were helping.

The first blog post I ever wrote was about my own sexual assault. It was 1997 and my audience was two people. I didn’t even know what I was doing would be called blogging. Years later, when many more people started reading my blog, I erased many of those early blog posts because I didn’t want strangers to have to respond to those vulnerable posts. I obfuscated my history to make others more comfortable.

I was at the MIT Media Lab from 1999–2002. At the incoming student orientation dinner, an older faculty member sat down next to me. He looked at me and asked if love existed. I raised my eyebrow as he talked about how love was a mirage, but that sex and pleasure were real. That was my introduction to Marvin Minsky and to my new institutional home.

My time at the Media Lab was full of contradictions. I have so many positive memories of people and conversations. I can close my eyes and flash back to laughter and late night conversations. But my time there was also excruciating. I couldn’t afford my rent and did some things that still bother me in order to make it all work. I grew numb to the worst parts of the Demo or Die culture. I witnessed so much harassment, so much bullying that it all started to feel normal. Senior leaders told me that “students need to learn their place” and that “we don’t pay you to read, we don’t pay you to think, we pay you to do.” The final straw for me was when I was pressured to work with the Department of Defense to track terrorists in 2002.

After leaving the Lab, I channeled my energy into V-Day, an organization best known for producing “The Vagina Monologues,” but whose daily work is focused on ending violence against women and girls. I found solace in helping build online networks of feminists who were trying to help combat sexual assault and a culture of abuse. To this day, I work on issues like trafficking and combating the distribution of images depicting the commercial sexual abuse of minors on social media.

By 2003, I was in San Francisco, where I started meeting tech luminaries, people I had admired so deeply from afar. One told me that I was “kinda smart for a chick.” Others propositioned me. But some were really kind and supportive. Joi Ito became a dear friend and mentor. He was that guy who made sure I got home OK. He was also that guy who took being called-in seriously, changing his behavior in profound ways when I challenged him to reflect on the cost of his actions. That made me deeply respect him.

I also met John Perry Barlow around the same time. We became good friends and spent lots of time together. Here was another tech luminary who had my back when I needed him to. A few years later, he asked me to forgive a friend of his, a friend whose sexual predation I had witnessed first hand. He told me it was in the past and he wanted everyone to get along. I refused, unable to convey to him just how much his ask hurt me. Our relationship frayed and we only talked a few times in the last few years of his life.

So here we are… I’m receiving this award, named after Barlow less than a week after Joi resigned from an institution that nearly destroyed me after he socialized with and took money from a known pedophile. Let me be clear — this is deeply destabilizing for me. I am here today in-no-small-part because I benefited from the generosity of men who tolerated and, in effect, enabled unethical, immoral, and criminal men. And because of that privilege, I managed to keep moving forward even as the collateral damage of patriarchy stifled the voices of so many others around me. I am angry and sad, horrified and disturbed because I know all too well that this world is not meritocratic. I am also complicit in helping uphold these systems.

What’s happening at the Media Lab right now is emblematic of a broader set of issues plaguing the tech industry and society more generally. Tech prides itself in being better than other sectors. But often it’s not. As an employee of Google in 2004, I watched my male colleagues ogle women coming to the cafeteria in our building from the second floor, making lewd comments. When I first visited TheFacebook in Palo Alto, I was greeted by a hyper-sexualized mural and a knowing look from the admin, one of the only women around. So many small moments seared into my brain, building up to a story of normalized misogyny. Fast forward fifteen years and there are countless stories of executive misconduct and purposeful suppression of the voices of women and sooooo many others whose bodies and experiences exclude them from the powerful elite. These are the toxic logics that have infested the tech industry. And, as an industry obsessed with scale, these are the toxic logics that the tech industry has amplified and normalized. The human costs of these logics continue to grow. Why are we tolerating sexual predators and sexual harassers in our industry? That’s not what inclusion means.

I am here today because I learned how to survive and thrive in a man’s world, to use my tongue wisely, watch my back, and dodge bullets. I am being honored because I figured out how to remove a few bricks in those fortified walls so that others could look in. But this isn’t enough.

I am grateful to EFF for this honor, but there are so many underrepresented and under-acknowledged voices out there trying to be heard who have been silenced. And they need to be here tonight and they need to be at tech’s tables. Around the world, they are asking for those in Silicon Valley to take their moral responsibilities seriously. They are asking everyone in the tech sector to take stock of their own complicity in what is unfolding and actively invite others in.

And so, if my recognition means anything, I need it to be a call to arms. We need to all stand up together and challenge the status quo. The tech industry must start to face The Great Reckoning head-on. My experiences are all-too common for women and other marginalized peoples in tech. And it it also all too common for well-meaning guys to do shitty things that make it worse for those that they believe they’re trying to support.

If change is going to happen, values and ethics need to have a seat in the boardroom. Corporate governance goes beyond protecting the interests of capitalism. Change also means that the ideas and concerns of all people need to be a part of the design phase and the auditing of systems, even if this slows down the process. We need to bring back and reinvigorate the profession of quality assurance so that products are not launched without systematic consideration of the harms that might occur. Call it security or call it safety, but it requires focusing on inclusion. After all, whether we like it or not, the tech industry is now in the business of global governance.

“Move fast and break things” is an abomination if your goal is to create a healthy society. Taking short-cuts may be financially profitable in the short-term, but the cost to society is too great to be justified. In a healthy society, we accommodate differently-abled people through accessibility standards, not because it’s financially prudent but because it’s the right thing to do. In a healthy society, we make certain that the vulnerable amongst us are not harassed into silence because that is not the value behind free speech. In a healthy society, we strategically design to increase social cohesion because binaries are machine logic not human logic.

The Great Reckoning is in front of us. How we respond to the calls for justice will shape the future of technology and society. We must hold accountable all who perpetuate, amplify, and enable hate, harm, and cruelty. But accountability without transformation is simply spectacle. We owe it to ourselves and to all of those who have been hurt to focus on the root of the problem. We also owe it to them to actively seek to not build certain technologies because the human cost is too great.

My ask of you is to honor me and my story by stepping back and reckoning with your own contributions to the current state of affairs. No one in tech — not you, not me — is an innocent bystander. We have all enabled this current state of affairs in one way or another. Thus, it is our responsibility to take action. How can you personally amplify underrepresented voices? How can you intentionally take time to listen to those who have been injured and understand their perspective? How can you personally stand up to injustice so that structural inequities aren’t further calcified? The goal shouldn’t be to avoid being evil; it should be to actively do good. But it’s not enough to say that we’re going to do good; we need to collectively define — and hold each other to — shared values and standards.

People can change. Institutions can change. But doing so requires all who harmed — and all who benefited from harm — to come forward, admit their mistakes, and actively take steps to change the power dynamics. It requires everyone to hold each other accountable, but also to aim for reconciliation not simply retribution. So as we leave here tonight, let’s stop designing the technologies envisioned in dystopian novels. We need to heed the warnings of artists, not race head-on into their nightmares. Let’s focus on hearing the voices and experiences of those who have been harmed because of the technologies that made this industry so powerful. And let’s collaborate with and design alongside those communities to fix these wrongs, to build just and empowering technologies rather than those that reify the status quo.

Many of us are aghast to learn that a pedophile had this much influence in tech, science, and academia, but so many more people face the personal and professional harm of exclusion, the emotional burden of never-ending subtle misogyny, the exhaustion from dodging daggers, and the nagging feeling that you’re going crazy as you try to get through each day. Let’s change the norms. Please help me.

Thank you.

Acceptance Speech by Oakland Privacy

Audio

Mike Katz-Lacabe:
So I first have to confess I'm not just a member of the EFF. I'm also a client. Thank you to Mitch Stoltz and your team for making sure that public records that I unearth remain available on the Internet for others to see.

So as Nash said, Oakland Privacy's strength comes not just from the citizens that volunteer as part of its group, but also from the coalitions that we build. And certainly every victory that is credited to us is the result of many, many other coalition members, whether in some cases it's the EFF or the ACLU or local neighborhood activists. It's really a coalition of people that makes us stronger and helps us get the things done that sometimes we not always deservedly get as much credit for. So I want to make sure to call out those other groups and to recognize that their work is important as well and critical for us.

EFF's nash presents a 2019 Barlow Award to members of Oakland Privacy



My work for Oakland Privacy comes from the belief that only from transparency can you have oversight, and from oversight derives accountability. So many examples of technology that have been acquired and used by law enforcement agencies in the Bay Area were never known about by the city councils that oversaw those police agencies.

In the city of Oakland, it was seven years after the city of Oakland acquired its stingray cell site simulator that the city of Oakland and the city council became aware of the use of that device by the police. In my city, I live in San Leandro, it was five years before the city council became aware of our city's use of license plate readers and a very notorious photo of me getting out of my car that was taken by a passing license plate reader got published on the Internet.

We do our best work when working together. That's been said. Let me give you ... speaking of stories, I'll take take off from Adam's talk here. For example, recently journalist Caroline Haskins obtained a bunch of documents pertaining to Ring, you may know the Ring doorbell, and its relationship with police departments. A post about a party that Ring held at the International Association of Chiefs of Police meeting with basketball player Shaquille O'Neal, where each attendee got five free Ring doorbells. That was highlighted by EFF Senior Investigative Researcher Dave Maass.

I, or we as Oakland Privacy, we then found a social media post by the police chief of Dunwoody, Georgia saying, "Hey, look at this great party with Ring, and there's Shaq." Dave then went and took that information, went back and looked at Dunwoody and found that subsequently, a few months later, Dunwoody was proud to announce the first law enforcement partnership with Ring in the state of Georgia. What a coincidence.

Oftentimes it's these coalitions working together that result in prying public records free and then establishing the context around them. The work we do involves very, very exciting things: Public records requests, lobbying of public officials and meeting with public officials, speaking at city council meetings and board of supervisors meetings. We're talking, this is, primo excitement here.

So, as was mentioned, our work with Oakland Privacy was helpful in getting the first privacy advisory commission, an actual city of Oakland commission going, within the city of Oakland. It's this organization, led by chair Brian Hofer, that passes policies regarding surveillance technologies, and not only passes policies but actually digs down and finds out what surveillance technologies the city of Oakland has. It has been a model for cities and counties, and we're proud that our work will continue there in addition to working on many other issues surrounding surveillance.

In fact, I would be very happy to tell you that we've had ... just recently the California assembly and the Senate passed a ban on the use of face surveillance on body-worn cameras. Again, our work with coalitions there makes the difference. And now, I would like to introduce another member of Oakland Privacy, Tracy Rosenberg.

Tracy Rosenberg:
Thank you, Mike, and hi, everyone, and thank you so much for this wonderful award. We are honored.

We're splitting up the speaking here because Oakland Privacy is a coalition and is a collective, and that's important to us. We have no hierarchy after all these years, and I've been doing this for five years. All that I get to call myself is a member. That's all I am.

I want to highlight, there are people in the audience that are not coming up on stage. J.P. Massar, Don Fogg, Leah Young. There are people that are not here whose names I won't mention since they're not here, but it's always a coalition effort.

And this week I've been jumping up and down because the broader coalition that includes EFF and Consumer Reports and ACLU and a bunch of other people, we just stood down the Chamber of Commerce, the tech industry, and pretty much every business in California in order to keep the Consumer Privacy Act intact.

There were six people on a whole bunch of conference calls, you don't want to know how many, and somehow we actually did it. It's official as of today. There is power in coalition work.

I'm incredibly grateful to Oakland Privacy because I was incredibly upset about the encroaching surveillance state, and I didn't know what to do. And in the end, in 2013, Oakland Privacy showed me what I could do, and I will never be able to repay the group for that.

I was thinking back to our first surveillance transparency ordinance in Santa Clara. EFF actually came down, and they took a picture of me speaking at that meeting and put it on their blog, and I thought, I wish I could put into words what lay behind that picture, which was 11 stinking months of going down to Santa Clara and sitting in that room with the goddamn Finance and Governmental Operations Committee where they were trying to bury our ordinance because let's face it, the powers that be don't want transparency. And every month standing there and saying, "I'm not going to let you do that. I'm just not."

We succeeded. It became law, I think it was June 7th, 2016, which doesn't feel like that long ago. And now there are 12. Eight of them are here in the Bay Area, a couple in Massachusetts, Seattle, and somehow Nashville did it without us and more power to them.

So I think that's pretty much what I kind of want to say here. I mean, what Oakland Privacy does fundamentally is we watch. The logo is the eye of Sauron, and well, I'm not a Tolkien geek, but I deal with what I am a part of. Hey look—I went to a basement, it was all guys. It is what it is. It's a little more gender-balanced now, but not entirely. But the point is that eye kind of stands for something important because it's the eye of "we are watching," and in really mechanical terms, we try to track every single agenda of God knows how many city councils there are in the Bay Area. I think we're watching about 25 now, and if a couple more of you would volunteer, we might make that 35.

But the point is, and every time there's a little action going on locally that's just making the surveillance state that much worse, we try to intervene. And we show up and the sad truth is that at this point, they can kind of see us coming from a mile away, and they're like, "Oh, great. You guys came to see us." But the point is, that's our opportunity to start that conversation. Oakland is a laboratory, it's a place where we can ... And Oakland's not perfect. All that you need to do is take a look at OPD and you know that Oakland's not perfect. Right? But it's a place where we've been able to ask the questions and we're basically trying to export that as far as it possibly can, and we go there and we ask the questions.

And really, the most important part to me and the part that gives me hope is we get a lot of people that come to the basement to talk to us and basically share with us how dystopia is coming, which we know. It's here. There's no hope, right? But when those people find the way to lift up their voices and say no, that's what gives me hope. So thank you. Thank you and Brian Hofer is also going to make a final set of comments. Thank you.

Brian Hofer:
So my name is Brian Hofer, I recently left Oakland Privacy. I founded Secure Justice with a handful of our coalition partners that are, some of who are in this room tonight. And we're going to continue carrying on the fight against surveillance, just like Oakland Privacy. I also had the privilege of chairing the city of Oakland's Commission, as you heard earlier, and it's an honor and a privilege to be recognized by EFF for the same reasons that my former colleagues have been saying, because you've been standing next to us in the trenches. You've seen us at the meetings, lobbying, joined in the long hours waiting at city council meetings late at night just for that two minute opportunity that Nash is now an expert at. You know how much labor goes into these efforts, and so I really want to thank you for standing next to us.

This path has been pretty unexpected for me. I quit a litigation job, was unemployed, and I read this East Bay Express article by Darwin BondGraham and Ali Winston based on public record requests that Oakland Privacy members had founded. And there's a little side bar in that journal that the very next day, just fate I guess, that this upstart group Oakland Privacy was meeting and that I could attend it. It's even more strange to me that I stayed. It was a two hour discussion about papier-mache street puppets and the people asking me if I was a cop when I walked in. Nobody wanted to sit next to me.

So when I finally spoke up and asked how many city council members they spoke to, the room got quiet. And so that became my job, because I was the one guy in the suit. At the honorable Linda Lye's going away party a couple months ago, I remarked that if we had lost the Domain Awareness Center vote, I would have never become an activist. I would have returned to my couch. I spent hundreds of hours on that project, and I would have been really disillusioned. But March 4th, 2014, which was the vote, is still the greatest day of my life. We generated international headlines by defeating the surveillance state in the true power to the people sense.

It was quite a contrast the following morning, on the Oakland Privacy list, when the naysayers thought the world had ended in calamity. Little did they know, that was the formation of the ad hoc privacy commission; we were about to change the conversation around surveillance and community control. EFF is directly responsible for helping us form that privacy commission in Oakland, and so it's my turn to congratulate you. Matt Cagle of the ACLU, Dia Kayyali, and myself were sitting around trying to figure out how to make it a permanent thing, and we noticed that another piece of technology was on the agenda. We didn't have any mandate or authority to write a privacy policy for it. But Dia signed a letter with me asking that we be given that task.

It worked, and that established the Privacy Commission as a policy writing instrument that remains today. As our colleagues were saying, that's been the launching pad for a lot of this legislative success around the greater Bay Area. It's the first of many dominoes to fall. I want to close with a challenge to EFF——and not your staff—like any non-profit, they're overworked and underpaid, because I'm sending them work and I don't pay for it. I was supposed to insert an Adam Schwartz joke there.

I believe that we're in a fight for the very fabric of this nation. Trump, people think he's a buffoon. He's very effective at destroying our civic institutions. The silent majority is silent, secure in their privilege, or too afraid or unaware how to combat what's going on. So I'm going to tell you a dirty secret about Oakland Privacy: we're not smarter than anyone else. We have no independently wealthy people. We have no connections. We didn't get a seat at the table via nepotism or big donations. We have no funding for the tens of thousands of volunteer hours spent advocating for human rights. And yet as you heard from the previous speakers, the formula of watching agendas, which anyone with an Internet connection can do in their pajamas, submitting public record requests, which anyone can do in their pajamas, and showing up relentlessly, which in Berkeley and Oakland, you can do in your pajamas—that led to a coalition legislative streak that will never be duplicated. That four year run will never happen again. So I ask that you challenge your membership to do the same, pajamas optional. We need numbers. We need people to get off their couch, like me, for the first time. The Domain Awareness Center was literally the first time I ever walked inside the open city hall, and I apologize for the police lingo, but your membership is the force multiplier and it's critical that more folks get involved. If you don't already know, somehow next week turned onto facial recognition ban week. Berkeley, Portland, Emeryville, we have our Georgetown national convening where I know EFF will be. It's critical that new diverse faces start showing up instead of the same actors. As Tracy said, they can see us from a mile away. We need more people.

In October, we expect four more cities to jump on board. Only one is in California, demonstrating that this isn't just a Bay Area bubble. It's got legs. And like the Domain Awareness Center moment, we've got a chance to change the national conversation, and we better take advantage of it. Thank you for this honor and thank you for this award.

Acceptance Speech by William Gibson

Audio
Thank you, Cory. And thank you, danah boyd. I will confess, I was actually ... I will confess I was actually a bit worried about coming down here and getting to this part of the evening and not having heard what she said or something very like it. And I found that a dismaying worry, and it's now been dismissed. So thank you.

This is the second time this year that I've received an award I wasn't expecting. The first one, Science Fiction Writers of America's Grand Master Award, I foolishly assumed I was too young for. With this one, though, I'd not thought it a possibility because I'm very probably, and I'm sure I could win a big bet with this, the least technically literate person in this room.

I seem to be here, though, I seem to myself to be here, because in the early 80s, knowing nothing whatever about computers, I began to listen to those who did, drawn not by their understanding, but by their vernacular poetics. Because I'm an English major. I got my B.A. in it, my specialty is in comparative literary critical methodologies. And when that also comes in really handy for a novelist is when we get a really shitty review. But what I actually did to come up with that stuff was sit in the bar at '80s SF cons in Seattle and eavesdrop, really really intensely. And then I would deconstruct the poetics of the computer literate.

Author William Gibson accepts his 2019 Barlow


The first time, for instance, that I heard interface used as an active noun, I physically swooned. Likewise, virus as a term of digital technology. That was where I first heard that as well. Made my eyes bug out, visibly. And if you don't believe me, I'll refer you to a scene in Neuromancer where Case, my street-smart cyberspace cowboy, finding that the going's just gotten particularly rough, issues an urgent call for a modem. Because I had, I confess, no idea what a modem was. But I loved the sound of the word. However, there's another scene in Neuromancer, one in which Case overhears sort of in background, partly what seems to the reader to be an infomercial for children, and it's describing something it calls, "The Matrix," with a capital M, which seems in context to be the sum of all this cyberspace thing that Case is always running around in.

But there's also in that little infomercial, there's a strong suggestion that the majority of that, of cyberspace, the majority of the content, is banal, everyday, absolutely quotidian. And by putting that in, I think I actually got that right. I somehow guessed that it all wouldn't be shit-hot cowboys versus a new order of giant corporations. So tonight, receiving this award from EFF, which by the way, I first heard of as a twinkle in John Perry Barlow's eye, though probably over the phone because he could do that. I'm very, very grateful that EFF exists, that it exists today to confront, among other things, the threat of the new order of giant corporations making it their business to gather magnitudes of utterly banal little bits of business about all of us. So thank you, EFF.

Special thanks to our sponsors: Airbnb; Dropbox; Matthew Prince; Medium; O'Reilly Media; Ridder, Costa & Johnstone LLP; and Ron Reed for supporting EFF and the 2019 Pioneer Award Ceremony. If you or your company are interested in learning more about sponsorship, please contact nicole@eff.org.

Victory! Individuals Can Force Government to Purge Records of Their First Amendment Activity

EFF - Fri, 09/13/2019 - 1:12pm

The FBI must delete its memo documenting a journalist’s First Amendment activities, a federal appellate court ruled this week in a decision that vindicates the right to be free from government surveillance.

In Garris v. FBI, the United States Court of Appeals for the Ninth Circuit ordered the FBI to expunge a 2004 memo it created that documented the political expression of news website www.antiwar.com and two journalists who founded and ran it. The Ninth Circuit required the FBI to destroy the record because it violated the Privacy Act of 1974, a federal law that includes a provision prohibiting federal agencies from maintaining records on individuals that document their First Amendment activity.

EFF filed a friend-of-the-court brief in the case that called on the court to robustly enforce the Privacy Act’s protections, particularly given technological changes in the past half century that have vastly increased the power of government to gather, store, and retrieve information about the expression and associations of members of the public. For example, law enforcement can use the Internet to collect and store vast amounts of information about individuals and their First Amendment activities.

Congress passed the Privacy Act after documenting a series of surveillance abuses by the FBI and other federal agencies, including tracking civil rights leaders like Martin Luther King, Jr., and spying on political enemies by President Richard Nixon. The law established rules about what types of information the government can collect and keep about people. The Act gives individuals the right to access records the government has on them and change or even delete that information.  One of the most protective provisions is a prohibition against maintaining records of First Amendment activity. Law enforcement was given a narrow exception for records that are “pertinent to and within the scope of an authorized law enforcement purposes.”

As EFF’s brief argued, “The prescient fears of the Act’s authors have been proven true by forty years of technological innovation that have given the federal government unprecedented ability to capture and stockpile data about the public’s First Amendment activity.”

In reversing a trial court’s ruling that the FBI did not have to delete the 2004 memo, the Ninth Circuit reviewed the language of the statute and concluded that the FBI did not have an authorized law enforcement purpose for keeping the memo. As the court explained, the Privacy Act’s expungement provision defines “maintain” as "maintain, collect, use, or disseminate."

The court said that because the definition is broad, Congress intended for the statute’s protections to apply to all those distinct activities. Simply put, an agency facing an expungement claim under the Privacy Act must show that the record at issue is pertinent to an authorized law enforcement activity both (1) during the initial collection of the record, and (2) during the ongoing storage of that record.

Or as the court put it: “That is, if the agency does not have a sufficient current ‘law enforcement activity’ to which the record is pertinent, the agency is in violation of the Privacy Act if it keeps the record in its files.”

The decision is a big win in the fight against ever-expanding federal law enforcement surveillance because it provides a meaningful mechanism for individuals to force the deletion of records that document their protected First Amendment activities.

This is essential in an era when so much political and social advocacy takes place online. As EFF argued in its brief:

As with political spying throughout our nation’s history, police scrutiny of First Amendment activity on the Internet chills and  deters expression in this critical democratic forum, and leads to unfairly disparate snooping on the speech of minority communities and political dissidents.

Given EFF’s commitment to fighting surveillance, we look forward to building on this case to protect individuals’ rights to speak out against the government. Congratulations to the ACLU of Northern California, who represented the plaintiff in the case, for working to meaningfully restrict the government’s surveillance powers.

Encrypted DNS could help close the biggest privacy gap on the Internet. Why are some groups fighting against it?

EFF - Thu, 09/12/2019 - 6:37pm

Thanks to the success of projects like Let’s Encrypt and recent UX changes in the browsers, most page-loads are now encrypted with TLS. But DNS, the system that looks up a site’s IP address when you type the site’s name into your browser, remains unprotected by encryption.

Because of this, anyone along the path from your network to your DNS resolver (where domain names are converted to IP addresses) can collect information about which sites you visit. This means that certain eavesdroppers can still profile your online activity by making a list of sites you visited, or a list of who visits a particular site. Malicious DNS resolvers or on-path routers can also tamper with your DNS request, blocking you from accessing sites or even routing you to fake versions of the sites you requested.

A team of engineers is working to fix these problems with “DNS over HTTPS” (or DoH), a draft technology under development through the Internet Engineering Task Force that has been championed by Mozilla. DNS over HTTPS prevents on-path eavesdropping, spoofing, and blocking by encrypting your DNS requests with TLS.

Alongside technologies like TLS 1.3 and encrypted SNI, DoH has the potential to provide tremendous privacy protections. But many Internet service providers and participants in the standardization process have expressed strong concerns about the development of the protocol. The UK Internet Service Providers Association even went so far as to call Mozilla an “Internet Villain” for its role in developing DoH.

ISPs are concerned that DoH will complicate the use of captive portals, which are used to intercept connections briefly to force users to log on to a network, and will make it more difficult to block content at the resolver level. DNS over HTTPS may undermine plans in the UK to block access to online pornography (the block, introduced as part of the Digital Economy Act of 2017, was planned to be implemented through DNS). 

Members of civil society have also expressed concerns over plans for browsers to automatically use specific DNS resolvers, overriding the resolver configured by the operating system (which today is most often the one suggested by the ISP). This would contribute to the centralization of Internet infrastructure, as thousands of DNS resolvers used for web requests would be replaced by a small handful. 

That centralization would increase the power of the DNS resolver operators chosen by the browser vendors, which would make it possible for those resolver operators to censor and monitor browser users’ online activity. This capability prompted Mozilla to push for strong policies that forbid this kind of censorship and monitoring. The merits of trusting different entities for this purpose are complicated, and different users might have reasons to make different choices. But to avoid having this technology deployment produce such a powerful centralizing effect, EFF is calling for widespread deployment of DNS over HTTPS support by Internet service providers themselves. This will allow the security and privacy benefits of the technology to be realized while giving users the option to continue to use the huge variety of ISP-provided resolvers that they typically use now. Several privacy-friendly ISPs have already answered the call. We spoke with Marek Isalski, Chief Technology Officer at UK-based ISP Faelix, to discuss their plans around encrypted DNS.

Supporting privacy-protecting technologies is a moral imperative.

Faelix has implemented support for DNS over HTTPS on their pdns.faelix.net resolver. They weren’t motivated by concerns about government surveillance, Marek says, but by ”the monetisation of our personal data.” To Marek, supporting privacy-protecting technologies is a moral imperative. “I feel it is our calling as privacy- and tech-literate people to help others understand the rights that GDPR has brought to Europeans,” he said, “and to give people the tools they can use to take control of their privacy.”

EFF is very excited about the privacy protections that DoH will bring, especially since many Internet standards and infrastructure developers have pointed to unencrypted DNS queries as an excuse to delay turning on encryption elsewhere in the Internet. But as with any fundamental shift in the infrastructure of the Internet, DoH must be deployed in a way that respects the rights of the users. Browsers must be transparent about who will gain access to DNS request data and give users an opportunity to choose their own resolver. ISPs and other operators of public resolvers should implement support for encrypted DNS to help preserve a decentralized ecosystem in which users have more choices of whom they rely on for various services. They should also commit to data protections like the ones Mozilla has outlined in their Trusted Recursive Resolver policy. With these steps, DNS over HTTPS has the potential to close one of the largest privacy gaps on the web.

EFF to Third Circuit: Off-Campus Student Social Media Posts Should be Entitled to Full First Amendment Protection

EFF - Thu, 09/12/2019 - 2:02pm

Special thanks to legal intern Maria Bacha who was the lead author of this post.

EFF, Student Press Law Center (SPLC), Pennsylvania Center for the First Amendment (PaCFA), and Brechner Center for Freedom of Information filed an amicus brief in B.L. v. Mahanoy Area School District urging the U.S. Court of Appeals for the Third Circuit to close a gap in the law to better protect off-campus student speech.

B.L., a student at Mahanoy Area High School, had tried out for the varsity cheerleading squad but had been placed on junior varsity. Out of frustration, she posted on Snapchat a selfie with the text “fuck school, fuck softball, fuck cheer, fuck everything” off school grounds on a Saturday. One of B.L.’s friends on Snapchat came across the “snap,” took a screen shot, and shared it with the cheerleading coaches. As a result, the coaches suspended B.L. from the junior varsity squad for one year. B.L.’s father appealed to the school board, which declined to get involved. B.L., through her parents, then filed a lawsuit against the district.

The U.S. District Court for the Middle District of Pennsylvania correctly held that B.L.’s off-campus speech was constitutionally protected. Thus, her public high school—a government institution bound by the First Amendment—could not lawfully punish her by suspending her from an extracurricular activity for her profanity. The school district appealed to the Third Circuit.

The district court relied on the Third Circuit’s prior decision in Snyder v. Blue Mountain School District (2011) to hold that B.L.’s profanity-laden “snap,” posted off campus and outside of school hours, was fully protected by the First Amendment. In Snyder, the Third Circuit interpreted the Supreme Court’s decision in Bethel School District No. 403 v. Fraser (1986) to hold that a public school may punish a student for vulgar on-campus speech—but that Fraser does not apply to off-campus speech.

One issue left open by the Third Circuit in Snyder is whether another Supreme Court student speech decision applies off campus: Tinker v. Des Moines Independent Community School (1969). That case involved only on-campus speech: students wearing black armbands on school grounds, during school hours, to protest the Vietnam War. The Supreme Court held that the school violated the student protestors’ First Amendment rights by suspending them for refusing to remove the armbands because the students’ speech did not “materially and substantially disrupt the work and discipline of the school,” and school officials did not reasonably forecast such disruption.

The Third Circuit in Snyder expressly declined to address the question of whether Tinker’s substantial disruption test applies to off-campus student speech. The district court in this case correctly concluded, if Tinker were to apply off campus, that B.L.’s off-campus speech could not be punished under Tinker’s substantial disruption test, because her “snap” did not cause a likelihood of substantial disruption or actual substantial disruption in her high school.

EFF’s amicus brief endorsed the district court’s decision in support of B.L., and further urged the Third Circuit to reach the question left open by Snyder and expressly hold that Tinker’s substantial disruption test does not apply to off-campus student speech.

We also wrote that because social media is an increasingly important medium for off-campus student expression, it is even more important today than it was when the Third Circuit issued its decision in Snyder that the court reach this open question.

Our brief provided the court with statistics and examples of how social media has increasingly become an important platform for advocacy and activism for young people all over the world, who use it as a tool to promote causes they believe in and advocate for change. Given the high barriers to entry of traditional communication channels, such as broadcast television, young people use social media to raise awareness, disseminate information, and garner supporters for the issues they care about. Social media is also a powerful tool for students seeking to discuss and criticize aspects of their lives at school.

Students should be free to express themselves online, from off-campus locations, outside of school hours, about even potentially controversial topics, without having to worry that school officials will claim that their speech somehow caused or may cause a disruption at school. Tinker’s substantial disruption rule does not offer sufficient protection for off-campus student speech and thus the Third Circuit should take this opportunity to hold that students’ off-campus speech is entitled to full First Amendment protection.

Victory! California Senate Votes Against Face Surveillance on Police Body Cams

EFF - Wed, 09/11/2019 - 7:48pm

The California Senate listened to the many voices expressing concern about the use of face surveillance on cameras worn or carried by police officers, and has passed an important bill that will, for three years, prohibit police from turning a tool intended to foster police accountability into one that furthers mass surveillance.

A.B. 1215, authored by Assemblymember Phil Ting, prohibits the use of face recognition, or other forms of biometric technology, on a camera worn or carried by a police officer in California for three years.  The Assembly passed an earlier version of the bill with a 45-17 vote on May 9. Today’s vote of the Senate was 22-15. We are pleased that the Senate has listened to the growing number of voices who oppose the way government agencies use face surveillance.

The government's use of face surveillance—particularly when used with body-worn cameras in real-time— has grave implications for privacy, free speech, and racial justice. For example, face recognition technology has disproportionately high error rates for women and people of color. Making matters worse, law enforcement agencies conducting face surveillance often rely on images pulled from mugshot databases, which include a disproportionate number of people of color due to racial discrimination in our criminal justice system.

As EFF activist Nathan Sheard told the California Assembly in May, using face recognition technology “in connection with police body cameras would force Californians to decide between actively avoiding interaction and cooperation with law enforcement, or having their images collected, analyzed, and stored as perpetual candidates for suspicion.” Stopping the use of face surveillance on police cameras for three years gives the state time to evaluate the effect that this technology has on our communities. We hope the California Legislature will follow-up with a permanent ban.

Thank you to everyone who contacted their legislators to support this bill. We also wish to thank the bill's sponsor, Assemblymember Ting, as well as the American Civil Liberties Union of Northern California and our many coalition partners for all of their hard work on this bill.

Lawmakers and community members across the country are advancing their own prohibitions and moratoriums on their local government’s use of face surveillance, including the San Francisco Board of Supervisors’ historic May ban on government use of face recognition. We encourage communities across the country to enact similar measures in their own cities.

A.B. 1215 will now head back to the Assembly for a procedural vote on its latest amendments, before being sent to the governor’s desk. We urge Governor Newsom to sign this important bill into law.

With the Intimidation of Tech Expert, Ola Bini Case Hits A New Low

EFF - Wed, 09/11/2019 - 6:52pm

In the early hours of last Thursday in Ecuador, members of the Judicial Police, assisted by the Prosecutor’s Office, broke down the door of Fabián Hurtado at his apartment in Quito. Hurtado is a cybersecurity expert at the International University (UISEK) in Ecuador and a digital forensics expert currently employed by Ola Bini’s defense. The police refused to let Hurtado read or have a copy of their warrant, and by immediately seizing his mobile phone and other digital equipment prevented him from contacting an attorney.

This raid was prompted, according to the authorities, by a belief that Hurtado “incorporated misleading information in his resume to try to mislead the authorities and citizens.” If so, the police action was wildly disproportionate to the alleged crime. Hurtado is a well-known, impartial forensics expert in Ecuador, employed by businesses, law enforcement, and defendants and plaintiffs alike. If there is a problem with Hurtado’s resume, the correct step would be to move to have him rejected as a court expert – not to storm into his home in the middle of the night.

Technical expertise will be vital ensuring a fair verdict in the Bini case, and the legitimacy of the trial will ultimately revolve around the technical knowledge of the court. The prosecution, after months of no obvious leads or reasons for Bini’s arrest and detention, settled last week on a charge based on a single screenshot obtained from Bini’s phone, showing a brief telnet connection years ago to an open router.

Bini’s case, from the very start, has been splayed across Ecuador’s media by the government, prosecutors and other political actors, with public opinion and short-term political expediency playing a greater part in the prosecution’s strategies rather than evidence and a fair trial. Given the high stakes initially described by government ministers and the prosecution – of squads of malicious hackers, and of Russian agents plotting to bring down the country’s infrastructure – the level of media attention is unsurprising. The danger to justice arises when one side uses their power as public servants to distort and undermine the legal process. With the raid on Fabián Hurtado, the prosecution give the impression of wanting to intimidate those most able to shine light on a very shady political affair.

Along with a growing number of civil liberties and international human rights groups, we urge the Ecuadorian authorities to step back and let the facts determine the course of the trial, with fairness and due process, and without intimidation or misuse of prosecutorial powers.

That Time EFF Got a Copyright Takedown Demand

EFF - Wed, 09/11/2019 - 4:56pm

Earlier this week, EFF received an email claiming that our body-camera police officer illustration (shown in the banner above) violated the sender’s copyright in a graphic they used to illustrate a tweet (cropped screenshot shown on the right). The email demanded we remove the image or provide a link to their e-commerce website, which sells police body cameras. For those interested in Search Engine Optimization (SEO), a link from EFF can be very beneficial to their page ranking. The funny thing was, the police officer illustration is an original EFF work.

It’s not a problem for someone to use our works in their own—they are available to the public under a Creative Commons attribution license—but that certainly doesn’t give a claim against our original. And their graphic had no attribution. (The Action Camera skateboarder illustration on the left appears to be an Adobe stock image.)

For EFF this was more amusing than threatening. We knew instantly that we needn’t worry about the implied threat, and if things went badly, we probably have more IP litigators per capita than any entity that’s not a boutique IP litigation firm. So we wrote back explaining the situation, and expect that will be the end of this.

But for many entities, it can be quite scary. Even if they are secure in their rights, the potential for a costly or time-consuming conflict may lead to a rational choice that a link is a low-cost solution. They might wonder if this misunderstanding will escalate into a DMCA takedown, potentially interfering with the availability of the page until the improper notice is resolved. Even if they disregard such a weak threat, dealing with it has the serious potential to take time away from running their operation.

We have not named the email’s sender. There is no indication that they are in the business of copyright trolling, it likely was a simple mistake, and we had no desire to use our platform to mobilize a shame campaign. Moreover, we’re well aware of the Streisand effect and see no need to provide the very link they seek in our discussion of why they shouldn’t have demanded a link. Instead, we hope that this example serves to show how copyright demands can be misused. Below is our response:

Dear [NAME REDACTED],

I am the General Counsel of the Electronic Frontier Foundation (EFF) and am writing in response to your email of September 10, in which you asserted that the illustration of a police officer in our Body Warn Camera page, https://www.eff.org/pages/body-worn-cameras, violated your company’s rights in the image used in your July 4 tweet, and demanded that we remove the illustration or provide a commercial link to your company on the eff.org website.

As an initial matter, please allow me to correct a fundamental mistake. The illustration on our page is an original image created by the Electronic Frontier Foundation, specifically our talented Art Director Hugh D’Andrade. Accordingly, you have no right to ask EFF, or anyone else, to remove our illustration, much less to provide your company with links or other benefits in exchange for its use. To the extent that you have sent similar demands to anyone else regarding our illustration, you will need to retract them immediately.

This is not to say that you can never use our illustration. In addition to your rights under fair use and fair dealing, EFF makes its content available under the Creative Commons Attribution 3.0 United States (CC BY 3.0 US) license, see https://www.eff.org/copyright. Thus, you would have had permission to use our illustration in your tweet if you had simply complied with the attribution requirement. If you wish to continue to use the police officer illustration, please be sure to comply with these license terms.

Finally, while we understand your desire to get links to your company’s website (and the SEO value of a link from EFF), we are disappointed and surprised that you would use copyright threats to try to make that happen. Baseless copyright threats are an ongoing problem for the Internet (see examples in our Takedown Hall of Shame https://www.eff.org/takedowns). Rather than contributing to that problem, we suggest that you and your company endeavor to earn that attention through the quality of your offerings.

In order to better educate the public about the issues with copyright demands, please note that we are blogging about your email and this response, but will endeavor to keep your name and the name of your company out of it.

Please let me know if you have any further questions.

best regards,

Kurt

-------------------------------
Kurt Opsahl, kurt@eff.org
Deputy Executive Director and General Counsel
Electronic Frontier Foundation https://www.eff.org/
ph: +1 415.436.9333 x 106 \\ fx: +1 415.436.9993 \\ @kurtopsahl

Congress Continues to Ignore the Dangerous Flaws of the CASE Act

EFF - Wed, 09/11/2019 - 1:36pm

The Copyright Alternative in Small-Claims Enforcement Act (CASE Act) is one of those (mostly) bad ideas that just won’t go away. It feels like a simple and easy solution to a thorny problem in copyright law: streamlining the dispute process. But as often happens, this solution is neither simple nor easy.

The U.S. House of Representatives’ Committee on the Judiciary followed its counterpart in the Senate by passing the CASE Act out of committee. This means that the whole House could vote to pass it, without bothering to fix any of its many flaws.

That would be a profound mistake.

prevent copyright trolling

Tell Congress not to treat free expression like a traffic ticket

The CASE Act’s goal is to make it simple and fast for copyright holders to get paid for infringement claims. The method it employs is to create a quasi-judicial body in the Copyright Office called the “Copyright Claims Board,” which would be able to award damages as high as $30,000 per proceeding, while also strictly limiting the ability of parties to appeal the decisions. $30,000 judgments issued by people who are not judges but rather officers of the Copyright Office, who see copyright holders—not the general public—as their customers, are not “small claims”. These are judgments that could ruin the lives of regular people; people who are engaging in the things we all do when we’re online: sharing memes, sharing videos, and downloading images.

During the mark up hearing, we once again heard the CASE Act described as a “voluntary” small claims system.

The CASE Act is not as “voluntary” as its boosters say it is. It cannot be emphasized enough how inadequate an “opt-out” system is. The way the CASE Act is currently structured, the Copyright Office sends a notice about the complaint to someone along with information about how to opt out. If they don’t opt out within 60 days of the notice—in whatever way the Copyright Office decides is the proper way to opt out—then the person is bound to whatever decision is made by the Claims Board, even if they don’t respond at all or don’t show up.

This is hardly a clear and easy process, particularly since it will come from a board most regular people have never heard of. Companies and people who have lawyers will know if they should to opt out. But the average Internet user will not, which means they will risk facing a huge judgment, potentially without ever having presented their side of the story.

Instead of a friendly and “voluntary” dispute system, CASE will create a new procedure for copyright trolls. Copyright trolls don’t make money through creating art, but through litigation and legal threats that target infringers and non-infringers alike. Courts have been reining in this activity, but the Copyright Claims Board will have none of the protections that those courts have established. Thus, CASE creates a quick and cheap way to collect on a large number of claims, without the review they might receive in a real court. And, given the amounts the CASE Act empowers the Claims Board toward, it will also make it easy for trolls to file a claim and then simply tell their targets to settle for what is still a significant amount, but less ruinous than the worst-case scenario of going through the full CASE Act process.

All of these problems don’t even include another huge flaw in the CASE Act: its new dispute system has to pass Constitutional muster. First, it’s not clear that Congress can just assign some copyright disputes to an administrative tribunal—particularly given that they may implicate fundamental speech concerns. Second, the proposed “opt-out” process is not just unfair—it’s potentially a violation of your constitutional right to due process.

There are just too many problems with the CASE Act for it to be a viable solution to the problem it’s supposed to solve. Tell your representatives to vote “no” on the CASE Act.

New Report Finds Border Communities Inundated with Surveillance Technologies

EFF - Wed, 09/11/2019 - 9:42am
EFF Partnership with University of Nevada, Reno Reynolds School of Journalism Launches ‘Atlas of Surveillance’

San Francisco - The Electronic Frontier Foundation (EFF) today published “The Atlas of Surveillance: Southwestern Border Communities,” the first report from a new research partnership with the University of Nevada, Reno’s Reynolds School of Journalism.

EFF and a team of students compiled profiles of six counties along the U.S.-Mexico border, outlining the types of surveillance technologies deployed by local law enforcement—including drones, body-worn cameras, automated license plate readers, and face recognition. The report also includes a set of 225 data points marking surveillance by local, state, and federal agencies in the border region.

Student researchers found a heavy concentration of surveillance technology, even among smaller municipalities. These technologies were often funded through federal grants, particularly Operation Stonegarden, which transfers money to local law enforcement to assist in border security operations.

“The federal government’s push to conduct persistent surveillance along the border has also accelerated adoption of advanced technology by police and sheriff departments in border town communities,” EFF Senior Investigative Researcher and Visiting Reynolds Professor of Media Technology Dave Maass says. “Our research paints a picture of a region beset by surveillance. Homeland Security agencies are using sensor towers and blimps. Meanwhile, we identified more than 30 law enforcement agencies on the ground using automated license plate readers, body-worn cameras, and mobile face recognition devices.”

The new report focuses on: San Diego County, Calif.; Pima and Cochise counties, Ariz.; Doña Ana County, N.M.; and El Paso and Webb counties, Texas. EFF and the Reynolds School are now embarking on a larger scale project to inventory police surveillance across the country, using crowdsourcing to collect news articles and public records and aggregating existing surveillance datasets.

“Because surveillance issues are increasingly relevant to every aspect of our lives, this collaboration between the Reynolds School at the University of Nevada, Reno and EFF provides great educational and engagement opportunities to our students,” Reynolds School Associate Professor and Director of the Center for Advanced Media Studies Gi Yun says.

To assist with large-scale newsgathering, EFF has launched a new tool, Report Back, to streamline research assignments. Report Back allows students to quickly receive small online reporting tasks, such as searching for a news article or government policy document about a certain technology in a particular jurisdiction. Once the student identifies a record about the surveillance technology, they enter the information into the greater database. Currently limited to Reynolds School students, EFF plans to expand its user base to grassroots organizations in 2020.

The Atlas of Surveillance project is part of a new partnership with the Reynolds School, which will also involve developing a course on Cybersecurity and Surveillance and leading students in regular workshops on filing Freedom of Information Act and other public records requests.

For the full report:
https://eff.org/atlas-border

Contact:  DaveMaassSenior Investigative Researcherdm@eff.org

Victory! California's Legislature Pulls AT&T and Comcast Bill That Protected Their Monopolies

EFF - Tue, 09/10/2019 - 6:59pm

AT&T and Comcast lobbyists fought hard this year to pass A.B. 1366, a bill that would have protected their broadband monopolies. Thanks to your support, that bill will not move forward this year.

The California legislature in 2012 decided to eliminate the authority of its own telecom regulator, the California Public Utilities Commission (CPUC) through the end of 2019—on the promise that such a move would produce an affordable, widely available, high-speed broadband network. What happened instead: over the past several years, California’s broadband market has been heading into a high-speed monopoly. For many, that’s led to more expensive and slower service than many other markets. In fact, all this law has done is protect broadband monopolies. As a result, the major ISPs were working hard to get it renewed through a new bill introduced this session, A.B. 1366.

EFF has opposed A.B. 1366 from the beginning, making clear that extending this law would leave the majority of Californians with only one option for high-speed broadband—if they had any at all. There was nothing good about the bill and nothing in it for Internet users. It exempted VoIP calls from privacy protections, it deregulated the prison telecom industry to allow for charging inmates' families insane rates, it hindered state public safety efforts, and it prevented the state from addressing its broadband monopoly problem.

Yet our opposition alone was not enough to stop this bill. It also took California residents all across the state contacting their legislators in Sacramento by phone and email to tell them to vote no. When the public speaks out clearly, all the money and influence loses. 

With a Very Bad Law Expiring, We Now Must Fight for a Better Future

With A.B. 1366 defeated—and the law restraining state authority over broadband set to expire at the end of this year—it is critical the state of California use its power to implement a plan to connect all Californians to affordable high-speed fiber access to the Internet. Such an effort is more important now than ever, as companies like AT&T and Verizon have abandoned fiber to the home, allowing cable companies like Comcast to remain unchallenged on high-speed access. Policymakers must do the hard work of identifying why the largest telecoms with billions in capital have willfully decided not to invest in future-proof networks. They must also promote policies that will accelerate the work of the small ISPs and local governments that shoulder the burden of building fiber networks.

As the fifth-largest economy in the world, and home to some of America’s largest cities, California can be just as connected as South Korea or Japan. Our rural markets can be connected to a 21st century infrastructure. Every major economy that is roughly the size of California has already adopted universal fiber plans. There is no reason this state can’t follow that example, or even step into a leadership role. In fact, it was historical efforts by California to inject competition in the telecom market in the 90s that lead to the 1996 Telecommunications Act’s foundational competition policies. Many of those federal laws inspired by California’s past efforts are what empower the handful of small fiber providers that exist today. As the FCC and Congress continue to fail to produce the national fiber policy this country desperately needs, California should move forward with its own plan to connect its residents.

Victory! Ruling in hiQ v. Linkedin protects scraping of public data

EFF - Tue, 09/10/2019 - 1:45pm

In a long-awaited decision in hiQ Labs, Inc. v. LinkedIn Corp., the Ninth Circuit Court of Appeals ruled that automated scraping of publicly accessible data likely does not violate the Computer Fraud and Abuse Act (CFAA). This is an important clarification of the CFAA’s scope, which should provide some relief to the wide variety of researchers, journalists, and companies who have had reason to fear cease and desist letters threatening liability simply for accessing publicly available information in a way that publishers object to. It’s a major win for research and innovation, which will hopefully pave the way for courts and Congress to further curb abuse of the CFAA.

The Trouble with the CFAA

Passed in 1986, the CFAA is the federal anti-hacking law, which imposes both criminal and civil liability on anyone who accesses a computer connected to the Internet “without authorization” or “exceeds authorized access.” Because the statute does not define “without authorization,” interpreting its meaning in the context of modern Internet usages has been notoriously difficult for courts around the country. The hiQ case is just the latest in a series of high-profile Ninth Circuit decisions about the CFAA, in which the appeals court has too often vacillated between limiting the CFAA to its original purpose and adopting more expansive interpretations that risk criminalizing widespread, innocuous online-behavior.

A key question in many early cases was whether companies and websites could enforce their computer use policies, like terms of service or corporate computer policies, through the CFAA’s concept of unauthorized access. In 2012, the Ninth Circuit issued a strong ruling in United States v. Nosal (Nosal I)  explaining that it refused to turn the CFAA “into a sweeping Internet-policing mandate.” The court instead chose to “maintain[] the CFAA’s focus on hacking,” holding that violating a company’s or website’s terms of use cannot give rise to liability. Otherwise, nearly anyone who used the Internet would face potential of criminal liability, for example by violating a social media site’s terms of service that prohibited even lying on a user profile.

Unfortunately, the Ninth Circuit muddied its own clear rule in two subsequent decisions, a second decision in the Nosal case (Nosal II) and Facebook v. Power Ventures, both involving password sharing. In Nosal II, the court found that “without authorization” is not limited to the circumvention of technical access mechanisms, like password barriers, and concluded that using someone else’s valid login credentials may violate the statute. Then, in Power Ventures, the court found that a data aggregator that had consent to access Facebook users’ accounts using their passwords nevertheless violated the CFAA by continuing to scrape data after Facebook sent a cease and desist letter and blocked one of Power Ventures’ IP addresses.

The dispute between hiQ and LinkedIn

EFF warned that the Ninth Circuit’s misguided decisions in Nosal II and Power Ventures would enable further abuse of the CFAA, and LinkedIn provided an example of why just weeks later.

HiQ Labs’ business model involves scraping publicly available LinkedIn data to create corporate analytics tools that could determine when employees might leave for another company, or what trainings companies should invest in for their employees. Perhaps because it intended to develop its own products that would compete with hiQ, LinkedIn served a cease and desist letter, stating it would implement technical measures to stop hiQ from accessing the website at all and relying on the Power Ventures case to argue that any further access to this public information would violate the CFAA. Rather than waiting to be sued, hiQ itself filed suit, obtaining a preliminary injunction in the district court, which found that hiQ was “likely to succeed” on its claims and holding that automated access to public information is likely not a violation of the CFAA. (The court used conditional “likely” language because preliminary injunctions are assessed on the chance a party will succeed after a full determination of the merits.)

On appeal, EFF filed an amicus brief, along with the search engine DuckDuckGo and the Internet Archive, urging the court to recognize that scraping is a commonplace technique that supports research in the public interest, among other beneficial uses. As a technical matter, web scraping is simply machine-automated web browsing, and accesses and records the same information, which a human visitor to the site might do manually. So-called good bots allow researchers to investigate racial discrimination on Airbnb, journalists to reveal price disparities on Amazon, and companies like DuckDuckGo and Google to use bots to make search engines return useful results.

Thankfully, the Ninth Circuit recognized how damaging it would be to extend its prior rulings to publicly available information as with LinkedIn profiles scraped by hiQ. As the court rightly pointed out, authorization commonly means that something is not generally available, and that access requires permission of some sort, whereas here, “the default is free access.” Thus, using automated scripts to access publicly available data is not the sort of "breaking and entering" into computers that the Computer Fraud and Abuse Act is intended to police. This ruling upholds the district court’s grant of a preliminary injunction, but the case could proceed to a further stage.

This is an extremely important holding that limits the mistakes in the Ninth Circuit’s earlier rulings in Nosal II and Power Ventures. The court says that those earlier cases control in situations where authorization is generally required—because data is not public—and the website owner either revokes that authorization or never gives it in the first place. But the court relies on a very narrow interpretation of public information that may not hold up in practice. Once someone logs on to Facebook, for example, a wealth of “private” information is available to every user of the service, making this information essentially publicly available. And, as we pointed out in these earlier cases, if a user grants a third party access, the third party has a form of authorization, even if the website itself would prefer the third party not have access. In any case, if authorization turns on whether or not someone has to log in to a free service, then this incentivizes a move to shield public information behind a log-in page.

Next Steps

Too often, the CFAA is used to chill speech and paint benign and even competitive uses of technology as malicious. While this decision represents an important step to putting limits on  using the CFAA to intimidate researchers with the legalese of cease and desist letters, the Ninth Circuit sadly left the door open to other claims, such as trespass to chattels or even copyright infringement, that might allow actors like LinkedIn to limit competition with its products. And even with this ruling, the CFAA is subject to multiple conflicting interpretations across the federal circuits, making it likely that the Supreme Court will eventually be forced to resolve the meaning of key terms like “without authorization.” Meanwhile, EFF will be on the lookout for more opportunities to protect research and innovation, and we’ll continue to protect security researchers as part of our Coders Rights Project.

Related Cases: hiQ v. LinkedIn

New Chair of EFF’s Board of Directors: Renowned Legal Expert Pamela Samuelson

EFF - Mon, 09/09/2019 - 4:54pm

EFF is proud to announce our newest Chair of our Board of Directors, renowned legal expert Pamela Samuelson. Pam has served on EFF’s board for nearly 20 years, and her deep knowledge of digital copyright law, intellectual property, and information policy has made EFF a stronger organization.

Pam is a co-director of the Berkeley Center for Law and Technology—an internationally respected research center at the University of California, Berkeley, School of Law. Pam is also co-founder and chair of the board of the Authors Alliance, a non-profit group that promotes the public interest in access to knowledge. She has written and spoken extensively about the challenges that new information technologies pose for traditional legal regimes, as well as about privacy, the First Amendment, and other cyberlaw issues.

Pam’s scholarship complements the ongoing work at EFF, demonstrating why fair use and other exceptions and limitations are important to achieving copyright's constitutional goals, why the CASE Act bill to create a small claims board for adjudicating copyright cases is flawed, why the anti-circumvention rules outlawing reverse engineering of technical protection measures should be revised, and why courts should not extend copyright protections to application program interfaces. In addition to writing scholarly articles on these issues, she writes a regular column as a Contributing Editor of the ACM on legal and policy issues affecting computing professionals and frequently files amicus curiae briefs in important cases on behalf of intellectual property professors in cases such as the Oracle v Google case, review of which is pending before the Supreme Court. Pam’s influential work has brought honors from around the world, including a Women of Vision Award from the Anita Borg Institute and the IP3 award from Public Knowledge.

Pam succeeds former board chair Brian Behlendorf, who will now be the vice chair. We are very grateful for Brian’s time as leader of our Board of Directors, and are thrilled that he will continue to bring his technology expertise to our board. Thank you both Pam and Brian!

Americans Deserve Their Day in Court About NSA Mass Surveillance Programs

EFF - Mon, 09/09/2019 - 1:45pm

EFF continues our fight to have the U.S. courts protect you from mass government surveillance. Today in our landmark Jewel v. NSA case, we filed our opening brief in the Ninth Circuit Court of Appeals, asserting that the courts don’t have to turn a blind eye to the government’s actions. Instead, the court must ensure justice for the millions of innocent Americans who have had their communications subjected to the NSA’s mass spying programs since 2001. Just this spring the Ninth Circuit Court of Appeals ruled in a case called Fazaga v. FBI that the state secrets privilege does not apply to cases challenging domestic electronic surveillance for national security.  Instead such cases must go forward to the merits of whether the spying is illegal. Today we asked the appeals court to apply that same reasoning to Jewel v. NSA and reverse a judge’s order of dismissal so our clients, and the American people, can finally have their day in court.

We argue in our brief:

“For over a decade, plaintiffs have sought a determination of whether the government’s acknowledged mass surveillance of the Internet communications and telephone records of hundreds of millions of Americans violates the Constitution and federal statutory law. But the district court refused to do so, defying Congress’s express command that such claims be decided on the merits—a command recently confirmed by this Court.”

This appeal challenges two separate orders of the district court dismissing first our Fourth Amendment claims, and later our statutory claims. Both dismissals were based in substantial part on the district court’s belief that the legality of the spying could not be adjudicated, even under protective court procedures, without revealing to the Judge at least, secret information which the government claims would harm national security.

The district court dismissed our Fourth Amendment claims in February 2015, finding that Jewel and the other plaintiffs could not prove on the available public evidence that they had been caught up in the spying. And the district court dismissed our remaining statutory claims in April 2019, claiming that it would be impossible to analyze the legality of the mass spying without revealing state secrets, and ruling again that the plaintiffs could not prove they were spied on based on the public evidence.

As we argue, the district court’s decisions wrongly deny the American people a ruling on whether the spying programs are legal:

“The district court’s dismissal hands the keys to the courthouse to the Executive, making it impossible to bring any litigation challenging the legality of such surveillance without the Executive’s permission.  It blinds the courts to what the Executive has admitted: the NSA has engaged in mass surveillance of domestic communications carried by the nation’s leading telecommunications companies, and this surveillance touches the communications and records of millions of innocent Americans.

“At stake are the statutory and constitutional bulwarks created to protect “the privacies of life” from the prying eyes of an all-seeing government.  Carpenter v. U.S., __ U.S. __, 138 S. Ct. 2206, 2214 (2018) (citation omitted). From the founding of the Republic, the Executive’s power to surveil has required robust constitutional and statutory limitations—including searching judicial review of the legality of surveillance—to ensure the privacy and freedom of all Americans.”

Our opening brief makes three main arguments:

  • First, the state secrets privilege cannot prevent consideration of whether the spying is legal because Congress created special secrecy procedures to enable courts to decide the legality of electronic communications surveillance. The district court was required to use those procedures (contained in section 1806(f) of FISA). Indeed, the Ninth Circuit ruled just this past February that the state secrets privilege does not apply in these types of cases. We urge the court’s panel of judges to apply the same rule here.
  • Second, even if the secret evidence is excluded, there is ample public evidence, including extensive government admissions, from which a judge could conclude that it is more probable than not that plaintiffs’ phone records were collected, that their Internet communications were intercepted and searched, and that metadata records of their Internet communications were collected. This is all that is needed to establish legal “standing” to bring the lawsuit; the trial judge must thus consider the legality of the spying programs.
  • Third, the Ninth Circuit should rule that the government’s interception of our clients’ Internet communications off of the Internet backbone without a warrant violated the Fourth Amendment.

Amicus briefs in support of our position will be filed next Friday and the government will file its responding brief in the weeks after that. After briefing is completed, the court will schedule a hearing, likely not for several months, with a decision thereafter.

This fight has been long and hard, and it’s likely to continue for some time. But stopping the modern-day version of the general warrants that the founders of the U.S. fought against is tremendously important. EFF is determined to ensure that the network we all increasingly rely on in our daily lives—for communicating with our families, working, participating in community and political activities, shopping, and browsing—is not also an instrument subjecting all of our actions to NSA mass surveillance.

Related Cases: Jewel v. NSAFirst Unitarian Church of Los Angeles v. NSAACLU v. Clapper

Watering Holes and Million Dollar Dissidents: the Changing Economics of Digital Surveillance

EFF - Fri, 09/06/2019 - 8:27pm

Recently, Google’s Project Zero published a report describing a newly-discovered campaign of surveillance using chains of zero day iOS exploits to spy on iPhones. This campaign employed multiple compromised websites in what is known as a “watering hole” attack. The compromised websites would automatically run the chain of exploits on anyone who visited, with the aim of installing a surveillance implant on the device. Google didn’t reveal the names of the websites or indeed who was being targeted but it soon became clear through other reporting that the likely target of this campaign was the Uyghur community, a Turkic Muslim minority in China facing mass detention and other harsh crackdowns perpetrated by the Chinese government with the most repressive policies coming into place in recent years.

Security company Volexity followed up the week after with detailed reports of similar website exploit chains targeting Android and Windows devices, again hosted on websites with a primarily Uyghur readership. This week, another publication confirmed that the Chinese government had compromised several international telcos in order to perform yet more invasive surveillance on expatriated Uyghurs.

Resetting Our Thinking on States and Zero Days

There are many important things to take away from these astonishing reports by Google and others. The biggest lesson is that we have to re-consider our understanding how state actors use zero days. The dominant thinking among security researchers has long been that governments and law enforcement would only want to use zero-day exploits sparingly and with very specific targets, to reduce the risk that an exploit would be discovered by security researchers or companies, who would then fix the bugs underlying the exploit, thus rendering it useless. 

Zero day exploits can be expensive, with iPhone exploits used against a single activist reportedly fetching upwards of 1 million dollars. Google’s report seemingly upends the traditional logic of zero day economics. This time a zero day was being used to exploit thousands of users, indiscriminately targeting all visitors to a specific set of websites. But if we consider the targets of this campaign and the likely actors behind it, the economics make perfect sense. While it is new to observe a state sponsored actor burning zero-days to target an entire community instead of one individual in the community it is a reasonable tactic in this case.

These attacks likely have the goal of spying on the Uyghur diaspora outside China, to gain as much intelligence as possible on anyone associated with this movement within China or supporting the community from outside of China’s national borders. In the past, China has already arrested many community leaders, Uyghur activists, human rights defenders, as well as their relatives, and is likely interested in discovering any nascent leaders before they become a problem.

Google’s report and Apple’s recent response both miss the mark on the impact of this attack. Google’s Project Zero post  was vague about the targeted nature of the attack saying “There was no target discrimination; simply visiting the hacked site was enough for the exploit server to attack your device … we estimate that these sites receive thousands of visitors per week.” Apple’s response understates the impact of the vulnerability stating, “the sophisticated attack was narrowly focused, not a broad-based exploit of iPhones ‘en masse”’as described." The reality is more complicated, this was a highly targeted attack against every member and supporter of the Uyghur community. Though this is technically a “watering hole” attack, the websites reported by Volexity as having been compromised were all hyper-targeted at the Uyghur community and its supports. Some were written in the Uyghur language, a Turkic language written with the Arabic script, which very few modern Turkic languages use today.

Google's post was light on specifics, but Project Zero researcher and report author Ian Beer highlighted an important way in which this discovery impacts the way we think about device security:

Real users make risk decisions based on the public perception of the security of these devices. The reality remains that security protections will never eliminate the risk of attack if you're being targeted. To be targeted might mean simply being born in a certain geographic region or being part of a certain ethnic group. All that users can do is be conscious of the fact that mass exploitation still exists and behave accordingly; treating their mobile devices as both integral to their modern lives, yet also as devices which when compromised, can upload their every action into a database to potentially be used against them. I hope to guide the general discussion around exploitation away from a focus on the million dollar dissident and towards discussion of the marginal cost for monitoring the n+1'th potential future dissident. I shan't get into a discussion of whether these exploits cost $1 million, $2 million, or $20 million. I will instead suggest that all of those price tags seem low for the capability to target and monitor the private activities of entire populations in real time.

If you are targeting one activist, it might cost one million dollars for the necessary zero day exploit, but if you are able to monitor thousands of activists or an entire ethnic population with a single  exploit suddenly the cost per person drops down to a much more affordable price. It’s unreasonable to think that economics of scale don’t apply to zero day exploits as they do to everything else. Many countries have an interest in targeting specific populations for surveillance (Palestinians in Israel, undocumented immigrants in the US, Kurds in Iran.) With that in mind, it’s likely that this is not the last time we will see a state actor targeting an entire ethnic or activist group  en masse with zero day exploits.

EFF's DEF CON 27 T-Shirt Puzzle

EFF - Fri, 09/06/2019 - 5:39pm

At Hacker Summer Camp 2019, EFF unveiled our 10th-annual limited edition member shirt—available only during the three-day event, and inspired by the DEF CON theme of Technology’s Promise: “a break from the dystopian imagery into a major-key, blue-sky thoughtscape, full of color and light...a future where we have tamed some of the more intractable problems that plague us in the present, where technology supports and inspires instead of controlling and surveilling.” We took cues from the DEF CON 27 Theme Guide, an illustrated ePub detailing thought exercises, media that inspired the theme, and color/style breakdowns. The theme was heavily influenced by the French comic artist Moebius’ piece entitled Alice, a piece that envisions “a future where tech lives up to our highest hopes.”

Shirt Design

EFF’s shirt design is an homage to Moebius, including artwork of a user in a digital future where the downsides of technology have been overcome. She's got a flying machine that is so efficient that it doesn't require her attention; she's got a vintage laptop from the early 21st century that still works thanks to the interoperability of her systems; she can communicate freely, using archaic Morse code, thanks to strong encryption on all of her devices; and of course, she has the ability to change her hair color at will.

As in previous years, we’ve included a secret puzzle built into the design of our exclusive member shirt as a special thanks for the clever, curious EFFers who support our work. Read on for a breakdown of the puzzle design and a walkthrough of the puzzle elements. Or, try to solve it yourself! The puzzle can be found at https://www.eff.org/shr/ and will be available through September 30th. After the 30th, the puzzle can be found on the Internet Archive’s Wayback Machine.

Puzzle Design

We established a technological utopia in our shirt design, and used only one rule to create a fantasy world map of this future: draw like Moebius. With this rule in mind, we wondered - what was under the clouds in the shirt design? Where would this user park her flying machine? We gathered reference materials to inspire us, and studied Le Monde d'Edena to capture Moebius’ distinct architectural and landscape style. We wanted the fantasy map to draw players into the world, where they would get lost in the tiny paths and buildings of this utopian civilization, eventually finding doors and hidden passages to the elements of the puzzle.

Printable version of EFF's DEF CON 27 puzzle map



Puzzle Walkthrough

Players access the puzzle via the morse code URL on the screen of the intrepid user in our shirt design (. ..-. ..-. .-.-.- --- .-. --. -..-. ... .... .-.). After arriving on the website, mousing over the map reveals glowing links in four different locations. The tower holds the final goal, which can only be deciphered after solving the other three puzzles. We decided to make progression through the puzzle non-linear this year in order to facilitate collaboration, and to give solvers time to do other DEF CON activities in between working on the EFF puzzle.

Knitting

The “first” puzzle can be found by hovering over the entrance to the palace. The page displays a knitting pattern in chart form. Knitting was chosen as a theme due to the history of women using knit fabrics for “steganography”—to hold concealed messages in times of war, particularly World War I. We also wanted to highlight skill sets not often represented in the cybersecurity space—someone familiar with knitting can immediately recognize something wrong with this pattern. There are entire rows of open circles, the symbol for a “yarn over” (skipping a stitch to create a hole), which doesn’t make sense. Inspecting the pattern reveals a hint, however: “the yarn overs are blanks”. Here are some examples of common patterns—you can compare to see the difference.

If you aren’t familiar with knitting, searching “yarn over” and identifying the symbol is the first step to solving this puzzle. The next is recognizing that the page title “SOS” is hinting at more morse code. Looking at the image, there are three different kinds of rows: all knits (blank squares), all purls (filled circles), and all yarn overs. We already know that the yarn overs are blanks from the hint, so that means trying the pattern in two ways: one with the knits as dots, and one as dashes. The correct method is converting knits to dots, which gives the morse code string “.-- .- -. -.. . .-.” which translates to “wander”.

“ASCII” Art

The next puzzle’s page is a throwback to an older Internet era, and is located at the docks tunnel. The page contains (mostly) ASCII art, like the following header that was used in EFF’s EFFector mailings (if you haven’t signed up to receive them, you can do so here):

There are a few different ways to identify where the puzzle is on this page: noticing that the whitespace is off in some of the images, such as the computer, or by taking a closer look at the characters in the pieces themselves. Looking at the character codes, the non-ASCII character U+2800, a braille space, can be found in place of a standard ASCII space 32. A hint in the page reads: “the distribution doesn’t matter”— clueing the solver into the idea that the order of the braille spaces within the art doesn’t hold the solution. Using a script to count the occurrences of U+2800 in each art piece gives the following: ”99 101 112 104 97 108 111 112 111 100”, which are ASCII codepoints for “cephalopod”.

If you’re curious, check out the script used to encode the braille. You can count the characters with the following line of ruby, where the text is a single piece of art:

`text.count("\u2800").chr`

Syzygy

The third puzzle, found in the entrance to the temple, consists of this image:


The first hint towards solving this puzzle is in the URL: “5y2y6y” is “leetcode” for “syzygy”, which is a phenomenon where many celestial bodies (at least three) are collinear in the same gravitational system. The image, keeping with the theme of exploration, shows an imagined system of planets. The answer to the puzzle comes from the first number of rotation steps where the planets will all align under the arrow. There are a number of ways to find this number. The number is small enough that you can write a program to brute force the answer. Or, since the periods of the planets are relatively prime, you can use the Chinese remainder theorem. The first full syzygy of this planetary setup is at step 74800, or leetcode for “taboo”.

Final Puzzle

The last puzzle, located at the top of the tower, simply displays:

Inspecting the page gives the hint “O(ne) T(rue) P(airing)”, a reference to the one time pad cipher, a relatively old message encryption technique that is difficult to crack as long as the key is random, as long or longer than the plaintext, never reused, and kept secret. A tool like this one can be used to decrypt the text, or the adventurous can read about one time pads and try their hand at translating it themselves. The key to this one time pad is “wandercephalopodtaboo”, a combination of the answers from the three other puzzles. Using this key to decrypt the ciphertext translates it from “oerbslutpjenclprr” to  “seeyouspacecowboy”, marking the completion of the puzzle!

Winners

The winners of this year’s puzzle are @aaronsteimle,  @_pseudoku, and @0xCryptoK! Their team members also solved our 2013, 2015, 2017, and 2018 DEF CON shirt puzzles.

Thank you to everyone who came by the booth to say hello, donated, and played through our puzzle! Creating this interactive art for our supporters is one of the highlights of our year, and we could not do it without you. If you’d like to support our work, consider becoming a member—and don’t forget to stop by the booth next year.

Until next time: “See you, space cowboy.”

Donate

ENJOYED THE PUZZLE? SUPPORT EFF!

Pages