Symposium - 2013 - Privacy and Big Data

Consumer Subject Review Boards

A Thought Experiment

Ryan Calo *

The adequacy of consumer privacy law in America is a constant topic of debate. The majority position is that United States privacy law is a “patchwork,” that the dominant model of notice and choice has broken down,[1] and that decades of self-regulation have left the fox in charge of the henhouse.

A minority position chronicles the sometimes surprising efficacy of our current legal infrastructure. Peter Swire describes how a much-maligned disclosure law improved financial privacy not by informing consumers, but by forcing firms to take stock of their data practices.[2] Deirdre Mulligan and Kenneth Bamberger argue, in part, that the emergence of the privacy professional has translated into better privacy on the ground than what you see on the books. [3]

There is merit to each view. But the challenges posed by big data to consumer protection feel different. They seem to gesture beyond privacy’s foundations or buzzwords, beyond “fair information practice principles” or “privacy by design.” The challenges of big data may take us outside of privacy altogether into a more basic discussion of the ethics of information.[4] The good news is that the scientific community has been heading down this road for thirty years. I explore a version of their approach here.

Part I discusses why corporations study consumers so closely, and what harm may come of the resulting asymmetry of information and control. Part II explores how established ethical principles governing biomedical and behavioral science might interact with consumer privacy.

I. Rationales for Studying Behavior

There are only a handful of reasons to study someone very closely. If you spot a tennis rival filming your practice, you can be reasonably sure that she is studying up on your style of play. Miss too many backhands and guess what you will encounter come match time. But not all careful scrutiny is about taking advantage. Doctors study patients to treat them. Good teachers follow students to see if they are learning. Social scientists study behavior in order to understand and improve the quality of human life.

Why do corporations study consumers? An obvious reason is to figure out what consumers want so as to be in a position to deliver it—hopefully better and cheaper than a competitor. I assume the reason that Microsoft employs the second greatest number of anthropologists in the world (after the United States government)[5] has to do with designing intuitive and useful software. But is that the only reason companies study consumers? And if not, how should we think about consumers as subjects of scientific scrutiny?

Were you to play the market equivalent of tennis against a corporation, it seems fair to think you would lose. They have several advantages. The first advantage is superior information. The websites and stores you visit gather whatever data they can about you and may supplement that information with profiles they purchase from third-party data brokers.[6] They also run data through powerful algorithms in a constant quest for novel insight.[7] The second advantage is that firms tend to control the circumstances of their transactions with consumers, sometimes entirely. Apple does not divulge its preferences and travel to a website you created from scratch in order to sell you music.[8] Firms hire people with advanced degrees and give them access to cutting-edge technology and rich datasets. These people write the legal terms and design the virtual and physical spaces in which our interactions with the firms occur.

Such advantages are fine in a win-win situation. The truth, however, is that sometimes consumers lose. The well-documented use of software by banks to maximize consumer overdraft fees by manipulating when ATM and debit transactions get processed is a simple enough example.[9] But pause to consider the full universe of possibility. Recent research suggests that willpower is a finite resource that can be depleted or replenished over time.[10] Imagine that concerns about obesity lead a consumer to try to hold out against her favorite junk food. It turns out there are times and places when she cannot. Big data can help marketers understand exactly how and when to approach this consumer at her most vulnerable—especially in a world of constant screen time in which even our appliances are capable of a sales pitch.[11]

If this sort of thing sounds far-fetched, consider two recent stories published by the New York Times. The first articleobligatory in any discussion of big data and privacy—focuses on how the retail giant Target used customer purchase history to determine who among its customers was pregnant, following which Target added ads related to babies in their direct marketing to those customers.[12] A second article describes the “extraordinary” lengths to which food manufactures go to scientifically engineer craving.[13] Either story alone raises eyebrows. But taken together they bring us closer than is comfortable to the scenario described in the previous paragraph.

My current writing project, Digital Market Manipulation, discusses the incentives and opportunities of firms to use data to exploit the consumer of the future.[14] But it is easy to take such concerns too far. The ascendance of big data will likely improve as many lives as it impoverishes.[15] The same techniques that can figure out an individual consumer’s reservation price or pinpoint a vulnerability to a demerit good can filter spam, catch terrorists, conserve energy, or spot a deadly drug interaction.[16] And big data may never deliver on its extraordinary promise. Both its proponents and detractors have a tendency to ascribe near magical powers to big data. These powers may never materialize.[17] Yet the possibility that firms will abuse their asymmetric access to and understanding of consumer data should not be discounted. I believe changes in this dynamic will prove the central consumer protection issue of our age.[18]

II. Ethical Principles

People have experimented on one another for hundreds of years. America and Europe of the twentieth century saw some particularly horrible abuses. In the 1970s, the U.S. Department of Health, Education, and Welfare commissioned twelve individuals, including two law professors, to study the ethics of biomedical and behavioral science and issue detailed recommendations. The resulting Belmont Report—so named after an intensive workshop at the Smithsonian Institute’s Belmont Conference Center—is a statement of principles that aims to assist researchers in resolving ethical problems around human-subject research.[19]

The Report emphasizes informed consent—already a mainstay of consumer privacy law.[20] In recognition of the power dynamic between experimenter and subject, however, the Report highlights additional principles of “beneficence” and “justice.” Beneficence refers to minimizing harm to the subject and society while maximizing benefit—a kind of ethical Learned Hand Formula. Justice prohibits unfairness in distribution, defined as the undue imposition of a burden or withholding of a benefit. The Department of Health, Education, and Welfare published the Belmont Report verbatim in the Federal Register and expressly adopted its principles as a statement of Department policy.[21]

Today, any academic researcher who would conduct experiments involving people is obligated to comply with robust ethical principles and guidelines for the protection of human subjects, even if the purpose of the experiment is to benefit those people or society. The researcher must justify her study in advance to an institutional, human subject review board (IRB) comprised of peers and structured according to specific federal regulations.[22] But a private company that would conduct experiments involving thousands of consumers using the same basic techniques, facilities, and personnel faces no such obligations, even where the purpose is to profit at the expense of the research subject.[23]

Subjecting companies to the strictures of the Belmont Report and academic institutional review would not be appropriate. Firms must operate at speed and scale, protect trade secrets, and satisfy investors. Their motivations, cultures, and responsibilities differ from one another, let alone universities. And that is setting aside the many criticisms of IRBs in their original context as plodding or skewed.[24] Still, companies interested in staying clear of scandal, lawsuit, and regulatory action could stand to take a page from biomedical and behavioral science.

The thought experiment is simple enough: the Federal Trade Commission, Department of Commerce, or industry itself commissions an interdisciplinary report on the ethics of consumer research. The report is thoroughly vetted by key stakeholders at an intensive conference in neutral territory (say, the University of Washington). As with the Belmont Report, the emphasis is on the big picture, not any particular practice, effort, or technology. The articulation of principles is incorporated in its entirety in the Federal Register or an equivalent. In addition, each company that conducts consumer research at scale creates a small internal committee comprised of employees with diverse training (law, engineering) and operated according to predetermined rules.[25] Initiatives clearly intended to benefit consumers could be fast-tracked whereas, say, an investigation of how long moviegoers will sit through commercials before demanding a refund will be flagged for further review.

The result would not be IRBs applying the Belmont Report. I suspect Consumer Subject Review Boards (CSRBs) would be radically different. I am not naïve enough to doubt that any such effort would be rife with opportunities to pervert and game the system. But the very process of systematically thinking through ethical consumer research and practice, coupled with a set of principles and bylaws that help guide evaluation, should enhance the salutary dynamics proposed by Mulligan, Bamberger, Swire, and others.

Industry could see as great a benefit as consumers. First, a CSRB could help unearth and head off media fiascos before they materialize. No company wants to be the subject of an article in a leading newspaper with the title How Companies Learn Your Secrets. Formalizing the review of new initiatives involving consumer data could help policy managers address risk. Second, CSRBs could increase regulatory certainty, perhaps forming the basis for an FTC safe harbor if sufficiently robust and transparent. Third, and most importantly, CSRBs could add a measure of legitimacy to the study of consumers for profit. Any consumer that is paying attention should feel like a guinea pig, running blindly through the maze of the market. And guinea pigs benefit from guidelines for ethical conduct.[26]

I offer CSRBs as a thought experiment, not a panacea. The accelerating asymmetries between firms and consumers must be domesticated, and the tools we have today feel ill suited. We need to look at alternatives. No stone, particular one as old and solid as research ethics, should go unturned.

  1. See Daniel J. Solove, The Digital Person: Technology and Privacy in the Information Age 71 (2006) (“Thus, the federal privacy statutes form a complicated patchwork of regulation with significant gaps and omissions.”); Daniel J. Solove,Introduction: Privacy Self-Management and the Consent Dilemma, 126 Harv. L. Rev. 1880, 1880-82 (2013).
  2. See Peter P. Swire, The Surprising Virtues of the New Financial Privacy Law, 86 Minn. L. Rev. 1263, 1264, 1316 (2002).
  3. See Kenneth Bamberger & Deirdre Mulligan, Privacy on the Books and on the Ground, 63 Stan. L. Rev. 247 (2011); cf. Omer Tene & Jules Polonetsky, Big Data for All: Privacy and User Control in the Age of Analytics, 11 Nw. J. Tech. & Intell. Prop. 239 (2013) (urging a cautious approach to addressing privacy in big data).
  4. My topic here is the intersection of corporate ethics and consumer privacy. There is a rich literature around the ethics of privacy, but it tends to focus on the importance of privacy as a value. See, e.g., Anita L. Allen, Unpopular Privacy: What Must We Hide? (2011); James H. Moor, The Ethics of Privacy Protection, 39 Libr. Trends 69 (1990).
  5. See Graeme Wood, Anthropology Inc., The Atlantic (Feb. 20, 2013),
  6. See Julia Angwin, The Web’s New Gold Mine: Your Secrets, Wall St. J. (Jul. 30, 2010),
  7. See Ira S. Rubinstein et al., Data Mining and Internet Profiling: Emerging Regulatory and Technical Approaches, 75 U. Chi. L. Rev. 261 (2008) (describing the capabilities of data mining).
  8. The ability to design the interface means, for instance, that Apple can update the look of its progress bar to create the appearance of faster download times. See Chris Harrison et al., Faster Progress Bars: Manipulating Perceived Duration with Visual Augmentations (2010), available at (finding Apple’s new progress bar reduces perceived duration by 11% in subjects). Apple even brings psychology to bear in its physical store. See, e.g., Marcus Morretti, Revealed: These 10 Extraordinary Rules Make Apple Stores the Most Profitable Retailers in the World, Bus. Insider (June 18, 2012),
  9. See Halah Touryalai, Are Banks Manipulating Your Transactions to Charge You an Overdraft Fee?, Forbes (Feb. 22, 2012), (reporting on the launch of a Consumer Finance Protection Bureau investigation into how banks process overdraft fees). Several banks eventually settled multimillion-dollar class actions lawsuits.
  10. For a popular account of this literature, see generally Roy Baumeister & John Tierney, Willpower: Rediscovering the Greatest Human Strength (2012).
  11. Objects, from watches to refrigerators, will increasingly be networked and have interfaces. A report by the Swiss mobile device company Ericsson and the Alexandra Institute estimates about fifty billion devices will be networked by 2020 into an “Internet of Things.” See Inspiring The Internet of Things! 2 (Mirko Presser & Jan Holler, eds., 2011),available at
  12. Charles Duhigg, How Companies Learn Your Secrets, N.Y. Times Mag. (Feb. 16, 2012), Michael Moss, The Extraordinary Science of Addictive Junk Food, N.Y. Times Mag. (Feb. 20, 2013), Ryan Calo, Digital Market Manipulation (Univ. of Wash. Sch. of Law, Research Paper No. 2013-27, 2013), available at
  13. For a definition of big data and an optimistic account of its impact on society, see Viktor Mayer-Schönberger & Kenneth Cukier, Big Data: A Revolution that Will Transform how We Live, Work, and Think (2013).
  14. See id; see also Jane Yakowitz, Tragedy of the Data Commons, 25 Harv. J.L. & Tech. 1, 8-10 (2011). “Reservation price” and “demerit good” are economic terms referring, respectively, to the highest price a person is willing to pay and a product that is harmful if over-consumed.
  15. See Paul Ohm, Response, The Underwhelming Benefits of Big Data, 161 U. Pa. L. Rev. Online 339, 345 (2013),available at Already much consumer protection law focuses on asymmetries of information and bargaining power, which big data stands to dramatically enhance.
  16. Nat’l Comm’n for the Prot. of Human Subjects of Biomedical & Behavioral Research, The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research (1978).
  17. See M. Ryan Calo, Against Notice Skepticism in Privacy (and Elsewhere), 87 Notre Dame L. Rev. 1027, 1028, 1032 (2012).
  18. Protection of Human Subjects, 44 Fed. Reg. 23,192 (Apr. 18, 1979).
  19. See Protection of Human Subjects, 45 C.F.R. §§ 46.103, 46.108 (2012) (describing IRB functions and operations).
  20. Cf. Evgeny Morozov, To Save Everything, Click Here: The Folly of Technological Solutionism 148 (2013) (“What institutional research board would approve Google’s quixotic plan to send a fleet of vehicles to record private data floating through WiFi networks or the launch of Google Buzz . . . ?”). Morozov’s point seems to be that technology companies should think before innovating. I’m not sure I agree with this frame. His examples are also curious—there is no evidence that Google sniffed WiFi on purpose and the problem with Google Buzz was not enough advanced consumer testing. See also Ohm, supra note 16, at 345 (noting that hospitals examining health records should conform to human subject research rules).
  21. See, e.g., Dale Carpenter, Institutional Review Boards, Regulatory Incentives, and Some Modest Proposals for Reform, 101 Nw. U. L. Rev. 687 (2007).
  22. Without delving into issues of standards or structure, Viktor Mayer-Schönberger and Kenneth Cukier briefly suggest that firms employ “internal algorithmists” akin to ombudsman that vet big data projects for integrity and societal impact. See Mayer-Schönberger & Cukier, supra note 14, at 181-82.
  23. Nat’l Research Council, Guide for the Care and Use of Laboratory Animals (8th ed. 2011).
Back to the Top