every era of measuring people
was rebuilt on a new kind of data.

Papers tallied informal votes taken at militia musters, juries and July 4 gatherings in the four-way 1824 race. The tallies put Andrew Jackson ahead of John Quincy Adams. [1][2]

Curtis hired Parlin for a job no one had held before. He named the work "commercial research," which became market research. [3]

The Literary Digest mailed ballots across the country and called Woodrow Wilson's win, starting two decades of mail-in polls. [2]

Starch founded Daniel Starch and Staff. His recognition method measured which print ads readers saw and remembered. [4]

Rensis Likert scored agreement with each statement from 1 to 5 and dropped the panel of judges older attitude scales needed. It became the most used response scale in survey research. [5][6]

George Gallup founded the American Institute of Public Opinion in Princeton, New Jersey. [7]

The Digest mailed ten million ballots and 2.38 million came back. It predicted Landon with 57%. Roosevelt carried every state but Maine and Vermont. [8][1]

lesson A huge sample of the wrong people loses to a small sample of the right ones.

Gallup sampled 50,000 voters chosen to match the country and called Roosevelt's win. The magazine folded by 1938. [8]

Tom Harrisson, Charles Madge and Humphrey Jennings founded Mass-Observation to record everyday life in Britain. Paid observers watched people in Bolton while volunteers kept diaries. [9]

Paul Lazarsfeld, Bernard Berelson and Hazel Gaudet interviewed 600 people in Erie County, Ohio each month of the 1940 campaign. Their panel found that conversation moved votes more than newspapers or radio did. [10]

Robert K. Merton and Patricia Kendall set down the group interview method from wartime propaganda studies with Paul Lazarsfeld. It became the focus group. [11][12]

Ernest Dichter founded the Institute for Motivational Research to find the hidden reasons people buy. He is credited with the term "focus group." [13]

Polls forecast a Dewey landslide, and the Chicago Daily Tribune printed the wrong winner. Gallup and Roper had stopped polling early and used quota samples. Truman won by 4.4 points. [14][15]

lesson Pollsters stopped measuring before people decided. The industry moved to probability sampling.

Ford built the Edsel on years of surveys and motivation research into what buyers wanted from a mid-priced car. Buyers stayed away, and Ford ended the brand in November 1959. [16][17]

lesson Asking buyers what they wanted years ahead missed what they bought on the day.

Warren Mitofsky's CBS News unit asked Kentucky voters how they had voted as they left the polls in the governor's race. He ran the first national exit poll in 1972. [18][19]

Paul Green and Vithala Rao showed how to measure the trade-offs people make between product features. Conjoint became marketing's standard trade-off method. [20]

Coca-Cola changed its formula after nearly 200,000 taste tests favored the new recipe. Drinkers revolted, and Coca-Cola Classic returned 79 days later. [21][22][23]

lesson A sip in a test told Coca-Cola what people liked, not what they would do when it took their Coke away.

Final polls put the Conservatives about 1 point behind Labour. John Major's party won by 7.6 points, and the Market Research Society opened an inquiry. [24][25]

lesson Quota samples missed late deciders and the voters who would not say.

The networks called Florida for Al Gore from Voter News Service exit polls, retracted it, then called it for George W. Bush and retracted that too. VNS's review found flawed precinct samples and an absentee count close to twice its estimate. [26][27][28]

lesson A thin sample of the day's voters cannot call a race decided by hundreds.

Polls underestimated the Conservative lead over Labour, and the party won a majority. An inquiry led by Patrick Sturgis found the samples held too many Labour supporters, and weighting did not fix it. [29][30]

lesson Weighting cannot repair a sample that reaches the wrong people.

Gallup, the firm that called 1936, said it would not poll the 2016 presidential primaries. Editor-in-chief Frank Newport said it would study the issues instead. [31]

National polls landed close, but state polls showed the wrong winner in Pennsylvania, Michigan and Wisconsin. AAPOR blamed late deciders and samples not weighted for education. [32]

lesson Who answers a poll is not who votes. Weighting by demographics missed what moved people.

Pew Research Center reported that its typical phone survey had a 6% response rate in 2018, down from 36% in 1997. It pointed to robocalls to cellphones as one cause. [33]

AAPOR's task force found the national polls' error was the largest in 40 years, and nearly every contest overstated the Democratic margin. It could not name a single cause. [34][35]

lesson The people who answer polls differ from those who don't in ways weighting cannot see.

Arthur C. Nielsen founded the company that would put a meter in American homes. [36]

Archibald Crossley's Cooperative Analysis of Broadcasting called homes and asked what they had heard the day before. Advertisers bought airtime on what listeners remembered. [37]

C. E. Hooper's firm phoned homes while a program was on the air and asked what was playing. Asking in the moment beat asking about yesterday, and Hooperatings overtook the Crossleys. [38][37]

Nielsen acquired the Audimeter, a device that clipped to a radio and logged what it was tuned to. Recording replaced recall. [36][39]

Nielsen launched a national radio rating built on meters in homes, not on what listeners remembered. [36]

Jim Seiler founded the American Research Bureau, later Arbitron. It measured radio with seven-day paper listening diaries. [40]

Nielsen began measuring U.S. television audiences with a meter on the set. Ratings set the price of TV advertising for decades. [36]

The People Meter recorded which person in the house was watching, along with the channel. [36]

Arbitron's pager-size meter detected the radio people heard wherever they went. Nielsen bought Arbitron in 2013. [40][36]

Attorney General Andrew Cuomo sued Arbitron, saying its Portable People Meter undercounted minority listeners and threatened minority-owned stations. Arbitron settled in January 2009 and agreed to response targets for every group. [41][42]

lesson A better meter still fails if the panel carrying it leaves people out.

The Media Rating Council suspended accreditation of Nielsen's national TV ratings after networks said it undercounted viewers during the pandemic, when Nielsen struggled to keep its panel homes. The MRC restored it in April 2023. [43][44]

lesson A panel that cannot reach its homes cannot count them.

A consortium led by Evergreen Coast Capital and Brookfield closed its buyout of Nielsen. [45]

Paramount let its Nielsen contract lapse at the end of September 2024 and made VideoAmp, which measures from set-top boxes and smart TVs, its primary measurement partner. [46][47]

The U.S. grocery industry chose George Laurer's IBM design as the Universal Product Code. [48][49]

At 8:01 a.m. a Marsh supermarket in Troy, Ohio scanned a 10-pack of Wrigley's Juicy Fruit. Every purchase could now leave a record. [49][50]

Information Resources launched BehaviorScan in Marion, Indiana and Pittsfield, Massachusetts. It tied the TV ads a household saw to what it bought. [51]

Catalina Marketing, founded in 1983, linked a printer to the register that read what a shopper bought and printed a coupon to match, often for a rival brand. [52]

InfoScan became the first nationwide supermarket tracking service built on barcode scans. By 1993 it drove over half of IRI's revenue. [51][53]

Nielsen's Homescan panel gave households handheld scanners to log the barcode of every purchase they brought home. It started with 15,000 homes. [54][55]

Tesco and dunnhumby tied purchases to named shoppers. Chairman Lord MacLaurin: "you know more about my customers after three months than I know after 30 years." [56]

The New York Times reported that Target statistician Andrew Pole scored shoppers' odds of pregnancy from about 25 products tied to their Guest ID, and estimated due dates for coupons. [57][58]

lesson A model that infers what people did not share costs more trust than it earns.

Kroger took over dunnhumby's U.S. assets and more than 500 staff to form 84.51°, ending its 12-year joint venture with Tesco's data arm. [59][60]

Information Resources and The NPD Group completed their merger and took the name Circana in March 2023. The combined firm tracks products across 500,000+ stores in 20 countries. [61][62]

Lou Montulli built the cookie at Netscape so a website could remember a visitor's shopping cart. Mosaic Netscape 0.9 shipped with it, and sites began to recognize returning visitors. [63]

Gian Fulgoni and Magid Abraham, both from IRI, founded comScore to measure online behavior from a large opt-in panel of computers. [64][65]

Ryan Finley founded SurveyMonkey. Anyone could now write and send a survey. [66]

Stephan Shakespeare and Nadhim Zahawi founded YouGov in the UK to run polls on a panel of people who answer online. [67]

Qualtrics began as an online survey platform in Provo, Utah. [68]

Greg Linden, Brent Smith and Jeremy York described Amazon's item-to-item filtering, which matched products bought together instead of matching customers. IEEE Internet Computing later named it the paper that best stood the test of time. [69][70]

Fred Reichheld's Harvard Business Review article asked customers how likely they were to recommend a company. A MeasuringU replication found his evidence tracked past growth, not future growth. [71][72]

Google bought Urchin and released Google Analytics free. Demand forced Google to pause sign-ups within a week. Every site could now record its visitors. [73][74]

Amazon opened Mechanical Turk, a market for small paid tasks. Social scientists adopted it as a cheap pool of survey respondents, most of them in the U.S. [75][76]

Adobe agreed to buy the web analytics company Omniture for $21.50 a share in cash and made it a business unit inside Adobe. [77][78]

SVMK, SurveyMonkey's parent, priced its IPO at $12 a share, above its $9 to $11 range, and began trading on Nasdaq. [79]

SAP bought Qualtrics days before its planned IPO, betting on "experience management." [80]

SAP took Qualtrics public at $30 a share and kept majority control, two years after buying it. [81][82]

SAP sold Qualtrics to Silver Lake and CPP Investments. The deal closed that June. [83][84]

Google dropped its plan, announced in 2020, to remove third-party cookies from Chrome and said it would offer users a choice instead. [85][86]

AOL posted searches from about 650,000 users with names swapped for numbers. The New York Times traced user 4417749 to Thelma Arnold of Lilburn, Georgia, and AOL's CTO resigned. [87][88]

lesson Removing names does not make behavior anonymous.

Netflix released 100 million real ratings and offered $1M to anyone who beat its recommender by 10%. A team won in 2009 with 10.06%. [89]

Dan Siroker's team tested 24 button and image combinations. The winner lifted sign-ups from 8.26% to 11.6%. Siroker estimates it raised $60M more. [90]

Google and the CDC launched Google Flu Trends, which estimated flu activity from search queries. Google said it could spot outbreaks up to 2 weeks before existing surveillance. [91][92]

Dan Siroker and Pete Koomen founded Optimizely to bring campaign-style experiments to any website. [93]

Arvind Narayanan and Vitaly Shmatikov had matched "anonymous" Netflix Prize ratings to public IMDb reviews to identify subscribers. After a lawsuit and an FTC inquiry, Netflix shelved the sequel. [94][95]

lesson A few ratings can name a person, so open behavioral data needs consent.

Twitter gave the Library of Congress its archive of every public tweet since 2006, for noncommercial research after a 6-month delay. [96][97]

An engineer's shelved idea, run as an A/B test, raised Bing's revenue 12%, more than $100M a year in the U.S. [98]

Facebook changed News Feeds for a week to see whether emotions spread. When the paper came out in 2014, the backlash drew an editorial expression of concern. [99]

lesson Measuring people without their consent costs the trust the data depends on.

Google Flu Trends estimated more than twice the share of flu doctor visits the CDC reported. David Lazer and colleagues later found it ran high in 100 of 108 weeks. [100][101]

lesson Fitting 50M search terms to 1,152 data points found noise, not flu.

Michal Kosinski, David Stillwell and Thore Graepel showed that Likes alone predict traits people never shared, including party. [102]

A model built on 300 Likes judged personality better than a person's spouse. Ten Likes beat a coworker. What people did revealed more than what they said. [103]

Harvard Business Review reported that Microsoft, Amazon, Booking.com, Facebook and Google each run more than 10,000 controlled experiments a year. [98]

Facebook disclosed that data on up to 87 million people may have gone to Cambridge Analytica, which built voter profiles from it. [104]

lesson Files on individuals, taken without consent, ended the era of open social data.

The EU's General Data Protection Regulation took effect and gave people a right to clear, affirmative consent over their data. Fines can reach 4% of global turnover. [105][106]

Apple's App Tracking Transparency made iPhone apps ask permission before tracking people across other apps and sites. Meta said the change would cost it about $10B in 2022. [107][108]

Google says it ran more than 700,000 experiments on Search in 2023, which led to 4,781 changes. [109]

Twitter replaced its free academic access, which allowed 10M tweets a month, with paid tiers. The $100 basic tier allowed 10,000 reads a month. [110][111]

Reddit set API prices of $12,000 per 50M requests. More than 6,000 communities went dark in protest, and the Apollo app shut down. [112][113]

Meta closed CrowdTangle, the tool researchers and journalists used to track public posts, months before the U.S. election. It pointed them to its Content Library instead. [114][115]

John Horton replayed classic behavioral economics experiments on GPT-3 and got results like the originals. He called the simulated subject homo silicus. [116][117]

Lisa Argyle and colleagues gave GPT-3 the backstories of real survey respondents. Its answers tracked real subgroups, which they called "algorithmic fidelity." [118]

James Brand, Ayelet Israeli and Donald Ngwe asked GPT to answer as a shopper. Its answers produced downward-sloping demand, and simulated higher earners tolerated higher prices. [119][120]

Joon Sung Park and colleagues built 25 agents in a simulated town that planned, remembered and reflected, and behaved like believable people. [121]

Gati Aher, Rosa Arriaga and Adam Tauman Kalai simulated whole samples of study participants with language models. They reproduced the Ultimatum Game and the Milgram study, and found a hyper-accuracy distortion on wisdom-of-crowds questions. [122][123]

Vanderbilt researchers had ChatGPT answer as personas and compared it with the American National Election Study. Averages matched, but answers varied less than people's, shifted with prompt wording and changed over 3 months. [124]

lesson A model that matches the average can still get the people wrong.

Aaru built 1,000 or more AI voters in each of seven swing states for Semafor and gave Kamala Harris Michigan, Nevada, Pennsylvania and Wisconsin. Donald Trump won all seven. [125][126][127]

lesson Agents built from demographics and prompts, not recorded behavior, inherit the blind spots of polls.

Park and colleagues interviewed 1,052 Americans for two hours each and built an agent for each one. The agents matched each person's survey answers 85% as well as the people matched themselves two weeks later. [128][129]

Qualtrics launched Edge Audiences in early access: simulated respondents built on its survey data, sold beside human panels. [130]

Olivier Toubia and colleagues asked 2,058 Americans 500 questions over four waves and released the data for building and testing simulated respondents. A repeat wave gives a test-retest baseline. [131][132]

The Y Combinator startup, founded by behavioral scientists James He and Patrick Sharpe, raised $5.35M in total to simulate how social groups react to messages. [133][134]

Redpoint led Aaru's Series A. Some shares sold at a $1B valuation and others lower. [135]

The UK startup raised $10M to build synthetic audiences that model how people think. [136][137]

The generative agents authors founded Simile to simulate how people respond to products and decisions. Index led $100M in February; Greenoaks led $200M at $2B in July. [138][139]