Last February, Forbes’ Kashmir Hill got our attention (over 1.9 million views by now) by pulling a golden big data nugget—“How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did”—from a long New York Times article by Charles Duhigg. While the article was actually an introduction to Duhigg’s book on the power of habit, the Times editorial team saw the traffic-generating potential of Duhigg’s encounter with Target’s super-statistician and gave it the title “How Companies Learn Your Secrets.”
We were treated last month to another excellent predictive analytics story with some privacy implications, but this time it was about how politicians learn our secrets. Writing in Technology Review, Sasha Issenberg summed up his ground-breaking work on the use of data by political campaigns in “A More Perfect Union: How President Obama’s Campaign Used Big Data to Rally Individual Voters.”
In meticulous detail, Issenberg shows how politicians are way ahead of companies in not just learning our secrets and predicting our actions, but in using data to change behavior. He says: “The campaign didn’t just know who you were; it knew exactly how it could turn you into the type of person it wanted you to be.” The Obama campaign had data on “as many as one thousand variables” for every voter, “drawn from voter registration records, consumer data warehouses, and past campaign contacts.” The clever mining of our “digital shadow,” the data on us that has been accumulating in private and public databases over the last two decades, is what now could make a political campaign successful.
If you are not focused on optimal data mining and using all the latest data science tools, you are missing an important success factor. The Romney “data science team was less than one-tenth the size of Obama’s analytics department.” Unlike the Obama campaign, it relied mostly on outsiders for data analysis, specifically TargetPoint Consulting, the firm that coined the term “microtargeting” to describe the process, used first in the Bush 2004 campaign and “modeled after the corporate world’s approach to customer relationship management,” of linking information from consumer data warehouses to voting registration records in order to develop individual-level predictive models. But, according to Issenberg, “Republicans have done little to institutionalize that advantage in the years since. By 2006, Democrats had not only matched Republicans in adopting commercial marketing techniques; they had moved ahead by integrating methods developed in the social sciences.”
Instead of deepening their understanding of individual behavior, Romney’s data scientists “fixated on trying to unlock one big, persistent mystery…’How can we get a sense of whether this advertising is working?’”
That was a question that was taken also by the Obama campaign and they found the answer, starting with the right hiring decision, of Carol Davidsen, “whose previous work had left her intimately familiar with the rich data sets held in [TV] set-top boxes.” She negotiated with research firms that aggregate this information to have them “repackage their data in a form that would permit the campaign to access individual histories without violating the cable provider’s privacy standards.” The matching of the Obama campaign’s list of persuadable voters and their addresses with cable providers’ billing files, and the resulting viewing data by household (without personally identifiable information), allowed the Obama campaign to create its “own television ratings system” and optimize its media buys.
In her Forbes piece, Kashmir Hill called “a quote for our times” what Target’s statistician Andrew Pole told Duhigg: “We are very conservative about compliance with all privacy laws. But even if you’re following the law, you can do things where people get queasy.” Does the use of personal data by political campaigns also make us queasy or does it depend on which political campaign is doing the sleuthing?
If you think that political campaigns are different from companies because their data science efforts, experiences, and innovations are short-lived, Issenberg explains that here too the Obama campaign is an innovator. The post-mortem report produced by the campaign’s staff was “inspired by the idea that the innovations of Obama 2012 should be translated not only to the campaign of the next Democratic candidate for president but also to governance. Obama had succeeded in convincing some citizens that a modest adjustment to their behavior would affect, however marginally, the result of an election. Could he make them feel the same way about Congress?”
In a companion piece to Issenberg’s article, “How Technology Has Restored the Soul of Politics,” no less an authority than Joe Trippi (who, as campaign manager for Howard Dean’s 2004 presidential bid, brought the Web to political campaigns), provides historical context. Stating that “it requires no hyperbole to say that Obama 2012 changed everything,” Trippi explains what has changed:
“By late 2002, political professionals from both sides of the political spectrum believed that it might be possible to take on the top-down, money-driven, television-ad-centric approach to politics and instead use technology to build a bottom-up, people-centered politics.
By 2007, Americans had begun participating in politics in numbers no one had imagined possible. TV ads would have almost nothing to do with Barack Obama’s election, although more would be spent on them than ever before. Hillary Clinton lost the Democratic nomination for the simple reason that she ran an old-fashioned campaign. But Obama’s victory in 2008 was remarkable not only because he raised a half-billion dollars online and had over 13 million people sign on to his campaign. His win in 2008 was most remarkable because it allowed his campaign staff to do something truly novel in 2012: build a national campaign armed with big data…
Mitt Romney’s campaign and its allies made the same mistake Hillary Clinton made in 2008: they ran a top-down campaign preoccupied with buying television ads and influencing the media.”
Trippi offers a number of reasons why “the Republican Party may struggle to catch up.” It is lagging in building a national network and its “top-down message discipline” impedes the flourishing of a grassroots organization. More convincing (to me) and clearly evident in Issenberg’s article is this—“the GOP is unloved by a key demographic group it will need: the technically educated, creative young people who like to build software and do data analysis.”
Trippi concludes: “There will be plenty of actors in both politics and business who will use the innovations of the Obama 2012 campaign as tools to manipulate people. But for me, right now, it feels as if technology has empowered people and given politics back its soul.”
Would Joe Trippi also feel the soul of the new machine rather than its manipulation if Romney had won by using big data “to rally individual voters”?
Wow, great post. Is this the emergence of the Big Data technocrats?