How Google Lost Hundreds of Thousands of Dollars in an Hour and Half

Google headquarters in Mountain View, California. Photo: Trey Ratcliff/Flickr

For a brief window this morning, something strange happened to the internet.

At approximately 9:15am ET, display ads disappeared across mega-trafficked media websites such as BuzzFeed, Time, Forbes, Gawker and Vox, replaced instead by big, blank spaces. Some were understandably pleased at the mini-vacation they got from companies making money from their eyeballs, but those running the websites weren’t so happy—and neither was Google. It was the one who was supposed to supply those ads.

The end result? A ton of lost money for all those websites—and for Google. How much? That’s hard to say. But Google alone likely lost several hundred thousands dollars in about an hour and a half—and the other sites lost a comparable amount, if not more.

In the wake of the ads outage, Google scrambled to figure out what caused it, and by 10:45am, most websites were back to normal. Google posted a quick memo—which reads like a sigh of relief—saying things were back up and running.

The glitch was caused by the ad tool called DoubleClick for Publishers, or DFP—a Google advertising service used by many websites to manage ad operations, and whose content is hosted on Google machines rather than the servers that hold the publisher’s own content.

Google says it doesn’t publicly reveal its revenue stream for DoubleClick for Publishers, instead lumping it in under “other” revenue on its earnings reports. That includes anything that doesn’t come from ad revenue on its own websites or from websites that use its AdSense ad network. Based on the company’s 2013 annual report, it made $568,000 per hour in “other” revenue across the internet. Though this includes more than DoubleClick dollars, we can roughly estimate that the company lost something approaching $1 million from this morning’s ad server hiccup.

The last time Google’s DFP system failed was more than a year ago, in March 2013, when the service was down for several hours. This sort of outage may please you in some way, but don’t get used to it. The internet, after all, runs on ads.