Tuesday Newsday: Data News Sites

Data journalism, news apps, open government data. In this age of Anonymous, Wikileaks, and proactive sources publishing freely available data, it is becoming critical that we have journalists and developers working together to make sense of it all and understand how it affects us.

Today there are a growing number of news outlets doing a fantastic job of both publishing data and creating interactions and visualizations to make sense of it. I wanted to showcase a handful of those sites so that people can understand why this type of journalism is important and how it works. And maybe even how to get involved if it’s of interest to you.



There is no shortage of people providing data sets today, the only thing to learn is how to narrow it down to the most useful one for you.

Fingal Open Data is doing a nice job in Ireland encouraging councils to open up their data. Their site http://data.fingal.ie/ provides data in CSV, XML and KML formats, and they’re currently running a competition for people to build useful apps given the available data.

Additionally in Ireland, sites like Gavin Sheridan’s The Story and John Handelaar’s Kildare Street are excellent resources for presenting data but also learning more about the types of data available in Ireland

I mentioned The Guardian’s Data Blog in a recent post about news developer blogs as a great example of community building. But clearly it is also one of the best sources in the UK for a wide range of freely available data sets. For each set of data they post, they offer a download of the data and recommend that if you do anything with it, you post it to their Flickr group. They often have additional visualizations and comparisons as well.

Example of data provided by The Guardian Data Blog


There are loads of additional resources for finding data, the trick is knowing how to search for the information you need. Sites like ScraperWiki (https://scraperwiki.com/), DataSift (http://datasift.com/), The World Bank (http://data.worldbank.org/),   BuzzData (http://buzzdata.com/) and more sites are growing and becoming available to the public for research and data mining.



It’s no coincidence that a lot of my news app examples have shown up in prior blog posts – there are some organizations who are very seriously ahead of the pack in terms of understanding how news and technology work together. On one hand I am delighted to have bright minds like these folks paving the way! On the other hand, it makes me a little depressed when I realize that Ireland is nowhere near this level of thinking.

NPR’s StateImpact (http://stateimpact.npr.org/) is a collaboration among NPR and local public radio stations in eight pilot states to examine public policy issues in depth. They provide explanatory, data-driven stories focused on how people’s lives are affected by government decisions.

NPR's StateImpact

The New York Times definitely has a well-respected team of interactive news developers and infographic designers. However it’s not always very easy to find them on their site. The Multimedia page is the best resource I’ve found for taking a look at cool interactive apps and images they create, but Small labs Inc has also put together a nice collection here: http://www.smallmeans.com/new-york-times-infographics/

NY Times Multimedia / Photos page

The Chicago Tribune has some of the busiest & best news app builders in the United States today and they are constantly kicking out new, interesting information in nice visual formats. Check out their site which has apps in many areas including community, schools, business, politics and more.

Chicago Tribune Maps & Apps

ProPublica creates not only fantastic interactive news applications like Dollars for Docs (which I referenced in my Quantified Self talk on skin problems), they also create excellent reusable tools like DocDiver that allow readers to work with the ProPublica reporters.

ProPublica Tools & Data



To learn more about data journalism, here are a few useful sites to check out and blogs to follow:

The Data Journalism Handbook: a work-in-progress coordinated by the European Journalism Centre & the Open Knowledge Foundation launched at the Mozilla Festival in London on November 5th 2011.

Data Journalism Blog: http://www.datajournalismblog.com/

Hacks/Hackers: http://hackshackers.com A grassroots journalism organization on a mission to create a network of journalists and technologists who rethink the future of news and information.

ProPublica’s Dan Nguyen’s Scraping for Journalism: A Guide for Collecting Data http://www.propublica.org/nerds/item/doc-dollars-guides-collecting-the-data – a great introduction to how to grab data that is in maybe less-than-optimal formats.

Also from Dan Nguyen, The Bastards Book of Ruby http://ruby.bastardsbook.com/ – an introduction to programming and its practical uses for journalists, researchers, scientists, analysts and anyone else who needs to make sense of data.

At the Society of News Design conference earlier this year, I went to some excellent talks by newsroom app developers who recommended that if you are thinking of going into this area, you need to know either Ruby on Rails or Django/Python, as those seem to be the dominant requested skills for news apps. I also liked this article recently on Poynter about using Backbone to create data news apps: http://www.poynter.org/how-tos/digital-strategies/147605/how-journalists-can-use-backbone-to-create-data-driven-projects/. You don’t need to know everything, just pick one and start creating projects.


And lastly, from the perfect timing department, here’s Matt Stiles, who is a data journalist from NPR, talking very recently about best practices in building news apps:

Data Best Practices from StateImpact on Vimeo.

Just in time for Last-Minute Gift Givers: Subscriptions to Idea!

Since Idea went live almost two weeks ago, the biggest complaint we heard was, “Why can’t I give you money?” We’ve gone back and forth on what’s the right thing to do for the community and how best to make sure what we’re doing is reasonably priced but sustainable too.


Subscribe to Idea Magazine


So today we’re answering your calls & opening up for subscriptions!  Subscriptions will begin with our next issue in February. And we have two reasons why you should get your subscription sooner rather than later:

1) Prices for annual subscriptions will go up shortly before the next issue (February 15th) launches, so make sure you get your subscription now to lock in a fantastic deal.

2) We have a special Christmas gift this week: one lucky subscriber will win a gorgeous DODOcase from our friends at http://clickcase.ie! If you haven’t taken a look at the DODOcase, it is stunning, constructed from black Moroccan cloth and bamboo, and was even chosen by Barack Obama for the presidential iPad. You’ll be the envy of everyone for your superb taste and quality.

DODOcase - iPad 2

Anyone who purchases a subscription for themselves or a friend between now and 12:00am December 25th (for the really last-minute gift givers) is entered to win the DODOcase, and we’ll post it on our site and email the winner on Christmas day.


I was surprised by the number of people who have honestly asked me, “Can you just put up a donation thing so we can make sure this keeps going?” It seems so unbelievable that today, in the middle of a tough recession, while governments and corporations are terrified of piracy, that there are people who are actually looking for ways to not get things for free. But along those lines we’ve also seen a lot of great websites and services go out of business because they never had a business model. This excellent blog post from Maciej at Pinboard describes exactly that phenomenon: http://blog.pinboard.in/2011/12/don_t_be_a_free_user/.

So while we have a lot of creative plans in store for Idea, we’re also focusing on making sure it’s a sustainable effort and sticks around for a while. We’re committed to making this work and appreciate the kind words and support from everyone who liked what they saw and read so far. I won’t promise the business model will be the same this time next year, but I do promise that the magazine will still be here and hopefully by then fully supporting itself.

Thanks again for your enthusiasm!

And hey, the next issue is already kicking off! For those of you thinking about a change of pace to a startup or other small, fast-paced project and trying to run the numbers to see if it might work, we think we can help.

What’s in an Idea?

What all goes into an idea?

Well, our Idea consisted of:

  • 3 core team members
  • Over 30 contributors
  • 21 logos
  • 24 interviews
  • 20MB of production files
  • 6 digital formats
  • Over 150 images, icons, logos, illustrations and photos
  • Upwards of 15 frameworks evaluated
  • Hundreds of emails, Twitter DMs, text messages and phone call
  • Less than 20 in-person meetings
  • Some Macs, a couple of PCs, some iPhones, a few iPads and a Windows Phone 7
  • A handful of messy, convoluted platforms & workflows

And I’m not going to try to think about the collective hours. There are only so many hours in a day, and we all have full-time jobs and families. It was definitely a labour of love.

Cover Issue1

Cover Issue 1

Launching Idea Magazine was tough. I’ve been dreaming of creating Idea for years. On one hand, I just wanted to kick it out the door. On the other hand, I would have killed for another couple of weeks to make more fixes, edits, changes, revisions. But at some point you have to ship.

In the last year, I’ve learned a lot about digital publishing and creating an environment that both accommodates and enhances reading experiences. I’ve learned there are a lot of religious wars about reading formats, but that there isn’t “The One Best Way.” Mostly, though, I’ve learned how difficult it is to build, reformat and distribute these publications. I’ve watched as people struggle with CMS after CMS, built to publish but not built FOR the people who are publishing.

This can be better.

Stewart Curry, the incredibly talented designer that I somehow convinced to work on Idea with me, saw these same problems. As we worked together on building the platform for Idea, we realized we were going to need a system and that the method we were using was unsustainable. We decided to take some time to build a platform not just for Idea, but for other publications who are struggling with similar issues of usability, functionality and sustainability. We’re calling it WOOPIE, for Write Only Once, Publish It Everywhere. I’m going to take a much-needed break between now and the new year, and when I come back I’ll be focusing on that.

Making Idea come to life after thinking about it and believing in it for the last few years is honestly a dream come true. Building a platform to enable other people create something like this for their own communities is an amazing opportunity, and I can’t wait to get started.

[Oh, and yes, Tuesday Newsday will resume next week.]