Some thoughts after a couple months with tabletop and handlebars
Published 2013-02-02
tl;dr - Learning how I can use tabletop.js & handlebars.js together has opened up some really cool custom mini-CMS possibilities and ways to manage data projects.
But thanks to the experiments of others, I have learned that flat json files can be better to use in production, especially for small one-off projects.
While I'd like to eventually write a script to take the tabletop object and write it to a flat file, I made some adjustments to some python code so I can take a csv, create a flat json file from it and drop that file into a handlebars function with minimal effort.
While it works for me, I'm curious to know if it works for others…
I'm happy to say that I still have access to some gray days since moving to Southern California, and on a recent one I set about streamlining some project structures, code snippets & templates for two code libraries that I've been using more and more since coming to Southern California Public Radio: tabletop.js and handlebars.js.
I'm only now getting back to the write up and posting code.
Overview
tabletop.js allows you to use a Google spreadsheet as a data backend. By simply publishing the spreadsheet and adding its key to a tabletop javascript function you can retrieve an an array of objects to be used on a webpage or in a web application. The library was released back in February -- right around the time of NICAR -- thanks to work by WNYC and Balance Media.
handlebars.js is a templating library -- much like mustache.js -- that "provides the power necessary to let you build semantic templates" based on data that is formatted as -- get this -- javascript objects. Using an example from the handlebar.js website, the library allows you to do things like this...
<div class="entry">
<h1>{{title}}</h1>
<div class="body">
{{body}}
</div>
</div>… where {{title}} and {{body}} represents information stored in an object like this:
var context = {title: "My New Post", body: "This is my first post!"}Practical Applications
Learning how these libraries can work with each other has been a lot of fun, especially in the context of a newly-created position at a new-to-me news organization where I have had a role to play in building a foundation for the presentation of data projects.
For instance, using handlebars.js ability to load precompiled handlebars templates from file we've been able to construct a basic project wrapper that can live outside CMS, minimizes the amount of code that needs to be changed at any given time and allows us to manage a single header and one footer file. DRY FTW.
I've also been able to use the combination of these libraries to come to having a custom mini-CMS that can run some of these projects, which can then be updated by anyone that can use a spreadsheet.
For instance, one of my first collaborations was with our business and economy reporter Matt DeBord on this project for Explaining the Monthly Jobs Report.
This project combines tabletop.js with handlebars.js -- and some highcharts.js for good measure -- to show the differences between "Initial," "Revised," and Final monthly jobs numbers released by the U.S. Bureau of Labor Statistics.
While my "design" judgement and sensibilities continue to evolve, for my money, the magic is in the fact that Matt can update this presentation for our audience by opening a Google spreadsheet and adding the new numbers. Lightweight, custom mini-CMS!
Tasneem Raja, the Interactive Editor at Mother Jones, has written a solid explainer on how the magazine has leveraged tabletop.js and a Google spreadhseet for collaborative projects such as their You Might Be the 47 Percent If… scoop.
Some of the projects I've worked on over the past couple months that use tabletop.js and handlebars.js include a presentation on the purchasing of Capital Appreciation Bonds by local school districts, a "By The Numbers" look at the impact of the so-called fiscal cliff on Emergency Unemployment Compensation benefits and a comparison of California and federal gun laws
But a post on NewsBeast Labs I found while writing this pointing to a post from Jeremy Singer-Vine -- appropriately titled "Why I love Tabletop.js but don't use it in production" -- and some words from Mark Boas on an Open News call a couple weeks ago reminded me of potential obstacles and issues with relying too much on tabletop.js for projects.
And a tweet this week a few weeks back from Aaron Williams, and one today from Matt Waite, reminded me I have to finish this post you are reading.
All of the issues that Jeremy listed are real, and Aaron's experience is real. Thinking about the projects I listed above, none of them really require the kind of dynamic data display that tabletop.js can pull off. It's a nice feature to have available on the monthly jobs report, but it isn't crucial, and certainly isn't required on the others.
A flat file very much would work just fine.
And so I went about finding a way to create a flat json file from a Google spreadsheet that I could use in a handlebars.js template.
My Workaround
I already had Andy Boyle's python script from a year ago that creates json from a csv. And I noticed in the comments section on Andy's post that Christopher Groskopf's csvkit has a csvjson utility that can also convert a csv to a json file.
After flirting with the idea of adding a custom argument to the csvjson utility to allow me to specify a key, I adjusted Andy's script to create a file that looks like this…
{"objects": [{"Source": "National Employment Law Project", "DataOrder": "1", "SourceLink": "http://www.nelp.org/", "Data": "12.29.2012", "Title": "The last day anyone will receive benefits from the Emergency Unemployment Compensation program unless Congress acts to renew it."}, {"Source": "Congressional Budget Office", "DataOrder": "2", "SourceLink": "", "Data": "$30,000,000,000", "Title": "Estimated cost to renew the Emergency Unemployment Compensation program through the end of 2013."}, {"Source": "National Employment Law Project", "DataOrder": "3", "SourceLink": "http://www.nelp.org/", "Data": "400,000", "Title": "Estimated number of Californians receiving benefits from the Emergency Unemployment Compensation program, which is set to expire Jan. 2."}, {"Source": "National Employment Law Project", "DataOrder": "4", "SourceLink": "http://www.nelp.org/", "Data": "2,100,000", "Title": "Estimated number of Americans receiving benefits under the Emergency Unemployment Compensation program that would lose their unemployment benefits come January if Congress doesn\u2019t act."}]… which is something that allows me to plug my new file name into a variable in this script and fly.
Snags
Now, admittedly it works because most of my handlebars work thus far has used simple data structures -- largely an object that contains an array of objects -- but my workflow might fall apart quickly should I need to access multiple data streams, etc. (UPDATE: This has already happened!)
One thing I have noticed through this experiment is tabletop.js strips uppercase letters and underscores from object keys, while the csv to json version doesn't. So that's a thing as it's the difference between a handlebars.js template that uses…
//what tabletop brings
{{projecttitle}}and
// created from a csv to json method
{{Project_Title}}Another thing I've noticed is the python script that converts the csv to json renders everything as a string, which has caused some issues when using something like highcharts, which requires is data objects to be integers. So I continue to explore a way to fix this.