(Failed) Lockdown Hack #4: Prontissimo

(About Lockdown Hacks)

Screenshot 2020-04-23 at 15.05.25


Amazon Product Advertising API, Ruby on Rails, Vue.js


Not burying the lede here, I failed to achieve what I set out to do with this one. (The screenshot above only reflects how far I got in terms of the frontend – yes, I spent 30 minutes designing a logo, what of it?)

I still think the idea was pretty good: Hack Amazon search to sort results by fastest delivery. Amazon does not let you sort this way, and I think I understand why. It wouldn’t normally be a useful feature because you can just filter by Prime eligibility which would indicate a one- or two-day delivery. Furthermore, delivery estimates for non-Prime items where Amazon doesn’t tightly control the supply chain are probably unreliable. But during this pandemic I noticed that Prime delivery has been all over the place here in the UK, sometimes weeks out. When online shopping is the most convenient or the only way to procure certain stuff, you do sometimes care intensely to pick the item that will be delivered the fastest.

I skimmed the documentation of Amazon’s Product Advertising API and decided I would be able to cobble together at least a proof of concept. Then over the next day and a half I faced enough hurdles that I eventually gave up. There were two major issues: Getting access to Amazon’s Product Advertising API is an involved process, and the API does not return the kind of data (delivery estimates) I needed, as far as I can tell.

I had skipped over the part in the documentation where it explains that to use the API you need to be an Amazon Associate. Not only that, you need to have made at least 3 qualified sales through the associate program and need “final acceptance” which involves human review. I did sign up as an associate in the US and in the UK and even managed to broker 3 sales in the US (with a little help from my friends). But the review process is opaque enough that I suspect it will take days to get final acceptance. I did contact the US and UK associate programs separately about fast-tracking but the replies did not give me confidence the rules are bendable in any way.

Because I hadn’t been able to try out real API requests and responses it also took me too long to realize that the API doesn’t provide the delivery estimates I needed (I think). This is the major takeaway for me from this failed hack: Good documentation is clearly a prerequisite for a successful API but it’s just as important to make it trivial for users to start exercising it with real data, even if it’s under restricted trial conditions like for example Twilio does it. It’s hard for a developer to project what an API will enable them to do by just looking at documentation.

I understand Amazon might have valid strategic reasons for restricting API access. The API has some potential as an abuse vector where somebody could recreate, for example, the Amazon homepage in a way that undermines the Amazon brand. But then again, don’t the tight rate limits – 1 request per second to start with – protect against most abuse? My product intuition is that the upside of removing barriers to entry trumps potential downsides here.

Lockdown Hack #3: Whence

(About Lockdown Hacks)

Screenshot 2020-04-20 at 20.12.29




A game of guessing random locations on Earth based only on satellite views


Google Maps JS API, Vue.js


Perhaps obviously, the popular game GeoGuessr inspired this hack, where you move around Street View to guess your location. It’s fun enough that they managed to spin it into a whole franchise, complete with a paywall and professional leagues. I remember when it was a simple free game, years ago.

Having worked a lot with satellite imagery over the last years, I had carried around this idea for a long time – applying a similar game mechanic to just satellite views rather than Street View. Scoring is based on how close your guess is and how many times you zoomed out.

During my brief research into similar games I also found the Map Zoom Quiz. In my view they made a fatal design decision to have players type in their answer. That makes it too hard, plus reaching for the keyboard in a game like this just feels wrong. Whence is still hard but you do get points for making an educated guess. I also observed I got better at it over time as I indeed learned to differentiate the world’s terrains more precisely and to latch onto specific clues (agricultural landscape looks very different in Asia vs North America!).

I didn’t face any real technical challenge during this – roughly two days of programming and game design, then one day of UI design and polish. The Google Maps API is mature and rich yet simple, and it was really comfortable to use with an appropriate Vue wrapper. I think I spent a disproportionate time on UI design and game design, both disciplines out of my comfort zone of course. I’m still not satisfied with the UI. It’s particularly awkward on small screens but I got tired of fiddling with it.

How does this hack tie into the lockdown condition? Just think of it as virtual travel, since real travel hasn’t been in the cards for most of the world recently. But really, it was just an idea that I had been stuck in my head for quite some time and I needed to scratch the itch.

Lockdown Hack #2: Towards proving that social distancing works

(About Lockdown Hacks)

How effective is social distancing at combating the spread of COVID-19? I explained in my previous post how I was inspired to look at correlating mobility data with COVID-19 growth to begin to answer that question. How much people move around is a decent proxy for how strictly social distancing rules are being observed and we can exploit differences in mobility between regions to evaluate the impact of social distancing.

This is still me riffing on a paper by actual epidemiologists I came across (Estimating the effect of physical distancing on the COVID-19 pandemic using an urban mobility indexJean-Paul R. SoucyShelby L. SturrockIsha BerryNick DanemanDerek R. MacFaddenKevin A. Brown). I have to stress once more that I’m not a scientist, much less an epidemiologist. I have experience building tools for scientists though, so I decided to contribute dashboards that maybe would allow an expert to explore the connection between mobility and COVID-19 spread.

There are lots of tools out there for creating interactive dashboards and even just in the interest of rapid progress I wanted to use them rather than build something custom from scratch. I settled on three fundamentally different tools: Google Spreadsheets as a traditional spreadsheet application, Tableau as the most prominent business intelligence application and Jupyter Notebooks with widgets as an interactive programming environment widely used by scientists.

Yes, I built essentially the same thing three times because I suspected I could bang these out quickly. I did not know ahead of time which approach was going to work best or which one would be most practical for others to build on – I had never used Tableau or Jupyter widgets before.

The requirements for each dashboard were:

  • Import the Descartes Labs mobility and New York Times COVID-19 data, preferably in realtime
  • Draw a scatter plot of mobility vs pandemic growth by US county, and make it as interactive as possible
  • Make the result available in a way that allows other people to easily use it and build on it

Google Spreadsheets

View the spreadsheet

Screenshot 2020-04-14 at 16.46.52

Working on this was like a warm bath of nostalgia because I worked on this product at Google from 2010 to 2012. I hadn’t done anything so fancy in a spreadsheet for a while but this came together quickly and intuitively.

However, Google Spreadsheets is a collaborative spreadsheet editing application – it’s not designed for interactive dashboards that multiple people can use at the same time. The shared spreadsheet is view-only; to interact with the “dashboard” you need to create your own copy of the document (and need to be signed into a Google account).

I also found out that the raw data CSV files are too big to import dynamically from a URL, so sadly the raw data in the spreadsheet is static and I will have to update it manually once in a while.


View the dashboard

Screenshot 2020-04-14 at 16.51.05

As a complete Tableau newbie this was rough going. I’m used to thinking about data in relational terms (tables, aggregations, joins) but Tableau has invented its own universe of concepts that only vaguely map to relational concepts. I can’t say I understand their choices from either a product point of view or a technical one. In the beginning I spent way too much time struggling with idiosyncratic concepts, and there exists now half a century of literature about relational-style backends, APIs and UIs. Google Spreadsheets manages to implement a full relational algebra in a browser. What’s your excuse, Tableau?

I struggled the hardest exactly at the edges of Tableau’s concept of relations where I needed to aggregate and join the two raw data sources. I realized Tableau is indeed very much a visualization tool, not a number crunching tool. I think in a real project I am meant to pre-aggregate the data either myself or in an add-on called Tableau Prep. In order to keep the workbook as self-contained as possible I stuck with ingesting the raw data and accepted the limitations, mainly that I had to fix the date ranges for the mean mobility index and mean pandemic growth rate. (I may be missing or misunderstanding features, any suggestions for improvements are very welcome.)

On the upside, it was easy to build the interactive features of the dashboard and it was trivial to publish and share it through Tableau Public.

Jupyter Widgets

Launch the notebook in Binder (may be slow to start up – run all cells to get the interactive plot at the bottom)
Or view the source (not interactive and does not show the plot)

Screenshot 2020-04-14 at 16.55.31

There are good reasons why Jupyter Notebooks with Python have become the de facto standard programming environment in certain science communities. They are well suited to the kind of explorative, iterative programming that is often at the core of teasing insights out of data. To me, as a veteran Python programmer, out of the three approaches this one felt the most extensible and powerful.

But I have mixed feelings about this solution. The power unlocked by notebooks also makes this “dashboard” intimidating or plain unusable to a general audience. I doubt that somebody who is not a programmer or has never seen a running notebook can figure out what to do when they launch it.

I’ve also held for a long time that the notebook model has some inherent flaws. The statefulness of the runtime makes collaborating on or sharing a notebook fundamentally tricky. I appreciate an effort like Binder, which gives you an ad-hoc runtime for notebooks and which works nicely for my purposes here, but collaboration, sharing and version control still look to me like largely unsolved problems in the Jupyter ecosystem. Not to go on too wide a tangent but I have wondered if Jupyter people have huddled on these topics with Smalltalk people who have some decades of experience with very stateful programming environments.


Ultimately, none of the dashboarding tools I explored were able to completely fulfil all requirements, not in the short time available anyway. On the other hand, I did manage to prototype a usable dashboard in each of the tools in the space of less than three days total. I can’t tell which of these might be most useful to a casual or expert user – let’s see what sticks.

Any feedback, comments or suggestions for improvements? Any other tools I should have explored? Let me know.

Lockdown Hack interlude: Correlating mobility stats & pandemic growth

I crunched some data related to the COVID-19 pandemic, following an impulse after I came across two separate news items:

The paper uses the Citymapper Mobility Index which is being published for 41 major cities across the world. The Descartes Labs data contains a very similar mobility index but it’s limited to the US and a lot denser, covering individual US counties. (Both datasets likely have problematic biases since they come from specific populations of mobile phone users but that’s a discussion for another day.)

I’m no epidemiologist so I can’t judge the merit of the paper but it did give me a glimpse into how an epidemiologist might think about putting these kinds of datasets together. So I set out to recreate this figure from the paper using the Descartes Labs data:

Screenshot 2020-04-11 at 16.40.13

I used Python with pandas, and the New York Times as a source for COVID-19 data at the county level. The full code is available in this Jupyter notebook. In the resulting graph the dots now represent counties of New York state with at least 20 cumulative confirmed cases of COVID-19 by March 29 (excluding New York City counties), while the axes correspond directly to the original figure:


At least the trend looks right.

In the notebook I included a lot of intermediate results to make it easy to follow along but stripped down the code is quite compact, an awesome demonstration of the power of pandas and the SciPy ecoystem in general:

# This code released under an MIT license. In notebook form with more explanation:
# https://github.com/nikhaldi/covid-notebooks/blob/master/Mobility%20vs%20COVID-19%20growth%20plot.ipynb
import numpy as np
import pandas as pd
us_counties = pd.read_csv(
dtype={"fips": str},
mobility = pd.read_csv(
dtype={'fips': str},
us_counties_march_week4 = us_counties.set_index('date').loc["2020-03-23":"2020-03-29"]
mobility_march_week2 = mobility.set_index("date").loc["2020-03-09":"2020-03-15"]
us_counties_march_week4_ny = us_counties_march_week4[(us_counties_march_week4["state"] == "New York")]
mobility_march_week2_ny = mobility_march_week2[
(mobility_march_week2["admin_level"] == 2) & (mobility_march_week2["admin1"] == "New York")
def mean_daily_growth(series):
return np.mean(series / series.shift(1) 1)
us_counties_march_week4_ny_mean_growth = us_counties_march_week4_ny[["fips", "cases"]].groupby("fips").aggregate(
max_cases=pd.NamedAgg(column="cases", aggfunc=np.max),
mean_daily_growth=pd.NamedAgg(column="cases", aggfunc=mean_daily_growth)
mobility_march_week2_ny_mean = mobility_march_week2_ny.groupby("fips").mean()
min_cases_threshold = 20
merged = pd.merge(
us_counties_march_week4_ny_mean_growth["max_cases"] >= min_cases_threshold
slope, intercept = np.polyfit(merged.m50_index, np.log(merged.mean_daily_growth), deg=1)
y_log_estimated = slope * merged.m50_index + intercept
ax = merged.plot.scatter(x="m50_index", y="mean_daily_growth", figsize=(10,10))
ax.plot(merged.m50_index, np.exp(y_log_estimated))

Even though I have experience with numpy and other parts of the SciPy ecosystem, I hadn’t used pandas before. Another motivation for this mini-project was to apply pandas to something practical. The bulk of this work was done within 3 hours.

This isn’t serious science of course – I’m not a scientist (but some of my best friends are!). It’s a proof of concept, proving to myself that I understand these datasets and that I roughly understand what was happening in that paper. I’m now thinking about how to build more serious tools around this idea of correlating physical distancing measures and COVID-19 outcomes.

Lockdown Hack #1: Hugster

Screenshot 2020-04-09 at 17.56.43




Get gentle reminders by text to reach out to friends while social distancing.


Ruby on Rails, Vue.js, BootstrapVue, Twilio


It took me about 3 days to arrive at the feature complete version as it is now. At that point it looked like this though:

Screenshot 2020-04-09 at 09.46.57

All the interactivity was in place and the UI elements were roughly lined up where I wanted them but I hadn’t spent any time on styling or visual considerations. I spent almost another full day just on UI/UX polishing. I know I said I wouldn’t mind unpolished outcomes but in this case I really felt the impact of decent design would be significant.

I can’t overstate how much BootstrapVue helped with the UX, even when I hadn’t ever used it before. It accounts for the somewhat cookie cutter look & feel but in earlier times it would have taken me days to recreate its prefab components (tab navigation, form input, modals etc.) with similar fidelity.


The biggest caveat from a product point of view is that you have to enter your contacts manually. It’s an obvious feature to import contacts from Google as users are already authenticated against it. Implementing that and dealing with Google’s restrictions around it would be another multi-day project though.

Arguably this should be a mobile app which can tightly integrate with the contact data on users’ phone and use native push notifications rather than texts. I ruled that out for practical reasons: I estimate it would take me more than a week of scrambling to learn React Native while putting a mobile app together and then I would still be faced with Apple’s app store barriers, delaying an iOS launch even further. Mobile platforms are not great for rapid prototyping, especially if, like me, you haven’t built a mobile app in years.

Easy parts

Twilio handles all the sending of text messages. I’m only using a single API call of theirs – sending a text message – so it should be easy, but their sign up flow and developer experience was slicker than I expected. Total time spent on the Twilio integration was probably less than 1 hour.

Hard parts

I’ve known this for a long time but the point was definitely driven home: Frontend development is a lot more time consuming than backend development for something like this where scalability on the backend is not a concern. I would say the split of my time was about 70% frontend, 30% backend.

This UI is about as simple as it can get but the number of states that the UI needs to account for is still kinda too high to hold in my head at once. This means countless loops of manual testing and a lot of code for what looks trivial on the surface.

I don’t think this is any fault of the tooling (the Vue.js developer experience has been very nice), it’s a systemic issue with UI development. Effort doesn’t scale linearly as you ramp up the interactivity (reactivity?) of the UI, it’s more like quadratic. I think people, including me, often underestimate the effort required to build a modern web frontend because that ramp has been steep lately – what’s technically possible and what users expect has evolved so rapidly.

Lockdown Hacks

The COVID-19 pandemic began to seriously affect life in London just a couple of weeks after I had moved here and started a new job. I had deliberately chosen an employer with a business model rooted in physical space – in real estate – since I had tired a bit of businesses that are, let’s say, mainly virtual.

You can guess what happened next. Virtuality is an advantage in a worldwide pandemic, and real estate business has collapsed for the time being. So less than a month into the new job I was laid off. No hard feelings from my side – I would have let myself go too. My skill set mainly pays off in a growth phase, not in a hunker-down phase.

This marks week four under lockdown and social distancing conditions in London, meaning I have not been leaving the flat other than for grocery shopping or for occasional exercise. Considering the world’s ever gloomier backdrop I’m doing fine. I’m healthy. I’m not lacking anything (other than social contact!). I’m quite good at self-imposing structure on lots of free time. I’m sticking to a lean media diet to keep anxiety at bay. Most importantly, I will probably be employed again soon enough.

I stewed for a couple of days after my abrupt layoff but then a plan presented itself as obvious in my head: I will spend the next few lockdown weeks working on a series of hacks with some connection to the pandemic or the lockdown/quarantine conditions. I will pick ideas off the top of my head and see how far I can drive them with my software generalist expertise.

I don’t know if this will result in anything immediately useful. Some hacks will turn out unpolished or silly or they will fail completely, and that’s ok! The journey is the destination. What matters are the meta-goals:

  1. To flex my mind while stuck indoors without a real job.
  2. To try out some new languages, frameworks and tools.
  3. To plant seeds for potential projects beyond the pandemic.
  4. To bait potential collaborators, co-founders, employers.
  5. To relive a happy part of my youth, before I had all-consuming full-time jobs, when this mode of open-ended play constituted a large part of my life.

That last point is probably the most important one, to be honest. I miss the pleasure of just following where my mind is leading me, somewhat unfiltered, free from the constraints of long-term maintainability. I’m very good at building maintainable, scalable software over a long period of time in a collaborative manner – but this break in the world calls for some solo rapid prototyping.

I’m giving myself about 3 days to explore any given idea before I move on to the next. I will publish the results as I go and document them on this blog. Watch this space.

Interviews as no-code zones

I wrestled with many new challenges during the last three years, as Head of Engineering at Placemeter building a product and an engineering team from scratch, and one of the toughest ones was hiring. I had little experience initially. How do you get good candidates to even apply at an early-stage startup? How do you decide who to interview? I thought at least I knew how to conduct interviews, since I had spent five years at Google where I averaged about one interview per week.

The specific format of the Google software engineer interview during that time (2007 – 2012) was impressed on me quite strongly: The interviewer states a problem (generally some form of algorithmic challenge), asks the candidate to describe possible solutions verbally and then to write out a solution in code on a whiteboard or on paper. One interview had to fit into 45 minutes, so these problems couldn’t be all that involved.

That last part of writing code in an interview is colloquially known as whiteboarding. I like to use the term live coding since it may happen in a code editor or specialized tool rather than on a literal whiteboard.


Because Google was obviously very successful with it and it was part of my routine for so long, I never questioned this interview format. When I found myself in charge of the interview process at a startup, when I actually had to stop and think about the qualities I want to evaluate in a software engineer, I soon realized the limitations of this format, particularly of the live coding aspect.

My main objection to live coding is that it’s a highly artificial activity which does not resemble anything a software engineer does day-to-day. In the real world, code is written on a keyboard with the support of an IDE and StackOverflow. In the real world, code is not written under time pressure, especially not while also being watched over. In the real world, code is not written in small isolated chunks; it almost always has to integrate with an existing large system. Live coding is so unnatural, I think it very weakly predicts how well someone is going to perform at a tech startup.

I’m not claiming this as an original insight. Google has reportedly stopped relying so much on whiteboard coding, with more focus on just problem solving. Other people have raised issues with live coding, for example Laurie Voss in this excellent post on interviewing.

Programming is easy

Apart from the artifice of live coding, there is a deeper problem with placing coding at the core of interviewing: Programming is easy.

Many people bristle at this idea, so let me explain. Here is a non-exhaustive list of things that I think are harder than programming but are often just as important when creating software:

  • Collaborating in a team. Working in a team requires nuanced communication on an ever-growing list of channels (meetings, email, Slack, git, issue trackers and so on). Being part of a team means making compromises, and that may be hard to swallow.
  • Product design and UI design. An engineer is rarely just a voiceless executing arm of a product manager, especially not in a startup. There has to be a feedback loop, with engineers for example reporting early when a feature is too hard to implement or a UI is too complex or underspecified. Developing a sense for product and UI design (or API design, in backend systems) doesn’t come easy to many.
  • Reading code. In any non-trivial codebase you are going to read a lot more code than you’re going to write. The corollary is that every bit of code is read many more times than it is written, so knowing how to organize and write code for readability is also important (and hard).
  • Debugging. The natural state of software is buggy. Reproducing bugs locally is hard, so is mastering debugging and profiling tools. It gets even trickier with production systems in the mix where you have to interpret live data, logging and monitoring output.

I can illustrate this well with my experience as an intern host. I’ve personally mentored about a dozen interns over the years. They were generally Computer Science undergrads, they were very bright and motivated, and they were already good programmers. I believe that interns get the most out of an internship if you put them to work on real features that are going to be launched to real users (no toy projects or internal-only tools). When I say they were good programmers, I mean that I would give them specs for such a feature, they would get to implementing for a couple of weeks or so, and then they would call me over to their computer to proudly demo what they presumed was the complete feature.

None of them initially realized that they may have completed most of the coding — which they were already capable of without much help from me — but they were far from done. Once they merged their code they would discover that it conflicted with somebody else’s ongoing work, and/or it broke an integration test, and/or when running against production data their code couldn’t handle some users’ messy data, and/or QA revealed their UI component can get into an inconsistent state, and/or spurious exceptions started happening when their code was deployed to production. Sorting out these issues would take at least as much time as they initially spent “programming”.

It’s a long way from a local demo of a feature to getting it into the hands of real users. Most of the actual mentoring I ever did happened during this stage of the software lifecycle, between the first local demo and the public launch. Not by accident this is precisely the stage into which most of the hard activities from above fall.

I see the advancement of a junior engineer to a senior engineer mainly as a shift of focus away from programming to those other, harder activities. Organizing a team of engineers for high productivity is about finding ways to emphasize those concerns in the overall development process.

In the no-code zone

If the correlation between live coding skills and actual engineering skills is weak and testing for coding will give you a one-dimensional picture of a candidate anyway — how should you interview engineers? Here are some techniques I found useful.

Take-home coding challenge. Obviously it’s necessary to evaluate coding skills somehow, but the test should be designed to minimize the drawbacks of live coding and to recreate realistic conditions as much as possible. I like to give candidates a coding challenge that a) they do at home in their own preferred development environment, b) has no deadline, c) and is building on some existing code. I think 2 to 4 hours is appropriate, so it takes some effort to design a challenge that is not too trivial but can be completed in that timeframe.

Take-home challenges have become pretty common in tech startups and can be controversial because of the time commitment required. I have never had a candidate refuse a challenge because I frame it like this: We think working in your own environment without a deadline is the best way for you to demonstrate your coding skills and we will not make you write code at any other point during the hiring process.

Various services have sprung up for administering take-home challenges, none of which I’ve tried. A very simple model worked for me: I share a new private GitHub repo with every candidate, containing instructions and the code they have to build on.

Design discussions. Whiteboards are better for drawing than for coding. I find a good way to ground a design discussion is to base it on a real system that a candidate would work on if hired. For example, I would sketch part of a system on the whiteboard and ask the candidate to design some extension to it. During the discussion I try to move from high-level architecture to very specific low-level details, including possibly UI or API design. If the conversation stalls I ask about tradeoffs and compromises. Under what circumstances would you use C++ as opposed to Python? How do your design decisions change if you are constrained by memory or by bandwidth?

Code reading. I have experimented with showing candidates a piece of code and asking concrete questions about what it does and how they could improve on it. Almost everybody I’ve tried this on has struggled with it, underscoring my point that reading code is really hard. I still think this can lead to revealing discussions.


Writing code during interviews is appealing because it produces something we can supposedly judge objectively. I think a carefully designed take-home challenge can satisfy this desire for a code artifact more effectively. Even then, code only says so much about how a candidate is going to fare at software engineering, which includes many activities other than programming. This leaves you with mostly design discussions during in-person interviews, the results of which are unfortunately more nuanced, more subjective, and harder to interpret.

Nobody says hiring is easy. Evaluating a person’s potential in a complex job like software engineering, after spending just a few hours with them, is inherently hard. It’s better to acknowledge that than to act as if there are fixed rules that always lead to the right hiring decision. You should be constantly questioning if your process is working.

I’ve rarely met an interviewee who likes live coding, so explicitly renouncing live coding may work to your advantage as a startup competing for scarce talent. I know I will push for that at my next job.

Lessons on Mobile Platforms from an Ancient JavaScript Game

I recently realized that I’ve been continuously programming in JavaScript for about 16 years. That is, I’ve been writing at least little snippets (and sometimes heaps) of JavaScript at least once a month for 16 years. 16 years! That’s insane. That’s a much longer run than I’ve had with any other language, and it doesn’t look like it’s going to end anytime soon. The funny part is that I’ve never been particularly fond of JavaScript as a language, but there is just no way around it as long as I continue do some sort of web development. I suspect that a lot of other developers who came of age in the late 90s have reached similar accidental JavaScript seniority and that would partly explain why by some metrics it’s on track to become the world’s most popular programming language.

Reminiscing about my early exploits in JavaScript, I searched through my backups to unearth a game I wrote in 1998 and included in my very first application for a paid programming job (I got the job). I was shocked to discover that in a modern browser the game still works exactly as intended, with zero modifications to the code. Here it is, in all its original low-res 90s glory: AsteroidMan, a PacMan clone written in JavaScript.

Screen Shot 2013-11-10 at 1.39.29 PM

In 1998, Netscape 4 and Internet Explorer 4 were the browsers at the cutting edge. They didn’t support much manipulation of the DOM yet, but it was possible to dynamically change the value of a form field — and the source of an image, which enabled games like this one based on a grid of images. One other neat technique for client-side apps I remember from that time was to use frames (yeah, frames, not those new-fangled iframes) to keep global JS state in the parent document, which was preserved across page navigation in the child frames. In my backups I also found a LucasArts-style adventure game I wrote that way. No, it’s too embarrassing to post now.

What do we learn from this exercise in JavaScript archaeology? The web is an insanely good application distribution and execution platform. I opened that HTML page from my backups in a browser and it looked and worked exactly as it did 15 years ago. There was zero installation or configuration required. The game does not only work in recent Chrome, Safari and Firefox on my laptop, it runs on my Android phone and on my iPad. I doubt any more traditional client software written 15 years ago would still run in a modern incarnation of the target platform, especially with zero installation and configuration by the user. Imagine a Windows 98 app running on Windows 8 (that’s a brilliant versioning scheme right there, Microsoft). The web platform is singular in how extremely portable and backwards compatible it is.

Here’s what this means for today. In case you hadn’t noticed, there is currently a big trend away from web apps towards native apps, namely iOS and Android apps. I seriously doubt a native iOS or Android app written today will work in a modern iOS/Android environment in, say, 4 years from now. I can guarantee you that neither iOS or Android will even still be around in a recognizable form 10 years from now. As platforms they are very likely a dead end while the web has a proven track record of being very open ended.

Unfortunately the current trend is unlikely to reverse soon because for now those platforms are a very profitable dead end, both for the platform stewards and for the programmers who serve them. If you have only rudimentary Android or iOS skills you should have no problems finding a well-paying job right now. The trend is a bit less profitable or downright onerous for creators of consumer products (that would be me and also my employer) because building natively for both iOS and Android and maybe even Windows Mobile is an expensive time sink.

I don’t expect any innovation from Apple in this regard since they have always been all about user and developer lock-in. But in my mind Google has a huge opportunity here. Google should push to make web apps first-class citizens in Android. Basically, an Android app may be a light-weight wrapper around a web app with some JavaScript APIs to call out to device APIs. This isn’t a novel idea, Phonegap and others have been trying to tack something very similar on top of Android, but it would help immensely if this approach was sanctioned and widely promoted by Google and ideally developed into an open standard. Such a fairly simple extension to the web has a much better shot at a long shelf life and at getting adopted by other mobile environments. It should also dramatically smooth out the pretty steep learning curve for Android development and thus increase Android’s developer mindshare.

The lesson for developers should be obvious: Think twice before you embark on a mobile-first strategy with native apps only. Consider Phonegap or other hybrid frameworks. Especially if you’re building for longevity or for low maintenance cost, for heaven’s sake, just build a web app.

Robolectric 2.2: Some Pages from the Missing Manual

When I have a conversation about Android development with anyone I usually end up complaining about how hard and annoying testing is. I think Google did a huge disservice to the community by neglecting testing infrastructure since as a result there is no culture of testing in Android development. There are some open source efforts to improve the situation though, and the main tool that has been preserving bits of my sanity while coding for Android is Robolectric.

I use it mostly to write unit tests for activity and fragment classes, since doing that with the tools provided by Android is really rather painful (and slow). With the just released Robolectric version (2.2) I feel its API, especially around maintaining the activity lifecycle, has finally evolved to where tests are really easy and intuitive to write. There is some basic documentation for the new API but I haven’t come across good real world examples, so I thought it could be useful to document some patterns I’ve been using.

Testing Activities

My activity tests tend to have the following shape, here for a fictional class MyActivity:

public class MyActivityTest {

    private ActivityController controller;
    private MyActivity activity;

    public void setUp() {
    	controller = Robolectric.buildActivity(MyActivity.class);

    public void tearDown() {

    private void createWithIntent(String myExtra) {
        Intent intent = new Intent(Robolectric.application,
    	Bundle extras = new Bundle();
    	extras.putString("myExtra", myExtra);
    	activity = controller

    public void createsAndDestroysActivity() {
        // Assertions go here

    public void pausesAndResumesActivity() {
        // Assertions go here

    public void recreatesActivity() {
        // Assertions go here


In Robolectric 2.2, ActivityController is now the canonical API (actually, the only API) for this sort of test. It has methods that make the underlying activity go through state changes. To fully create an activity as it would be presented to a user you generally want to call create(), start() and visible() on it. I’m also putting a destroy() call in an @After method to ensure tearing down the activity always works properly.

As a rule, every activity test I write has at least the 3 tests above, covering the most common state transitions – create/destroy, pause/resume, and recreate. You might be surprised how often you initially implement broken recreate logic. On an actual device, a recreate happens for example when the screen rotates, something that’s quite common but easily overlooked in manual testing.

Often I’ll want to write multiple tests that start an activity with different intents. This is hinted at in the createWithIntent() method which takes a customizable extra parameter for the intent. For a very simple class you might omit that and just create the activity without any intent in the setUp() method.

Testing Fragments

For fragment unit tests I am using a base class that can make any fragment go through the lifecycle of a parent activity:

public class FragmentTestCase {

    private static final String FRAGMENT_TAG = "fragment";

    private ActivityController controller;
    private FragmentActivity activity;
    private T fragment;

     * Adds the fragment to a new blank activity, thereby fully
     * initializing its view.
    public void startFragment(T fragment) {
        this.fragment = fragment;
        controller = Robolectric.buildActivity(FragmentActivity.class);
        activity = controller.create().start().visible().get();
        FragmentManager manager = activity.getFragmentManager();
        		.add(fragment, FRAGMENT_TAG).commit();

    public void destroyFragment() {
        if (fragment != null) {
            FragmentManager manager = activity.getFragmentManager();
            fragment = null;
            activity = null;

    public void pauseAndResumeFragment() {

    public T recreateFragment() {
        // Recreating the activity creates a new instance of the
        // fragment which we need to retrieve by tag.
    	fragment = (T) activity.getFragmentManager()
    	return fragment;


The general idea is that the best way to put a fragment through realistic state transitions is to add it to a real activity and then make that activity change states. Using a blank FragmentActivity works well for this purpose.

The helper methods allow me to very quickly write at least 3 tests covering the most common transitions, same as above for activities. This is an example for a MyFragment class:

public class MyFragmentTest extends FragmentTestCase<MyFragment> {

    private MyFragment fragment;

    public void setUp() {
        fragment = new MyFragment();

    public void createsAndDestroysFragment() {
        // Assertions go here

    public void pausesAndResumesFragment() {
        // Assertions go here

    public void recreatesFragment() {
        fragment = recreateFragment();
        // Assertions go here



I’ve had pretty solid results with these patterns so far. I’m beta testing an app with real users right now, fully covered with tests like the above, and I haven’t seen any crashes yet (knock on wood). But I’m sure this can be improved upon. I’d be curious to hear how others use Robolectric since I can find so few real world examples out there.

The Test-Infected Programmer’s Android Apothecary

Earlier this week I gave a lightning talk about testing at an Android meetup in New York. Because, on the surface, testing is not the most exciting topic in the world, I tried to spice it up with a snazzy title and a slightly gimmicky structure. The jury is out whether it worked or not (it didn’t draw many laughs), but decide for yourself based on the slides.

Since the slides are not that useful without the accompanying stage show, here’s a very quick summary. The premise is that test infrastructure provided by the Android SDK out of the box has quite a lot of shortfalls and for someone serious about unit testing there are a bunch of 3rd-party libraries that help ease the pain.

These are the libraries I presented:


  • Fixes the problem of having to run tests on an Android emulator, which can significantly improve test execution speed and plays nicer with most Continuous Integration environments
  • Works by replacing some Android core classes with “shadows” which are hand-written fakes, so some caution is warranted because behavior can be subtly different from the real implementations in the Android VM

FEST Android

  • Simplifies assertions about Android objects such as views
  • Based on FEST fluent assertions, a way to make assertions more readable, type-safe and intuitive


  • The self-promotion part of the talk: This is a library I wrote myself, based on concepts from assert_select in Rails and Android’s own UiSelector
  • Makes assertions about complex Android UIs easier by selecting views with CSS-style selectors
  • Alpha-quality software, in the sense that the API is not completely stable (I’ll have a more in-depth blog post once I’m happy with the API)