Testing RESTful APIs the not-quite-as-hard way

Last week I wrote briefly about using wget and curl to test RESTful interfaces. A couple people at CERN wrote in to suggest I look at a tool they were quite happy with, called httpie.

I’m impressed. It seems to strike a lovely balance between expressiveness and simplicity. What’s especially brilliant is that it’s written for the common case of needing to customize headers and set specific parameters; you can do it straight off the command line. For what I was doing last week:

$ http GET http://localhost:8000/muppet/6 Accept:application/json

sets the Accept header in your request; sending data is unbelieveably easy. Want to post to a form? -f gets you url encoding, and meanwhile you just set parameters on the command line:

$ http -f POST http://localhost:8000/ name="Kermit" title="le Frog"
Accept: */*
Accept-Encoding: gzip
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:8000
User-Agent: HTTPie/0.2.7



If you’re sending JSON it does things like set the Content-Type and Accept headers to what they should be by simply specifying -j (which sensibly is the default if you POST or PUT and have name=value pairs). And, -v gets you both request and response headers; if you’re testing at this level you usally want to see both. Good show.

$ http -v -j GET http://localhost:8000/muppet/6
GET /muppet/6 HTTP/1.1
Accept: application/json
Accept-Encoding: gzip
Host: localhost:8000
User-Agent: HTTPie/0.2.7

HTTP/1.1 200 OK
Cache-Control: max-age=42
Content-Encoding: gzip
Content-Type: application/json
Date: Thu, 09 Aug 2012 03:52:27 GMT
Server: Snap/0.9.1
Transfer-Encoding: chunked
Vary: Accept-Encoding

    "name": "Fozzie"
    "title": "Bear"

Speaking of bears, I’m afraid to say it turned out to be quite the bear getting httpie installed on Ubuntu. I had to backport pygments, requests, python-oathlib, and pycrypto from Quintal to Precise, and meanwhile the httpie package in Quintal was only 0.1.6; upstream is at 0.2.7 and moving at a rapid clip. I finally managed to get through dependency hell; if you want to try httpie you can add my network tools PPA as ppa:afcowie/network. I had to make one change to httpie: the default compression header in python-requests is

Accept-Encoding: identity, deflate, compress, gzip

which is a bit silly; for one thing if the server isn’t willing to use any of the encodings then it’ll just respond a normal uncompressed entity, so you don’t need identity. More importantly, listing deflate and compress before gzip is inadvisable; some servers interpret the order encodings are specified as an order of preference, and lord knows the intersecting set of servers and clients that actually get defalate right is vanishingly small. So,

Accept-Encoding: gzip

Seems more than sufficient as a default; you can always change it on the command line if you have to for testing. Full documentation at github; that said, once it’s installed, http --help will tell you everything you’d like to know.