Digging around the Github v3 API

Series: worklog July 09, 2011

I’ve had a little project idea brewing for a while that would require read and write access to a Github repository with client side Javascript, which eliminates the ability to use existing libraries like grit and libgit2.

A few of the libraries have Node.js bindings, but that’s not exactly what I wanted. There is a cool project called git.js that looked like it would be perfect, but it doesn’t (yet) support any write operations.

Instead, I’ll have to use the Github HTTP API.

The real problem was that, prior to v3, the Github API was pretty useless for working with repositories. You could get the stats (forks, watchers, title, etc) but the file data wasn’t accessible. I guess you could just scrape the website to get the data but that’s a little hacky and wouldn’t work for making commits or pushing changes (theoretically possible now that you can Edit files online and commit the changes through the web interface?).

(Addendum: the v2 did have read access to git objects, but not write access, as technoweenie from Github points out here)

Anyways, the Github v3 API exposes all the raw git data now, so that seems like a much more robust way to go.

Unfortunately, there is a lack of useful examples on the new API methods; instead of showing you how to get file at a given commit from the API, you get pointed to the ProGit ‘Git Internals’ chapter. So I began to dig into the git internal plumbing with the goal of getting the latest version of a file from one of my repos.

I first started looking at the /blobs/ API route since that’s where a file is stored in a git repo. Before I went digging around with the command line to find the SHA hash for the file, I tried to see if I could get it from the Github website — however most of the SHA identifiers are for commits, not files.

So that was a dead-end, but a bit of googling later, I found that you can get the SHA for a file by running git hash-object FILEPATH (aside: this StackOverflow question was really insightful too). So I picked a random file from my blog repo and found the SHA (957e6b4efb22fa921d0e6b17a1fbf46788c97ed3).

Then I pinged the /blobs/:sha route with hurl.it and got this response:

Interesting, the content field looks promising, but I think it is the git loose object, which is the content deflated with zlib. but it is base64 encoded. Not exactly ideal for showing the file contents to the user.

A bit more digging and I found that the API has some custom MIME types. Adding the Accept: application/vnd.github-blob.raw to the HTTP Header, I got this response:

Bingo! Access to the file data and it’s human readable. I suspect that using that MIME type causes the API to call git cat-file or something similar on the server.

Now, I need to be able to get the data for any file in the repo and I would prefer to not have to use the command line to get the SHA every time. So it looks like the /trees/ API method will be useful — it displays the directories and files present at a given commit. And since you can use branch names (like master) instead of SHA (read more here), I figured I could try plugging that into the Github API.

Cool, it worked. And better yet, the url field gives me the API URL to call for to get the file contents (or the tree if it’s a directory).

So given a Github username and repo name, I can pull down the file structure with /tree/master and then drill down on the individual files with /blobs/ and the special MIME type.

More to come later as I try to figure out how to make an edit to a file and commit it to the repo and more details on the project this is all for once I make more progress.


built with , Jekyll, and GitHub Pages — read the fine print