Node.js::Getting around Github Raw resources limitations

Node.js::Getting around Github Raw resources limitations

It’s a very common scenario: you finally find the file you’re looking for to start using on your website (be it a jquery plugin, an angular directive, or whatever you can include on your website) on Github, navigate to that file, hit the “raw” link and then copy the URL.

But when you try it on your site, it won’t work… , you check the Network tab on your favourite browser and you see that the resource is being interpreted as a plain text file, so your browser doesn’t know what to do with it.

Not cool Github! (well, to be fair, I don’t think they intended their site to act as a CDN).

Two easy ways to get around raw.github.com limitations to use files on our websites.

I’ll quickly go over two solutions to this problem…

Online solution: rawgithub.com

This is a very cool solution, you change a small bit on your file’s url (instead of usingraw.github.com/… you use rawgithub.com/… ) and the file is delivered to you with the correct content type.
Awesome! But wait, there is a catch… the author of Rawgithub.com states:
Use this for testing and for sharing demo code with friends, but don’t use it for anything production-related. It’s slow, it might break from time to time, and it’s intended as a convenience for development, not free static hosting.
So, testing / development: cool, production: not cool.

And that is when it hit me, for some cases, doing what this service is doing for us, is really, actually, super easy, so I went ahead and created a small npm module, called “rawhub“.

Offline solution: rawhub

This is a node.js only solution, at least for now. What this module does, is simply download the resource from raw.github.com into a local folder on your server and then serve it from there.

Usage

On node, you setup the express middleware like this:

app.use(rawhub.setUp({folder: '/rawhub'}));

On your front-end, you change:

This: https://raw.github.com/user/repo/master/filename.js
For this: http://myserver.com/[FOLDERNAME]/user/repo/master/filename.js

Lets see a quick example of use

On the node.js side of things:

var express = require("express");
var rawhub = require("rawhub");
var app = express();

app.use(rawhub.setUp({folder: '/rawhub'}));
app.listen(3000);

That is a very simple express app, and it’s using rawhub. The setUp method is createing the middleware and the folder option is so rawhub knows which urls are meant to actually be fromraw.github.com.
That’s it, now on every request, rawhub will query github to get the file and will return it to you with the correct content type header.

On the HTML side of things:

<html>
  <head>
   <script src="/rawhub/user/repo/master/filename.js"></script>
<!-- ... -->

The change on the front-end is very simple, just treat the resource as if it was local on the folder you configure when setting up the middleware, and that’s it.

Other options

Doing a request to github on every request your site gets, can be a little too much, and it’ll add to your site’s loading time, so there is a cache option:

app.use(rawhub.setUp({
  folder: '/rawhub',
  cache: {
    folder: '/cache',
    ttl: 100
   }
}));

Now, with the above code, we’re telling rawhub to download and save the files from raw.github.com into ourproject/cache for 100 seconds.

And that’s it!

At least for now, this module was created in 2 hours, so there is probably a lot of room for improvement, I just hope someone can find it useful.

github raw user content

Leave a Reply

Your email address will not be published. Required fields are marked *