Last week I wrote a tool for reverse engineering protobuf data called protofudger. I wanted to show it off to a few people, but didn’t want them to have to download it and run an unsigned binary on their system, or set up a whole go toolchain just to build one program. If only there were a better way!
TL;DR:take a look at this if you want to look at the end result. If you’d like a file to test with, try this one from node-protobuf’s test suite .
Enter GopherJS . If you haven’t heard of it, here’s an excellent description right from their home page:
First task: get the program to compile nicely.
This was trivial. I ran
in my project directory, and it gave me a
file, along with a source map. GopherJS scores major points for being pain-free. Five stars, easy transaction, would download again.
Second task: refactor the program to operate more like a library.
I had a typical go command line thing going on, using the
package for arguments, the
package to write stuff to the terminal, and everything happening from start to end in a
function. This… Did not transfer so well to the web. You know, obviously. Makes sense in hindsight.
The meat of the matter was in refactoring all the
statements out of the parsing functions, and instead returning strings to be printed by the
function. This was pretty straightforward, so I won’t go into too much detail. Take a look at this commit
if you’d like to see the actual changes. I also had to make sure the parsing functions weren’t reading stuff from the environment (e.g. command line args), instead receiving them as parameters.
Third task: somehow bind the program to the UI.
I decided that I wanted the UI to remain largely unchanged. This means text output, some very limited binary options, and an input file.
On the command line, the most obvious way to do the “input file” bit was to read from standard input, e.g.
./protofudger < file.dat
. I figured the best conversion to the web would be to drop a file onto the web page. This meant that I was going to have to get the data from a
to look like a
for the go code. Again, most of the heavy lifting here is done by GopherJS. If you define a go function as
and it just works as you’d expect. In my case, this looked like
The next thing would be the options. There are currently only two options for protofudger - one to display all possible interpretations of numeric fields, and one to display byte offsets alongside field numbers. These mapped pretty well to checkboxes. It’s pretty easy to get the state of a checkbox, so I just do this:
parse(new Uint8Array(reader.result), showAll.checked, showOffsets.checked)
Output was easy. That
function returns an array of strings that I can join together and shove into a
element for that fixed-width terminal feel.
Fourth task: build a bundle that I can deploy.
First I tried to build the
go code as a module that I could
from the main program. This actually works
, but it’s not what I settled on. This would be a boring post if I ended it with “yep, the first thing I tried was perfect,” wouldn’t it?
GopherJS actually provides hooks to build CommonJS modules - in addition to a
handle, it provides a
handle to go programs. This allows you to do something like
, and whatever you expose will be available via
. This is awesome, and it’s something that I think would be totally manageable if it weren’t for some limitations in webpack and Babel. What it would
work very well for would be Node.js programs. With GopherJS you could very easily build a wrapper around your go program for use in Node.
So, obvious solution: don’t run Babel over the GopherJS output. Yep! That worked. It built correctly, and I had a bundle that I could put into a page. But sourcemaps were broken!
Fifth task: get source maps working.
Well, yeah. There kind of is. There’s a package called source-map-loader , but it hasn’t seen any love for a while. There were a couple of bugs in it that I thought prevented it from operating at all (see issue 18 ), but even once I fixed them it still didn’t give me great results.
I really wanted to have working source maps, so I decided to go back to basics. When I included the GopherJS output directly in the page, they worked fine. The actual source code doesn’t show up, because GopherJS doesn’t include it in the source map, but all the line numbers and filenames were correct. I figured it wasn’t too horrible to have two files if it meant I had working source maps.
Conclusion: JS like it’s 2004.
What I ended up doing was treating the GopherJS output like a third party library. I switched from setting
in the GopherJS entry file, then I access that explicitly within the UI code. This has upsides and downsides. One downside is that there are multiple files now. This makes deployment a tiny bit more complex, as I need to put two
tags in the HTML file. One upside is that there are multiple files now (lol). The browser can download the UI file at the same time as the GopherJS code, which has (anecdotally) resulted in slightly faster load times. The UI code makes sure that
exists before it tries to use it, so I could even load the GopherJS output asynchronously if I wanted to optimise things further.
I hope this is helpful for someone out there, and I hope I’ve littered this post with enough keywords to catch people googling for “gopherjs webpack source maps,” to warn them off the path I tried to walk!