Babelfish SDK contains the tools and libraries required to create a Babelfish driver for a programming language.

The Babelfish SDK has the following dependencies:

• Docker
• Go

Make sure that you've correctly set your GOROOT and GOPATH environment variables.

Babelfish SDK gets installed using either Go:

$ go get -t -v gopkg.in/bblfsh/sdk.v2/...

or make command:

$ make install

These commands will install both bblfsh-sdk and bblfsh-sdk-tools programs at $GOPATH/bin/.

The SDK provides scaffolding templates for creating a new language driver. These templates are converted to Go code that ends up in bblfsh-sdk tool. Use make to update these templates:

$ make
go get -v github.com/jteeuwen/go-bindata/...
go get -v golang.org/x/tools/cmd/cover/...
cat protocol/internal/testdriver/main.go | sed -e 's|\([[:space:]]\+\).*//REPLACE:\(.*\)|\1\2|g' \
	> etc/skeleton/driver/main.go.tpl
chmod -R go=r ${GOPATH}/src/github.com/bblfsh/sdk/etc/build; \
go-bindata \
	-pkg build \
	-modtime 1 \
	-nocompress \
	-prefix ${GOPATH}/src/github.com/bblfsh/sdk/etc/build \
	-o assets/build/bindata.go \
	${GOPATH}/src/github.com/bblfsh/sdk/etc/build/...
chmod -R go=r ${GOPATH}/src/github.com/bblfsh/sdk/etc/skeleton; \
go-bindata \
	-pkg skeleton \
	-modtime 1 \
	-nocompress \
	-prefix ${GOPATH}/src/github.com/bblfsh/sdk/etc/skeleton \
	-o assets/skeleton/bindata.go \
	${GOPATH}/src/github.com/bblfsh/sdk/etc/skeleton/...

You can validate this process has been properly done before submitting changes:

$ make validate-commit

If the code has not been properly generated, this command will show a diff of the changes that have not been processed and will end up with a message like:

generated bindata is out of sync
make: *** [Makefile:66: validate-commit] Error 2

Review the process if this happens.

On the other hand, If you need to regenerate proto and proteus files, you must run go generate from protocol/ directory:

$ cd protocol/
$ go generate

It regenerates all proto and proteus files under protocol/ and uast/ directories.

Babelfish SDK helps both setting up the initial structure of a new driver and keeping that structure up to date.

Let's say we're creating a driver for mylang. The first step is going to the location where we want the repository for the driver to be bootstrapped:

$ cd $GOPATH/src/github.com/bblfsh

Now the driver should be bootstrapped with bblfsh-sdk. This will create a git repository, and some directories and files required by every driver. They will be overwritten if they exist, like the README.md file in the example below.

$ bblfsh-sdk init mylang alpine
initializing driver "mylang", creating new manifest
creating file "manifest.toml"
creating file "Makefile"
creating file "driver/main.go"
creating file "driver/normalizer/normalizer.go"
creating file ".git/hooks/pre-commit"
creating file ".gitignore"
creating file ".travis.yml"
creating file "Dockerfile.build.tpl"
creating file "driver/normalizer/normalizer_test.go"
creating file "Dockerfile.tpl"
creating file "LICENSE"
managed file "README.md" has changed, discarding changes
$ git add -A
$ git commit -m 'initialize repository'

Note that this adds a pre-commit git hook, which will verify these files are up to date before every commit and will disallow commits if some of the managed files are changed. You can by-pass this with git commit --no-verify.

You can find the driver skeleton used here at etc/skeleton.

Whenever the managed files are updated, drivers need to update them. bblfsh-sdk can be used to perform some of this updates in managed files. For example, if the README template is updated, running bblfsh-sdk update will overwrite it.

$ bblfsh-sdk update
managed file "README.md" has changed, discarding changes

bblfsh-sdk doesn't update the SDK itself.

For further details of how to construct a language driver, take a look at Implementing the driver section in documentation.

bbflsh-sdk also includes a testing framework for a driver. In order to run test for a particular dirver, change to it's directory and run:

$ bblfsh-sdk test

This will:

• compile a "test binary" that parses content of the ./fixtures directory of the driver
• create a docker image with all dependencies, native driver and a test binary
• run this test binary inside a Docker container, using that image

Overall, SDK supports 3 different kind of tests for a driver:

• UnitTests, parsing content of ./fixtures and applying UAST transformations. Described above.
• Integration tests, using content of ./fixtures/_integration*
• Benchmarks, using content of ./fixtures/bench_*

First two always run, benchmarks are only triggered by bblfsh-sdk test --bench.

GPLv3, see LICENSE