July 15, 2016 · Heroku

Heroku Buildpacks for Vendored Binaries

Heroku offers a functionality called buildpacks that helps to install vendored binaries in Heroku applications. Recently I needed to use the Kindlegen binary for an app hosted on Heroku. I didn't want to check a large binary into the application, plus each operating system likely requires a different version of the binary. Buildpacks are a great solution for this scenario because you can properly install dependencies for Heroku.

To create the buildpack, I just needed to write some shell scripts to download the binary, move it to the appropriate directory and set environment variables. Here's a link to the final Kindlegen buildpack I created. I'll go through the different scripts that are required and how the binary is installed.

detect

This is the first script that's run. It can be used to determine whether or not the deployment should continue based on the presence of certain files. In my case I always want the buildpack to run, so I just made it print the name of the buildpack and exit with status code 0.

echo "Kindlegen"  
exit 0  

compile

This is where most of the action happens in the buildpack. It's used to download a tarball from S3, extract it and then install the binary.

indent() {  
  sed -u 's/^/       /'
}

BUILD_DIR=$1  
KINDLEGEN_DIR=$BUILD_DIR/vendor/kindlegen  
BIN_DIR=$KINDLEGEN_DIR/bin  
PROFILED_DIR=$BUILD_DIR/.profile.d  
APP_BIN_DIR=/app/vendor/kindlegen/bin  
URL="https://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz" 

echo "-----> Installing Kindlegen"  
mkdir -p $BIN_DIR

echo Vendoring $URL | indent  
curl -s $URL | tar -xz -C $KINDLEGEN_DIR

echo Moving binary | indent  
mv $KINDLEGEN_DIR/kindlegen $BIN_DIR/

echo Setting PATH | indent  
mkdir -p $PROFILED_DIR  
echo PATH=\$PATH:$APP_BIN_DIR > $PROFILED_DIR/kindlegen.sh  

The indent function just prints out text in a nicer format. Next some variables are created. APP_BIN_DIR is ultimately where the binary will be placed and URL is the link to S3 with the Kindlegen tarball. This can be any publically available URL. mkdir -p $BIN_DIR creates the directory where the binary will be moved. curl -s $URL | tar -xz -C $KINDLEGEN_DIR downloads and extracts the tarball. This directory will contain documentation and license files, as well as the binary. In the next step only the binary file is moved into the bin directory.

The last step is to set the path environment variable. This is needed so that Kindlegen is available on the command line. Previously you could set the environment variables in the release script, however the API changed and now environment variables need to be set using .profile.d scripts. APP_BIN_DIR, which contains the full path to the Kindlegen binary directory will be appended to the path variable. This will be set in the file kindlegen.sh in the .profile.d directory. Scripts in this directory are sourced when the application is deployed.

release

The release script lists addons and defaultprocesstypes for Heroku. For vendored binaries you typically don't need to set any of theses values. For the Kindlegen buildpack this is just echo "--- {}".

Add the buildpack to Heroku

To actually use the buildpack you need to add it to your application. The buildpack URL should point to a repository. To add the Kindlegen buildpack I ran heroku buildpacks:add --index 1 https://github.com/benmel/kindlegen-buildpack.git.

Now whenever your application is deployed to Heroku the buildpack will run. The echo commands from the scripts will print to the console. For example the Kindlegen buildpack will print out:

remote: -----> Fetching set buildpack https://github.com/benmel/kindlegen-buildpack.git... done  
remote: -----> Kindlegen app detected  
remote: -----> Installing Kindlegen  
remote:        Vendoring https://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz  
remote:        Moving binary  
remote:        Setting PATH  

For more detailed information about buildpacks check out Heroku's API documentation.

Comments powered by Disqus