Presence Insights: Context

IBM Presence Insights consists of almost 30 libraries and iterates on a two-week cycle. During that two-week cycle, the default branch of these libraries changes rapidly as multiple teams deliver features. Because we rely on npm to deploy these libraries to the staging environment, handling this traditionally would result in many versions being cut during each development cycle, most of them being unimportant or extremely minor. This caused overly complicated version histories that were difficult to parse at times when regressions popped up.

Through npm dist-tags, you can establish and manage multiple streams of development without complicating the versioning history of the library. dist-tags are valid install targets, meaning that a given dist-tag named unstable could be installed through npm via npm install @pi/library@unstable. This also means that it’s a valid target for a package.json.

When code is checked in to any branch during the iteration and passes regression tests, Travis CI creates a version based on the current (“latest”) version. The latest version contains additional metadata, including the branch name and timestamp. This version is published under a tag that is based on the branch name.

Resources

This automation is handled through:

  • travis.yml
  • package.json
  • @pidev/publish

.travis.yml

Travis.yaml from PI logger controls the configuration of the build run by Travis on code check-in. Here’s an example:


            deploy:
                provider: script
                script: npm run travis‑publish
                skip_cleanup: true
                on:
                    all_branches: true
                    node: '0.12'
                    tags: false

This block declares that deploy is managed by a custom script that we provide via npm run travis-publish. on:d is uded to define which modules to publish. Because we want to manage prerelease versions, we publish from all branches and do not publish on tags (the release script pushes the on tags at the end of each iteration). skip_cleanup is enabled because our custom publish script relies on some Node dependencies.

package.json

Defines what npm run travis-publish means. An example of package.json from Presence Insights logger is:

"travis-publish": "publish npm --platform travis"

publish is a CLI written in Node specified as a dev dependency:

"@pidev/publish": "0.x"

@pidev/publish

Defines the custom logic for our deployments, including cobbling the metadata from branch name and timestamp. It is a simple commander + shelljs node CLI that exposes a command for publishing to npm via publish npm. When the --platform travis is present, a routine accessing Travis environment variables is used.

For Presence Insights, we named the staging environment YS1-dev. When changes were checked into the default branch, the tag used was dev to match the name of the staging area. For other teams or situations, rc is recommended to make it clear that anything on the default branch is a candidate for being released.

// change on master branch, main development stream
if (branch === 'master') {
    sh.echo(ch.blue('creating dev version and publishing'));
    version += '‑dev‑' + moment().tz('America/New_york').format('YYYYMMDD‑HHmm');
    tag = 'dev';
}

The version is created from the dist-tag, and the timezone in the format YYYYMMDD-HHmm. For example: @pilib/logger@1.0.5-dev-20160416-0945.

Example output from a build ran on Travis:

creating dev version and publishing
exec npm version 1.2.11‑dev‑20160421‑1754
v1.2.11‑dev‑20160421‑1754
exec npm dist‑tag add @pilib/logger@1.2.11‑dev‑20160421‑1754 dev
+dev: @pilib/logger@1.2.11‑dev‑20160421‑1754
exec npm publish ‑‑tag dev
+ @pilib/logger@1.2.11‑dev‑20160421‑1754

If it’s not the default branch, we do a little normalization to ensure that the branch name is safe to use in the version metadata.

  • Remove /
  • Replace _ with –
  • Use normalize()



else {
    // if its not master branch, its a feature branch, create dist‑tag for feature
    // based on branch name, publish to that.
    sh.echo(ch.blue('creating a feature version and publishing'));
    
    var normalizedBranch = branch.split('/').join('‑');
    normalizedBranch = normalizedBranch.split('_').join('‑');
    normalizedBranch = normalizedBranch.normalize();

    sh.echo(ch.green(normalizedBranch) + ' is the dist‑tag for this feature');
    version += '‑' + normalizedBranch + '‑' + moment().tz('America/New_york').format('YYYYMMDD‑HHmm');
    tag = normalizedBranch;
}

In this scenario, a branch named feature/geofences_39284 (where 39284 is the work item id) is normalized to the tag feature-geofences-39284, with a full version of @pilib/logger@1.0.5-feature-geofences-39284-20160416-0945, for example. This version is available via npm install @pilib/logger@feature-geofences-39284.

After parsing out the relevant information and cobbling together our tag and prerelease version string, we publish to npm without updating the latest tag.

// properly exit if version fails 
if (sh.exec('npm version ' + version).code !== 0) {
    sh.exit(1);
}

// properly exit if adding the dist‑tag fails
if (sh.exec('npm dist‑tag add ' + scope + '/' + name + '@' + version + ' ' + tag).code !== 0) {
    sh.exit(1);
}

// properly exit if publish fails
if (sh.exec('npm publish ‑‑tag ' + tag).code !== 0) {
    sh.exit(1);
}

First, cut the version that we created. Add the dist-tag explicitly in the case that this is the first time, then publish with the --tag flag to tell npm not to update the latest version.

With that, the code checked in is available for local and staging environments through the dist-tag created. Projects that specify dev as the version for private Presence Insights dependencies will automatically pick up changes to the default branch throughout the iteration. Projects that specify feature versions will pick up changes to that branch.

Next steps

  1. Slim down @pidev/publish currently relies on things such as moment-timezone, but timezone is only needed once. It also uses commander, which is a somewhat large framework. You can slim down @pidev/publish to only rely on shelljs, yargs, and chalk.
  2. Parameterize the default branch and the tag that represents it in @pidev/publish. We use the master branch for our default branch but the name of our staging environment is dev, so it is somewhat confusing. Change the name from dev to staging or rc.

Conclusion

In this article, you learned how to automate pre-release version management by combining npm dist-tags with Travis CI. Pre-release versions can be a powerful tool to allow quick feedback in deployed environments while also maintaining a concise and easy to understand version history, as well as allowing work to happen iteratively for a single release more easily.