Using the IBM Watson Machine Learning Community Edition (WML CE) auxiliary channels

IBM® Watson™ Machine Learning Community Edition (WML CE) now includes two auxiliary conda channels – the early access channel and the supplementary channel. In this tutorial, we provide step-by-step instructions and best practice recommendations for using each of these conda channels.

Early access channel

With the WML CE focus on providing a consumable distribution for enterprise-level machine learning tasks, there is often a delay between when the upstream community releases a new version of a framework or tool and when it is included in an official WML CE release. The WML CE early access channel provides packages ahead of an official WML CE release. Packages in this channel have been built and preliminarily tested by IBM, but are not fully supported.

Using the early access channel

The recommended way to use the early access conda channel is to add it to specific environments that need access to the experimental packages, instead of adding it to the global conda channel list. This ensures that only the environments intended to work with experimental packages have access to them and production environments are not compromised. Because the early access packages have the same names as the released packages, it is also important to have the early access channel come before the main channel:

conda create -n wml_ce_test
conda activate wml_ce_test
conda config --env --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access/

The early access channel will now be at the top of ${CONDA_PREFIX}/.condarc:

channels:
  - https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access/
  - defaults

Early access packages can now be installed in this environment using the normal commands. For example:

conda activate wml_ce_test
conda install tensorflow-gpu=2.1
Collecting package metadata (repodata.json): done
Solving environment: done
...
The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ...
    tensorflow-2.1.0           |gpu_py37_914.g4f6e601           3 KB  https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access
    tensorflow-base-2.1.0      |gpu_py37_e5bf8de_72632.gbc9303f       597.3 MB  https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access
    tensorflow-estimator-2.1.0 |py37_7ec4e5d_1461.g4f6e601         603 KB  https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access
    tensorflow-gpu-2.1.0       |     914.g4f6e601           2 KB  https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access
    ...
    ------------------------------------------------------------
...
The following NEW packages will be INSTALLED:

  ...
  tensorflow         ibmdl/export/pub/software/server/ibm-ai/conda-early-access/linux-ppc64le::tensorflow-2.1.0-gpu_py37_914.g4f6e601
  tensorflow-base    ibmdl/export/pub/software/server/ibm-ai/conda-early-access/linux-ppc64le::tensorflow-base-2.1.0-gpu_py37_e5bf8de_72632.gbc9303f
  tensorflow-estima~ ibmdl/export/pub/software/server/ibm-ai/conda-early-access/linux-ppc64le::tensorflow-estimator-2.1.0-py37_7ec4e5d_1461.g4f6e601
  tensorflow-gpu     ibmdl/export/pub/software/server/ibm-ai/conda-early-access/linux-ppc64le::tensorflow-gpu-2.1.0-914.g4f6e601

Note: Confirm that the packages are the correct version and are coming from


## Supplementary channel

Packages in the [WML CE supplementary channel](https://anaconda.org/powerai) are curated and tested, but they are not supported with fixes and security updates like the packages in the main channel. The conda recipes for the packages in this channel are hosted on [our Github.com repository](https://github.com/IBM/powerai/blob/master/conda-recipes) and are open for [contributions](https://github.com/IBM/powerai/blob/master/conda-recipes/README.md) or [suggestions](https://github.com/IBM/powerai/issues).

### Using the supplementary channel

The supplementary channel does not provide newer copies of packages in the main channel. So, adding it to the global channel list shouldn't cause a problem, and order is not as important, although it should still probably come before the default channels.

`conda config --add channels https://anaconda.org/powerai`

The resultant `~/.condarc` file may look something like:

channels:

Alternative usage of the supplementary channel

Because the supplementary channel is hosted at Anaconda.org, it is fairly straightforward to install packages directly from the channel, instead of modifying .condarc:

conda create -n wml_ce_test
conda activate wml_ce_test
conda install <package> -c powerai