Environments for running AI in Node.js

Previous tutorials introduced you to building, training, and running TensorFlow.js in Node.js (and Node-RED). While the possibility of using the GPU was hinted at, the tasks were primarily focused on running TensorFlow.js on the CPU. The CPU provides the quickest and easiest approach to integrating AI into Node.js. However, it comes at the cost of performance and full operation parity with TensorFlow.

For use cases where the CPU performance is a deterrent, other bindings and architectures can be considered. In this tutorial, we go beyond the CPU and explore some of the environments and back ends that are available to better run machine learning tasks in Node.js with TensorFlow.js. We look at customizing and optimizing TensorFlow.js in Node.js for specific environments.

Learn how to take advantage of the GPU for increased performance over the CPU, and also achieve complete support of all operations using native execution of SavedModels. In addition, we review alternative back ends for TensorFlow.js, and see how to successfully run TensorFlow.js in Node.js on a high performing IBM® Power® machine.

Native SavedModel execution in Node.js

In the first tutorial of this series, we explained how to install TensorFlow.js in a Node.js environment.

npm install @tensorflow/tfjs-node

Trained models in various formats can then be converted to the JavaScript-friendly format to be used in Node programs. In particular, models converted from the SavedModel format can be used for inference through the tf.loadGraphModel API.

const tf = require('@tensorflow/tfjs-node')
...
model = await tf.loadGraphModel(modelUrl, {fromTFHub: true})
...
const output = model.predict(input_tensor);
...

When running in this mode, the back end loads the TensorFlow C library (libtensorflow) that contains a fuller set of TensorFlow kernels and APIs to execute the operations in the model. This is different from a browser environment where the model is executed using the JavaScript or WebGL implementations of the operations. The C library provides acceleration of the linear algebra computations on CPU hardware. As a result, CPU inference performance of the same model in Node.js and Python are comparable, and in some cases the Node.js version can even be slightly better thanks to the interpreter.

Because the full TensorFlow C library is available, it’s also possible to load the TensorFlow SavedModel directly and run inference on the model. This capability was added recently through the new tf.node.loadSavedModel API.

const model = tf.node.loadSavedModel(dirname, 'serve', 'serving_default');
const output = model.predict([input_tensor]);

This approach offers two significant advantages:

  1. You can use the full set of operations from the TensorFlow C library instead of being limited to the subset of operations supported in TensorFlow.js. This allows SavedModels in both TensorFlow 1.x and even some in TensorFlow 2.x to run in the Node.js environment.

  2. You do not need to use the converter to convert from the SavedModel format to the JavaScript format. This eliminates the need to install the converter tool, which is a Python program and requires installing a Python environment if you don’t already have one.

The following image shows the difference between the two APIs.

native mode

Under the hood, this approach uses the TensorFlow C TF_LoadSessionFromSavedModel API to load the SavedModel directly. In support of this new capability, several additional utilities were added to make it easier to manage the SavedModel in JavaScript such as inspecting the signature, and deleting the model. You can find more details on the implementation in Native SavedModel execution in Node.js.

Note that the SavedModel format can be used only for inference in Node.js by either the loadGraphModel or loadSavedModel API. Training is not supported. One reason is that models are typically exported to the SavedModel format for inference usage in production, and the inference graph is usually different from the training graph. However, it is still possible to use SavedModel for transfer learning in Node.js. In this case, the SavedModel graph is used to generate an embedding vector from the input, and additional layers are added on top of the SavedModel graph to be trained using the embedding vector.

Taking advantage of the GPU

One of the major reasons that machine learning has become so popular is the accessibility of parallel computation on the GPU. GPUs have become cheaper and more powerful, reducing both training and inference times significantly. Without a doubt, if you’re running machine learning models in an environment that is equipped with GPU, you definitely should take advantage of it.

Due to the availability of JavaScript, TensorFlow.js can be used in many environments, for example, the browser, Node.js, and React. On top of this, TensorFlow.js uses several technologies underneath to boost the performance of model training and inference by using the GPU on these supported environments. In this section, we focus on the Node.js environment. But before we jump to it, let’s take a quick view of two GPU-accelerated back ends in the browsers: tfjs-backend-webgl and tfjs-backend-webgpu.

WebGL

WebGL is a cross-platform web standard for a 3D graphics API that is based on OpenGL ES. WebGL is supported in most modern browsers, including the browsers on your mobile devices. TensorFlow.js uses general-purpose computing on GPU (GPGPU) techniques, implementing its unique vertex and fragment shader programs for performing tensor computations on the GPU. WebGL is one of the registered back ends when running TensorFlow.js on the browser. It has a higher priority than tfjs-cpu-backend, which means your device’s GPU should accelerate your machine learning applications on the browser in most circumstances.

WebGPU

WebGPU is the upcoming successor to WebGL. It’s a JavaScript API developed by the W3C GPU for the Web Community Group, and was made with the intention to improve on WebGL. The companies behind most of the major browsers are helping in this effort.

The WebGPU standard and implementations are still being developed, so it might be a few years before WebGPU really replaces WebGL in browsers. Currently, it’s available in Chrome Canary, which is the developmental version of Chrome. Safari also has a developer experimental feature for this. The WebGPU back end itself in TensorFlow.js is labeled experimental, so many operations are still missing, and performance might not be up to par with WebGL yet. If you want to try it out, the recommended browser is Chrome Canary.

GPU on Node.js

Back to the main topic: how to use the GPU with Node.js. Node.js provides C++ addons to expose C/C++ APIs to JavaScript. TensorFlow.js uses this mechanism and provides an npm package, @tensorflow/tfjs-node-gpu, accelerated by the prebuilt TensorFlow C shared library under the hood. Because of the dependency of the prebuilt TensorFlow C binary, the npm package supports a specific set of environments. You can find the list here. Make sure that your operating system and NVIDIA software match the listed criteria before installing the npm package.

If your platform is not in the list but you still want to use the NVIDIA GPU on your system, you can follow these instructions to build the Tensorflow C binary on your own. It involves lots of dependencies, for example, Bazel, and build tools. However, it’s still doable if your platform is supported by TensorFlow. After you finish the build and unpack the tarball file to the deps folder of the @tensorflow/tfjs-node-gpu npm folder, run the following command to generate the C++ addon.

yarn build-addon-from-source

You can also use pre-built TensorFlow binaries for specific platforms from the community by performing the following steps.

  1. Compose a JSON file named custom-binary.json, and put it into the scripts directory under the @tensorflow/tfjs-node-gpu npm source folder.

  2. Run yarn or npm install to trigger the download and build procedures from the @tensorflow/tfjs-node-gpu npm folder.

The following code is an example custom-binary.json that uses the prebuilt TensorFlow binary for the NVIDIA Jetson Nano (arm64 architecture).

{
  "tf-lib": "https://developer.ibm.com/tfjs-cos/libtensorflow-gpu-linux-arm64-1.15.0.tar.gz"
}

Combining the Node.js C++ Addon and TensorFlow C binary lets TensorFlow.js boost its machine learning performance with GPU acceleration. In the custom-binary.json file, you can specify the prebuilt TensorFlow C binary as well as the prebuilt C++ addon. If the C++ addon is omitted, it downloads the prebuilt TensorFlow C binary from the URL and builds the C++ addon locally.

Usage

As with other npm packages, you must run either of the following commands to install the @tensorflow/tfjs-node-gpu package.

npm install @tensorflow/tfjs-node-gpu

or

yarn install @tensorflow/tfjs-node-gpu

If you can successfully install @tensorflow/tfjs-node-gpu without any issue, you can verify the TensorFlow.js GPU acceleration by simply using the require() API to load the module.

const tfjs = require('@tensorflow/tfjs-node-gpu');

You should see log messages like the following.

tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7

This means that TensorFlow.js can load the C++ addon and TensorFlow C binary. Indirectly, it uses the NVIDIA driver and software to interact with the GPU.

However, if you can’t install @tensorflow/tfjs-node-gpu properly and see the following error messages, you must check to see whether you can build the Tensorflow C binary for your system yourself, or find an existing prebuilt binary for your system from the community.

* Downloading libtensorflow
(node:7840) UnhandledPromiseRejectionWarning: Error: Unsupported system: gpu-linux-arm64

An alternative back end with WebAssembly

In some cases, using the Node back end for TensorFlow.js (tfjs-node) might not be doable due to issues getting the libtensorflow dependency installed. An alternative back end that you can use is the WebAssembly back end for TensorFlow.js (tfjs-backend-wasm).

With support in Node since version 8, WebAssembly, or WASM, is a binary format that provides a compilation target for languages like C, C++, or Rust. This format can then be loaded and run from JavaScript, whether it be in the browser or on the server, offering near native performance. Another major benefit of WebAssembly is its portability, allowing code to be run on various operating systems and instruction set architectures both on and off the web.

With regards to TensorFlow.js, the WASM back end offers much faster CPU execution than the plain JavaScript CPU back end, making it a good choice for Node applications when tfjs-node isn’t an option, or if you don’t want to build libtensorflow from source.

WASM performance

To compare the performance between the different back ends in a Node environment, some measurements were taken (on a 2015 MacBook Pro with a 2.8 GHz Quad-Core Intel™ Core i7) to evaluate the inference times for a MobileNet v2 model. The average inference time of 50 inferences for each back end is listed below.

tjfs-node tfjs-backend-wasm tfjs-backend-cpu
49 ms 109 ms 966 ms

While not quite as fast as the tfjs-node back end, the WASM back end still boasts considerable improvement over the JavaScript CPU back end. TensorFlow.js recently enabled WASM SIMD support, which further improves performance. For their browser tests, the TensorFlow.js team brought MobileNet v2 inference times down from 98.4 ms to 30.2 ms on a MacBook Pro. SIMD is still an experimental feature, both in browsers and in Node. However, with improved and increased support underway, it’s not a stretch to assume that the WASM back end will eventually surpass the tfjs-node back end in CPU execution speed.

Caveats

Current development for the WASM back end has been focused on inference operations, so there are still a number of operations that are typically used in training that have yet to be implemented. If you simply want to perform inference using the WASM back end in Node, then you will likely not encounter any problems. However, when trying to train, you might hit issues like Error: '<op>' not yet implemented or not found in the registry. This indicates the WASM implementation for that op is missing. You can either open an issue on the TensorFlow.js repository or try implementing the op yourself, contributing it to the community. In any case, when using this back end, keep in mind the operation disparities for it.

Usage

Using the WASM back end for TensorFlow.js in Node is straightforward. Here, we go through some of the steps to get a simple app running.

  1. Install dependencies. We use the main TensorFlow.js package instead of tfjs-node.

     npm install @tensorflow/tfjs @tensorflow/tfjs-backend-wasm
    
  2. In a new file, import the previously installed dependencies.

     const tf = require('@tensorflow/tfjs');
     require('@tensorflow/tfjs-backend-wasm');
    
  3. Then, in the same file, we create a function to run a quick inference using a MobileNet model that we downloaded from TFHub.

     async function run() {
       await tf.setBackend('wasm');
       console.log(tf.getBackend());
       const img = tf.ones([1, 224, 224, 3]).toFloat();
       const model = await tf.loadGraphModel(
         'https://tfhub.dev/google/tfjs-model/imagenet/mobilenet_v2_130_224/classification/3/default/1',
         { fromTFHub: true });
       const prediction = await model.predict(img);
       console.log(prediction);
     }
    
     run();
    
  4. Save and run the file on the command line.

     node test-wasm.js
    

You should see an inference being performed using the WASM back end for TensorFlow.js. Some of the convenience functions for turning images into tensors (for example, tf.node.decodeImage) are missing because we are no longer importing the tfjs-node package, but other means for turning images into tensors can be used instead.

Achieving high performance with IBM Power

In this section, you learn about running the tfjs-node package with GPU support on IBM Power. The instructions are based on Ubuntu 18.04 LTS ppc64le on an IBM POWER9 with NVIDIA GPUs. As mentioned, the tfjs-node package is a binding for the TensorFlow C shared library and provides native TensorFlow execution for server-side JavaScript applications running under Node.

Because tfjs-node does not provide a precompiled binding for POWER, you must build it from source. The build process requires the libtensorflow and CUDA shared libraries. Fortunately, IBM Watson™ Machine Learning Community Edition provided these libraries. We only need to focus on building the C binding.

Start Watson Machine Learning Community Edition container

This tutorial assumes that you have a basic knowledge of using Docker. To jump-start the build process, we start a Watson Machine Learning Community Edition Docker container.

docker run --runtime nvidia -it --env LICENSE=yes --rm ibmcom/powerai:1.6.2-tensorflow-ubuntu18.04-py37-ppc64le bash

Alternatively, you can use the following.

nvidia-docker run -it --env LICENSE=yes --rm ibmcom/powerai:1.6.2-tensorflow-ubuntu18.04-py37-ppc64le bash

To verify that the container has access to the GPU, run the nvidia-smi command inside the container. You should see an output similar to the following.

(wmlce) pwrai@7e039567d38c:/$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000004:04:00.0 Off |                    0 |
| N/A   31C    P0    39W / 300W |      0MiB / 32510MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000004:05:00.0 Off |                    0 |
| N/A   33C    P0    41W / 300W |      0MiB / 32510MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla V100-SXM2...  On   | 00000035:03:00.0 Off |                    0 |
| N/A   31C    P0    39W / 300W |      0MiB / 32510MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla V100-SXM2...  On   | 00000035:04:00.0 Off |                    0 |
| N/A   34C    P0    40W / 300W |      0MiB / 32510MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Run the following command to update the TensorFlow package in this Docker container.

conda install -y tensorflow=1.15.3

Build the C binding

To compile tfjs-bindings.node, follow these steps.

  1. Install npm. The npm command-line interface is available with the installation of Node.js. To install Node.js, run the following commands.

     cd ~
     curl --output ./node-v12.16.1-linux-ppc64le.tar.xz https://nodejs.org/dist/v12.16.1/node-v12.16.1-linux-ppc64le.tar.xz
     tar xvf node-v12.16.1-linux-ppc64le.tar.xz
     export PATH="/home/pwrai/node-v12.16.1-linux-ppc64le/bin:$PATH"
    

    Verify Node.js and npm versions.

     node -v
     npm -v
    
  2. Install tfjs-node. Installing tfjs-node should fail because the precompiled libtensorflow shared C library for ppc64le is not hosted by the official TensorFlow.js team. However, the installation will still download the tfjs-node source code for you.

     cd ~
     mkdir tfjs; cd tfjs
     npm install @tensorflow/tfjs-node
    

    It should fail with this message:

     * Downloading libtensorflow
     (node:1049) UnhandledPromiseRejectionWarning: Error: Unsupported system: cpu-linux-ppc64
    
  3. Make the libtensorflow shared libraries available to the tfjs-node package.

     cd ~/tfjs
     mkdir -p node_modules/\@tensorflow/tfjs-node/deps/lib
     cp /opt/anaconda/envs/wmlce/lib/python3.7/site-packages/tensorflow_core/libtensorflow* node_modules/\@tensorflow/tfjs-node/deps/lib/
    
  4. Make TensorFlow C library headers available to the tfjs-node package.

     sudo mkdir -p /usr/local/include/tensorflow
     sudo cp -r /opt/anaconda/envs/wmlce/lib/python3.7/site-packages/tensorflow_core/include/tensorflow/c /usr/local/include/tensorflow/
    

    Note: At the time of this writing, tfjs-node depends on libtensorflow 1.15.x. You can check the supported libtensorflow version.

  5. Build the tfjs-node.binding.

     sudo apt update
     sudo apt install -y build-essential
     cd  ~/tfjs/node_modules/\@tensorflow/tfjs-node
     npm run build-addon-from-source
    

    Verify that /home/pwrai/tfjs/node_modules/@tensorflow/tfjs-node/lib/napi-v5/tfjs_binding.node exists.

  6. Make additional libraries available for runtime.

     export LD_LIBRARY_PATH="/opt/anaconda/envs/wmlce/lib"
    
  7. Verify that tfjs-node can be imported. First, open a Node shell.

     cd ~/tfjs
     node
    

    You should see the following.

     Welcome to Node.js v12.16.1.
     Type ".help" for more information.
     >
    
  8. Enter the following code in the shell.

     const tf = require('@tensorflow/tfjs-node')
     tf.version
    

    You should see an output similar to the following. Verify that an object is returned with the version information.

     Overriding the gradient for 'Max'
     Overriding the gradient for 'OneHot'
     Overriding the gradient for 'PadV2'
     Overriding the gradient for 'SpaceToBatchND'
     Overriding the gradient for 'SplitV'
     node-pre-gyp info This Node instance does not support builds for N-API version 6
     node-pre-gyp info This Node instance does not support builds for N-API version 6
     2020-07-08 23:53:55.477732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
     2020-07-08 23:53:55.611085: I tensorflow/core/platform/profile_utils/cpu_utils.cc:101] CPU Frequency: 3783000000 Hz
     2020-07-08 23:53:55.638838: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x33c66f00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
     2020-07-08 23:53:55.638934: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
     2020-07-08 23:53:55.644219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
     2020-07-08 23:53:56.110275: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties:
     name: Tesla V100-SXM2-32GB major: 7 minor: 0 memoryClockRate(GHz): 1.53
     pciBusID: 0004:04:00.0
     2020-07-08 23:53:56.112323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 1 with properties:
     name: Tesla V100-SXM2-32GB major: 7 minor: 0 memoryClockRate(GHz): 1.53
     pciBusID: 0004:05:00.0
     2020-07-08 23:53:56.114339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 2 with properties:
     name: Tesla V100-SXM2-32GB major: 7 minor: 0 memoryClockRate(GHz): 1.53
     pciBusID: 0035:03:00.0
     2020-07-08 23:53:56.116349: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 3 with properties:
     name: Tesla V100-SXM2-32GB major: 7 minor: 0 memoryClockRate(GHz): 1.53
     pciBusID: 0035:04:00.0
     2020-07-08 23:53:56.116374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
     2020-07-08 23:53:56.118026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
     2020-07-08 23:53:56.119548: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
     2020-07-08 23:53:56.119887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
     2020-07-08 23:53:56.121373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
     2020-07-08 23:53:56.122499: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
     2020-07-08 23:53:56.125816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
     2020-07-08 23:53:56.141703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0, 1, 2, 3
     2020-07-08 23:53:56.141725: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
     2020-07-08 23:53:57.485893: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
     2020-07-08 23:53:57.485950: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 1 2 3
     2020-07-08 23:53:57.485963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N Y Y Y
     2020-07-08 23:53:57.485977: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 1:   Y N Y Y
     2020-07-08 23:53:57.486006: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 2:   Y Y N Y
     2020-07-08 23:53:57.486018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 3:   Y Y Y N
     2020-07-08 23:53:57.497925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 30487 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-32GB, pci bus id: 0004:04:00.0, compute capability: 7.0)
     2020-07-08 23:53:57.503382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 30487 MB memory) -> physical GPU (device: 1, name: Tesla V100-SXM2-32GB, pci bus id: 0004:05:00.0, compute capability: 7.0)
     2020-07-08 23:53:57.508492: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:2 with 30487 MB memory) -> physical GPU (device: 2, name: Tesla V100-SXM2-32GB, pci bus id: 0035:03:00.0, compute capability: 7.0)
     2020-07-08 23:53:57.513843: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:3 with 30487 MB memory) -> physical GPU (device: 3, name: Tesla V100-SXM2-32GB, pci bus id: 0035:04:00.0, compute capability: 7.0)
     undefined
     {
       'tfjs-core': '2.0.1',
       'tfjs-backend-cpu': '2.0.1',
       'tfjs-backend-webgl': '2.0.1',
       'tfjs-data': '2.0.1',
       'tfjs-layers': '2.0.1',
       'tfjs-converter': '2.0.1',
       tfjs: '2.0.1',
       'tfjs-node': '2.0.1'
     }
    

Conclusion

In this tutorial, you’ve seen several ways to maximize the performance of your TensorFlow.js applications in Node. You can execute native TensorFlow SavedModels for faster inference speed and wider operation availability. You can also use the GPU-enabled library tfjs-node-gpu, which has NVIDIA CUDA support for even faster performance.

Furthermore, if a supported GPU is unavailable or if you are having issues finding or building a libtensorflow binary, then the WebAssembly back end for TensorFlow.js is also a great option, boasting significantly better CPU performance over plain JavaScript. Lastly, you learned how to get TensorFlow.js working with Node on IBM Power machines, which is another alternative platform for running AI apps in Node. As you can see, TensorFlow.js has a wide range of availability, and this certainly does a lot to bolster AI in Node use cases. Hopefully, this tutorial serves as a good reference point for guiding you in your own setup of TensorFlow.js.