Download and install
Download and install NodeJS to your machine (node v0.8.0 or newer)
Check that it works okay from a terminal command-line:
- node –version
- npm –version
Setup a couple of GMail email accounts
- First one is to read incoming emails from
- Second is the one to send emails to
For a full demonstration of this, you could have multiple “Second” emails to demonstrate sending to different support groups.
Github download of sample code
- npm install
The app has a sample UI web page to invoke the NodeJS code for demonstration purposes, this is written in AngularJS v1 (for simplicity), it does not do anything amazing and should be very easy to understand and extend (if you want to upgrade it to AngularJS v2, go ahead and do so and share it back with me).
Read the following, but remember to modify the emailpoc.js and common.js files with YOUR credentials!
Application Architecture & POSTMAN testing
This application is built using NodeJS and is currently built to just execute once, for demonstration purposes. You can easily extend (and I encourage you to do so) to either have an event driven mechansim to be aware of new emails arriving in the Inbox or worst-case use a polling mechanism to request if their are any new emails (just don’t do it ever 5milliseconds or GMail might not like you)
Whilst we are using GMail IMAP/SMTP to read/send the emails, this could easily be switched out to connect to an Enterprise email system – the concept is the same, you will just have to modify the details based on the connectivity you need.
I’ve also built a very simple UI web page that invokes the Express REST API calls to perform each of the individual component elements:
You can also just as easily use POSTMAN to call the APIs directly passing the content required and see the returned responses.
We can call /getEmails and this will respond with the email parsed and extracted as shown below:
We can extract the “text” and place this into the BODY to POST to the /classify API:
This will respond with the following:
This data was provided by passing the “text” to the NLC and the “top_class” and “classes” are passed back as-is.
This is where the /classify API nodeJS code can be extended to have which group email address to use depending upon the “confidence” level and the “class_name”, or just from the “top_class” value.
We can test the /sendToGroup API code by using the “text” and “subject” data from the initial call to /getEmails earlier. We will determine the “from” email address as explained above. For this example, I’ll use a default address.
When the /sendToGroup API is called, it will then send the email to the defined email address. You can then check in GMail to see the email.
Setup the Natural Language Classifier
As defined in the ingredients section, it is assumed that you have an IBM Bluemix account. Sign in to your account and select [Catalog] [Watson]. The service that we are interested is highlighted below:
Select the [Natural Language Classifier] service to create a new instance:
Once created, select the NLC service:
Now, we need to know the Service Credentials to insert into the emailpoc.js file (so we know which NLC service to connect to). In order to do that we need to select “Service Credentials” and click on “View Credentials”:
This is 2 of the 3 pieces of data that we need to enter into the emailpoc.js file (lines 24/25).
If we click “Manage” and go back to the previous screen, select the [Access the beta toolkit], this will open in a new window.
The “Classifier ID” is the last value that we need to enter into the emailpoc.js file (line 28).
I have created a sample CSV file that contains some initial ground truth data (support_data_train.csv):
The format is simple. Enter the Utterance with a comma and then the category name. I’ve included a limited sample here, but feel free to add a lot more to obtain a greater confidence scoring.
We shall import this into the NLC like so, select “Training data” and the [import] icon. This is open a dialog for you to select the .csv file.
Once imported you will see the values like so:
We can perform some tests directly, just to make sure all is okay:
Walkthrough and breakdown of code
As the application is written in NodeJS, you can run this locally on your own machine or you can “cf push” it up to the IBM Cloud Bluemix platform and run it from there – either way, you’ll get the same result. I prefer to run it locally to start off with and to debug until I’m happy.
As shown above, you can manually call the REST APIs from POSTMAN or you can use the sample UI demo.
In a terminal window, navigate to the project folder and execute
You will get an output that will tell you what PORT has been assigned (usually :6013).
In a web browser, you can navigate to: http://localhost:6013/
The index.html page is the container page that via the common.js file content will then display the mainpage view and invoke the controller to display the mainpage.html page to you:
Here is the rendered web page:
If you take a look at the code for the mainpage.html you’ll see that the ng-click actions will invoke functions within the mainpage.js controller code.
By pressing the [Get Emails] button you will call the getEmails() function, as you can see this is the code that is executed:
This then calls the same API that we called with POSTMAN earlier on. This then executes the code within the emailpoc.js node application:
As you can see the return of that API in emailpoc.js is returned to the mainpage.js controller function.
We then split out the “text”, “from” and “subject” values and put them into $scope variables that are then populated and displayed in the web page:
If you now press the [Classify] button, this will invoke the function within the mainpage.js controller code:
This will then call the emailpoc.js API:
As you can see, this code is executing the Watson Developer Cloud – Natural Language Classifier library and passes the “text” to the NLC service. The response is then just passed back as-is to the calling mainpage.js controller function.
The returned value is then parsed and the values put into $scope variable and refreshed in the web page:
As you can see we are just using this for DEBUGGING information at the moment.
Now time for you to complete this sample:
The [Send To Group] button is not actually coded yet. But, you can guess what it is going to do. I’ll leave that piece of work for you to complete.
In essense, the mainpage.html page tells you the name of the function in the ng-click action to name the function in the mainpage.js controller. You can then pretty much copy & paste the same content from the .classify code (as we’re doing a POST call). The difference is that you will want to put different code into the “data” object that you are going to POST. Hint: Take a look at the POSTMAN call we made earlier to see what the emailpoc.js “/v1/sendToGroup” API call is expecting to receive.
The next logical step would be for you to extend the emailpoc.js “/v1/classify” API call to parse the response data (extract the top_class and classes content) and insert code to determine which email group the classified email should be sent to. Then you could pass this email address back to the calling mainpage.js controller function. This value can then be passed when the [Send To Group] button is pressed.