IBM SPSS Modeler supports Python scripting using Jython, a Java[tm] implementation of the Python language. Modeler versions 16 and 17 use Jython 2.5.1 which includes a number of useful and popular modules. However, many other modules are available and customers often want to use their own so a frequent question is how to include them.

There are two approaches:

  • Copy the module to the “site-packages” folder under modeler-installation/lib/jython/Lib. This has the advantage of making modules available to anybody who uses that Modeler installation but usually requires someone with a level of administrative privileges to update the Modeler installation.
  • Define a JYTHONPATH environment variable and add folders containing the modules to that. This allows each user to have their own module search paths without affecting other users and also does not require administrative privileges. However, it does mean that a script will only work for users who have the correct JYTHONPATH set.

We will add the BeautifulSoup HTML parser to the search path using each method. Note that we need to use BeautifulSoup 3 to be compatible with Jython 2.5.x (later versions of BeautifulSoup require Python 2.7 and 3.x). You can download the correct version from here:

http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-3.2.1.tar.gz

You can also check the BeautifulSoup 3 documentation.

Setup
Use your favourite archiving tool to extract the BeautifulSoup-3.2.1 folder containing so you can see:

BeautifulSoup.py
BeautifulSoupTests.py
PKG-INFO
setup.py

Using “site-packages”

  1. Locate the folder modeler-installation/lib/jython/Lib/site-packages.
  2. Copy the BeautifulSoup.py file from the BeautifulSoup-3.2.1 folder you extracted earlier to the site-packages folder (you may be prompted to authorise the copy).

Using JYTHONPATH

  1. Define a JYTHONPATH environment variable. On Windows 7:
    1. Open the Control Panel
    2. Click the “System” option
    3. Click the “Advanced System Settings” on the left.
    4. In the dialog you should see the Advanced tab is selected and this should have an “Environment Variables…” button at the bottom. Click on that.
    5. In the Environment Variables dialog, click the “New…” button to create the JYTHONPATH environment and provide the full path to the BeautifulSoup-3.2.1 folder e.g. C:\Users\jclinton\Downloads\BeautifulSoup-3.2.1.
  2. Restart Modeler.

Testing
To confirm the installation has worked, we will run a simple script that uses BeautifulSoup and the urllib2 modules to look for links on a web page (it’s not a realistic example but you could use something more complex to scrape web pages for values that that could be fed, say, to text analytics).

Create a new stream, open the stream dialog at the Execution tab and copy the following script to it.


import sys
import urllib2
from BeautifulSoup import BeautifulSoup

try:
    page = urllib2.urlopen("http://www.metoffice.gov.uk/public/weather/forecast")
    soup = BeautifulSoup(page)
    for link in soup('a'):
        print(link.get('href'))
except urllib2.URLError, e:
    print "URLError:", e.reason
except:
    print "Unexpected error:", sys.exc_info()[0]

Switch to the Debug sub-tab and when you run the script, you should see the values of the “href” attributes in each HTML “a” (link) element.

7 comments on"Adding New Modules To Jython Scripting In IBM SPSS Modeler"

  1. Hi Julian,

    regarding the first approach:
    i can find the site-packages-folder for my v17-installation, but not for v16. Is there a similiar approach for v16?

    Does the second approach work for v16?

    Thanks, best regards,
    Andreas

  2. JulianClinton January 25, 2016

    Hi Andreas

    There should be the same site-packages folder in Modeler 16 under /lib/jython/Lib already although possibly I added it post installation. If it’s not there, then you should be able to add the site-packages folder yourself and add your custom Modules to that.

    Could you confirm for me exactly which Modeler 16 you are using? From the Modeler main window, do Help -> About.. and in that dialog, click the ‘Additional Details’ button. Mine is 16.0.0.2.

    Regards

    Julian

  3. Hi Julian,

    thanks!
    i use version 16.0.0.0
    the first approach (using the site-packages-folder) has not worked. I added the path /jyton/Lib as there is no jython folder in /lib , but the execution script couldn´t find the module.
    Have i missed something?

    The second apporach (setting env. variable) worked on my local system.
    I hope it also works for Batch+Server…

    Thanks,
    Andreas

  4. JulianClinton January 26, 2016

    Hi Andreas

    I’m not sure why the site-packages is missing. At some point it would be worth updating to Modeler 16.0.0.2 or 17. Using JYTHONPATH should work for Modeler batch mode.

    Regards

    Julian

    • In Modeler V16 Jython was a jar file called sth like jython-stanfard-lib.jar, as it’s name suggests, it’s where Jython stores standard packages. What you can do is open the jsr file using a tool like 7zip, add the packages you need, and re-pavkage it as a jar file with the same filename. Note this approach does not work for all packages, e.g., those depends on c libraries.

  5. Hello Julian,
    Thanks for sharing these methods
    Method 2 with JYTHONPATH worked fine for me when running the script from the modeler client interface. However, running same script using the clemb command line is giving an error:
    Erreur : AEQMJ0132E: Le script ne parvient pas à charger le module Func sur ligne 1 colonne 1
    (translation: script not able to find module)
    Is there any additional step to be able to run on batch mode?
    Kind regards
    Nadjet

    • My mistake, the system variable was not initialized on the prompt command

Join The Discussion

Your email address will not be published. Required fields are marked *