This article explains how Streams Studio manages the build processes and how that compares to a typical command line compilation.
Understanding Command Line Compilations
For a typical command line compilation, a single invocation of the sc compiler will perform:
- Pre-processing of mixed-mode SPL source (*.splmm) files
- Creating the toolkit index file (toolkit.xml)
- Compiling a main composite operator
A single invocation of the sc command performs all three above steps as part of compiling a streams application.
Streams Studio Separation of Processing
The above three steps are managed separately within Streams Studio. Each of these steps are performed discretely based on file modification events explained below.
Pre-Processing of Mixed-Mode SPL Source Files
Anytime you edit a mixed-mode source file and save your changes, Streams Studio will perform mixed-mode pre-processing of that file to produce its associated generated SPL source file. Mixed-mode pre-processing is performed whenever you modify a mixed-mode source file, regardless of whether the Eclipse Auto-build setting is turned on. The output of the mixed-mode preprocessing is displayed in the SPL Build Console view.
In Streams Studio, you can modify the pre-processor options by clicking the “Edit Preprocessor Options” toolbar icon¬† at the top right of the SPL Mixed Mode Editor window. You can specify any number of arguments that are available as $ARGV values in Perl code during preprocessing.
A subtle difference with Streams Studio is that it allows you to maintain a separate list pre-processor arguments for each mixed-mode source file. In contrast, when you use the sc command from a command line, a single set of arguments from the command line are applied to all mixed-mode source files within the project.
The following will trigger mixed-mode pre-processing:
- Modifying a .splmm source file
- Modifying the pre-processor options for a .splmm source file
Creating The Toolkit Index File
The toolkit index file is critical to displaying accurate information in the Project Explorer view as well as providing the correct choices for content assistance within the SPL editors. Anytime you modify a file in an SPL project that could change the contents of the toolkit index, Streams Studio will rebuild the toolkit index by calling the spl-make-toolkit command. The toolkit index is rebuilt regardless of whether the Eclipse Auto-build setting is turned on.
The following will trigger toolkit indexing for an SPL project
- Modifying a .spl source file
- This includes modifications caused by mixed-mode pre-processing described above
- Deleting the toolkit index file (toolkit.xml)
- Modifying a .cgt file (C++ operator code template)
- Modifying a .xml file (operator or native function model)
- Creating a new namespace in the project
- Changes to a Java .class file (Java source compiled by JDT)
- Changes to a Java .jar archive file
- Creating or changing a sub-directory
- Modifying an active build configuration or setting a build configuration to be active
- Modifying a .xlf file in the project’s impl/nl directory (national language resources)
- Modifying C++ .h, .hpp or .a files in the project’s impl directory
- When using an external builder, any file modification within the source folders
Java Operators and Functions
New in version 3.2, Java primitive operators and Java native functions can be defined using annotations in the Java source code. When you use annotations to define a Java operator or function, the toolkit indexer will automatically create the model files (MyOperator.xml or javaFunction.xml) from the compiled Java classes containing the annotations. The annotation processing greatly simplifies the creation and maintenance of the model files for Java language operators and functions. For more information on developing with Java in Streams Studio see¬†Creating Java primitive operators and¬†Creating Java native functions¬†in the streams InfoCenter.
Compiling A Main Composite Operator
This is where the work of translating SPL source into compiled executable files happens. The compiler invocation identifies the qualified name of a main composite operator (for example, com.foo::MyMain) within the project.
By default, applications are compiled using Streams Studio’s¬†internal builder. The internal builder provides for compiler configuration using “build configurations”. Each main composite can have multiple build configurations, only one of which can be active at any time. The internal builder will process the active build configuration, if any, for every main composite operator in the project.
When the build completes successfully, the build configuration can be used to open an application graph view or to launch the application. When the Eclipse Auto-build setting is turned on, Streams Studio will automatically build main composite operators when relevant files change in the project.
When Auto-build is turned on, the same triggers listed for toolkit indexing, will also cause the SPL applications to be recompiled. The internal builder will recompile every main composite operator in the project. For each main composite operator, the active build configuration is processed.
When Auto-build is turned off, you can use project explorer actions to build all active configurations in a project or just build a single configuration.
Using An External Builder
Instead of using the internal builder in Streams Studio, you can use an external builder, such as ‘make’, by specifying commands to be run to build the project.
To change a project to use an external builder:
- In the project explorer, right click on the project and select “SPL Application Options”.
- In the options dialog, change the “Builder type” to “External builder” and set the remaining fields as desired. The default values are filled in for using the ‘make’ command to build the project.
The typical implementation is to use ‘make’ along with a Makefile that you maintain to build the project as needed. Because Streams Studio always maintains control of the mixed-mode preprocessing and toolkit index processing described above, the external builder should not perform that processing. When writing a Makefile for use within Streams Studio, you need to be sure to specify the –no-mixed-mode-preprocessing and –no-toolkit-indexing options when compiling main composites. This is necessary to prevent the build process from looping. For more information see¬†Creating a Makefile for an external builder¬†in the streams InfoCenter.
Remote Development Considerations
When using Streams Studio remote development, all of the build steps described here are only performed when you have a connection to the remote host. If you do not have a connection or are in “offline” mode, the build steps are not performed. Before the build commands are executed on the remote host, the modified files in the project are synchronized to the remote context on the remote host. After the remote compilation completes, portions of the compiler output are synchronized from the remote context to the local workspace. This includes the generated ADL file along with generated source code for C++ operators.
For more detailed information on building applications in Streams Studio see¬†Building SPL Projects¬†in the streams InfoCenter.