After creating the splitter and Annotator, one needs to prepare the Insight Pack by doing the following:
- Create index configuration to map the fields extracted to the schema under which extracted values will be stored.
- Create the jar file containing the splitter and annotator.
- Create filesets and sourcetypes.
- Build the Insight pack.
- Awareness of Insight Packs (reference What is an Insight Pack?)
- Insight Pack tooling installed (reference Install and Configure Insight Pack Tooling)
- Insight Pack project created (reference Create your First Insight Pack Project)
- Splitter code completed ( reference Create Splitter using Java)
- Annotator code completed (reference Create Annotator using Java)
Insight pack artifacts
Index configuration defines the schema for storing the details extracted in the annotator. Indexing configuration specifies the following for each element.
Name : Name of the field
Type : TEXT, LONG, DOUBLE, DATE
- Filterable : Enable or disable facet counting and filtering on this field.
- Searchable : Controls whether the field is enabled for searching/matching against it.
- Sortable : Enable or disable the field for sorting and range queries.
- Retrievable: Determines whether the contents of this field are stored for retrieval.
- Retrievebydefault: When set to true, the contents of the field is always returned as part of any search response.
Click on indexconfig.spec.ucdk located under your project. The following form will be displayed:
Click on the Field configuration tab. Timestamp and logrecord fields are added to the index config by default. Change the format of the date to suit the format in your log file as shown below:
Add new fields. Each field is mapped to the field names extracted in the annotator.
Following are the fields extracted in the annotator. We will now map these to a field in our schema. The field names in the schema can be same as the field name in your annotator code for better manageability of the code.
private static final String SEVERITY = “Severity”;
private static final String EXCEPTION = “Exception”;
private static final String ERRORCODE = “ErrorCode”;
private static final String DETAIL = “Detail”;
private static final String RETURNCODE = “ReturnCode”;
private static final String RETURNMESSAGE = “ReturnMessage”;
To add a field, click on add, enter the field name and its type and attributes as shown below and click on NEXT as shown below.
In the Add Source Panel, click on add and enter the path of the field in the output json created in the annotator code. In our case it was annotations.Severity as shown below. Click on Finish for this and the previous form.
Similarly add all other fields. After adding you will see the fields as follows:
Defining of index config is complete now.
Create a jar
Create a jar file containing the created splitter and annotator code. From your project open the build_fileSetJar.xml file. Replace the existing content with the following and save the file. Change the name of the jar file to suite your project.
NOTE: In the Edit Configuration panel, select all the options and click on Run as shown below. Every time java code is changed, the jar needs to be created.
The following will be displayed in the console. Ignore the warnings!
C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build_fileSetJar.xml clean: [delete] Deleting directory
C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build compile: [mkdir] Created dir:
C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build_fileSetJar.xml:9: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds[javac] Compiling 2 source files to
C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build\classes run: compile: [javac]
C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build_fileSetJar.xml:9: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds jar: [jar] Building jar: C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\src-files\extractors\fileset\java\MyAppExtractor.jar BUILD SUCCESSFUL Total time: 16 seconds
Refresh the project (press F5 on the project name) to see the jar file under src-files as follows:
Source type defines the combination of splitter, annotator (Filesets) and index config used to parse, annotate and index a given type of file.
Define the required FIleset as follows.
Click on insightpack_spec.ucdkt. In the form shown on the right hand side, click on File sets. Click on Add and fill the form as follows:
Create Annotator fileset as shown below.
Click on Source types tab, click on add and add a source type as follow:
Save the changes done.
Build insight pack zip file
Right click on the project and click on Build Insight Pack as shown below.
Successful creation of insight pack will be shown in console as follows.
Congratulations! Your first insight pack is ready for testing now.