After creating the splitter and Annotator, one needs to prepare the Insight Pack by doing the following:

  1. Create index configuration to map the fields extracted to the schema under which extracted values will be stored.
  2. Create the jar file containing the splitter and annotator.
  3. Create filesets and sourcetypes.
  4. Build the Insight pack.

 Pre-requisites

Insight pack artifacts

Index config

Index configuration defines the schema for storing the details extracted in the annotator. Indexing configuration specifies the following for each element.

Name : Name of the field

Type : TEXT, LONG, DOUBLE, DATE

Attributes:

  • Filterable : Enable or disable facet counting and filtering on this field.
  • Searchable : Controls whether the field is enabled for searching/matching against it.
  • Sortable : Enable or disable the field for sorting and range queries.
  • Retrievable: Determines whether the contents of this field are stored for retrieval.
  • Retrievebydefault: When set to true, the contents of the field is always returned as part of any search response.

Click on indexconfig.spec.ucdk located under your project. The following form will be displayed:

Build Insight Pack Figure 1

 Click on the Field configuration tab. Timestamp and logrecord fields are added to the index config by default. Change the format of the date to suit the format in your log file as shown below:

Build Insight Pack Figure 2

 Add new fields. Each field is mapped to the field names extracted in the annotator.

Following are the fields extracted in the annotator. We will now map these to a field in our schema. The field names in the schema can be same as the field name in your annotator code for better manageability of the code.

private static final String SEVERITY = “Severity”;
private static final String EXCEPTION = “Exception”;
private static final String ERRORCODE = “ErrorCode”;
private static final String DETAIL = “Detail”;
private static final String RETURNCODE = “ReturnCode”;
private static final String RETURNMESSAGE = “ReturnMessage”;

To add a field, click on add, enter the field name and its type and attributes as shown below and click on NEXT as shown below.

 Build Insight Pack Figure 3

In the Add Source Panel, click on add and enter the path of the field in the output json created in the annotator code. In our case it was annotations.Severity as shown below. Click on Finish for this and the previous form.

Build Insight Pack Figure 4

 Similarly add all other fields. After adding you will see the fields as follows:

Build Insight Pack Figure 5

Defining of index config is complete now.

Create a jar

Create a jar file containing the created splitter and annotator code. From your project open the build_fileSetJar.xml file. Replace the existing content with the following and save the file. Change the name of the jar file to suite your project.


<project>
	    <target name="clean">
        <delete dir="build"/>

    </target>
		    <!-- Compile custom Java source code into class files -->
		    <target name="compile" description="Compile splitter/annotator Java source code into class files">
		        <mkdir dir="build/classes"/>
		        <javac srcdir="src" destdir="build/classes">
		            <classpath refid="compile.classpath"/>
		        </javac>
		        <copy file="log4j.properties" 
		              todir="./classes"/>
		    </target>

	<!-- Create JAR files containing custom Java class files -->
			    <target name="jar" depends="compile" description="Create JAR files containing custom Java class files">
			        <jar destfile="src-files/extractors/fileset/java/MyAppExtractor.jar" 
			             basedir="build/classes">
			        </jar>
			    </target>
       
    <target name="run">
    </target>

	<!-- Define the CLASSPATH -->
	    <path id="compile.classpath">
	        <fileset dir="./lib">
	            <include name="*.jar"/>
	        </fileset>
	        <fileset dir="build/classes">
	            <include name="*/**"/>
	        </fileset>
	    </path>
</project>

Right click on the content of the file, select Run As – > Ant Build as shown below.Build Insight Pack Figure 6

NOTE: In the Edit Configuration panel, select all the options and click on Run as shown below. Every time java code is changed, the jar needs to be created.

Build Insight Pack Figure 7

The following will be displayed in the console. Ignore the warnings!

Buildfile:

C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build_fileSetJar.xml clean: [delete] Deleting directory

C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build compile: [mkdir] Created dir:

C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build\classes [javac]

C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build_fileSetJar.xml:9: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds[javac] Compiling 2 source files to

C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build\classes run: compile: [javac]

C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\build_fileSetJar.xml:9: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds jar: [jar] Building jar: C:\geetha\2015\insightpack_tool\mars\workspace\MyAppLogInsightPack\src-files\extractors\fileset\java\MyAppExtractor.jar BUILD SUCCESSFUL Total time: 16 seconds

Refresh the project (press F5 on the project name) to see the jar file under src-files as follows:

Build Insight Pack Figure 8

Source Types

Source type defines the combination of splitter, annotator (Filesets) and index config used to parse, annotate and index a given type of file.

Define the required FIleset as follows.

Click on insightpack_spec.ucdkt. In the form shown on the right hand side, click on File sets. Click on Add and fill the form as follows:

Build Insight Pack Figure 9

Create Annotator fileset as shown below.

Build Insight Pack Figure 10

Click on Source types tab, click on add and add a source type as follow:

Build Insight Pack Figure 11

Save the changes done.

Build insight pack zip file

Right click on the project and click on Build Insight Pack as shown below.

Build Insight Pack Figure 12

Successful creation of insight pack will be shown in console as follows.

Build Insight Pack Figure 13

Congratulations! Your first insight pack is ready for testing now.

Join The Discussion

Your email address will not be published. Required fields are marked *