IBM Developer

Article

SPL to KQL converter: Bridging Splunk and QRadar gap

Explore seamless SPL to KQL conversion for enhanced SIEM integration

By Rahul K P, Honey Gidwani

On this page

As organizations transition between security information and event management (SIEM) solutions, migrating queries becomes a critical task. Migrating saved searches poses a challenge due to differences in query syntax, functionality, and the underlying data model between Splunk and IBM QRadar Suite. Additionally, ensuring that the migrated searches retain their intended functionality and accuracy in the new environment is crucial. This article explores the process of converting Security Information and Event Management (SIEM) queries from the Splunk Processing Language (SPL) to the Kusto Query Language (KQL) used in IBM QRadar Suite.

Component technologies

To convert SIEM queries from SPL to KQL, you'll use the following technologies:

Kusto Query Language (KQL)

Kusto Query Language (KQL) is a query language that you can use to query the IBM QRadar data lake. KQL allows you to build a variety of different type of queries from simple search queries to more complex aggregated queries.

Splunk Search Processing Language (SPL)

Splunk Search Processing Language (SPL) is a set of commands that you use to search data within the Splunk ecosystem. SPL includes all the search commands and their functions, arguments, and clauses. Initially rooted in the UNIX pipeline and SQL, SPL's syntax enables a broad scope of data operations, including searching, filtering, modification, manipulation, insertion, and deletion.

watsonx.ai

watsonx.ai is IBM's next-generation enterprise studio for AI builders to train, validate, tune, and deploy both traditional machine learning and new generative AI capabilities powered by foundation models. This platform combines top-tier architectures with a rigorous focus on data acquisition, provenance, and quality, to serve enterprise needs. It accelerates the development of AI applications while demanding less data and substantially reducing manual efforts.

A standout feature of watsonx.ai lies in its utilization of generative AI, capable of producing diverse content types. It supports a repository of open-source and IBM foundation models that users can prompt to yield desired outcomes. These foundation models cater to a wide range of use cases spanning natural languages and programming languages.

The watsonx.ai library leverages the llama-2-70b-chat (70B) model to facilitate the conversion of SPL to KQL queries, employing appropriate prompts. The llama-2-70b-chat represents a refined iteration of llama-2-70b, an auto-regressive language model that uses an optimized transformer architecture. These tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. Notably, Llama-2-Chat models outperform open-source chat models across various benchmarks and are comparable in terms of helpfulness and safety to popular closed-source models such as ChatGPT and PaLM.

Challenges in SPL to KQL conversion

During the conversion process from SPL to KQL, several challenges arise that need to be addressed to ensure a smooth transition:

  • Syntax differences: SPL and KQL have distinct syntaxes, requiring meticulous translation of functions, operators, and field references to maintain query integrity.

  • Functionality variations: Differences in functionality between SPL and KQL might lead to situations where certain functions or features lack direct equivalents. This requires the implementation of creative solutions or alternative approaches to preserve query functionality.

  • Data model variances: SPL and KQL often have different methods for representing and accessing data. Consequently, adjustments in query logic are necessary to accommodate these variances and ensure accurate data retrieval and manipulation.

Benefits of switching from SPL to KQL

When organizations switch from SPL to KQL, they can expect several advantages that enhance their overall SIEM experience:

  • Cross-platform compatibility: Converting queries facilitates seamless integration and migration between Splunk and IBM QRadar Suite, ensuring a smooth transition between platforms without compatibility issues.

  • Enhanced query performance: Optimizing queries for the target platform results in improved query performance and more efficient resource utilization, leading to faster and more accurate data retrieval.

  • Accuracy: Leveraging a generative AI model to convert SPL queries to KQL minimizes the risk of query errors. This ensures that security analysts obtain accurate results, enhancing the effectiveness of their threat detection and response efforts.

  • Increased Efficiency: By automating the conversion process, the converter saves analysts significant time and effort that would otherwise be spent manually translating queries. This streamlined approach allows analysts to focus more on analyzing security data and less on query syntax conversion, thereby increasing overall operational efficiency.

SPL to KQL conversion workflow

Navigating through the conversion process is easy. Here's how it works:

  1. Simply type your SPL query in the provided text box under the SPL Query section.

  2. Click on Translate to start the conversion process.

    Text box to enter SPL Query

  3. After the conversion completes, you will see the resulting KQL query displayed in a separate section titled KQL Query.

  4. To run the converted KQL query directly in IBM QRadar Suite, use the provided option or button. This could be a Run in QRadar button that opens a new tab or window with the QRadar interface and automatically fills the query field with the converted KQL query.

    Screen showing resulting KQL query

SPL to KQL converter demo

To see the SPL to KQL converter in action, watch the following demonstration video:

Deploying the SPL to KQL converter

The SPL to KQL converter application details are stored in a GitHub repository. For installation instructions, downloading the app, and installing it in IBM QRadar, go to the SPL to KQL Converter GitHub repository.

Summary

In conclusion, transitioning from SPL queries to KQL is a vital step when shifting between SIEM solutions. Despite the challenges it poses, the advantages of cross-platform compatibility and improved query performance make it a valuable pursuit for organizations aiming to harness the advanced capabilities of Azure Sentinel.

Next steps

For further details on the mentioned technology stacks, refer to the following resources: