Import Apache CouchDB logs to IBM Security Guardium Insights using a universal connector

IBM Security Guardium Insights is a hybrid cloud data security hub that helps you improve visibility into user data activity and risk. Guardium Insights helps you protect data more efficiently, enhance information technology flexibility, and reduce operational costs as you embrace new business paradigms such as moving data to the cloud.

The Guardium universal connector enables Guardium Insights to import data from potentially any data source's native activity logs. The universal connector includes support for various plug-in packages and requires minimal configuration to push the audit data into Guardium Insights for analysis and protection.

Learning objectives

This tutorial will show you how to use the Guardium universal connector to import Apache CouchDB activity logs to IBM Security Guardium Insights.

Prerequisites

• IBM Security Guardium Insights 3.2.x version
• Couch DB 3.3+ Linux version
• Filebeat 7.51.1 (installed in the machine where the database is located)

Estimated time

Completing this tutorial should take about 20 minutes.

Steps

The high-level steps covered in this tutorial are:

1. Create and configure a universal connector
2. Configure Filebeat to push logs to Guardium Insights
3. Generate traffic from CouchDB
4. Monitor logs in Guardium Insights
5. Generate database exception and access violation reports

Step 1. Create and configure a universal connector

1. Download the plugin for CouchDB from the Guardium Insights GitHub page. The plugin is contained in a .zip file with a name such as CouchdbOverFilebeatPackage.zip. Note: To locate the link, scroll the "Available plugins" section. The table in this section contains all the available plugins for Guardium Insights.
2. Locate the CouchDB row in the "Available plugins" table and click the GI link to download the .zip file. Download the file to any machine from which you can access and upload to Guardium Insights.
3. To upload the file into Guardium Insights, log in to Guardium Insights using your user name and password.
4. In the Guardium Insights left menu, click the Settings icon and then click Connections. The Connections page opens.
5. Click Manage > UC plugins > Add plugin. Select the CouchdbOverFilebeatPackage.zip file and then click Open. When the file is uploaded, you can configure the universal connector. Note: Be sure to upload the .zip file without unzipping it or extracting the files.
6. In Guardium Insights, click Settings > Connections to return to the Connections page and then click Add connection.
7. In the drop-down menu, select Universal Connector and click Configure. The “Connect data source via Universal Connector” window opens.
8. Enter a name and description for the connection and click Next.
9. In the Build Pipeline pane, click the Choose input plugin menu and select Filebeat.
10. Click Choose filter plugin and select CouchDB. Click Next.
11. In the Datasource Tag field, enter a unique name for this connector. In this example, the name is couchdb27. When you have entered a name, click Configure.
12. The universal connector is created. Copy the hostname and port and save in a file for future reference. Click Download certificate and save the certificate to a location where you can easily retrieve it. When you have saved your hostname, port, and certificate, click Done.
13. Copy or use FTP to save the certificate to the Linux database machine in a directory that is accessible by you. Usually, you will save to your home directory.

Step 2. Configure Filebeat to push logs to Guardium Insights

You now need to edit the Filebeat configuration file to push the database logs into Guardium Insights.

1. Log in to the Linux machine where the database and Filebeat are installed.
2. Locate the filebeat.yml file in the Filebeat installation directory. The file will normally be in the /etc/filebeat directory. Open the file using an editor (alternatively, you can use the vi editor from a terminal window.)
3. Under the filebeat.Inputs section, verify the following information:
   1. In the type: log attribute, ensure the enabled flag is set to “true”.
   2. Ensure that paths attribute points to the database log folder.
   3. In the tags attribute, enter the value of the unique identifier that you added earlier in the Datasource Tag field (see Step 1, no. 11). In this example, the unique name is couchdb27.
4. Scroll to the Outputs section. Ensure that the output.logstash is uncommented. (Note: All other output sections must be commented out; for example, #output.elasticsearch.)
5. In the output.logstash section, update the following attributes:
   1. In the hosts attribute, enter the hostname and port number that you copied when you were configuring the universal connector. (See Step 1, no. 12.) Note: The port number will be 443.
   2. Update the ssl.certificate_authorities attribute to point to the fully qualified path name of the certificate file you saved from Guardium Insights. (See Step 1, nos. 12 and 13.)
6. Save the filebeat.yml file.
7. Restart the Filebeat service using the following command in a terminal window: sudo service filbeat restart

Step 3: Generate traffic from CouchDB

You can now run queries in CouchDB to create database logs and ensure they are being pushed to Guardium Insights using Filebeat.

1. Ensure you have your database IP, port, and login credentials. (You might need to request them from your database administrator.)
2. Access the CouchDB Fauxton interface by entering the following URL in a browser: http://<DB IP>:<port>/_utils/ .
3. Log in to the database with your username and password. The page will look similar to the following image:
4. Click on the database where you want to run the run the query.
5. In the menu, click Run A Query with Mango. Edit the default query to use the query and conditions of your choice. In this example, the selector query uses the name attribute and $eq:mahesh condition, meaning the name must be "mahesh".
6. Click Run Query to execute the query. This example will display the records that have the name field matching the value mahesh.
7. The output is displayed in the table. You can run different DDL and DML queries to generate more database traffic. If necessary, check with your database administrator for assistance with running these queries.

Step 4: Monitor logs in Guardium Insights

When you generate traffic in the database, it's recorded in the database logs. The Filebeat service is configured to push these logs to a Logstash output destination, in this case your Guardium Insights instance. Now you'll monitor those database activity logs in Guardium Insights.

1. In the Guardium Insights main menu, click Activity Log. The log report displays all the activities that you've performed in the Guardium Insights. You can review any log for audit purposes.
2. In the Guardium Insights menu, click Reports.
3. In the search field, enter connection events. The "Connection events" window displays, including the Reports link.
4. Click Reports to view the connection events from CouchDB. The report provides a summary of the events logged into Guardium Insights over the last 24 hours. Note: You can adjust the time period using the filter option available at the top of the report.
5. Review the report, confirming that it displays the activities you performed to generate the logs. Note that the “Event source” column lists "Universal Connector" as the source.
6. In the Guardium Insights menu, click Reports and in the search field, enter Full SQL. The "Full SQL" window is displayed.
7. Click Reports. The detailed report on SQL activities is displayed, listing the DDL and DML activities that you performed in an earlier step.
8. Review the full report and to check all the database activities in the given time period.

The database logs are now available and Guardium Insights can start monitoring the events for the anomalies and risks.

Step 5: Generate database exception and access violation reports

Exception and access violation reports can help you identify potential attacks. For example, if you see multiple failed login attempts, it's possible that someone is breaking into your database to steal the data.

You can create security policies to log access violations and monitor database exceptions to gain insight into anomalies and risks.

1. In the Guardium Insight menu, click Policies and then click Create a policy.
2. Enter a name for the policy and then add access rules to log details and issue an alert in case of violation. In this example, all CouchDB activities are monitored using an If condition when the database type matches CouchDB. All database activities in the CouchDB database will be analysed for risks.
3. Exception reports will give you insight into database exceptions, which can help identify attacks. In the Guardium Insights menu, click Reports.
4. In the search field, type Exception. The "Exception details" window opens.
5. Click Reports. All exceptions that occurred for the given time period are displayed.

Summary

In this tutorial, you've learned how to create a universal connector and configure it for a CouchDB database. You've learned how to send your logs to Guardium Insights and use its reports to analyze anomalies and risks. Next steps

Find out how IBM Security Guardium Insights can help you set up more security policies and risk configurations to bring provide insight into database risks. Check out the following resources:

• IBM Security Guardium Insights product page
• Block malicious data attacks with IBM Security Guardium and IBM Security QRadar
• Protect against phishing emails using watsonx Assistant and the IBM threat intelligence platform