Introduction
If a user application is not working properly or goes down or gets killed, currently there are only limited ways to detect and monitor such issues. Also, most of the times monitoring and maintaining high availability of an application requires third-party software. This article provides a way to implement a kernel module on Linux, compile it, and explore ways in which a user application can monitor and communicate with this kernel module. The kernel module can continuously monitor the user application, take in commands from the user application, create raw sockets, bind on them, and send data.
Figure 1. Two-way interaction between a user-application and a kernel module
Figure 1 shows the user-application and kernel module interaction components.
Application: It can be any application running at user-level which can interact with kernel module.
Kernel module: It contains the definition of the system calls that can be used by the application and kernel level APIs to monitor application health, communicate with user level applications, and so on.
Prerequisites to use Linux kernel extension
Linux kernel extension is used to monitor application health, communicate with user-level applications, and manage kernel logs. The following prerequisites are required for users to make use of it.
- Knowledge of C programming language
- Knowledge of Linux operating system
- Understanding about GCC complier
Load/Unload and check status of kernel module
The following commands can be used to load or unload and list various kernel modules. These will be helpful in writing and using kernel extensions through user applications.
Loading kernel module
insmod: This command is used to insert a module into the kernel.
Usage: insmod ‘kernel_ext_binary’
# insmod helloWorld.ko
Welcome to Hello world Module.
Unloading kernel module
Usage: rmmod ‘kernel_ext_binary’
# rmmod helloWorld.ko
Goodbye, from Hello world.
Listing of all running kernel modules
lsmod: This command lists all the kernel modules that are currently loaded.
Usage: lsmod | grep ‘kernel_ext_binary’
# lsmod | grep hello
helloWorld 12189 1
Detailed information about kernel module
modinfo: This command displays additional information about the module.
Usage: modinfo hello*.ko
# modinfo helloWorld.ko
filename: /root/helloWorld.ko
description: Basic Hello World KE
author: helloWorld
license: GPL
rhelversion: 7.3
srcversion: 5F60F86F84D8477986C3A50
depends:
vermagic: 3.10.0-514.el7.ppc64le SMP mod_unload modversions
The above commands can be run on console and through a binary application using the system() call.
Communicate with user space
The user space must open a file specified by path name using the open() API. This file will be used by both, the user application and the kernel module to interact with each other. All commands and data from the user application are written to this file (from which the kernel module will read and act upon). The reverse is also possible.
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
int open(const char *pathname, int flags);
Usage:
int fd;
#define DEVICE_FILE_NAME "/dev/char_dev"
fd = open(DEVICE_FILE_NAME, 0);
The return value of open() is a file descriptor, a small non-negative integer that is used in subsequent system calls (in this case, ioctl).
Use of ioctl calls
The ioctl() system call can be invoked from a user space to manipulate the underlying device parameters.
#include <sys/ioctl.h>
int ioctl(int fd, int cmd, ...);
Here, fd is the file descriptor returned from open() and cmd is the same as what is implemented in the kernel module’s ioctl().
Usage:
#define IOCTL_SEND_MSG _IOR(MAJOR_NUM, 0, char *)
int ret_val;
char message[100];
ret_val = ioctl(file_desc, IOCTL_SEND_MSG, message);
if (ret_val < 0) {
printf("ioctl_send_msg failed:%d\n", ret_val);
exit(−1);
}
In the above example IOCTL_SEND_MSG is a command which is sent to the module.
_IOR means that the application is creating an ioctl command number for passing information from a user application to the kernel module.
The first argument, MAJOR_NUM, is the major device number we’re using.
The second argument is the number of the command (there could be several with different meanings).
The third argument is the type we want to get from the process to the kernel.
In the same way, user application can get a message from kernel with little modification to the ioctl arguments.
Thread handling at the kernel module
The following sections provide ways to handle multithreading in kernel context. We can create and manage multiple kernel threads inside a kernel module.
Thread creation
We can create multiple threads in a module using the following calls.
#include <linux/kthread.h>
static struct task_struct * sampleThread = NULL;
sampleThread = kthread_run(threadfn, data, namefmt, …)
kthread_run() creates a new thread and tells it to run.
threadfn is the function name to run.
data *is a pointer to the function arguments
namefmt is the name of the thread (in ps command output)
Thread stopping
We can stop running threads by using the following call:
kthread_stop(sampleThread);
This API stops the thread gracefully.
Establish socket communication
It is possible to create a raw socket in the kernel mode using the sock_create() function.
Through this socket, a kernel module can communicate with other user-level applications within or outside the host.
struct socket *sock;
struct sockaddr_ll *s1 = kmalloc(sizeof(struct sockaddr_ll),GFP_KERNEL);
result = sock_create(PF_PACKET, SOCK_RAW, htons(ETH_P_IP), &sock);
if(result < 0)
{
printk(KERN_INFO "[vmmKE] unable to create socket");
return -1;
}
//copy the interface name to ifr.name and other required information.
strcpy((char *)ifr.ifr_name, InfName);
s1->sll_family = AF_PACKET;
s1->sll_ifindex = ifindex;
s1->sll_halen = ETH_ALEN;
s1->sll_protocol = htons(ETH_P_IP);
result = sock->ops->bind(sock, (struct sockaddr *)s1, sizeof(struct sockaddr_ll));
if(result < 0)
{
printk(KERN_INFO "[vmmKE] unable to bind socket");
return -1;
}
Using sock_sendmsg(), the kernel module can send data using the computed message structure.
struct msghdr message;
int ret= sock_sendmsg(sock, (struct msghdr *)&message);
Generate signals to user space process
Signals can also be generated from the kernel module to user application. If the process identifier (PID) of the user process is known to kernel, using this pid the module can fill in the required pid structure and pass it on to send_sig_info() to trigger the required signal.
struct pid *pid_struct = find_get_pid(pid);
struct task_struct *task = pid_task(pid_struct,PIDTYPE_PID);
int signum = SIGKILL, sig_ret;
struct siginfo info;
memset(&info, '\0', sizeof(struct siginfo));
info.si_signo = signum;
//send a SIGKILL to the daemon
sig_ret = send_sig_info(signum, &info, task);
if (sig_ret < 0)
{
printk(KERN_INFO "error sending signal\n");
return -1;
}
Log rotation
A log rotation file is used by rsyslog to support log rotation of kernel module logs. If the user wants to redirect all specific kernel module related logs to a specific file, an entry needs to be added to rsyslog (/etc/rsyslog.conf) as follows.
Usage:
:msg,startswith,\"[HelloModule]\" /var/log/helloModule.log
The above entry allows rsyslog to redirect all kernel logs starting with [HelloModule] to the /var/log/helloModule.log file. You can look for more samples in the /etc/rsyslog.conf file.
Sample: Users can write their own rotation script (such as the following one) and place it in /etc/logrotate.d.
"/var/log/helloModule.log" {
daily
rotate 4
maxsize 2M
create 0600 root
postrotate
service rsyslog restart > /dev/null
endscript
}
This script checks daily if the log file has exceeded 2M (that is, 2 MB) size and supports 4 rotations of this file. If the log size exceeds 2 MB, it will create a new file with same name and 0600 file permissions, and the old file will be appended with the date time stamp.
Post rotation, it will restart the rsyslog service.
Makefile
Refer to the following content of the makefile to generate the binaries for a sample program, helloWorld.c
obj−m += helloWorld.o
all:
make −C /lib/modules/$(shell uname −r)/build M=$(PWD) modules
clean:
make −C /lib/modules/$(shell uname −r)/build M=$(PWD) clean
Note: This sample is based for RHEL flavor, other flavors may vary in implementation of makefile.
Integration of kernel module with user application
This section shows a sample user application and a kernel module. The user application will be able to communicate with the kernel module and exchange data.
The user application uses ioctl calls to send data to the kernel module. In the following example, these ioctl calls can be used to send application-specific details or send any updates at a later point of time. The following sample program incorporates all the concepts explained earlier.
Sample user application
# cat helloWorld.h
#ifndef HELLOWORLD_H
#define HELLOWORLD_H
#include <linux/ioctl.h>
// cmd ‘KE_DATA_VAR’ to send the integer type data
#define KE_DATA_VAR _IOR('q', 1, int *)
#endif
# cat helloWorld.c
#include <stdio.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <sys/ioctl.h>
#include <stdlib.h>
#include "helloWorld.h"
/* @brief: function to load the kernel module */
void load_KE()
{
printf ("loading KE\n");
if (system ("insmod /root/helloWorld.ko") == 0)
{
printf ("KE loaded successfully");
}
}
/* @brief: function to unload the kernel module */
void unload_KE()
{
printf ("unloading KE\n");
if (system ("rmmod /root/helloWorld.ko") == 0)
{
printf ("KE unloaded successfully");
}
}
/* @brief: method to send data to kernel module */
void send_data(int fd)
{
int v;
printf("Enter value: ");
scanf("%d", &v);
getchar();
if (ioctl(fd, KE_DATA_VAR, &v) == -1)
{
perror("send data error at ioctl");
}
}
int main(int argc, char *argv[])
{
const char *file_name = "/dev/char_device"; //used by ioctl
int fd;
enum
{
e_load, //load the kernel module
e_unload, //unload the kernel module
e_send, //send a HB from test binary to kernel module
} option;
if (argc == 2)
{
if (strcmp(argv[1], "-l") == 0)
{
option = e_load;
}
else if (strcmp(argv[1], "-u") == 0)
{
option = e_unload;
}
}
else if (strcmp(argv[1], "-s") == 0)
{
option = e_send;
}
else
{
fprintf(stderr, "Usage: %s [-l | -u | -s ]\n", argv[0]);
return 1;
}
}
else
{
fprintf(stderr, "Usage: %s [-l | -u | -s ]\n", argv[0]);
return 1;
}
if ((option != e_load) && (option != e_unload))
{
fd = open(file_name, O_RDWR);
if (fd == -1)
{
perror("KE ioctl file open");
return 2;
}
}
switch (option)
{
case e_load:
load_KE();
break;
case e_unload:
unload_KE();
break;
case e_send:
send_data(fd);
break;
default:
break;
}
if ((option != e_load) && (option != e_unload))
{
close (fd);
}
return 0;
}
Sample kernel module
# cat helloWorld.c
#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/version.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/errno.h>
#include <asm/uaccess.h>
#include <linux/time.h>
#include <linux/mutex.h>
#include <linux/socket.h>
#include <linux/ioctl.h>
#include <linux/notifier.h>
#include <linux/reboot.h>
#include <linux/sched.h>
#include <linux/pid.h>
#include <linux/kmod.h>
#include <linux/if.h>
#include <linux/net.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <linux/unistd.h>
#include <linux/types.h>
#include <linux/time.h>
#include <linux/delay.h>
typedef struct
{
char ethInfName[8];
char srcMacAdr[15];
char destMacAdr[15];
int ifindex;
}KEConfig_t;
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Owner Name");
MODULE_DESCRIPTION("Sample Hello world");
MODULE_VERSION("0.1");
static char *name = "world";
static struct task_struct *ke_thread;
static struct KEConfig_t KECfg;
module_param(name, charp, S_IRUGO);
MODULE_PARM_DESC(name, "The name to display in /var/log/kern.log");
/* @brief: create socket and send required data to HM
* creates the socket and binds on to it.
* This method will also send event notification
* to HM.
* */
static int createSocketandSendData(char *data)
{
int ret_l =0;
mm_segment_t oldfs;
struct msghdr message;
struct iovec ioVector;
int result;
struct ifreq ifr;
struct socket *sock;
struct sockaddr_ll *s1 = kmalloc(sizeof(struct sockaddr_ll),GFP_KERNEL);
if (!s1)
{
printk(KERN_INFO "failed to allocate memory");
return -1;
}
printk(KERN_INFO "inside configureSocket");
memset(s1, '\0', sizeof(struct sockaddr_ll));
memset(픦, '\0', sizeof(ifr));
result = sock_create(PF_PACKET, SOCK_RAW, htons(ETH_P_IP), &sock);
if(result < 0)
{
printk(KERN_INFO "unable to create socket");
return -1;
}
printk(KERN_INFO "interface: %s", KECfg.ethInfName);
printk(KERN_INFO "ifr index: %d", KECfg.ifindex);
strcpy((char *)ifr.ifr_name, KECfg.ethInfName);
s1->sll_family = AF_PACKET;
s1->sll_ifindex = KECfg.ifindex;
s1->sll_halen = ETH_ALEN;
s1->sll_protocol = htons(ETH_P_IP);
result = sock->ops->bind(sock, (struct sockaddr *)s1, sizeof(struct sockaddr_ll));
if(result < 0)
{
printk(KERN_INFO "Unable to bind socket");
return -1;
}
//create the message header
memset(&message, 0, sizeof(message));
message.msg_name = sockData->sock_ll;
message.msg_namelen = sizeof(*(sock_ll));
ioVector.iov_base = data;
ioVector.iov_len = sizeof(data);
message.msg_iov = &ioVector;
message.msg_iovlen = 1;
message.msg_control = NULL;
message.msg_controllen = 0;
oldfs = get_fs();
set_fs(KERNEL_DS);
ret_l = sock_sendmsg(sockData->sock, &message, sizeof(data));
return 0;
}
static long ke_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
{
int b;
switch (cmd)
{
case KE_DATA_VAR:
if (get_user(b, (int *)arg))
{
return -EACCES;
}
//set the time of HB here
mutex_lock(&dataLock);
do_gettimeofday(&hbTv);
printk(KERN_INFO "time of day is %ld:%lu \n", hbTv.tv_sec, hbTv.tv_usec);
printk(KERN_INFO "data %d\n", b);
//send data out
createSocketandSendData(&b);
mutex_unlock(&dataLock);
break;
default:
return -EINVAL;
}
return 0;
}
/* @brief: method to register the ioctl call */
static struct file_operations ke_fops =
{
.owner = THIS_MODULE,
#if (LINUX_VERSION_CODE < KERNEL_VERSION(2,6,35))
.ioctl = ke_ioctl
#else
.unlocked_ioctl = ke_ioctl
#endif
};
/* @brief The thread function */
int ke_init()
{
printk(KERN_INFO "Inside function");
return 0;
}
/* @brief The LKM initialization function */
static int __init module_init(void)
{
printk(KERN_INFO "module_init initialized\n");
if ((ret = alloc_chrdev_region(&dev, FIRST_MINOR, MINOR_CNT, "KE_ioctl")) < 0)
{
return ret;
}
cdev_init(&c_dev, &ke_fops);
if ((ret = cdev_add(&c_dev, dev, MINOR_CNT)) < 0)
{
return ret;
}
if (IS_ERR(cl = class_create(THIS_MODULE, "char")))
{
cdev_del(&c_dev);
unregister_chrdev_region(dev, MINOR_CNT);
return PTR_ERR(cl);
}
if (IS_ERR(dev_ret = device_create(cl, NULL, dev, NULL, "KEDevice")))
{
class_destroy(cl);
cdev_del(&c_dev);
unregister_chrdev_region(dev, MINOR_CNT);
return PTR_ERR(dev_ret);
}
//create related threads
mutex_init(&dataLock); //initialize the lock
KEThread = kthread_run(ke_init,"KE thread","KEThread");
return 0;
}
void thread_cleanup(void)
{
int ret = 0;
if (ke_thread)
ret = kthread_stop(ke_thread);
if (!ret)
printk(KERN_INFO "Kernel thread stopped");
}
/* @brief The LKM cleanup function */
static void __exit module_exit(void)
{
device_destroy(cl, dev);
class_destroy(cl);
cdev_del(&c_dev);
unregister_chrdev_region(dev, MINOR_CNT);
thread_cleanup();
printk(KERN_INFO "Exit %s from the Hello world!\n", name);
}
module_init(module_init);
module_exit(module_exit);
A module always begins with either init_module or the function you specify with the module_init() call. This is the entry function for modules. It tells the kernel what functionality the module provides and sets up the kernel to run the module’s functions when they’re needed.
All modules end by calling either cleanup_module() or the function you specify with the module_exit() call. This is the exit function for modules. It undoes whatever the entry function did. It unregisters the functionality that the entry function registered.
Note: Suppose a file is opened at user space using open(), which is used to communicate with the kernel module and
if any execve() call is executed by the user process, then set the FD_CLOEXEC socket option on fd (file descriptor).
fd = open(“/dev/char_device”, O_RDWR);
fcntl(fd, F_SETFD, FD_CLOEXEC);
If the FD_CLOEXEC option is not set on this fd, the file descriptor is set to remain open across an execve() call. The kernel module fails to unload complaining that it is already in use and is dependent on the process initialized through the execve() call.
Summary
This article can help you write, compile, and run Linux kernel extensions in general. In addition, it also provides specific ways to monitor user applications, their health, and restart user applications if it fails or hangs. It also provides ways to perform kernel logging, establish layer two socket communication with user applications, and perform synchronization between kernel extension and user application.