Tag Archives: Wait Queues

Waiting / Blocking in Linux Driver Part – 4

<< Previous Article

In the last article, we discussed the usage of wait queues in Linux kernel. We saw the variants of wait_event(). Just for completeness, we will discuss how the wait queues are implemented internally. First step is the creation and initialization of wait queue entry. This is done as below:

DEFINE_WAIT(wait_entry);

Next step is to add the wait queue entry to the queue and set the process state. Both of these things are being done by a single function declared below:

prepare_to_wait(wait_queue_head_t *wq_head, wait_queue_t *wq, int state);

After this, we schedule out the process by invoking the schedule() API. Once we are done with waiting, next step is to clean up with the API below:

finish_wait(wait_queue_head_t *wq_head, wait_queue_t *wq);

All the above are available from:

#include <linux/wait.h>

One more point worth noting is that the condition has to be tested manually as was done in our earlier article.

   Send article as PDF   

Waiting / Blocking in Linux Driver Part – 3

<< Previous Article

The last article in this series focused on implementing the basic wait mechanism. It was a manual waiting where everything, starting from putting the process to the sleep to checking for the wake up event, was done by driver writer. But, such kind of manual waiting is error prone and may at times result in synchronization bugs. So, does kernel provide some robust wait mechanism? No points for guessing the right answer, yes it does. So, read on to explore more on wait mechanism in kernel.

Wait Queues

Wait queue is a mechanism provided in kernel to implement the wait. As the name itself suggests, wait queue is the list of processes waiting for an event. Below are the data structures for wait queues:

#include <linux/wait.h>
// Data structure: wait_queue_head_t
// Created statically 
DECLARE_WAIT_QUEUE_HEAD(wait_queue_name);
// Created dynamically
wait_queue_head_t my_queue;
init_waitqueue_head(&my_queue);

As seen above, wait queues can be defined and initialized statically as well as dynamically. Once the wait queue is initialized, next step is to add our process to wait queue. Below are variants for this:

// APIs for Waiting
wait_event(queue, condition);
wait_event_interruptible(queue, condition);
wait_event_timeout(queue, condition, timeout);
wait_event_interruptible_timeout(queue, condition, timeout);

As seen, there are two variants – wait_event() and wait_event_timeout(). The former is used for waiting for an event as usual, but the latter can be used to wait for an event with timeout. Say, if the requirement is to wait for an event till 5 milliseconds, after which we need to timeout.

So, this was about the waiting, other part of the article is to wake up. For this, we have wake_up() family of APIs as shown below:

// Wakes up all the processes waiting on the queue
wake_up(wake_queue_head_t *);
// Wakes up only the processes performing the interruptible sleep
wake_up_interruptible(wait_queue_head_t *);

Below is modified code from the last article where we use wait queues:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/errno.h>
#include <asm/uaccess.h>
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/delay.h>

#define FIRST_MINOR 0
#define MINOR_CNT 1

static char flag = 'n';
static dev_t dev;
static struct cdev c_dev;
static struct class *cl;
static DECLARE_WAIT_QUEUE_HEAD(wq);

int open(struct inode *inode, struct file *filp)
{
	printk(KERN_INFO "Inside open\n");
	return 0;
}

int release(struct inode *inode, struct file *filp) 
{
	printk (KERN_INFO "Inside close\n");
	return 0;
}

ssize_t read(struct file *filp, char *buff, size_t count, loff_t *offp) 
{
	printk(KERN_INFO "Inside read\n");
	printk(KERN_INFO "Scheduling Out\n");
	wait_event_interruptible(wq, flag == 'y');
	flag = 'n';
	printk(KERN_INFO "Woken Up\n");
	return 0;
}

ssize_t write(struct file *filp, const char *buff, size_t count, loff_t *offp) 
{   
	printk(KERN_INFO "Inside write\n");
	if (copy_from_user(&flag, buff, 1))
	{
		return -EFAULT;
	}
	printk(KERN_INFO "%c", flag);
	wake_up_interruptible(&wq);
	return count;
}

struct file_operations pra_fops = {
	read:        read,
	write:       write,
	open:        open,
	release:     release
};

int wq_init (void)
{
	int ret;
	struct device *dev_ret;

	if ((ret = alloc_chrdev_region(&dev, FIRST_MINOR, MINOR_CNT, "SCD")) < 0)
	{
		return ret;
	}
	printk("Major Nr: %d\n", MAJOR(dev));

	cdev_init(&c_dev, &pra_fops);

	if ((ret = cdev_add(&c_dev, dev, MINOR_CNT)) < 0)
	{
		unregister_chrdev_region(dev, MINOR_CNT);
		return ret;
	}

	if (IS_ERR(cl = class_create(THIS_MODULE, "chardrv")))
	{
		cdev_del(&c_dev);
		unregister_chrdev_region(dev, MINOR_CNT);
		return PTR_ERR(cl);
	}
	if (IS_ERR(dev_ret = device_create(cl, NULL, dev, NULL, "mychar%d", 0)))
	{
		class_destroy(cl);
		cdev_del(&c_dev);
		unregister_chrdev_region(dev, MINOR_CNT);
		return PTR_ERR(dev_ret);
	}
	return 0;
}

void wq_cleanup(void)
{
	printk(KERN_INFO "Inside cleanup_module\n");
	device_destroy(cl, dev);
	class_destroy(cl);
	cdev_del(&c_dev);
	unregister_chrdev_region(dev, MINOR_CNT);
}

module_init(wq_init);
module_exit(wq_cleanup);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Pradeep");
MODULE_DESCRIPTION("Waiting Process Demo");

As seen, the earlier manual waiting has been replaced by single statement wait_event_interruptible() which is more robust.

Below is the sample run of the above program, assuming that the module is compiled as wait.ko:

$ insmod wait.ko
Major Nr: 250
$ cat /dev/mychar0
Inside open
Inside read
Scheduling out

This gets our process blocked. Open another shell to wake up the process:

$ echo 'y' > /dev/mychar0
Inside open
Inside write
y
Inside close
Woken up
Inside close

As seen above, this will wake up the process, since the condition of flag being ‘y’ is satisfied.

Next Article >>

   Send article as PDF   

Waiting / Blocking in Linux Driver

<< Previous Article

Continuing our journey with Linux kernel internals, the next few articles in this series will focus on wait mechanisms in kernel. Now, you might be wondering, why do we need to wait in Linux driver? Well, there can be quite a lot of reasons to wait in driver. Let’s say you are interfacing with hardware such as LCD, which requires you to wait for 5ms before sending a subsequent command. Another example is say you want to read the data from disk, and since disk is a slower device, it may require you to wait until valid data is available. In these scenarios, we have no option, but to wait. One of the simplest way to implement the wait is a busy loop, but it might not be efficient way of waiting. So, does kernel provide any efficient mechanisms to wait? Yes, of course, kernel does provide a variety of mechanisms for waiting. Read on to get the crux of waiting in Linux kernel.

Process States in Linux

Before moving on to the wait mechanisms, it would be worthwhile to understand the process states in Linux. At any point of time, a process can be in any of the below mentioned states:

  • TASK_RUNNING :- Process is in run queue or is running
  • TASK_STOPPED :- Process stopped by debugger
  • TASK_INTERRUPTIBLE :- Process is waiting for some event, but can be woken up by signal
  • TASK_UNINTERRUPTIBLE :- Similar to TASK_INTERRUPTIBLE, but can’t be woken up by signal
  • TASK_ZOMBIE :- Process is terminated, but not cleaned up yet

For a process to be scheduled, it needs to be in TASK_RUNNING state, while TASK_INTERRUPTIBLE and TASK_INTERRUPTIBLE states correspond to a waiting process.

Wait Mechanism in Linux Kernel

API schedule() provides the basic wait mechanism in the linux kernel. Invoking this API yields the processor and invokes the scheduler to schedule any other process in run queue. Below is the programming example for the same:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/device.h>
#include <asm/uaccess.h>
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/delay.h>

#define FIRST_MINOR 0
#define MINOR_CNT 1

static dev_t dev;
static struct cdev c_dev;
static struct class *cl;

int open(struct inode *inode, struct file *filp)
{
	printk(KERN_INFO "Inside open\n");
	return 0;
}

int release(struct inode *inode, struct file *filp)
{
	printk(KERN_INFO "Inside close\n");
	return 0;
}

ssize_t read(struct file *filp, char *buff, size_t count, loff_t *offp)
{
	printk(KERN_INFO "Inside read\n");
	printk(KERN_INFO "Scheduling out\n");
	schedule();
	printk(KERN_INFO "Woken up\n");
	return 0;
}

ssize_t write(struct file *filp, const char *buff, size_t count, loff_t *offp)
{
	printk(KERN_INFO "Inside Write\n");
	return 0;
}

struct file_operations fops =
{
	.read = read,
	.write = write,
	.open = open,
	.release = release
};

int schd_init (void)
{
	int ret;
	struct device *dev_ret;

	if ((ret = alloc_chrdev_region(&dev, FIRST_MINOR, MINOR_CNT, "wqd")) < 0)
	{
		return ret;
	}
	printk("Major Nr: %d\n", MAJOR(dev));

	cdev_init(&c_dev, &fops);

	if ((ret = cdev_add(&c_dev, dev, MINOR_CNT)) < 0)
	{
		unregister_chrdev_region(dev, MINOR_CNT);
		return ret;
	}

	if (IS_ERR(cl = class_create(THIS_MODULE, "chardrv")))
	{
		cdev_del(&c_dev);
		unregister_chrdev_region(dev, MINOR_CNT);
		return PTR_ERR(cl);
	}
	if (IS_ERR(dev_ret = device_create(cl, NULL, dev, NULL, "mychar%d", 0)))
	{
		class_destroy(cl);
		cdev_del(&c_dev);
		unregister_chrdev_region(dev, MINOR_CNT);
		return PTR_ERR(dev_ret);
	}
	return 0;
}

void schd_cleanup(void)
{
	printk(KERN_INFO " Inside cleanup_module\n");
	device_destroy(cl, dev);
	class_destroy(cl);
	cdev_del(&c_dev);
	unregister_chrdev_region(dev, MINOR_CNT);
}

module_init(schd_init);
module_exit(schd_cleanup);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Pradeep Tewani");
MODULE_DESCRIPTION("Waiting Process Demo");

Example above is a simple character driver demonstrating the use of schedule() API. In read() function, we invoke schedule() to yield the processor. Below is the sample run, assuming that the above program is compiled as sched.ko:

$ insmod sched.ko
Major Nr: 244
$ cat /dev/mychar0
Inside open
Inside read
Scheduling out
Woken up

So, what do we get? Does the usage of schedule() API serves the purpose of waiting? Not really. Why is it so?  Well, if you recall the definition of schedule(), it states that the process invoking this API voluntarily yields the processor, but only yielding the processor is not enough. Process is still in run queue and as long as process is in run queue, it would be scheduled again to run. This is exactly what happens in the above example, which makes our process to come out of wait quite immediately. So, the pre-condition for performing the wait with schedule() is to first move the process out of the run queue. How do we achieve this? For this, we have a API called set_current_state(). Below is the modified code snippet for the read() function:

ssize_t read(struct file *filp, char *buff, size_t count, loff_t *offp)
{
	printk(KERN_INFO "Inside read\n");
	printk(KERN_INFO "Scheduling out\n");
	set_current_state(TASK_INTERRUPTIBLE);
	schedule();
	printk(KERN_INFO "Woken up\n");
	return 0;
}

In the above example, before invoking the schedule() API, we are setting the state of the process to TASK_INTERRUPTIBLE. This will move the process out of run queue and hence it won’t be scheduled again to run. Below is the sample run for the modified example:

$ insmod sched.ko
Major Nr: 250
$ cat /dev/mychar0
Inside open
Inside read
Scheduling out

Conclusion

So, finally we are able to get the process blocked. But do you see the problem with this code? This process is indefinitely blocked. When and who will wake this process up? Another thing worth noting is that in real life scenarios, process always waits on some event, but our process is put to an unconditional sleep. How do we make a process wait on an event? To find out the answer to these questions, stay tuned to my next article. Till then, Happy Waiting!

Next Article >>

   Send article as PDF