Tag Archives: OSFY

Linux Character Drivers

This fourth article, which is part of the series on Linux device drivers, deals with the various concepts of character drivers and their implementation.

<< Third Article

Shweta at her hostel room in front of her PC, all set to explore the characters of Linux character drivers, before it is being taught in the class. She recalled the following lines from professor Gopi’s class: “… today’s first driver would be the template to any driver you write in Linux. Writing any specialized advanced driver is just a matter of what gets filled into its constructor & destructor. …”. With that, she took out the first driver code, and popped out various reference books to start writing a character driver on her own. She also downloaded the on-line “Linux Device Drivers” book by Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman from http://lwn.net/Kernel/LDD3/. Here follows the summary from her various collations.

W’s of character drivers

We already know what are drivers and why we need them. Then, what is so special about character drivers? If we write drivers for byte-oriented operations or in the C-lingo the character-oriented operations, we refer to them as character drivers. And as the majority of devices are byte-oriented, the majority of device drivers are character device drivers. Take for example, serial drivers, audio drivers, video drivers, camera drivers, basic I/O drivers, …. In fact, all device drivers which are neither storage nor network device drivers are one form or the other form of character drivers. Let’s look into the commonalities of these character drivers and how Shweta wrote one of them.

Figure 7: Character driver overview

The complete connection

As shown in Figure 7, for any application (user space) to operate on a byte-oriented device (hardware space), it should use the corresponding character device driver (kernel space). And the character driver usage is done through the corresponding character device file(s), linked to it through the virtual file system (VFS). What it means is that an application does the usual file operations on the character device file – those operations are translated to the corresponding functions into the linked character device driver by the VFS – those functions then does the final low level access to the actual devices to achieve the desired results. Note that though the application does the usual file operations, their outcome may not be the usual ones. Rather, they would be as driven by the corresponding functions in the device driver. For example, a read followed by a write may not fetch what has been written into, unlike in the case of regular files. Note that this is the usual expected behaviour for device files. Let’s take an audio device file as an example. What we write into it is the audio data we want to playback, say through a speaker. However, the read would get us the audio data we are recording, say through a microphone. And the recorded data need not be the played back data.

In this complete connection from application to the device, there are four major entities involved:

Application
Character device file
Character device driver
Character device

And the interesting thing is that, all of these can exist independently on a system, without the other being there. So, mere existence of these on a system doesn’t mean they are linked to form the complete connection. Rather, they need to be explicitly connected. Application gets connected to a device file by invoking open system call on the device file. Device file(s) are linked to the device driver by specific registrations by the driver. And the device driver is linked to a device by its device-specific low-level operations. Thus, forming the complete connection. With this, note that the character device file is not the actual device but just a placeholder for the actual device.

Major & minor number

Connection between the application and the device file is based on the name of the device file. However, the connection between the device file and the device driver is based on the number of the device file, not the name. This allows a user-space application to have any name for the device file, and enables the kernel-space to have trivial index-based linkage between the device file & the device driver. This device file number is more commonly referred as the <major, minor> pair, or the major & minor numbers of the device file. Earlier (till kernel 2.4), one major number was for one driver, and the minor number used to represent the sub-functionalities of the driver. With kernel 2.6, this distinction is no longer mandatory – there could be multiple drivers under same major number but obviously with different minor number ranges. However, this is more common with the non-reserved major numbers and standard major numbers are typically preserved for single drivers. For example, 4 for serial interfaces, 13 for mice, 14 for audio devices, …. The following command would list the various character device files on your system:

$ ls -l /dev/ | grep “^c”

<major, minor> related support in kernel 2.6

Type: (defined in kernel header <linux/types.h>)

dev_t // contains both major & minor numbers

Macros: (defined in kernel header <linux/kdev_t.h>)

MAJOR(dev_t dev) // extracts the major number from dev
MINOR(dev_t dev) // extracts the minor number from dev
MKDEV(int major, int minor) // creates the dev from major & minor

Connecting the device file with the device driver involves two steps:

Registering for the <major, minor> range of device files
Linking the device file operations to the device driver functions

First step is achieved using either of the following two APIs: (defined in kernel header <linux/fs.h>)

int register_chrdev_region(dev_t first, unsigned int cnt, char *name);
int alloc_chrdev_region(
	dev_t *first, unsigned int firstminor, unsigned int cnt, char *name);

First API registers the cnt number of device file numbers starting from first, with the name. Second API dynamically figures out a free major number and registers the cnt number of device file numbers starting from <the free major, firstminor>, with the name. In either case, the /proc/devices kernel window lists the name with the registered major number. With this information, Shweta added the following into the first driver code.

#include <linux/types.h>
#include <linux/kdev_t.h>
#include <linux/fs.h>

static dev_t first; // Global variable for the first device number

In the constructor, she added:

int ret;

if ((ret = alloc_chrdev_region(&first, 0, 3, "Shweta")) < 0)
{
	return ret;
}
printk(KERN_INFO "<Major, Minor>: <%d, %d>\n", MAJOR(first), MINOR(first));

In the destructor, she added:

unregister_chrdev_region(first, 3);

Putting it all together, it becomes:

#include <linux/module.h>
#include <linux/version.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/kdev_t.h>
#include <linux/fs.h>

static dev_t first; // Global variable for the first device number

static int __init ofcd_init(void) /* Constructor */
{
	int ret;

	printk(KERN_INFO "Namaskar: ofcd registered");
	if ((ret = alloc_chrdev_region(&first, 0, 3, "Shweta")) < 0)
	{
		return ret;
	}
	printk(KERN_INFO "<Major, Minor>: <%d, %d>\n", MAJOR(first), MINOR(first));
	return 0;
}

static void __exit ofcd_exit(void) /* Destructor */
{
	unregister_chrdev_region(first, 3);
	printk(KERN_INFO "Alvida: ofcd unregistered");
}

module_init(ofcd_init);
module_exit(ofcd_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Anil Kumar Pugalia <email@sarika-pugs.com>");
MODULE_DESCRIPTION("Our First Character Driver");

Then, Shweta repeated the usual steps, she learnt for the first driver

Build the driver (.ko file) by typing make
Load the driver using insmod
List the loaded modules using lsmod
Unload the driver using rmmod

Summing up

Additionally, before unloading the driver, she peeped into the kernel window /proc/devices to look for the registered major number with the name “Shweta” using cat /proc/devices. It was right there. But she couldn’t find any device file created under /dev with the same major number. So, she created them by hand using mknod, and then tried reading & writing those. Figure 8 shows all these. Please note that the major number “250” may vary from system to system based on the availability. Figure 8 also shows the results, Shweta got from reading & writing one of the device files. That reminded her that the second step for connecting the device file with the device driver – “Linking the device file operations to the device driver functions” is not yet done. She realized that she needs to dig further information to complete this step and also to figure out the reason for the missing device files under /dev. We shall continue further in our next article, to figure out what more is Shweta learning and how is she going ahead with her first character driver.

Figure 8: Character device file experiments

Fifth Article >>

Getting Recursive with Bench Calculator

2 Replies

This fourth article of the mathematical journey through open source, takes you through the recursive functional power of bench calculator.

<< Third Article

Equipped with the fundamentals of programming functions, we are all set to jump into the spiral of recursion. Frankly speaking recursion is not a spiral, as many would like to see it as. It is more to do with the way our brain works. So, let’s look into it as our brain suggests. Any recursion fundamentally has two parts: the recursive relation and the termination condition(s). For mathematics, recursion is basically to do with functions.

Recursive relation

A function when expressed or defined using the function itself, but of lower order, we refer that as a recursive relation. Typical lower order examples would be f(n) expressed in f(n – 1), f(n – 2), … where n is an integer; f(x) expressed in f(x / 2), … where x is a real number. In case of mathematics, a recursive relation is also called a recurrence relation. Here goes some examples:

Factorial: n! = n * (n – 1)! or fact(n) = n * fact(n – 1)
Fibonacci: fib(n) = fib(n – 1) + fib(n – 2)
Exponent: e^x = (e^x/2)² or exp(x) = (exp(x/2))^2

Termination Condition

Now, the point is, as we all understand, if we try to expand these, they will continue for ever, unless we decide to stop at some decent condition. For example, n! = n * (n-1)! = n * (n-1) * (n-2)! = n * (n-1) * (n-2) * (n-3)! = … and so on. So, we decide and say that let’s stop it, when we reach 0, as we assume n to be positive. And hence, n! becomes n * (n-1) * (n-2) * … * 2 * 1 * 0!. And then we define 0! to be 1. This is called the termination condition. And, we would notice that, whenever there is a recursive relation, we would need one or more such termination conditions to stop the relation going forever. The number of termination conditions typically depend on the different lower orders, we have in the recursive relation. For example, factorial of n has just one lower order factorial of (n-1); exponent of x has again just one lower order exponent of (x/2); but Fibonacci of n has two lower orders Fibonacci of (n-1) and Fibonacci of (n-2). Hence, factorial and exponent would have one termination condition and Fibonacci would have two termination conditions.

Together in formation

Based on this understanding, the above recursive relations can be written, as follows:

For factorial:
fact(n) = n * fact(n – 1), for n > 0
fact(n) = 1, for n = 0
For Fibonacci:
fib(n) = fib(n – 1) + fib(n – 2), for n > 1
fib(n) = 1, for n = 1
fib(n) = 0, for n = 0
For exponent:
exp(x) = (exp(x/2))^2, for x > 0.001
exp(x) = 1 + x, for x <= 0.001

What is this 0.001 for the exponent? Note that x is a real number and if it is positive, dividing by 2, how ever many times would never reach absolute zero, though it would become smaller and smaller approaching zero. Hence, we would never be able to stop, if we put x > 0. So, we take an approximation, depending on expected accuracy of the result. And for this reason, we cannot put e(x) just equal to 1 for x < 0.001, but need to have some variability based on x, and hence 1 + x. Moreover, as it makes sense to have x take all real values (positive, zero, and negative), the conditions could be further enhanced by replacing x by its absolute value. So, for exponent,

exp(x) = (exp(x/2))^2, for abs(x) > 0.001
exp(x) = 1 + x, for abs(x) <= 0.001

Together in action

Given that we have these mathematical formulations, putting them down into recursive functions of a programming language is a mere mechanical task. Here, it goes – all three in a single file recursion.bc:

define fact(n) {
    if (n <= 0) # < to avoid -ve value cases
        return 1
    return (n * fact(n - 1)) 
}
define fib(n) {
    if (n <= 0) # < to avoid -ve value cases
        return 0
    if (n == 1)
        return 1
    return (fib(n - 1) + fib(n - 2))
}
define abs(x) {
    if (x >= 0)
        return x
    return -x
}
define exp(x) {
    if (abs(x) <= 0.001)
        return (1 + x)
    return (exp(x/2) * exp(x/2))
}

There execution and usage is demonstrated here:

$ bc -ql recursion.bc
fact(0)
1
fact(5)
120
fib(1)
1
fib(2)
1
fib(3)
2
fib(4)
3
fib(5)
5
fib(6)
8
exp(0)
1
exp(1) # Shall be an approximation
2.71695572946643553835
e(1)
2.71828182845904523536
quit # Get out
$

Summing up

With all these, we have pretty much explored all the functionalities of the bench calculator, and if you are a programming geek, you may even do the next level of stuff with these. Next level stuff? Yes, I mean jumping to the next dimension, vectors, etc using arrays of bc. Aha! But isn’t there a way for non-programmers? Yes, there is octave.

Fifth Article >>

Kernel C Extras in a Linux Driver

2 Replies

This third article, in the series on Linux device drivers deals with the kernel’s message logging,
and kernel-specific GCC extensions.

<< Second Article

Enthused by how Pugs impressed professor Gopi, in the last class, Shweta decided to do something similar. And there was already an opportunity – finding out where has the output of printk gone. So, as soon as she entered the lab, she got hold of the best located system, logged into it, and took charge. Knowing her professor pretty well, she knew that there would be a hint for the finding, from the class itself. So, she flashed back what all the professor taught, and suddenly remembered the error output demonstration from “insmod vfat.ko” – dmesg | tail. She immediately tried that and for sure found out the printk output, there. But how did it come here? A tap on her shoulder brought her out of the thought. “Shall we go for a coffee?”, proposed Pugs. “But I need to …”. “I know what you are thinking about.”, interrupted Pugs. “Let’s go, yaar. I’ll explain you all about dmesg”.

Kernel’s message logging

On the coffee table, Pugs began:

As far as parameters are concerned, printf & printk are same, except that when programming for the kernel we don’t bother about the float formats of %f, %lf & their likes. However unlike printf, printk is not destined to dump its output on some console. In fact, it cannot do so, as it is something which is in the background, and executes like a library, only when triggered either from the hardware space or the user space. So, then where does printk print? All the printk calls, just put their contents into the (log) ring buffer of the kernel. Then, the syslog daemon running in the user space picks them for final processing & redirection to various devices, as configured in its configuration file /etc/syslog.conf.

You must have observed the out of place macro KERN_INFO, in the printk calls, in the previous article. That actually is a constant string, which gets concatenated with the format string after it, making it a single string. Note that there is no comma (,) between them – they are no two separate arguments. There are eight such macros defined in <linux/kernel.h> under the kernel source, namely:

#define KERN_EMERG	"<0>" /* system is unusable			*/
#define KERN_ALERT	"<1>" /* action must be taken immediately	*/
#define KERN_CRIT	"<2>" /* critical conditions			*/
#define KERN_ERR	"<3>" /* error conditions			*/
#define KERN_WARNING	"<4>" /* warning conditions			*/
#define KERN_NOTICE	"<5>" /* normal but significant condition	*/
#define KERN_INFO	"<6>" /* informational				*/
#define KERN_DEBUG	"<7>" /* debug-level messages			*/

Depending on these log levels (i.e. the first 3 characters in the format string), the syslog daemon in the user space redirects the corresponding messages to their configured locations – a typical one being the log file /var/log/messages for all the log levels. Hence, all the printk outputs are by default in that file. Though, they can be configured differently to say serial port (/dev/ttyS0) or say all consoles, like what happens typically for KERN_EMERG. Now, /var/log/messages is buffered & contain messages not only from the kernel but also from various daemons running in the user space. Moreover, the /var/log/messages most often is not readable by a normal user, and hence a user-space utility ‘dmesg‘ is provided to directly parse the kernel ring buffer and dump it on the standard output. Figure 6 shows the snippets from the two.

Figure 6: Kernel’s message logging

Kernel-specific GCC extensions

With all these Shweta got frustrated, as she wanted to find all these by her own, and then do a impression in the next class – but all flop. Pissed off, she said, “So as you have explained all about printing in kernel, why don’t you tell about the weird C in the driver as well – the special keywords __init, __exit, etc.”

These are not any special keywords. Kernel C is not any weird C but just the standard C with some additional extensions from the C compiler gcc. Macros __init and __exit are just two of these extensions. However, these do not have any relevance in case we are using them for dynamically loadable driver, but only when the same code gets built into the kernel. All the functions marked with __init get placed inside the init section of the kernel image and all functions marked with __exit are placed inside the exit section of the kernel image, automatically by gcc, during kernel compilation. What is the benefit? All functions with __init are supposed to be executed only once during boot-up, till the next boot-up. So, once they are executed during boot-up, kernel frees up RAM by removing them by freeing up the init section. Similarly, all functions in exit section are supposed to be called during system shutdown. Now, if system is shutting down anyway, why do you need to do any cleanups. Hence, the exit section is not even built into the kernel – another cool optimization.

This is a beautiful example of how kernel & gcc goes hand-in-hand to achieve lot of optimizations and many other tricks – we could see others, as we go along. And that is why Linux kernel can be compiled only using gcc-based compilers – a close knit bond.

Kernel function’s return guidelines

While returning from coffee, Pugs started all praises for the OSS & its community. Do you know why different individuals are able to come together and contribute excellently without any conflicts – moreover in a project as huge as Linux? There are many reasons. But definitely, one of the strong reasons is, following & abiding by the inherent coding guidelines. Take for example the guideline for returning values from a function in kernel programming.

Any kernel function needing error handling, typically returns an integer-like type and the return value again follows a guideline. For an error, we return a negative number – a minus sign appended with a macro included through the kernel header <linux/errno.h>, that includes the various error number headers under the kernel sources, namely <asm/errno.h>, <asm-generic/errno.h>, <asm-generic/errno-base.h>. For success, zero is the most common return value, unless there is some additional information to be provided. In that case, a positive value is returned, the value indicating the information like number of bytes transferred.

Kernel C = Pure C

Once back into the lab, Shweta remembered their professor mentioning that no /usr/include headers can be used for kernel programming. But Pugs said that kernel C is just standard C with some gcc extensions. Why this conflict? Actually this is not a conflict. Standard C is just pure C – just the language. The headers are not part of it. Those are part of the standard libraries built in C for C programmers, based on the concept of re-using code. Does that mean, all standard libraries and hence all ANSI standard functions are not part of ‘pure’ C? Yes. Then, hadn’t it been really tough coding the kernel. Not for this reason. In reality, kernel developers have developed their own needed set of functions, and they are all part of the kernel code. printk is just one of them. Similarly, many string functions, memory functions, … are all part of the kernel source under various directories like kernel, ipc, lib, … and the corresponding headers under include/linux directory.

“O ya! That is why we need to have kernel source for building a driver”, affirmed Shweta. “If not the complete source, at least the headers are a must. And that is why we have separate packages to install complete kernel source or just the kernel headers”, added Pugs. “In the lab, all the sources are setup. But if I want to try out drivers on my Linux system at my hostel room, how do I go about it?” asked Shweta. “Our lab have Fedora, where the kernel sources typically get installed under /usr/src/kernels/<kernel_version> unlike the standard place /usr/src/linux. Lab administrators must have installed it using command line ‘yum install kernel-devel‘. I use Mandriva and installed the kernel sources using ‘urpmi kernel-source‘, replied Pugs. “But, I have Ubuntu”. “Okay!! For that just use apt-get install – possibly, ‘apt-get install linux-source‘”.

Summing up

Lab timings were just getting over. Suddenly, Shweta put out her curiosity – “Hey Pugs! What is the next topic we are going to learn in our Linux device drivers class?”. “Hmmm!! Most probably character drivers”. With this information, Shweta hurriedly packed up her bag & headed towards her room to setup the kernel sources and try out the next driver on her own. “In case you are stuck up, just give me a call. I’ll be there”, called up Pugs from the behind with a smile.

Fourth Article >>

Notes:

The default syslog file /var/log/messages may vary from distro to distro. For example, in the latest Ubuntu distros, it is /var/log/syslog.

Other References:

Another possible pointer to the missing /var/log/messages in Ubuntu

Bench Calculator to Program Mathematics

5 Replies

This third article of the mathematical journey through open source, takes you through the functional power of bench calculator.

<< Second Article

After going through the basic programming on bench calculator, here’s the time to explore its functional power. As mentioned earlier, we can do functions in bench calculator. Unlike C, it has built-in functions, though the standard math functions and the user-defined functions are similar to as in C.

Built-in Functions

Complete list of bc‘s built-in functions is:

length(expr) – returns the number of significant digits in expr
read() – reads a number from standard input in the base dictated by the ibase variable
scale(expr) – returns the number of digits after the decimal point in expr
sqrt(expr) – returns the positive square root of expr, given that expr is non-negative

Here’s a sample execution of the above functions:

$ bc -ql
length(000023.450) # Number of significant digits
5
scale(000023.450) # Number of digits after the decimal
3
sqrt(2) # Square root of 2
1.41421356237309504880
sqrt(-1) # Square root of -1 is an error
Runtime error (func=(main), adr=4): Square root of a negative number
ibase=2 # Changing the input base to 2
x=read() # Wait to read the input in binary and then display
1100 # This is the input
x # Display the read value in the default output base 10
12
quit # Get out
$

Is that the complete list of built-in functions? But no talk of the previously used print. The reason is that print is not a function – didn’t you notice the missing () with print. print is actually a statement in bc, like if, for, … and the syntax of print is: print <list>, where <list> is a comma separated list of strings and expressions

If you have not yet got the hang of this word expression, it is a statement of numbers and variables operated with the various operators and functions.

Standard Math Functions

When bc is invoked with -l option, the math library gets loaded along with. And the following 6 math functions, also get available to use:

s(x) – returns sine of x, x is radians
c(x) – returns cosine of x, x is radians
a(x) – returns arctangent (in radians) of x
l(x) – returns the natural logarithm (base e) of x
e(x) – returns the value of e raised to the power of x
j(n, x) – Bessel function of integer order n of x

All these functions operate with the scale dictated by the built-in variable scale. By default, scale is set to 20. Here’s a sample execution:

$ bc -ql
scale # Show the current scale
20
pi=4*a(1) # Calculate pi as tan^-1(1) is pi / 4
pi # Show the value approx. to 20 decimals
3.14159265358979323844
s(pi/3) # Calculate sine of 60° - should sqrt(3)/2
.86602540378443864675
sqrt(3)/2 # value for comparison – note the approx. error
.86602540378443864676
c(pi/3) # Calculate sine of 60° - should be 0.5
.50000000000000000001
l(1) # log(1)
0
e(1) # Value of e¹ approx. to 20 decimals
2.71828182845904523536
quit # Get out
$

This all sounds too geeky and mathematical – all going over the head. Okay, let’s forget about that and do some simple stuff. Let’s write our own simple functions – yes user-defined functions.

User-defined functions

Here is how we write a user-defined function (to add two numbers) in bc:

$ bc -ql
define add(x, y) {
	return (x + y)
}
add(3, add(4, 5)) # Lets add 3 with the sum of 4 & 5
12
quit # Get out
$

Given that, the factorial code from our previous learnings can be converted into a function as follows (say in functions.bc):

define factorial(n) {
	product = 1
	for (current_num = 1; current_num <= n; current_num += 1)
	{
		product *= current_num
	}
	return product
}

And then, we can use that function as follows:

$ bc -ql functions.bc # Load the functions while invoking bc
factorial(10) # Compute the factorial of 10
3628800
quit # Get out
$

As now, we have factorial, we can even calculate the series of e, i.e. 1 + 1/1! + 1/2! + …, say upto 1/20! for a good enough approximation. Here’s how it would go

$ bc -ql functions.bc # Load the functions while invoking bc
exp=1
for (i = 1; i <= 20; i++)
{
	exp += (1/factorial(i))
}
exp # Display the computed value of e
2.71828182845904523525
e(1) # Compare with the standard math function
2.71828182845904523536
quit # Get out
$

And as in C, if we need a function only to do actions and not return anything, void is the way.

$ bc -ql
define void designer_print(v) {
	print "---{", v, "}---"
}
designer_print(100) # Print 100 with the designs
---{100}---
quit # Get out
$

With all these fundamentals of functions in bc, next we would dive into its recursive functional power.

Fourth Article >>

Writing your First Linux driver in the Classroom

7 Replies

This second article, which is part of the series on Linux device drivers, deals with the concept of dynamically loading drivers, first writing a Linux driver, before building and then loading it.

<< First Article

As Shweta and Pugs reached their classroom, they were already late. Their professor was already in there. They looked at each other and Shweta sheepishly asked, “May we come in, sir”. “C’mon!!! you guys are late again”, called out professor Gopi. “And what is your excuse, today?”. “Sir, we were discussing your topic only. I was explaining her about device drivers in Linux”, was a hurried reply from Pugs. “Good one!! So, then explain me about dynamic loading in Linux. You get it right and you two are excused”, professor emphasized. Pugs was more than happy. And he very well knew, how to make his professor happy – criticize Windows. So, this is what he said.

As we know, a typical driver installation on Windows needs a reboot for it to get activated. That is really not acceptable, if we need to do it, say on a server. That’s where Linux wins the race. In Linux, we can load (/ install) or unload (/ uninstall) a driver on the fly. And it is active for use instantly after load. Also, it is disabled with unload, instantly. This is referred as dynamic loading & unloading of drivers in Linux.

As expected he impressed the professor. “Okay! take your seats. But make sure you are not late again”. With this, the professor continued to the class, “So, as you now already know, what is dynamic loading & unloading of drivers into & out of (Linux) kernel. I shall teach you how to do it. And then, we would get into writing our first Linux driver today”.

Dynamically loading drivers

These dynamically loadable drivers are more commonly referred as modules and built into individual files with .ko (kernel object) extension. Every Linux system has a standard place under the root file system (/) for all the pre-built modules. They are organized similar to the kernel source tree structure under /lib/modules/<kernel_version>/kernel, where <kernel_version> would be the output of the command “uname -r” on the system. Professor demonstrates to the class as shown in Figure 4.

Figure 4: Linux pre-built modules

Now, let us take one of the pre-built modules and understand the various operations with it.

Here’s a list of the various (shell) commands relevant to the dynamic operations:

lsmod – List the currently loaded modules
insmod <module_file> – Insert/Load the module specified by <module_file>
modprobe <module> – Insert/Load the <module> along with its dependencies
rmmod <module> – Remove/Unload the <module>

These reside under the /sbin directory and are to be executed with root privileges. Let us take the FAT file system related drivers for our experimentation. The various module files would be fat.ko, vfat.ko, etc. under directories fat (& vfat for older kernels) under /lib/modules/`uname -r`/kernel/fs. In case, they are in compressed .gz format, they need to be uncompressed using gunzip, for using with insmod. vfat module depends on fat module. So, fat.ko needs to be loaded before vfat.ko. To do all these steps (decompression & dependency loading) automatically, modprobe can be used instead. Observe that there is no .ko for the module name to modprobe. rmmod is used to unload the modules. Figure 5 demonstrates this complete experimentation.

Figure 5: Linux module operations

Our first Linux driver

With that understood, now let’s write our first driver. Yes, just before that, some concepts to be set right. A driver never runs by itself. It is similar to a library that gets loaded for its functions to be invoked by the “running” applications. And hence, though written in C, it lacks the main() function. Moreover, it would get loaded / linked with the kernel. Hence, it needs to be compiled in similar ways as the kernel. Even the header files to be used can be picked only from the kernel sources, not from the standard /usr/include.

One interesting fact about the kernel is that it is an object oriented implementation in C. And it is so profound that we would observe the same even with our first driver. Any Linux driver consists of a constructor and a destructor. The constructor of a module gets called whenever insmod succeeds in loading the module into the kernel. And the destructor of the module gets called whenever rmmod succeeds in unloading the module out of the kernel. These two are like normal functions in the driver, except that they are specified as the init & exit functions, respectively by the macros module_init() & module_exit() included through the kernel header module.h

/* ofd.c – Our First Driver code */

#include <linux/module.h>
#include <linux/version.h>
#include <linux/kernel.h>

static int __init ofd_init(void) /* Constructor */
{
	printk(KERN_INFO "Namaskar: ofd registered");

	return 0;
}

static void __exit ofd_exit(void) /* Destructor */
{
	printk(KERN_INFO "Alvida: ofd unregistered");
}

module_init(ofd_init);
module_exit(ofd_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Anil Kumar Pugalia <email@sarika-pugs.com>");
MODULE_DESCRIPTION("Our First Driver");

Above is the complete code for our first driver, say ofd.c. Note that there is no stdio.h (a user space header), instead an analogous kernel.h (a kernel space header). printk() being the printf() analogous. Additionally, version.h is included for version compatibility of the module with the kernel into which it is going to get loaded. Also, the MODULE_* macros populate the module related information, which acts like the module’s signature.

Building our first Linux driver

Once we have the C code, it is time to compile it and create the module file ofd.ko. And for that we need to build it in the similar way, as the kernel. So, we shall use the kernel build system to do the same. Here follows our first driver’s Makefile, which would invoke the kernel’s build system from the kernel source. The kernel’s Makefile would in turn invoke our first driver’s Makefile to build our first driver. The kernel source is assumed to be installed at /usr/src/linux. In case of it to be at any other location, the KERNEL_SOURCE variable has to be appropriately updated.

# Makefile – makefile of our first driver

# if KERNELRELEASE is not defined, we've been called directly from the command line.
# Invoke the kernel build system.
ifeq (${KERNELRELEASE},)
	KERNEL_SOURCE := /usr/src/linux
	PWD := $(shell pwd)
default:
	${MAKE} -C ${KERNEL_SOURCE} SUBDIRS=${PWD} modules

clean:
	${MAKE} -C ${KERNEL_SOURCE} SUBDIRS=${PWD} clean

# Otherwise KERNELRELEASE is defined; we've been invoked from the
# kernel build system and can use its language.
else
	obj-m := ofd.o
endif

Note 1: Makefiles are very space-sensitive. The lines not starting at the first column have a tab and not spaces.

Note 2: For building a Linux driver, you need to have the kernel source (or at the least the kernel headers) installed on your system.

With the C code (ofd.c) and Makefile ready, all we need to do is put them in a (new) directory of its own, and then invoke make in that directory to build our first driver (ofd.ko).

$ make
make -C /usr/src/linux SUBDIRS=... modules
make[1]: Entering directory `/usr/src/linux'
CC [M] .../ofd.o
Building modules, stage 2.
MODPOST 1 modules
CC .../ofd.mod.o
LD [M] .../ofd.ko
make[1]: Leaving directory `/usr/src/linux'

Summing up

Once we have the ofd.ko file, do the usual steps as root, or with sudo.

# su
# insmod ofd.ko
# lsmod | head -10

lsmod should show you the ofd driver loaded.

While the students were trying their first module, the bell rang, marking the end for this session of the class. And professor Gopi concluded, saying “Currently, you may not be able to observe anything, other than “lsmod” listing showing our first driver loaded. Where’s the printk output gone? Find that out for yourself in the lab session and update me with your findings. Moreover, today’s first driver would be the template to any driver you write in Linux. Writing any specialized advanced driver is just a matter of what gets filled into its constructor & destructor. So, here onwards, our learnings shall be in enhancing this driver to achieve our specific driver functionalities.”

Notes

In most of today’s distros, one may safely have KERNEL_SOURCE set to /lib/modules/$(shell uname -r)/build, instead of /usr/src/linux i.e. KERNEL_SOURCE := /lib/modules/$(shell uname -r)/build in the Makefile.

Third Article >>

Explore the Power of the Bench Calculator

6 Replies

This second article of the mathematical journey through open source, takes you through the basics of bench calculator.

<< First Article

Faced with the limitations of the shell command expr and other shell constructs, here we are all set to explore the powerful command bc. bc stands for bench calculator. It is not just a command or tool but a complete language in itself. And its power lies in its arbitrary precision-ness with not only integers but with real number. Wondering what that means? It just means that its computation is mostly not limited by size of the integer or real number types, unlike in most programming languages. Thus, this is more closer to our day to day dealing of mathematics, keeping away the internal details of computer’s precision blah blah. So, let’s get started with the first usual maths. And then we will move onto more involved one with variables, conditionals, and later on with functions and recursion.

Basic operations

For integer-only math, you may invoke the bench calculator as bc. For a full-fledged real number math, you may invoke it as bc -l. Once invoked, it will print a welcome message and then wait for you to just type your math statements and press Enter to get your answer. For quitting the bench calculator, you need to press Ctrl-D (^D), on an empty line. All the basic arithmetic operations: addition (+), subtraction (-), multiplication (*), quotient (/), remainder (%), power (^), brackets (()) are just there – with the C-like precedence & associativity rules. An example with all of them in use:

$ bc
bc 1.06.95
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
2 + 2 * 3 - 5 + 21 / 4 * 6 # A basic maths statement
33
(2 ^ 2) ^ 3 # Another one with power & bracket
64
^D

Yes, you guessed it right. # starts a one-line comment as in shell, or like // in C++. For a multi-line comment, you may use /* */ as in C/C++. You may want that in case you are writing a complete program in bc. Yes, you can even do that. How? You may put your math statements (each one on a line by itself) in a file, say in prog.bc, as follows:

2 + 2 * 3 - 5 + 21 / 4 * 6 # A basic maths statement
(2 ^ 2) ^ 3 # Another one with power & bracket
quit # This will complete the program and not wait for more input

And then execute – yes I mean execute, you do not need to compile it – as follows:

$ bc prog.bc
bc 1.06.95
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
33
64
$

Aha! You actually got the welcome message from bc and then your results. To avoid the welcome message, you may add -q option to bc, as follows:

$ bc -q prog.bc
33
64
$

You may also try out the difference in the output of the same program with a -l option, i.e. with real numbers. Then, / would be treated as a complete division, not just a quotient provider. Here’s what you would get:

$ bc -ql prog.bc 
34.50000000000000000000
64
$

Programming with bc

As soon as we say programming, the first thing flashing to your mind might be variables. Yes, so they are there. All the variables must start with a small letter alphabet and then may contain numbers or small letter alphabets. Yes, you read it right – only small letter alphabets (a,b,c,…,z). This is because, capital letters (A,B,C,…) are used to represent numbers in other bases greater than 10. Yes, bc have support for various bases from 2 to 16, and two variables associated with them: 1) ibase: defines the base for input; 2) obase: defines the base for output. By default, both are set to 10, as in our day-to-day usual maths. But can be modified, for more fancier base conversions. Here’s a snippet:

$ bc -ql
ibase # Show the current input base
10
obase # Show the current output base
10
obase=16 # Set the output base to 16
108 # Input in base 10; Output should be in base 16
6C
obase=10 # Set the output base back to 10
ibase=16 # Set the input base to 16
11 # Input now in base 16; Output should be in base 10
17
ibase=10 # Set the input base to 16. 10 is 16 in input base 16.
ibase=A # Set the input base to 10.
obase=2 # Set the output base to 2, i.e. binary
x = 2 * 5 - 1 # Set the variable x to 9 (input base 10)
x # Display the value of x (in output base 2)
1001
x * 2 # This should display 18 in base 2, but x is still 9
10010
obase=10
x++ # Post increment: Display the current value of x and then increment it
9
x # Display the incremented value
10
--x # Pre decrement: Decrement x and then display its value
9
^D

From the demo above, you might have already observed that there is nothing like declaring variable type or so – just assign them using = and then use them. Moreover, bc also have basic conditional and loop constructs: if, for, while, break, continue. And along with are the usual C-like relational (<, <=, >, >=, ==, !=), logical (!, ||, &&), and the operation-assignment (-=, +=, *=, /=, %=, ^=) operators. Note of caution: Their precedence and associativity rules may not be as in C. If you do not understand that, forget about it – just make sure to use brackets for whatever you want to operate first. Here goes two simple programs to demonstrate: 1) Computing the sum of the first n numbers (sum.bc); 2) Computing the product of the first n numbers, i.e. factorial of n (factorial.bc)

-------------------------------- file: sum.bc -----------------------------

print "Enter a positive number: "
num = read()
sum = 0
current_num = 1
while (current_num <= num)
{
	sum += current_num
	current_num += 1
}
print "Sum of first ", num, " numbers is: ", sum, "\n"
quit

-------------------------------- file: factorial.bc -----------------------------

print "Enter a positive number: "
num = read()
product = 1
for (current_num = 1; current_num <= num; current_num += 1)
{
	product *= current_num
}
print "Product of first ", num, " numbers is: ", product, "\n"
quit

The above programs can be tried out by issuing shell commands bc -ql sum.bc and bc -ql factorial.bc, respectively. But, what are those two words: print & read, doing there in the code. What else; they are just two built-in functions, displaying the message to the user and taking a number from the user, respectively. Functions? Yes, bc can do functions as well. That’s what would be our next camping. So, right now, just go ahead and try the above programs.

Third Article >>

Linux Device Drivers for your Girl Friend

25 Replies

This is the first article of the series on Linux device drivers, which aims to present the usually technical topic in a way that is more interesting to a wider cross-section of readers.

“After a week of hard work, we finally got our driver working”, was the first line as Pugs met his girl friend Shweta.

“Why? What was your driver upto? Was he sick? And what hard work did you do?”, came a series of question from Shweta with a naughty smile.

Pugs was confused as what was Shweta talking about. “Which driver are you talking about?”, he exclaimed.

“Why are you asking me? You should tell me, which of your drivers, you are talking about?”, replied Shweta.

Pugs clicked, “Ah C’mon! Not my car drivers. I am talking about my device driver written on my computer.”

“I know a car driver, a bus driver, a pilot, a screw driver. But what is this device driver?”, queried Shweta.

And that was all needed to trigger Pugs’ passion to explain the concept of device drivers for a newbie. In particular, the Linux device drivers, which he had been working on since many years.

Of drivers and buses

A driver is one who drives – manages, controls, directs, monitors – the entity under his command. So a bus driver does that with a bus. Similarly, a device driver does that with a device. A device could be any peripheral connected to a computer, for example mouse, keyboard, screen / monitor, hard disk, camera, clock, … – you name it.

A pilot could be a person or automatic systems, possibly monitored by a person. Similarly, device driver could be a piece of software or another peripheral / device, possibly driven by a software. However, if it is an another peripheral / device, it is referred as device controller in the common parlance. And by driver, we only mean the software driver. A device controller is a device itself and hence many a times it also needs a driver, commonly referred as a bus driver.

General examples of device controllers include hard disk controllers, display controllers, audio controller for the corresponding devices. More technical examples would be the controllers for the hardware protocols, such as an IDE controller, PCI controller, USB controller, SPI controller, I2C controller, etc. Pictorially, this whole concept can be depicted as in figure 1.

Figure 1: Device & driver interaction

Device controllers are typically connected to the CPU through their respectively named buses (collection of physical lines), for example pci bus, ide bus, etc. In today’s embedded world, we more often come across microcontrollers than CPUs, which are nothing but CPU + various device controllers built onto a single chip. This effective embedding of device controllers primarily reduces cost & space, making it suitable for embedded systems. In such cases, the buses are integrated into the chip itself. Does this change anything on the drivers or more generically software front?

Not much except that the bus drivers corresponding to the embedded device controllers, are now developed under the architecture-specific umbrella.

Drivers have two parts

Bus drivers provides hardware-specific interface for the corresponding hardware protocols, and are the bottom-most horizontal software layers of an operating system (OS). Over these sit the actual device’ drivers. These operate on the underlying devices using the horizontal layer interfaces, and hence are device-specific. However, the whole idea of writing these drivers is to provide an abstraction to the user. And so on the other end, these do provide interface to the user. This interface varies from OS to OS. In short, a device driver has two parts: i) Device-specific, and ii) OS-specific. Refer to figure 2.

Figure 2: Linux device driver partition

The device-specific portion of a device driver remains same across all operating systems, and is more of understanding and decoding of the device data sheets, than of software programming. A data sheet for a device is a document with technical details of the device, including its operation, performance, programming, etc. Later, I shall show some examples of decoding data sheets as well. However, the OS-specific portion is the one which is tightly coupled with the OS mechanisms of user interfaces. This is the one which differentiates a Linux device driver from a Windows device driver from a MAC device driver.

Verticals

In Linux, a device driver provides a system call interface to the user. And, this is the boundary line between the so-called kernel space and user space of Linux, as shown in figure 2. Figure 3 elaborates on further classification.

Based on the OS-specific interface of a driver, in Linux a driver is broadly classified into 3 verticals:

Packet-oriented or Network vertical
Block-oriented or Storage vertical
Byte-oriented or Character vertical

Figure 3: Linux kernel overview

The other two verticals, loosely the CPU vertical and memory vertical put together with the other three verticals give the complete overview of the Linux kernel, like any text book definition of an OS: “An OS does 5 managements namely: CPU/process, memory, network, storage, device/io”. Though these 2 could be classified as device drivers, where CPU & memory are the respective devices, these two are treated differently for many reasons.

These are the core functionalities of any OS, be it micro or monolithic kernel. More often than not, adding code in these areas is mainly a Linux porting effort, typically for a new CPU or architecture. Moreover, the code in these two verticals cannot be loaded or unloaded on the fly, unlike the other three verticals. And henceforth to talk about Linux device drivers, we would mean to talk only on the later three verticals in figure 3.

Let’s get a little deeper into these three verticals. Network consists of 2 parts: i) Network protocol stack, and ii) Network interface card (NIC) or simply network device drivers, which could be for ethernet, wifi, or any other network horizontals. Storage again consists of 2 parts: i) File system drivers for decoding the various formats on various partitions, and ii) Block device drivers for various storage (hardware) protocols, that is the horizontals like IDE, SCSI, MTD, etc.

With this you may wonder, is that the only set of devices for which you need drivers, or Linux has drivers for. Just hold on. You definitely need drivers for the whole lot of devices interfacing with a system, and Linux do have drivers for them. However, their byte-oriented accessibility puts all of them under the character vertical – yes I mean it – it is the majority bucket. In fact, because of this vastness, character drivers have got further sub-classified. So, you have tty drivers, input drivers, console drivers, framebuffer drivers, sound drivers, etc. And the typical horizontals here would be RS232, PS/2, VGA, I2C, I2S, SPI, etc.

Multiple-vertical interfacing drivers

On a final note on the complete picture of placement of all the drivers in the Linux driver ecosystem, the horizontals like USB, PCI, etc span below multiple verticals. Why? As we have a USB wifi dongle, a USB pen drive, as well as a USB to serial converter – all USB but three different verticals.

In Linux, bus drivers or the horizontals, are often split into two parts, or even two drivers: i) Device controller specific, and ii) An abstraction layer over that for the verticals to interface, commonly called cores. A classical example would be the usb controller drivers ohci, ehci, etc and the USB abstraction usbcore.

Summing up

So, to conclude a device driver is a piece of software which drives a device, though there are so many classifications. And in case it drives only another piece of software, we call it just a driver. Examples are file system drivers, usbcore, etc. Hence, all device drivers are drivers but all drivers are not device drivers.

“Hey Pugs! Just hold on. We are getting late for our class. And you know what kind of trouble, we can get into. Let’s continue from here, tomorrow.”, exclaimed Shweta.

With that Pugs wrapped up saying, “Okay. This is majorly what the device driver theory is. If you are interested, later I shall show you the code & what all have we been doing for all the various kinds of drivers”. And they hurried towards their classroom.

Second Article >>

Mathematics through the Shell

4 Replies

This article, which is the first part of the mathematical journey through open source, takes you through the basic mathematics through a shell.

Mathematics is something which is knowingly or unknowingly part of every moment of our life, starting from the interpretation of the moment – the time to shopping to advanced engineering calculations and computations. We have learnt mathematics from our childhood by counting. With the age of computers, we have been taught how to do mathematics with them. In this journey of mathematics, we are going to tour through the various open source softwares with their mathematical capabilities. Today, we start with the most preliminary one the shell.

Shell command ‘expr’

expr supports the basic arithmetic operations: Addition (+), Subtraction (-), Multiplication (\*), Quotient (/), Remainder (%) – with the C-like precedence & associativity rules. So, if that’s what you want, you may use it as follows:

$ expr 2 + 3
5
$ expr 34 – 67
-33
$ expr 23 \* 27
621
$ expr 43 / 6
7
$ expr 43 % 6
1
$ expr 2 + 2 \* 3 – 5 + 21 / 4 \* 6
33

Note the spaces between every one of the expr, numbers, and the operator – expr is very space-sensitive. The power of this simple command is its precision-less-ness. What I mean is, try out the following:

$ expr 1024 \* 1024 \* 1024 \* 1024 \* 1024 \* 1024 \* 1024 \* 1024 \* 1024 \* 1024

And you still don’t get an overflow, but 1267650600228229401496703205376, i.e. 2¹⁰⁰.

Also, you may use variables. Here’s an example:

$ x=5; y=6; z=`expr $x + $y`
$ echo $z
11

However, it has its own limitation. Basic one being, you do not have the exponentiation operator. Moreover, you can’t change precedences by using brackets. And that’s where you start looking at the other options.

Shell’s arithmetic expansion

With this, we get the all the mathematical C operators, plus the exponentiation using **. We have the bracketing, and variable usage even without the cryptic $. It is achieved by putting the complete expression to be evaluated, between $((…)) – the arithmetic expander. Here’s a set of examples:

$ echo $(((2+3)*4-2**4/5%6))
17
$ x=y=8; echo $((3 << 4 | x | 2 << y))
568

There are some additional interesting ways to assign variables, as well, using let or declare -i.

$ let x=y=8 z='3 << 4 | x | 2 << y' w=z/3
$ echo $w
189
$ declare -i x=y=8 z='3 << 4 | x | 2 << y' w=z/3
$ echo $w
189

Note that the $((…)) has been replaced by single quoting in assigning the value to z. In fact, in most cases – see the assignment of w – it can be actually assigned directly as in C. Though the single quote for z is required to prevent the shell specially interpreting constructs like <<, >>, |, *, etc, and for it not to be space-sensitive. Otherwise, checkout, this example:

$ declare -i x=y=8 z=++x+y
$ echo $z
17

In fact, the variables can be declared as integer once, and then be operated any number of times, as follows:

$ declare -i x y z
$ x=y=8
$ z=++x+y
$ echo $z
17

And, finally here’s two simple ones:

$ echo $((2 ** 100))
0
$ echo $((2.6 + 3))
bash: 2.6 + 3: syntax error: invalid arithmetic operator (error token is ".6 + 3")

Oops!!! how come 2¹⁰⁰ is zero, and we are not able to do real number operations. It has its own limitation. It is limited by the C’s long integer math, typically upto 64-bit computations on today’s computers; and by not having support for non-integer math. That’s where we would look out for something more powerful: the bench calculator.

Second Article >>