OS Design, Implementation and Structures

Designing an operating system is one of the biggest software projects there is, and the slides open with a warning: no silver bullet, but some approaches have been successful. The internal structure of different OSs varies widely, and it is shaped by two things: the choice of hardware and the type of OS you are building (a real-time kernel for a pacemaker and a time-sharing server OS make very different choices).

The first job is to define goals and specifications. Who is this OS for? The slides split the answer into two camps that often conflict.

The most important design principle on these slides is to separate policy from mechanism. A mechanism determines how to do something. A policy decides what will be done. Keep them apart and you can change your mind about the policy without rewriting the mechanism.

Why does this matter so much? Because policies change and mechanisms do not. A university lab might want fair sharing today and priority for research jobs next semester. If the scheduling mechanism is general (a timer plus a ready queue), switching policy is a configuration change. If the policy is baked into the mechanism, you rewrite the kernel. The slides say the separation allows maximum flexibility if policy decisions are to be changed later.

A system as large and complex as a modern OS must be engineered carefully if it is to work and be modified easily. The common approach is to partition the task into small components rather than build one monolithic system. Each component should be a well-defined portion of the system with carefully defined inputs, outputs and functions. How these components are interconnected and melded into a kernel is what we call the operating system structure.

Many operating systems do not have well-defined structures. Such systems are small, simple and limited. The classic example is MS-DOS: it was written to fit in as little memory as possible, so application programs could call BIOS routines directly and write straight to the display. There was no protection because the Intel 8088 it ran on had no dual mode.

The monolithic structure is the next step up. The core of the OS is called the kernel, and in a monolithic design the kernel acts as a manager for everything: file management, memory management, device management and so on. That means a large number of functions at one level, all in one address space. The original UNIX used this structure, split into just two parts: system programs and the kernel.

In a layered structure the OS is divided into a number of layers (levels). The bottom layer (layer 0) is the hardware; the highest layer (layer N) is the user interface. Each layer is built only on the layers beneath it, and each layer hides the existence of certain data structures, operations and hardware from higher layers. Layer 3 does not know or care how layer 1 works; it just calls layer 2's functions.

The micro-kernel approach removes all non-essential components from the kernel and implements them as system and user-level programs. What is left is a micro (smaller) kernel that provides only minimal process and memory management plus a communication facility. The slide diagram shows this split clearly.

Notice what has moved: the file system and device drivers, which are inside the kernel in a monolithic design, now run as ordinary user-mode processes. When an application wants to read a file, it does not call into the kernel's file code. It sends a message to the file-system server, the micro-kernel delivers it, and the reply comes back the same way.

The slides call this the best current methodology: loadable kernel modules. The kernel provides core services, and other services are implemented dynamically, as the kernel is running. Concretely: CPU scheduling and memory management algorithms are implemented directly in the kernel, while support for different file systems is provided by loadable modules. Plug in a USB stick formatted as exFAT and Linux loads the exFAT module on the spot.

The slide example is the Solaris modular approach: a core kernel surrounded by seven kinds of loadable modules (scheduling classes, file systems, loadable system calls, executable formats, STREAMS modules, miscellaneous, device and bus drivers). Linux uses the same idea with .ko files; lsmod lists the modules currently loaded on your machine.

An important aspect of a modern system is its collection of system programs. They provide a convenient environment for program development and execution. Some of them are simply user interfaces to system calls (the rm command is a thin wrapper around the unlink() call); others are considerably more complex (a compiler). Most users' view of the OS is defined by system programs, not by the actual kernel. The slides list six types.