[ArabBSD] First Lecture in FreeBSD Summer Course, Ain Shams University

2.  Open Source Community.
 Why should we care about OS Programming?
 What is ArabBSD?
 OS Categories.
 FreeBSD Vs. Linux.
 FreeBSD Kernel Design Overview.
 FreeBSD File Hierarchy.
 FreeBSD Booting Process.
 FreeBSD Kernel Basics.

3. Kernel
Monolothic Microkernel Exokernel
Hybrid

4.  Composite Component-Based Operating
System.
 Web-Based OS (Chrome OS).

5.  Kernel has no scheduler.
 Scheduling policies and synchronization protocols
are defined by user-level schedulers.
 Mutable Protection Domains (MPD) instead of IPC.

8.  Kernel development.
 Different Systems Emulation Support (Binary
Emulation).
 Levels of Security .
 Maintainers.
 License.

12. BSD
FreeBSD OpenBSD NetBSD
PC-BSD DragonFlyBSD

13.  FreeBSD Installation.
 X Window System Installation.
 UNIX Basics.

15. User
Applications
Kernel
Hardware

16.  Paged Virtual Address Space.
 Software Interrupts.
 Timers and Counters.
 Identifiers for accounting, protection,
scheduling, ..etc.
 Descriptors
Higher layers provide “system” facilities such as
filesystem, terminals and sockets.

17.  CPU Time.
 Asynchronous Events.
 Memory.
 IO Descriptors.
Processes are the fundamental service provided
by UNIX.

19. User process
Blocking Wall
Top half of Kernel
Bottom Half of Kernel

20. Multiple User
Processes
Blocking Wall
Top half of Kernel
Bottom Half of Kernel

22. bin
sbin
boot
cdrom
dev
etc
lib
mount
media
/
proc
(Root sys kern
Directory) tmp home
src
ports
usr games
bin
bin
Local
sbin
share
crash
run
var
tmp
spool

23. Directory Description
 bin Essential command binaries
 boot Static files of the boot loader
 dev Device files
 etc Host-specific system configuration
 lib Essential shared libraries and kernel modules
 media Mount point for removable media
 mnt Mount point for mounting a filesystem temporarily
 sbin Essential system binaries
 tmp Temporary files
 usr Secondary hierarchy
 var Variable data

24.  home User home directories
 lib Alternate format essential shared libraries
 /sys/kern Kernel Directory.

25.  /bin contains tools required system to work (like 'ls',
'rm', 'mkdir' etc.) which would be available at any time
(for example if system crashes, they were available in
single user mode). Used by both root and users.
 /usr/bin contains tools which are not needed in
single user mode. Used by both root and users.
 /usr/local/bin is for local use - you install your
binaries there. Used by both root and users (Third
party binaries).

26.  /sbin : System binaries
Purpose
 Utilities used for system administration are stored
in /sbin, /usr/sbin, and /usr/local/sbin.
 /sbin contains binaries essential for booting, restoring,
recovering, and/or repairing the system in addition to
the binaries in /bin. Programs executed after /usr is
known to be mounted (when there are no problems) are
generally placed into /usr/sbin.
 Locally-installed system administration programs
should be placed into /usr/local/sbin.

27.  /boot : Static files of the boot loader
Purpose
 This directory contains everything required
for the boot process except configuration
files not needed at boot time. Thus /boot
stores data that is used before the kernel
begins executing user-mode programs. This
may include saved master boot sectors and
sector map files.
 Also, the compiled kernel should be located
in /boot/kernel/.

28. /dev : Device files
Purpose
The /dev directory is the location of special
or device files.

29. /etc : Host-specific system configuration
Purpose
The /etc hierarchy contains configuration
files.
A "configuration file" is a local file used to
control the operation of a program; it must
be static and cannot be an executable binary.

30.  File Description
 fstab Static information about filesystems
 ftpusers FTP daemon user access control list
 gateways File which lists gateways for routed
 host.conf Resolver configuration file
 inetd.conf Configuration file for inetd

31. /etc/X11 : Configuration for the X Window
System
Purpose
/etc/X11 is the location for all X11 host-
specific configuration.

32. /home : User home directories (optional)
Purpose
/home is a fairly standard concept, but it is
clearly a site-specific filesystem.

33. /media : Mount point for removable media
Purpose
This directory contains subdirectories which
are used as mount points for removable
media such as floppy disks, cdroms and zip
disks.

34.  /mnt : Mount point for a temporarily
mounted filesystem.
Purpose
 This directory is provided for temporarily
mounted filesystem. For example, partitions
from harddrives or network shares.

35.  /tmp : Temporary files
Purpose
 The /tmp directory must be made available
for programs that require temporary files.

36.  The /usr Hierarchy
Purpose
 This directory includes all the user related
stuff.

37.  /usr/include : Directory for standard include
files.
Purpose
 This is where all of the system's general-use
include files for the C programming language
should be placed.

38.  /usr/lib : Libraries for programming and
packages
Purpose
 /usr/lib includes object files, libraries, and
internal binaries that are not intended to be
executed directly by users or shell scripts.

39.  /usr/local : Local hierarchy
Purpose
 The /usr/local hierarchy is for use by the
system administrator when installing software
locally. It needs to be safe from being
overwritten when the system software is
updated. It may be used for programs and
data that are shareable amongst a group of
hosts, but not found in /usr.

40.  /usr/local/share
Purpose
 The requirements for the contents of this
directory are the same as /usr/share. The
only additional constraint is
that /usr/local/share/man and /usr/local/m
an directories must be synonomous (usually
this means that one of them must be a
symbolic link).

41.  /usr/sbin : Non-essential standard system
binaries
Purpose
 This directory contains any non-essential
binaries used exclusively by the system
administrator.

42.  /usr/share : Architecture-independent data
Purpose
 This hierarchy is intended to be shareable
among all architecture platforms of a given
OS.
 Any program or package which contains or
requires data that doesn't need to be
modified should store that data
in /usr/share (or /usr/local/share, if installed
locally).

43.  /usr/share/man : Manual pages
Purpose
 This section details the organization for
manual pages throughout the system,
including /usr/share/man.

44.  The /var Hierarchy
Purpose
 /var contains variable data files. This includes
spool directories and files, administrative and
logging data, and transient and temporary files.
 Some portions of /var are not shareable between
different systems. For instance, /var/log
and /var/lock. Other portions may be shared,
notably /var/mail, /var/cache/man, /var/cache/fo
nts, and /var/spool/news.

45.  /var/cache : Application cache data
Purpose
 /var/cache is intended for cached data from
applications. Such data is locally generated as
a result of time-consuming I/O or
calculation.

46.  /var/crash : System crash dumps (optional)
Purpose
 This directory holds system crash dumps.

47.  /var/mail : User mailbox files (optional)
Purpose
 The mail spool must be accessible
through /var/mail.
 User mailbox files in this location must be
stored in the standard UNIX mailbox format.

48.  /var/run : Runtime variable data
Purpose
 This directory contains system information
data describing the system since it was
booted. Files under this directory must be
cleared (removed or truncated as appropriate)
at the beginning of the boot process.

49.  /proc : Kernel and process information virtual
filesystem
Purpose
 The proc filesystem is the standard method
for handling process and system information.

50.  Take a tour in FreeBSD File Hierarchy.

52.  When a computer is powered on, there is
nothing running on the CPU. For a program to
be set running, the binary image of the
program must first be loaded into memory
from a storage device.
 Bootstrapping: cold start to the point at
which user-mode programs can be run.
 In FreeBSD, the binary image is in /vmunix or
/boot/kernel/kernel

53.  Initial bootstrapping is machine dependant.
 Usually 512 bytes.
 Used to load boot program.
 The boot program's task is to load and
initialize the executable image of a program
and to start that program running.
 Not required to be on the same storage
device.

54. [BITS 16]
ORG 0
int 0x18
TIMES 510-($-$$) DB 0
DW 0xAA55

55.  Binary file reading in terms of Assembly
language.
 FreeBSD Assembly Tools (as, nasm).

56.  The boot program reads the binary image of
a program to be bootstrapped into main
memory, and then initializes the CPU so that
the loaded program can be started.
 Boot -> several stages of hardware and
software initialization in preparation for
normal system operation

57.  The first stage is responsible for initializing
the state of the CPU, including the run-time
stack and virtual-memory mapping.
 Once virtual-memory mapping is enabled,
the system does machine-dependent
initializations, and then machine-
independent initializations.

58.  The machine-dependent operations include
setting up virtual-memory page tables and
configuring I/O devices; the machine-
independent actions include mounting the
root filesystem and initializing the myriad
system data structures. This order is
necessary because many of the machine-
independent initializations depend on the I/O
devices being initialized properly.
 Following the setup of the machine-
independent portions

59. Turn on PC
boot0
Read the value 0xfffffff0 from instruction pointer register
boot1
Hardware translates the value to read BIOS Memory Block
boot2
Jump to BIOS’s POST routines loader
POST performs different checks init
One of the POST instructions is INT 0x19 instruction which
reads 512 bytes from the first sector of boot device into
the memory at address 0x7c00

60. File •sys/boot/i386/boot0/boot0.S
Location
•INT 0x19 instruction loads an MBR, i.e. the boot0 content, into the
Memory memory at address 0x7c00
Location
•Starting from 0x1be, called the partition table, It has 4 records of 16 bytes each,
MBR called partition record
Structure
•the 1-byte filesystem type
•the 1-byte bootable flag
Partition
Record •the 6 byte descriptor in CHS format
Structure •the 8 byte descriptor in LBA format

61. LBA (Logical Block Addressing)
• has the starting sector for the
partition and the partition's length.
CHS (Cylinder Head Sector)
• has coordinates for the first and last
sectors of the partition.

62. The boot manager scans the partition table and prints the menu on the
screen so the user can select what disk and what slice to boot. By
pressing an appropriate key, boot0 performs the following actions:
modifies the bootable flag for the selected partition to make it
bootable, and clears the previous
saves itself to disk to remember what partition (slice) has been
selected so to use it as the default on the next boot
loads the first sector of the selected partition (slice) into memory and
jumps there

63.  Actually, there is a 512-byte file
called boot1 in the directory /boot as well. It is
used for booting from a floppy. When booting
from a floppy, boot1 plays the same role
as boot0 for a harddisk which is
locating boot2 and runs it.

64.  You may have realized that a
file /boot/mbr exists as well. It is a simplified
version of boot0. The code in mbr does not
provide a menu for the user, it just blindly
boots the partition marked active (i.e. No
Boot Manager Used).

65.  Configure boot file using boot0cfg.
 Note: Both boot0 and boot2 are invisible to
listing commands because they are not under
filesystem’s control.

66.  It’s the first specific file for FreeBSD since it’s
used for locating FreeBSD slice.

67. File • sys/boot/i386/boot2/boot2.c
Location
• It must scan the harddisk.
• knowing about the filesystem structure
• finds the file /boot/loader
• reads it into memory using a BIOS service
Procedure • passes the execution to the loader's entry point.
• boot2 prompts for user input so the loader can be booted
from different disk, unit, slice and partition
• passes the execution to BTX

68. • switches the processor to protected mode
• prepares a simple environment before calling the client
• boot0 passes the execution to BTX's entry point. BTX then switches the
processor to protected mode, and prepares a simple environment before
calling the client. This includes:
• virtual v86 mode. That means, the BTX is a v86 monitor. Real mode
BTX instructions like pushf, popf, cli, sti, if called by the client, will work.
Procedure
• Interrupt Descriptor Table (IDT) is set up so all hardware interrupts are routed
to the default BIOS's handlers, and interrupt 0x30 is set up to be the syscall
gate.
• Two system calls: exec and exit.
• Finally, BTX creates a Global Descriptor Table (GDT)
• boot2 defines an important structure, struct bootinfo. This structure is
initialized by boot2 and passed to the loader, and then further to the kernel.
Some nodes of this structures are set by boot2, the rest by the loader. This
Back to
structure, among other information, contains the kernel filename, BIOS
Boot2
harddisk geometry, BIOS drive number for boot device, physical memory
available, envp pointer etc.

71.  Open Boot2 file.
 Take a look in Boot2 make file.
 How do the two system calls exec and exit
are created within boot2? .
 Take a look in Global Descriptor Table.
 Bootinfo header file.
 Choosing the most appropriate choice in 10
seconds!!

72. File • sys/boot/i386/boot/loader
Location
• kernel bootstrapping final stage
• When the kernel is loaded into memory, it is being called by
Procedure the loader
• The main task for the loader is to boot the kernel.
• It provides a scripting language that can be used to automate tasks, do
Functionality
pre-configuration or assist in recovery procedures.

73.  Have a look in /boot/loader.conf

75. • sys/i386/i386/machdep.c
File
Location
•Initialize the kernel tunable parameters, passed from the bootstrapping
program.
•Prepare the GDT.
•Prepare the IDT.
•Initialize the system console.
•Initialize the DDB, if it is compiled into kernel.
Functionality
•Initialize the TSS.
•Prepare the LDT.
•Set up proc0's pcb

76. • sys/kern/init_main.c
File
Location
• This function performs a bubble sort of all the
system initialization objects and then calls the
entry of each object one by one.
Functionality • Call System Scheduler which represents a
process with PID 0.

78.  boot0 : The first stage is written entirely in
assembly language and does the work
necessary for non-assembly-language code
to operate.
 boot1: Used by floppy disks..
 boot2: The third stage does machine-
independent operations, completing its work
by starting up the system-resident processes
(Filesystems and kernel loading).

79.  Init : kernel starts Init as the first process. Init
starts shell and runs Resource Configuration
(rc) files. rc config files (for start and set
options of services) are /etc/rc ,
/etc/defaults/rc.conf , /etc/rc.conf

80.  Machine Dependent.
 Assembly-Language Startup.
 Machine-Dependent Initialization.
 Message Buffer.
 System Data Structures.
 Autoconfiguration.
 Device Probing.
 Device Attachment

81.  Machine Independent.
 Proc0 Creation.
 User-Level Initialization

82.  System-Startup Topics.
 Kernel Configuration.
 Passage of Information To and From the
Kernel.

84. Kernel
Static Modules KLDs
Compiled in the kernel Loaded during run time

85.  Some modules should be compiled in the
kernel and can’t be loaded during run time.
 Those modules are ones that used before control
turned to the user.
 Kernel Loadable Modules result large
flexibility in the kernel size since only needed
KLD’s are loaded.
 We can use Binary file for total KLD’s indexing and
make it easier for the user to determine what KLD’s
can be loaded at boot time without the need for
loading each one separately.

86. Application
Kernel
Hardware

87. • <sys/sysctl.h>
• <sys/socket.h>
• <vm/vm_param.h>
File
• <netinet/in.h>
Location • <netinet/icmp_var.h>
• <netinet/udp_var.h>
• get or set kernel state
• The sysctl utility retrieves kernel state and allows processes with
appropriate privilege to set kernel state. The state to be retrieved
or set is described using a “Management Information Base''
Functionality
(“MIB'') style name, described as a dotted set of components.
• To retrieve the maximum number of processes allowed in the system, one
would use the following request sysctl kern.maxproc
• Information about the system clock rate may be obtained with sysctl
Example kern.clockrate

88.  Check Sysctl files and test them

89.  Q. How do we can realize object oriented
approach in FreeBSD Kernel?
 A. Kernel Objects (Kobj)

90. Terminology Description
A set of data - data structure - data
Object
allocation.
Method An operation – function
Class One or more methods.
Interface A standard set of one or more methods

91. • Kobj works by generating descriptions of methods. Each description holds
a unique id as well as a default function. The description's address is used
Kobj to uniquely identify the method within a class' method table.
• A class is built by creating a method table associating one or more functions
with method descriptions. Before use the class is compiled. The compilation
allocates a cache and associates it with the class. A unique id is assigned to
each method description within the method table of the class if not already
Class
done so by another referencing class compilation
• To retrieve the maximum number of processes allowed in the system, one
would use the following request sysctl kern.maxproc
• Information about the system clock rate may be obtained with sysctl
Example kern.clockrate

92. New Method
Use Method
New Method
Use script to
generate function
associated function to qualify
The default within the class is arguments
function associated used
with the method
description is used
Found
Not Found
automatically
reference the
generated function proceeds to use the class'
method
table to find the method
description for a
lookup
looks up the method by using the
Not Cached unique id associated with the method
description as a hash into the cache
associated with the object's class

93.  Kobj Naming Conventions.

95.  FreeBSD Handbook Web Page
http://www.freebsd.org/doc/en_US.ISO8859-
1/books/handbook/
 McKusick, M., et al. The design and
implementation of the 4.4BSD operating system,
Addison-Wesley, 1996.
 ArabBSD Web Page
https://sites.google.com/site/arabbsd/
 This presentation is available on ArabBSD Website.