Kernel Development Learning Pipeline

Lecture 4 - 21 September 2023

Topics covered:

• Miscommunication regarding these notes
• Questions on P0
• Linux crash course and minimal distro

A time saver for vim users

Append these two lines to your ~/.vimrc to auto-highlight whitespace errors in your editor:

:highlight ExtraWhitespace ctermbg=red guibg=red
:match ExtraWhitespace /\s\+$/

Miscommunication regarding these notes

• Reading through these lecture notes and following any instructions given there is essential for students to succeed in this course.
• We apologize for the lack of emphasis on this
• We did things a bit differently in the past
• As such, we will relax our expectations of what videos you have watched so far.
• If you have not done so, please go through all of L0, L1, L2, L3
• We will announce lecture notes and any subsequent errata via the course matrix server
• In case anyone is having trouble finding a good client, try element. It’s got mobile, web, and desktop apps.

Questions on P0

• How do students make the commits while writing the code to later generate the patches?
  • Commit early and commit often when developing your shell
  • When in doubt about whether to commit, just do it
  • When your repo is in a state where your program builds and correctly implements stage N, make sure to make a commit and put a note about completing N in the log
  • Later, when you are preparing your work for submission, you will squash everything between these commits into single commits implementing a stage
  • Note that squash here is a technical term referring to the git-rebase option
  • You will run git format-patch ... on the branch containing only these squashed commits
  • This is a great opportunity to practice git rebase
• What functions can we use?
  • When P1 specifies a syscall for a scenario, you must use that
  • Beyond that, feel free to use anything provided by libc
• Do we need to check for memory leaks?
  • Yes
  • We will deduct credit from assignments for any memory leaks
  • Use valgrind to find the problems
  • Memory leaks are not as severe as build or runtime failures, which merit no credit
  • Try to fix these, but don’t let this stop you from submitting if you are very close to the deadline

Linux crash course and minimal distro

• We started with the Linux Crash Course slide deck.
• First, we gave the general what, where, why, and who of Linux
  • Free and open source operating system kernel and ecosystem
  • Running on many systems large and small, fast and slow, distributed and centralized
  • Many pre-packaged combinations of system components capable of running user applications are available
  • We call one of these a Linux distribution, or distro for short.
  • Some examples are Fedora, Ubuntu, Arch Linux, RHEL, Puppy, and TAILS
• Having motivated our discussion of system components, we proceeded to discuss each one briefly
  • We quickly went through the bootloader, kernel, C standard library, shared libraries, storage configuration, and filesystem hierarchy layout
  • As we were about to proceed to the demo, we didn’t go into great detail
  • Take note of the availability of manpages: run man man and read through the description.
• After finishing the slide deck, we build a minimal Linux distribution live
  • Using a ruthlessly minimal .config file, we built the kernel in noticably less time than it took to clone, even with the --depth=1 option. Barely minutes.
  • Then, we tested this kernel with qemu-system, generating a kernel panic as expected
  • This is because the kernel attempts to start an inital userspace process and no such thing exists on the system
  • On many Linux systems, systemd is this first “init” process, and it is given PID 1
  • We then created a root filesystem and a “hello world” init process
  • This native assembly simply prints hello world
  • Take note that the entry point is _start and not main
  • Normally this is hidden as libc does various init things between _start and the user-defined main
  • We used strace to see how our little app talks to the kernel
  • In order to pass the app binary into the kernel, we use cpio to create an archive usable as an inital ram disk for our system, which allows the kernel to run the app as an init process
  • This is generally refered to by the abbreviation initrd.
  • initrd is an area of RAM that provides a storage device interface to the rest of the kernel.
• Since there were no questions, we attempted to make our minimal Linux distro interactive
  • We implemented a parameter allowing a user to specify a string to print after “hello” instead of “world”
  • The feature started working right in time for the end of class

Before next class

• Make sure you have carefully read through L0, L1, L2, and L3 and seen all the provided video content.

Important Reminders

P0 is due at 11:59PM next Tuesday (26 September 2023)

See this video for more guidance on how to complete P0.