Calling the System

There are many things that a programmer will need to do that require functionality reserved to the operating system. This typically has to do with which functionality the operating system developer thinks is safe and secure to expose to a user. For example in linux the execve() system call is used to spawn processes as this is a task that requires loading a file into memory not owned by the calling program which is typically prohibited. It also allows the operating system to handle the scheduling and signaling of the second program.

Because system calls require a context switch to kernel code they are typically implemented using an interrupt. x86 has several unused interrupts which can be bound to interrupt service routines  and invoked in userland to jump into kernel code. In the case of linux, interrupt 0x80 is used.While system calls are all callable as C functions it is an interesting study to invoke one in assembly. The listing below is written in Intel syntax, and can be assembled using NASM, the netwide assembler. This choice is mostly motivated by my undying hatred of percent signs, and this listing can be fairly easily converted to the AT&T syntax used by the GNU assembler. We will be trying to print some text so we want to use sys_write. The C signature is

ssize_t write(int <i>fd</i>, const void *<i>buf</i>, size_t <i>count</i>)

Since all system calls are invocations of the same interrupt we will also need to pass an argument denoting which syscall we want (0x04 in the case of sys write). A full listing of system calls and associated register values can be found here.

section .text
    global _start
    mov eax, 0x04 ;int 0x80 causes the operating system to check register eax for the syscall's code 
    mov ebx, 1    ;fd=1 for stdout
    mov ecx, msg  ;set buf to the string allocated bellow
    mov edx, len  ;count=len(message)
    int 0x80      ;trigger interrupt
;we also have to invoke the exit system call (sys_exit is call 1)
    mov eax, 1    ;code 1 for sys_exit
    mov ebx, 0    ;equivalent to exit(0)
    int 0x80

section .data
    msg db "My Love's the Bogans!",0xa ;0xa is newline
    len equ $ - msg ;msg is a pointer to the head of the string $ is an alias in NASM for the previous address, in this case the end of the string

Finally, this can be compiled with

nasm -felf64 bogans.asm
ld bogans.o -o bogans

Running the resulting binary should print the text then exit. While this is an atypical amount of work to print some text it is a good example of how system calls work at a low level.

Speak softy, and carry a big fork

Fork is one of the many powerful tools you have at your disposal as a C developer. Take for example the code below. Here was an honest attempt at implementing an example using fork, but things went very wrong.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>

// main takes two parameters: argc is the number of command-line
// arguments; argv is an array of strings containing the command
// line arguments
int main (int argc, char *argv[])
    int status;
    pid_t pid;
    int i, num_children;

    // the first command-line argument is the name of the executable.
    // if there is a second, it is the number of children to create.
    if (argc == 2) {
        num_children = atoi (argv[1]);
    } else {
        num_children = 1;

    for (i=0; i<num_children; i++) {

        // create a child process
        printf ("Creating child %d.\n", i);
        pid = fork ();

        /* check for an error */
        if (pid == -1) {
            fprintf (stderr, "fork failed: %s\n", strerror(errno));
            perror (argv[0]);
            exit (1);

    /* see if we're the parent or the child */
    if (pid == 0) {
        printf ("Hello from child %d.\n", i);

    /* parent continues */
    printf ("Hello from the parent.\n");

    exit (0);

The code is supposed to take in a command line argument of the number of processes to create, but it doesn’t quite behave as expected. What’s wrong it? It may not be obvious from a first pass, but I recommend reading through and figuring out what this code actually does. Run the code with different arguments to get a better idea of what’s happening.

This behavior is the fundamental idea behind an exploit sometimes seen in malicious software called a fork bomb. The code in a fork bomb infinitely spawns new processes, starving the system of resources and effectively crashing it. The simplest fork bomb in C looks like this:

#include <unistd.h>

// WARNING! Run at your own risk!
int main(void)