Intermediate

Anatomy of a Syscall: What Happens When You Call write() in Linux?

Shatter the user-space illusion. Trace the exact hardware and kernel execution path of a write() call from glibc down to the CPU's MSR_LSTAR register.

write(fd, "Hello", 5) looks like a function call. It isn’t — not in the usual sense. Your process can’t write to a file descriptor any more than it can allocate physical RAM or send a network packet. The x86_64 architecture enforces privilege separation in hardware: your application runs in Ring 3, the least privileged mode. The kernel runs in Ring 0. There is no function pointer your process can follow to cross that boundary directly.

What happens instead is a deliberate hardware trap. Your code loads some registers, executes a single instruction, and control transfers to the kernel — which validates your request, does the work, and hands control back. The whole sequence is a system call, and tracing it from your C code to the hardware and back is one of the cleaner ways to understand what an operating system actually does.

Step 1: The glibc Wrapper

The write() in your C program is a thin wrapper in glibc. Its job is to marshal your arguments into the specific registers that the System V AMD64 ABI prescribes for syscalls, then execute the trap instruction.

Bypassing glibc and writing the call directly in x86_64 assembly makes the register protocol explicit:

raw_write.asm
section .data
msg db "Syscall executed.", 10 ; 18 bytes including newline
section .text
global _start
_start:
mov rax, 1 ; syscall number: sys_write (from unistd_64.h, x86_64 only;
; ARM64 uses a different table — check arch-specific headers)
mov rdi, 1 ; arg1: file descriptor (1 = stdout)
mov rsi, msg ; arg2: pointer to buffer
mov rdx, 18 ; arg3: byte count
syscall ; transfer to kernel
mov rax, 60 ; sys_exit
xor rdi, rdi ; exit code 0
syscall

The kernel sees none of your C types. It reads rax for the syscall number and rdi, rsi, rdx, r10, r8, r9 for up to six arguments. Everything else is the C compiler’s concern.

Step 2: The syscall Opcode

The two-byte opcode 0x0F 0x05 triggers a hardware state machine inside the CPU. In one operation, it:

  1. Saves the return address (next instruction) into rcx
  2. Saves RFLAGS into r11
  3. Elevates privilege from Ring 3 to Ring 0
  4. Loads the instruction pointer from MSR_LSTAR — a model-specific register written by the kernel during boot

MSR_LSTAR contains the address of entry_SYSCALL_64, the kernel’s universal syscall entry point in arch/x86/entry/entry_64.S. The CPU jumps there unconditionally.

On systems with KPTI (Kernel Page Table Isolation, enabled as a Spectre/Meltdown mitigation on most modern hardware), the privilege elevation also triggers a page table switch — from the user-space page tables to the kernel’s. This invalidates most TLB entries, which is one of the dominant costs in syscall overhead. More on that in the performance section.

Step 3: Kernel Entry — entry_SYSCALL_64

The kernel enters in an awkward state: Ring 0 privilege, but still using the user-space stack pointer. The first priority is switching to a safe per-CPU kernel stack before touching anything else.

entry_SYSCALL_64:
swapgs ; swap user GS base with kernel GS base
; (kernel uses GS for per-CPU data structures)
movq %rsp, PER_CPU_VAR(cpu_tss_rw + TSS_sp2) ; stash user RSP
movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp ; load kernel stack
; Save user register state onto the kernel stack
pushq %rcx ; user return address
pushq %r11 ; user RFLAGS
pushq %rdi
pushq %rsi
pushq %rdx
pushq %rax ; syscall number

swapgs exchanges the user-visible GS base register with a kernel-internal value that points to per-CPU data — including the kernel stack pointer. This is why GS is poisoned from the user’s perspective during a syscall.

After preserving the user’s register state, the kernel checks for any enabled seccomp filters. Seccomp-BPF allows processes (or container runtimes like Docker) to install BPF programs that inspect and potentially block or modify syscalls before the kernel dispatch table is consulted. If a seccomp filter denies sys_write, the kernel returns -EPERM here, and none of the code below ever runs. This is a major mechanism for syscall sandboxing in containers and browsers.

If seccomp passes, rax (value: 1) is used as an index into sys_call_table, an array of function pointers. Index 1 resolves to sys_write, which calls ksys_write().

Step 4: ksys_write() and the User Pointer Problem

ssize_t ksys_write(unsigned int fd, const char __user *buf, size_t count)
{
struct fd f = fdget_pos(fd);
if (!f.file)
return -EBADF;
ssize_t ret = vfs_write(f.file, buf, count, &pos);
fdput_pos(f);
return ret;
}

fdget_pos(fd) walks current->files->fd_array[fd] — your process’s open file table — to find the struct file associated with file descriptor 1. If it’s valid, vfs_write is called.

The __user annotation on buf is not just a comment. It marks the pointer as originating from user space, flagging it for static analysis tools (Sparse) that catch unannotated use of user pointers in kernel code. The underlying safety problem: the kernel cannot blindly dereference a user pointer.

  • The pointer might be invalid (would cause a kernel oops)
  • The pages it references might be swapped to disk (requires a page fault to resolve)
  • It might point into kernel memory — if the kernel were to copy from a kernel address into a write buffer and expose it to hardware, that’s an information leak

The solution is copy_from_user(). This function copies bytes from the user-space address into a kernel buffer (typically on the kernel stack or a pre-allocated kernel object — not a newly allocated page). On x86_64 with SMAP (Supervisor Mode Access Prevention) enabled, direct kernel access to user-space addresses will fault. copy_from_user() wraps the copy in __uaccess_begin_nospec / __uaccess_end macros, which temporarily permit access for exactly the duration of the copy, then re-enable SMAP protection.

If the target pages are swapped out, the copy triggers a hardware page fault mid-copy. The kernel’s page fault handler suspends the syscall, initiates disk I/O to bring the pages back into RAM, updates the page tables, and resumes. Your process has no visibility into this — from its perspective, the write() call just took longer than usual.

Step 5: The VFS and Driver Dispatch

vfs_write inspects the struct file’s associated file_operations table — a set of function pointers registered by whichever driver owns this file descriptor:

  • Pseudo-terminal (TTY): The string goes to the TTY line discipline, which may pass it to a terminal emulator via a Unix socket, or directly to a VGA/framebuffer driver
  • Regular file on ext4: The write goes through the page cache, and the ext4 driver eventually flushes dirty pages to the block layer as 4 KB blocks, queued for DMA transfer to the storage device
  • Socket (fd redirected to a pipe or network): The data enters the socket buffer and follows the network stack

The VFS abstraction means ksys_write is identical regardless of destination. The dispatch happens through the file_operations pointer, resolved at runtime.

Step 6: Return via sysretq

The driver returns a byte count up the call chain to ksys_write(), which places it in rax. The kernel’s exit path in entry_64.S then:

  1. Pops the saved user registers off the kernel stack
  2. Restores the user stack pointer
  3. Executes swapgs to restore the user GS
  4. Executes sysretq

sysretq loads rcx into rip (your return address) and r11 into RFLAGS, then drops privilege back to Ring 3. If KPTI is active, the page tables switch back to the user-space set here, with another TLB shootdown.

Your process resumes at the instruction after syscall. The glibc wrapper checks rax for a negative value (which would indicate an errno), and returns the count to your application.

Step 7: The Actual Cost

A system call is not a function call. The overhead has several additive components:

TsyscallTtrap+TKPTI+Tcache+TcopyT_{syscall} \approx T_{trap} + T_{KPTI} + T_{cache} + T_{copy}

  • TtrapT_{trap}: Fixed cost of the syscall/sysretq opcode pair, swapgs, privilege switch — on the order of 100–200 cycles on modern hardware
  • TKPTIT_{KPTI}: Page table switch + TLB flush — significant on systems with KPTI enabled; each flush means subsequent memory accesses must re-walk page tables from RAM rather than hitting TLB. This can add hundreds of additional cycles
  • TcacheT_{cache}: The kernel’s execution loads kernel code and data into L1/L2. When you return to Ring 3, your application’s working set has been partially evicted
  • TcopyT_{copy}: copy_from_user() cost, proportional to data size; includes SMAP enable/disable overhead

The practical implication: calling write() once per byte across one million bytes will spend the overwhelming majority of execution time in context switch overhead rather than I/O. This is precisely why stdio (printf, fwrite) maintains a user-space buffer: it accumulates writes until either the buffer is full or a flush is triggered, then makes a single write() call. The syscall happens once instead of a million times.

For cases where even copy_from_user overhead matters, Linux provides sendfile() and splice(), which move data between file descriptors entirely within the kernel without copying to user space at all. For network-intensive code, io_uring batches multiple syscalls into a single ring-buffer-based submission, amortizing the trap cost across many operations.

A quick way to observe all of this empirically:

Terminal window
strace -c ./your_program

strace intercepts every syscall your process makes and reports call counts and time. Running it on a program that writes byte-by-byte versus one that buffers will show the difference in write call counts immediately — and the wall-clock difference will be substantial.

The vDSO Exception

Not all syscalls cross the Ring 3/Ring 0 boundary. The kernel maps a small shared library — the vDSO (virtual Dynamic Shared Object) — into every process’s address space at a random address. For syscalls that only need to read kernel-maintained data (like clock_gettime and gettimeofday), the vDSO provides user-space implementations that read directly from a kernel-shared memory page. No privilege switch, no TLB flush, no copy_from_user. On a modern system, clock_gettime(CLOCK_MONOTONIC) completes in around 20 nanoseconds rather than the 200+ nanoseconds of a full syscall.

The vDSO is why glibc’s time() doesn’t appear in strace output — it never actually makes a syscall.

Understanding the syscall path makes the design of higher-level APIs legible. Buffered I/O, zero-copy interfaces, io_uring, vDSO — all of them exist as responses to specific, measurable costs in the path described above.

V

Author

Varkin Academy

Tags

#linux #kernel #c #assembly #os internals