Process depth control

Overview:

This patch provides a facility to limit the creation of child processes, therefore limiting the depth of the process tree.

That means if a process depth is set to:

0. it can't fork.
1. it can fork, but its children can't.
2. the process and its children can fork, but its grandchildren can't.
3. ...

Note: the "fork" word is used in a wider sense, so restriction applies to fork, vfork, and other functions that calls do_fork() on Linux or fork1() on DragonFly BSD.

The solution presented here uses the rlimit framework, so very few changes are needed.

Now it's time for some real world justification for this job. Let's have a look at a known exploit and see how this patch offers the tools needed to handle the situation.

Example:

mod_php under Apache 2.0.x leaks a critical file descriptor that can be used to hijack the https service.

The technique is simple.

1. fork() and daemonize yourself.
2. select() on the leaked descriptor and start serving pages.

Limit the depth of children processes, preventing the malicious code to daemonize itself.

A simple solution for Apache httpd is to write an rlimit module that set the RLIMIT_DEPTH parameter to 0 or 1 on its children processes.

The Patch:

The idea is to create a new rlimit contant so the user is just a setrlimit call away from restraining processes fork's abillity.

Linux

The kernel modifications consists in:

On include/`arch`/resources.h:

• add a new rlimit constant, RLIMIT_DEPTH
• bump RLIM_NLIMITS
• and add the default value

On kernel/fork.c:

• update child's depth limit
• check if child isn't trespassing the allowed depth bound

Note: This patch has not received feedback from the Linux community, and thus is not being maintained. It has a bug that happens when rlim_cur != RLIM_INFINITY but rlim_max == RLIM_INFINITY. On that case rlim_max is also being decremented after the fork, but it's value should not be changed, as it makes no sense to subtract 1 from infinity.

DragonFly

Kernel changes:

On src/sys/sys/resource.h:

• add a new rlimit constant, RLIMIT_FORK;
• bump RLIM_NLIMITS.

On src/sys/kern/kern_fork.c:

• check the current limit before allocating the new process;
• set the new limits on the child if the parent's limits weren't set to RLIM_INFINITY.

Userland changes:

• login(1): create a forkdepth resource that can be limited for login classes as described on login.conf(5).
• sh(1): allow the ulimit built-in command to set a fork limit, using the -r flag;
• csh(1): create a forkdepth resource that can be limited using the limit built-in command;
• limits(1): adapt the code for the new resource limit.

The relevant manual pages have also been modified to include a description of the changes.

Authors:

Linux patches:

• Michel Machado
• Marco Molinaro

DragonFly patches:

• Andre Nathan

Download:

For Linux 2.6.3

i386 diff file
all archs diff file

For DragonFly BSD:

resource.h patch
kern_fork.c patch
login(1) patch
sh(1) patch
csh(1) patch
limits(1) patch
libc getrlimit.2 patch
libcr getrlimit.2 patch
login.conf patch

Test program:

patch_test.c