Adapting Plan 9's listen to GNU Guix

The work presented herein yielded more than the listen bash script one can download with:

git clone git@the-dam.org:listen

A Golang 9P2000.L FUSE client was developed from scratch:

git clone git@the-dam.org:f29p

This client is needed to mount 9P servers in the containers in which listen isolates the network services (section 7.3). It is a seldom documented fact of Linux that unprivileged mounting is reserved to a select few filesystems, among which 9P is not present. No other piece of software has ever been made public that allows one to mount a 9P2000.L server from within a Linux container.

A pull request implementing UnlinkAt was merged into p9ufs, a Golang 9P2000.L server: https://github.com/hugelgupf/p9/pull/87/files

listen requires deep changes in the system it is installed on (for example, a change in the default privileged port ranges, see section 5), but we wrote a set of operating configuration functions (see section 4) that makes installing listen a literal one-line change in a configuration file. These functions can be added to Guix System through our channel (Klein 2023).

(channel
  (name 'beaverlabs)
  (url "https://gitlab.com/edouardklein/guix")
  (branch "beaverlabs"))

A network-transparent implementation of finger (see section 6.2) was developed to work with our listen. It should work with Plan 9's listen as well, as soon as somebody writes a Python 3 interpreter for Plan 9, which should be right around the time we sunset IPv4 globally.

Fetch it with:

git clone git@the-dam.org:fingerd

First comes namespace. A namespace in Plan 9 is the view a process has of the filesystem. Because every system resource is available as a file, a namespace is the view a process has of the system's resources (Pike et al. 1993).

On Linux, namespaces are an isolation mechanism bolted upon processes. Linux processes are not, by default, as isolated from one another as they are in Plan 9.

Because on Linux everything is definitely not a file, there are multiple kinds of namespaces. One can be in a network namespace, a mount namespace, a user namespace, etc. (Kerrisk 2013; Unshare(1) – Linux User’s Manual 2023, 2023)

To remove this ambiguity, we will explicitly specify "Linux namespace", "Linux mount namespace", etc. or use the improper term of containers when talking about the specific Linux isolation mechanism, and reserve the bare namespace term for the Plan 9 generic notion of a process' view of the system resources.

When we talk about the namespace of a Linux process, we talk about what it sees of the underlying system. This view is constructed with the help of different, potentially nested Linux namespaces, and 9P mounts through FUSE.

GNU Guix is a package manager (Courtès 2013), that can be installed on any Linux distribution.

Guix System is a Linux distribution that uses GNU Shepherd as its daemon manager, lacks systemd, and of course uses Guix as its package manager. It goes further by providing a declarative configuration system for the whole operating system, with atomic updates, roll-backs, etc. (Neidhart 2019)

Waters are muddied by the fact that the guix system command, provided by GNU Guix the package manager (which can be installed on any Linux distribution) allows one to instantiate a Guix System operating system as a VM, a container, a docker image, etc.

We will use Guix to refer to the package manager, and Guix System to the operating system.

In UNIX parlance, a service is a background process (a daemon), typically launched by a service manager (SysVinit, rc, shepherd, systemd, etc.), examples include network services such as a web server, but also the cron daemon.

In the context of listen, services refer to the network services that listen manages.

On Guix System, a service is a broader notion, that encompasses network services, daemons, but also any aspect of the system configuration, such as udev rules, user accounts, etc.

We will always specify, unless the context makes it absolutely clear, whether we are talking about network/listen services, or Guix services.

We will avoid calling UNIX services services, and use daemon instead.

The echo protocol is implemented by linking cat to /srv/listen/tcp7:

ls -l /srv/listen/tcp7
lrwxrwxrwx 1 root root 65 Mar 21 21:14 /srv/listen/tcp7 -> \
    /gnu/store/mppp9hwxizx9g9pikwcvvshb2ffxyq7p-coreutils-9.1/bin/cat

This link is created by Beaver Lab's listen/echo function:

(define (listen/echo os)
  "Return a copy of OS, in which listen's echo (tcp7) service is active."
  (extend-service
     os
     activation
     #~(begin
         (when (file-exists? "/srv/listen/tcp7")
           (delete-file "/srv/listen/tcp7"))
         (symlink #$(file-append coreutils "/bin/cat") "/srv/listen/tcp7"))))

This service can run in the empty namespace that listen provides by default.

The above echo example ranks among the simplest things one can do with listen. Let's study now a more complex service, the finger service listening on port 79. Our finger implementation is a simple 68-lines python script that reads a query on its standard input, parses it, sets some of the env vars from the CGI specification⁵5: Allowing queries like finger 'hello?name=alice&greeting=Howdy@the-dam.org' , and execs the requested script in /srv/finger/. This script's output is what the remote finger client gets.

This network service is active now, and one can simply query it with a finger client or by running:

echo | nc the-dam.org 79

to get a list of the available user names.

finger illustrates how to clear two hurdles most real life use-case will meet:

• it needs to access data from outside the container,
• it relies on custom software, unavailable from the vanilla version of guix provided by listen's container.

Accessing data from outside listen's container is done, as it is in Plan 9, by mounting 9P servers in the service's namespace.

listen will look for a /srv/finger/tcp79.namespace file. This is not, as it is in Plan 9, a namespace file understood by newns or addns (Auth(2) – 9front’s Manual 2024). It is a proper script, called by listen, and expected to setup its own environment, its own view of the filesystem, and its own network before calling exec $@, relinquishing control to a command built by listen.

The default namespace script, /srv/finger/listen.namespace, called in the absence of a service-specific script, is quite simple:

#!/gnu/store/aslr8ym1df4j80ika5pfxy5kbfv4iz3w-bash-5.1.16/bin/bash
set -euo pipefail
# This is the default namespace file for listen services. This is where you
# mount the 9P services all your listen services need.
exec "$@"  # Once done, call the command provided as an argument by listen

The declarative configuration will not overwrite this file if it already exists, making it easy to configure the default namespace in a site-specific manner.

finger's namespace file contains:

#!/usr/bin/env bash
set -euo pipefail
mkdir -p /srv/finger
f29p unix!/srv/9p/finger /srv/finger &
exec "$@"

This mounts the 9P server listening on /srv/9p/finger to /srv/finger.

This 9P server daemon is started and managed by shepherd, Guix System's daemon manager, thanks to a call to os/9p-serve in the function that configures finger:

(os/9p-serve "/srv/finger/" "/srv/9p/finger" #:name '9p-finger
             #:user "listen" #:owner "listen" #:group "listen" #:mode "700")

This call creates a socket on /srv/9p/finger. On the socket listens an instance of p9ufs, a modern Golang 9P2000.L implementation. This process is owned by listen, and so is the socket.

On Plan 9, namespace operations fail or succeed thanks to the authentication mechanism embedded in 9P. This authentication mechanism relies on the factotum process mounted on /mnt/factotum having the required credentials (Cox et al. 2002).

Despite a previous attempt at porting this mechanism to Linux (Klein and Gette 2023), this mechanism is not readily available to listen. Instead, to control the operations on files outside of the service container, listen relies on the ownership of the p9ufs process and the ownership and permissions of the socket file.

In this particular case, listen gets a read-only by default (one has to pass #:read-write "1" to os/9p-serve to get read-write access) view of /srv/finger, with the same rights as it would have outside of the container. In the next section we will see an example of user git "lending" its read rights on /srv/git to listen (which can't read /srv/git whose owner, group, and mode are git:users r-xrwx---).

This choice makes it possible for e.g. alice to provide finger information to her servermates only, who, belonging to the users group, may call her script while logged-in, whereas anonymous users from the internet, being confined to listen's identity, won't see her script as executable:

ls -l /srv/finger/alice
-rwxr-x--- 1 alice users 121 Mar 18 10:57 /srv/finger/alice

Using custom software is done with a Guix profile. While on Plan 9 one makes software accessible by binding various directories over /bin, the story is more involved on Linux. Due to constraints imposed by ubiquitous dynamic linking, interpreted languages, and mostly-standardized-but-not-quite defaults paths, one has to rely on a myriad of search-paths. search-paths are environment variables that tell the software where to find the resources it needs.

guix can setup profiles (Courtès 2018) that are links to its immutable, content-addressed, store. In it are synthetic directories with links to all the needed resources (them, too, living in the store), as well as a profile script that will set the search-paths to those resources.

listen will load the profile in /run/listen/tcp79/profile before calling the service namespace script and the service script. A default profile is used if /run/listen/tcpXXX/profile does not exists.

In the case of finger, the profile is created by a call to os/profile:

(os/profile "/run/listen/tcp79/profile"
            #:name 'finger-profile
            #:packages '("python" "the-dam-org-f29p"))

which sets up two packages: the f29p 9P FUSE client, and the Python interpreter.

As revealed by an analysis of the fourth edition source code, 9front's source code, and a question asked on 9fans (“Content of Your /Rc/Bin/Service or /Dis/Svc ? – 9fans Thread” 2024), Plan 9 relies, for the following protocols (as well as a few others), on servers reading and writing on the standard streams: ftp, (ssh), telnet, smtp, http, pop, imap, samba, rlogin, lp, 9P (of course).

Some of those servers, such as rc-httpd have been ported to BSD or Linux, but others rely for example on /net (Presotto and Winterbottom 1993), which does not exist (yet ?) on Linux.

On the-dam.org, git repositories are made available through a user called git, as which anybody can connect through ssh, e.g.:

git clone git@the-dam.org:listen

However, a bug (“Can’t Clone a Git Repo over Anonymous SSH –- Issues.Guix.Gnu.Org” 2023) prevents Guix from using this kind of access, and so we must open port 9418 to let git use its own unauthenticated, unencrypted protocol (but Guix checksums the code after fetching so those two points are not that big of a deal).

Git does provide an easy way to run a server speaking this protocol: just run git daemon. However, git daemon binds to port 9418, and does not use the standard streams.

One could extract the git protocol from the code and create a standalone standard stream network transparent server, but that is a lot of work.

What we do instead is abuse the namespace script. Instead of relinquishing control to a command crafted by listen, here is what tcp9418.namespace does:

#!/usr/bin/env bash
set -euxo pipefail
mkdir -p /srv/git
f29p unix!/srv/9p/git /srv/git &
ip link set lo up
git daemon --base-path=/srv/git --export-all&
exec socat -dd UNIX-listen:/run/socket,fork TCP-CONNECT:127.0.0.1:9418

Lines 3 and 4 play the same role as for finger: making the necessary data available to the service by mounting a 9P server.

Said 9P server is started thanks to this line in the operating system configuration function listen/git-daemon:

(os/9p-serve "/srv/git/" "/srv/9p/git" #:name '9p-git
             #:user "git" #:owner "listen" #:group "listen" #:mode "700")

Here we see that only user listen can use the socket to mount the 9P server, but the process listening on the other end is not owned by listen but by git. This is because listen is not privileged enough to read the actual /srv/git. By making user git run the 9P server, listen gets exactly the kind of access it needs, on a very specific and well-defined subset of the global filesystem.

Line 5 of the git daemon namespace script sets up a loopback network interface. Behind the generic "container" term hide Linux' namespaces (same name as Plan 9, different feature). Important here is a Linux network namespace, that makes a process believe it owns all network interfaces (it does, but not the actual ones). Setting a dummy loopback interface on line 5 lets us start git daemon on line 6, where it will happily listen on port 9418, blissfully unaware that nobody but its parent can see the interface it is listening on.

Instead of calling exec $@ as expected, the namespace script instead execs socat in such a way than any connection to the /run/socket socket is forwarded to the container's port 9418. Creating this socket is the signal listen is waiting for before it starts forwarding incoming data from the internet to the service. Failure to make it appear within a handful of second is a sign that the service failed to start.

Usually listen calls socat inside the container like so:

socat -dd UNIX-listen:/run/socket,fork EXEC:/srv/$service

where /srv/$service is of course the service script, which will communicate with the internet through its standard stream.

By hijacking the expected behaviour and calling socat by itself, tcp9418.namespace can forward data to git daemon's port instead.

As for the tcp9418 script, it is just a link to true, it is never called anyway, it is only used so that by detecting its presence listen will start the service.

To make git, ip, etc. available to the service, /run/listen/tcp9418/profile is created by a call to:

(os/profile "/run/listen/tcp9418/profile"
            #:name 'git-profile
            #:packages '("git" "iproute2" "coreutils" "bash" "socat"
                         "the-dam-org-f29p"))

From the user's point of view, service scripts are fully network transparent. They read data on their standard input and write answers on their standard output. One instance of the script will be launched for each new connection.

Service script may specify software dependencies by setting a Guix profile in /run/listen/tcpXXX/profile, either manually (by calling guix install --profile=/run/listen/tcpXXX/profile ...) or with the declarative operating system configuration function os/profile.

Service scripts are run in a container fully isolated from the rest of the system. To access data from the outside, one mounts a 9P server in the service's companion namespace script. This 9P server can be started manually (calling p9ufs), or through the use of the declarative operating system configuration function os/9p-serve.

Abusing the service namespace script is possible to let non-network transparent servers run without having to rewrite them to use the standard streams. This facility allows typical web-applications to be run with no modifications. the-dam.org redirects https://alice.the-dam.org to alice's first port in her range, and does so for all users, making web application deployment easy, and rootless.

Additional configuration abstractions are available, for example the unprivileged counterparts to os/profile and os/9p-serve are home/profile and home/9p-serve. They allow non-root users to run their own 9P servers and create their own profile, so that the service script they own can access them without root having to intervene. Describing them is outside the scope of this document, but they are available to the-dam.org users.

Also, the register-listen-service function is available to inform the shepherd that some daemons (typically, the 9P servers) shall be started before listen.

The bareness of the container in which the service script runs provides two main advantages:

• It increases security by isolating the service script from the rest of the system. An attacker gaining remote code execution in the container has no obvious way to impact the rest of the system except a denial of service through resource exhaustion (e.g. a fork bomb.).
• It forces the explicit declaration of all used resources (software dependencies, files) and processes (in our example, the git server). This explicitness makes it trivial to run the exact same script in a slightly different container for development, testing, or failover purposes.

listen will loop over the files in /srv/listen/ whose name matches tcpXXX where XXX denotes a port number.

When it finds an executable file, for example /srv/listen/tcp7, listen:

• creates an isolated container in which it:
• activates the service's profile at /run/listen/tcp7/profile if it exists, or the default one at /run/listen/profile otherwise,
• runs the service namespace script /srv/listen/tcp7.namespace if it exists, the default namspace script /srv/listen/listen.namespace otherwise,
• runs socat to link the /run/listen/tcp7/socket socket to the service script /srv/listen/tcp7 standard streams,
• logs the standard error in /var/log/listen/tcp7/service.log.
• Then outside the container, it launches a daemon that will listen on port 7, and forward data back and forth to and from the service script (via /run/listen/tcp7/socket).

listen uses inotify to watch the /srv/listen/ dir for changes. When notified of a change, listen scans the whole directory again:

• listen leaves alone running daemons
  • whose service script remains executable,
  • and whose content has not changed since it started⁶6: To achieve this, listen caches the hash of the script at start time, and compares the current hash with the cached one during the new scan. ,
• listen restarts running daemons
  • whose service remains executable,
  • but whose content changed since they started,
• listen stops every running daemon whose service script has lost its execution bit.
• listen starts every service script that has become executable.

The listen process runs owned by nobody-like user listen. This protects user data from unauthorized reads or writes should listen become compromised or misbehaving.

As a second line of defence, we would have liked listen to run in a container, seeing of the system only what its job requires ⁷7: Note that Plan 9 or Inferno make this trivial :

• GNU Guix's paths:
  • the read only store /gnu/store/ where GNU Guix installs all software,
  • the /var/guix/ dir to communicate with the guix daemon,
• the service scripts in /srv/listen/,
• the log directory in /var/log/listen,
• a runtime directory in /run/listen/,
• the host's network.

Alas, Linux mirrors our low-growth,high-capital-returns economy: privilege remains an inherited property(Piketty and Zucman 2015). Despite Linux namespaces and capabilities, this inheritance model prevents unprivileged access to non-file resources like ports from a container [personal communication with the kernel developers]. One example: Linux does not honor setuid or setcap binaries within containers. Solving this issue would require

• either moving away from the inheritable root-owns-everything model, or its somewhat finer-grained capabilities substitute,
• or exposing non-file resources like ports as files.

We must for now fall back on basic Linux inter-process isolation.

To prevent any other user from messing with the ports listen manages, we set all ports as privileged.

Despite their complexity and coarse-grainedness, we make use of capabilities: user listen can remain wholly unprivileged but for its ability to bind to all ports. A good step in the right direction from inetd mandatorily running as root !

We use Linux capabilities to grant user listen the right to bind to privileged ports (i.e. all the ports), using a setcap-ed wrapper named with-cap-bind, that only user listen can execute.

Despite listen having quite a low footprint on the system, it remains a sensitive target:

• it can access the internet,
• it can listen on any port,
• it can kill or mess with existing service script processes.

Yet, the service script processes constitute a more probable target than listen itself. Indeed they do not just pass data around as listen does: they handle this untrusted user data. Pirates use complex operations on untrusted data, e.g. parsing, as a typical attack route.

Because service scripts present a big attack surface, they must run in an environment so isolated that a full takeover would not negatively impact the rest of the system much.

To this effect they run in containers. GNU Guix's guix shell creates the service containers. They only expose:

• GNU Guix paths,
• the service-specific (e.g. /run/listen/tcp7) runtime directory,
• the service-specific (e.g. /var/log/listen/tcp7) log directory,

Note that the script can not access the network ! Communication with the outside happens through the UNIX domain socket /run/listen/tcp7/socket.

The /srv/listen/tcp* service scripts get their data via the /run/listen/tcp*/socket ⁸8: Note that this path appears as /run/socket to the service script, see figure 4. socket.

The brilliant utility socat acts as plumbing between any two kinds of process input/output (sockets, files, standard input/output/error, etc.).

For e.g. port 7, listen calls it this way:

/run/setuid-programs/with-cap-bind socat -dd \
    TCP4-LISTEN:7,fork,reuseaddr \
    UNIX-connect:/run/7/socket \
    >> /var/log/tcp7/listen.log 2>&1 &

socat runs with the following wrapper, arguments, and options:

with-cap-bind

grants socat the right to bind to privileged ports (see section 7.2).

-dd

enables logging the remote IP addresses. This then feeds tools like fail2ban to react against abuse.

fork

connect to the UNIX domain socket each time a client connects to the tcp socket.

reuseaddr

the socat daemon might die with its TCP socket on port 7 in the TIME_WAIT state. With the socket in this state, no new socat daemon can start again: it would get an "address already in use" error. Going out of the TIME_WAIT state can take up to 4 minutes. To avoid such a delay, socat ignores the TIME_WAIT state thanks to the reuseaddr option.

Interested readers will find an explanation of the TIME_WAIT state in the UNIX socket FAQ.

Port numbers make for a terrible user interface. To use service names instead, one just has to symbolically link the service name to the service script, e.g.

ln -s /srv/listen/finger /srv/listen/tcp79

This allows the owner to edit /srv/listen/finger to change the finger service, instead of having to remember that finger clients connect to port 79.

It also allows authenticated users to start the script by name:

ssh alice@the-dam.org /srv/listen/finger

The script will have an effective user id of alice ⁹9: Unless the script's author has set its setuid bit. , and might therefore give more information than the anonymous version accessible to anyone. By contrast, listen starts the anonymous version with an effective user id of user listen, who can not read much on the system.

A study of the source code of the fourth edition of Plan 9, 9front, and inferno, as well as asking for listen service script examples of 9fans (“Content of Your /Rc/Bin/Service or /Dis/Svc ? – 9fans Thread” 2024) make some usage patterns emerge.

Namespace files: while in the default installation few services appear to have a custom tcpXXX.namespace file, some services like ftp and http do. The others use the site-specific default namespace file /lib/namespace. These put the files to be served where the service expect them, without any complex software configuration mechanism.

Seamless authentication: On Plan 9 and inferno, a user's identity spans a whole cluster, not just a single machine. Authentication is baked into the 9P protocol, and requires either less than 15 lines of boilerplate, or direct handling by some standard middleware. Switching a process' identity, say from alice to bob, requires the caller to prove to the auth server that it knows bob's secret. This is automagically handled by the factotum at both ends of the connection. The namespace file benefit implicitly from that mechanism, provided the currently mounted factotum knows the required secrets.

Simple scripts: Only the two simplest services (the echo and discard protocols) in the default installation are fully implemented as shell scripts. The rest are custom servers written usually in C for Plan 9 and Limbo for inferno, or thin shell wrappers around those custom servers. Reports from the 9fans mailing list, however (“Content of Your /Rc/Bin/Service or /Dis/Svc ? – 9fans Thread” 2024), praise the ease with which one can write a custom network service for a specific use case, such as collecting vac hashes after a backup, a text-based zine, a connection helper to some MUDs, etc.

/net usage: The synthetic /net hierarchy (Presotto and Winterbottom 1993) is wildly used among the listen service scripts, which get it as an argument. As with all the resources on Plan 9, this part of the filesystem can be exported and shared, in total or in part, between different processes on different machines.

We want to transpose the patterns to Linux, despite the lack on integration of two of the basic primitives of the Plan 9 experience:

• per-process view of the filesystem (namespaces),
• unprivileged modification of said view.

This lack of integration on Linux was compensated by

• the use of Guix containers (which offer a per-container view of the filesystem, see section 7.3),
• and our new f29p FUSE 9P client, which allow for unprivileged mounting of 9P2000.L servers from within a container, something that was absolutely impossible before, see section 2.2 ¹⁰10: While technically true, this sentence disregards the fact that some couples of server/clients will work from within a container, see the aside in section 2.2. .

Namespace scripts replace namespace files, as there are no newns or addns on Linux.

Process and file ownership and permission make a poor, but workable, substitute to the seamless 9P authentication mechanism of Plan 9. As detailed in section 6.3, one can run a 9P server as one identity, and use the socket's ownership and permission to allow another user to benefit from this identity's rights. The alternative, which inetd uses, would be to run as root.

/net is replaced by a single socket from which the service script can talk to the incoming connection. A Linux network namespace isolates the service script from the host's network. Compromises made due to the complex and mostly undocumented interactions between Linux namespaces and Linux capabilities (see section 9.2) make porting /net to Linux a very worthwhile target to improve listen.

On Linux and BSD, inetd used to be the tool that would, like listen, launch a network service when a client connects and forward data back and forth between the connection and the service's standard streams.

inetd has fallen out of fashion:

• most servers have grown in complexity, they do request management on their own.
• inetd listens on behalf of several servers, these servers start on-demand when a request comes. This saves CPU and memory, but increases latency, and costs one process per request.
  • The CPU and memory gains decreased to insignificance because of decreasing hardware costs.
  • Conversely, from a business perspective, the cost of latency has gone up.
  • Processes cost more than threads (on Linux). Self-managed servers (such as e.g. nginx) use a single process (sometimes one process per core) and handle each request in a thread. This makes inetd more expensive under load than a single more complex server.
  • Nowadays, people use containers (with podman, docker, kubernetes, etc.). Containers typically simulate a whole operating system to run one or a small handful of services. Container orchestration software fulfill inetd's role and more.
  • Like most good things from UNIX, inetd's role now eschews to systemd, the cancer that will eat Linux's userspace. Almost all modern distributions lack an inetd (except for a handful of indomitable Gauls sensible Linux distributions like GNU Guix). BSDs have kept their sanity and their versions of inetd.

inetd reads its configuration from a text file, typically /etc/inetd.conf. Editing this file requires privileged access. This leads to difficulties in a multi-user context. listen addresses this.

inetd also provides each service process with a complete view of the system, limited only by the ownership under which said service process starts. In contrast, listen places security first by running the service scripts in a initially void namespace.

authbind exists since 1998. It allows access to privileged ports on a per-user and per-port basis. It does so by masking an application's call to bind. The application ends up calling authbind's version, which uses a privileged program to call the real bind in the application's stead.

systemd offers to run ephemeral services, as inetd does. However using this functionality means you also have to chose systemd for everything else the systemd developers bullied their way into (including, but not limited to: your login manager, your desktop environment, your DNS, your logs).

When we begin this adaptation work, we naively believed that Linux capabilities, containers, and the Linux kernel's support of 9P would allow us to emulate Plan 9's per-process namespace on Linux.

The first bad surprise happened when we discovered that within a Linux mount namespace, one can only mount a handful of filesystem types, the only non trivial of which is FUSE. 9P is excluded. We therefore had to write a 9P-to-FUSE wrapper (see section 2.2) instead of relying on the kernel's v9fs.

We would like to work in the direction of a patch that would allow in-container unprivileged mounts of 9P filesystems on Linux.

Another bad surprise happened when we discovered that, first setuid binaries, and worst, setcap binaries, had no effect in Linux user namespaces. After emailing the kernel developers themselves we got confirmation that what we wanted to achieve: use a setcap binary to grant listen the ability to bind to privileged ports from within a Linux user namespace, was impossible.

This makes it impossible to run listen in its own namespace, as we initially wanted to do (see section 7.2), instead it gets a full view of the system, limited only by its identity.

Porting /net to Linux, would allow us to run listen in a container in which the host's /net is 9P-imported, and stop caring the Linux capabilities feature of which was mildly said that "coherence in its design and implementation are not particularly evident"¹¹11: https://lwn.net/Articles/632520 .

After running their initialization code, and just before exec-ing into listen's provided command, namespace scripts should freeze their namespaces, and prevent the service script from doing anything not strictly necessary to answer the clients' requests. One way to do that is to port OpenBSD's pledge to bash. Justine has written an awesome port of pledge() to Linux. We wish to incorporate it in a loadable bash builtin, which would allow pledges to happen mid-process in bash, and not just as a wrapper to a command.

listen's ability to bind on privileged ports hinges on the with-cap-bind wrapper. As of <2024-01-08 Mon> this wrapper supports any command. Its only use consists of a socat invocation that redirects data from a port to a service container's /run/socket:

/run/setuid-programs/with-cap-bind "$(which socat)" -dd \
    TCP4-LISTEN:"$port",fork,reuseaddr \
    UNIX-connect:/run/listen/"$service"/socket \
    >>/var/log/listen/"$service"/listen.log 2>&1 &

We should make the port number the only argument to the wrapper, and bake the socat invocation into the wrapper. That way, when an attacker gains remote code execution as user listen, it can not bind arbitrary services to any port unless it also gains write rights to /srv/listen (which user listen does not have).

Our work on listen started when looking for a clean way to write a finger server and getting frustrated that none existed. Our finger service being operational, our sights are now set on new protocols aiming for simplicity like gemini and nostr.

As mentioned in section 6.5, an attacker taking over a service process could exhaust the system's resources. We are working on a fix to that, but Linux' API for resource limits has complex interactions with Linux namespaces, that we do not fully understand yet.

Securing the listen process proved the most frustrating part of the design work.

For example: the question "how do unprivileged users bind to privileged ports ?" admits no good answer, because the question itself is wrong on so many levels:

• Port number themselves stem from TCP emerging from earlier protocols (see the early RFCs 322, 349, 433 and those that obsolete them), and a clean design would probably elect to eschew them, leveraging a \(2^{128}\) address space to allow process-to-process communication, instead of the route-to-host, then route-to-process dance we do know.
  • The host to process frontier should be an implementation detail on the receiving end, not baked so deeply in the stack.
  • This barrier may even change from request to request as new hosts come up or down depending on load.
  • This already happens anyway with e.g. kubernetes, but we would have less cruft if it was baked into the protocol.
• We should use strings instead of numbers to specify the protocol.
• Apart from ease of implementation and historically underpowered hardware, why does one need to specify privileged ports as a single range ? One should be able to restrict access on a per-port basis.
• Even then, why should one manage ports as a monolithic entity that either all users, or no user but root, can access ?
• Ports should benefit from a fine-grained access control API, like files do. Exposing ports as virtual files would allow that.