Peer-to-peer Git: Radicle Seed Node on OpenBSD

Even though Git is decentralized by design, in reality, it is very much centralized, with plenty of popular projects being in the hands of just a few organizations: GitHub (Microsoft), GitLab, Launchpad (Canonical), and Codeberg, just to name some the most recognized ones. If any of these organizations decide to boot a project – or worse, shut the whole service down – it will definitely lead to at least some headaches and inconveniences for maintainers, as well as users.

Ideally, the Git infrastructure that we use would instead also be fully decentralized, using a peer-to-peer architecture in which no repository lives on only a single server, but is being replicated by many different servers, hosted by different individuals and organizations. This is where Radicle comes in:

Radicle is an open source, peer-to-peer code collaboration stack built on Git. Unlike centralized code hosting platforms, there is no single entity controlling the network. Repositories are replicated across peers in a decentralized manner, and users are in full control of their data and workflow.

In this write-up we’ll be looking at how to set up our seed node, allowing us to not only replicate existing projects on Radicle but also host our repositories and make them available to the rest of the network.

Preparation

Usually, when I build an OpenBSD-based infrastructure, I use a Vultr VPS instance. You can start with the lowest VPS configuration and upgrade over time if necessary. Anything with at least 1GB of RAM and 10GB of storage should be fine.

We’re going to be using the fresh-out-the-oven OpenBSD 7.5 for this setup.

Note: I won’t explicitly mention under which user I’m running each command, hence please read carefully. When the prompt shows seed#, I’m acting as root user, otherwise, when it shows seed%, I’m using the _seed user.

As soon as the OpenBSD VPS has booted, we can log in via SSH and perform a quick update of the system:

seed# syspatch

seed# pkg_add -u

Next, perform a system upgrade to -current. We want this because -stable releases especially for Rust (which we will require for building Radicle v0.9.0) are a bit too outdated:

seed# sysupgrade -s

The system will reboot and when you log in you should be greeted with a version that says -current:

OpenBSD 7.5-current (GENERIC) #23: Thu Apr 11 12:18:37 MDT 2024

Welcome to OpenBSD: The proactively secure Unix-like operating system.

After that, let’s do a quick upgrade of the existing packages and install some handy tools:

seed# pkg_add -uvi
seed# pkg_add git zsh neovim wget mosh rsync htop

What I like to do is link nvim to vim, because typing vim is in my muscle memory. I also like to use zsh as shell. Since that’s all preference, these steps are optional:

seed# ln -s /usr/local/bin/nvim /usr/local/bin/vim
seed# chsh -s /usr/local/bin/zsh root

Another optional but useful thing that I like to do is to change the SSH port and disable password authentication / enable pubkey authentication. Make sure you have an authorized_keys entry with your pubkey in place before applying this change:

seed# sed -i 's/^#Port 22/Port 31231/g;\
              s/^#PubkeyAuthentication .*/PubkeyAuthentication yes/g;\
              s/^#PasswordAuthentication .*/PasswordAuthentication no/g' \
                /etc/ssh/sshd_config
seed# rcctl restart sshd

Afterwards, disconnect from SSH, and re-connect, ideally using mosh, and launch tmux for the sake of comfort. See this post on how to make the mosh experience even smoother.

Radicle

First, let’s install the required software to build Radicle:

seed# pkg_add rust

Then, install Radicle using cargo, the Rust package manager:

FYI: This documentation is using the root user to build the required components. If you don’t feel comfortable with this, feel free to jump to creating the _seed user first and then come back to perform these steps underneath that user.

seed# #https://github.com/rust-lang/cargo/issues/11435#issuecomment-1740163332
seed# ulimit -n 1024
seed# cargo install --force --locked \
  --git https://seed.radicle.xyz/z3gqcJUoA1n9HaHKufZs5FCSGazv5.git \
  radicle-cli \
  radicle-node \
  radicle-remote-helper
seed# mv ~/.cargo/bin/* /usr/local/bin/

FYI: Depending on the performance of your machine, this process might take more than a cup of coffee in time. If you’re setting up a single vCPU VPS, consider running it on a beefier machine and transferring the built binaries to your server.

2024-07-30T15:10:28.883896Z  INFO starting http daemon..
2024-07-30T15:10:28.884217Z  INFO version pre-release (a73c5490)
2024-07-30T15:10:28.891133Z  INFO git version 2.45.2
2024-07-30T15:10:28.891779Z  INFO listening on http://127.0.0.1:8080
2024-07-30T15:10:28.910664Z  INFO using radicle home at /home/_seed/.radicle
2024-07-30T15:10:33.220206Z DEBUG request{id=0}: started processing request
2024-07-30T15:10:33.224040Z ERROR request{id=0}: Error getting node config: command error: (de)serialization failed: missing field `type` at line 1 column 20

/usr/local/bin/rad config init --alias seed.xn--gckvb8fzb.com
✓ Initialized new Radicle configuration at /home/_seed/.radicle/config.json
seed% radicle-node --listen 0.0.0.0:8776 --force
2024-07-30T15:28:26.698Z INFO  node     Starting node..
2024-07-30T15:28:26.699Z INFO  node     Version pre-release (574ac356)
2024-07-30T15:28:26.699Z INFO  node     Unlocking node keystore..
2024-07-30T15:28:26.699Z INFO  node     Node ID is z6MksHwp2VUdawHNWNnd8YwDEkP1E6LuERxCGvTHHarEjYPi
2024-07-30T15:28:26.701Z ERROR node     Fatal: failed to load configuration from /home/_seed/.radicle/config.json: missing field `target` at line 20 column 6

<OBSOLETE>

Normally we would also compile the radicle-httpd package together with all the others. Unfortunately, there’s currently an issue on OpenBSD that prevents it from building:

   Compiling radicle-term v0.9.0 (/root/.cargo/git/checkouts/z3gqcJUoA1n9HaHKufZs5FCSGazv5-cab4735f10a16009/574ac35/radicle-term)
error[E0425]: cannot find value `cmd_name` in this scope
   --> radicle-httpd/src/commands/web.rs:200:36
    |
200 |         let mut cmd = Command::new(cmd_name);
    |                                    ^^^^^^^^ not found in this scope

error[E0425]: cannot find value `cmd_name` in this scope
   --> radicle-httpd/src/commands/web.rs:215:71
    |
215 |                 term::error(format!("Could not open web browser via `{cmd_name}`"));
    |                                                                       ^^^^^^^^ not found in this scope

For more information about this error, try `rustc --explain E0425`.
error: could not compile `radicle-httpd` (lib) due to 2 previous errors

We can, however, manually fix that issue. For that, we need to clone the heartwood repository and patch a single file within the radicle-httpd folder:

seed# git clone https://seed.radicle.xyz/z3gqcJUoA1n9HaHKufZs5FCSGazv5.git
seed# cd z3gqcJUoA1n9HaHKufZs5FCSGazv5/
seed# git diff radicle-httpd/
diff --git a/radicle-httpd/src/commands/web.rs b/radicle-httpd/src/commands/web.rs
index 3229ebf5..fb246539 100644
--- a/radicle-httpd/src/commands/web.rs
+++ b/radicle-httpd/src/commands/web.rs
@@ -196,6 +196,8 @@ pub fn run(options: Options, ctx: impl term::Context) -> anyhow::Result<()> {
         let cmd_name = "open";
         #[cfg(target_os = "linux")]
         let cmd_name = "xdg-open";
+        #[cfg(target_os = "openbsd")]
+        let cmd_name = "echo";

         let mut cmd = Command::new(cmd_name);
         match cmd.arg(auth_url.as_str()).spawn() {

We can then proceed to manually build and install the binary for radicle-httpd:

seed# cd radicle-httpd
seed# cargo build --release
seed# mv ~/.cargo/bin/* /usr/local/bin/

<\OBSOLETE>

We can build and install radicle-httpd using cargo and the radicle-explorer repository:

seed# cargo install --force --locked \
  --git https://seed.radicle.xyz/z4V1sjrXqjvFdnCUbxPFqd5p4DtH5.git \
  radicle-httpd
seed# mv ~/.cargo/bin/* /usr/local/bin/

Seed User

Next, we create the _seed group and user on the system:

seed# groupadd _seed
seed# useradd -d /home/_seed -m -c "Radicle Seed" \
  -g _seed -L daemon -s /sbin/nologin _seed

After that, we can log in as _seed user and configure Radicle:

seed# su -l -s /usr/local/bin/zsh _seed -
seed% /usr/local/bin/rad auth --alias seed.domain.com

As seed nodes do not typically sign permanent artifacts with their key, it is not generally necessary to set up a passphrase, so we can simply skip the prompt by pressing enter.

Now, let’s open the Radicle node configuration to adjust it:

seed% /usr/local/bin/rad config edit

First of all, we need to decide what seeding policy we’d like our seed to have. We can decide between permissive and selective seeding. To understand the differences, let’s have a quick look at the official Radicle manual:

A permissive or “open” policy is said to be fully-replicating, meaning your seed will try to have a fully copy of all repository data available on the network.

A selective or restricted policy requires you, the operator, to manually allow repositories to be seeded. This means that the node will ignore all repositories, except the ones that are pre-configured to allow seeding.

Tl;dr: If you’re looking to simply host a public seed to support the Radicle network, you’d want to go with a permissive policy. If, however, you’re looking to build more of a private seed with only your repositories, or repositories you care about, the selective policy is what you’d want.

For the permissive policy, configure the following values:

{
  "node": {
    ...
    "policy": "allow",
    "scope": "all"
  }
}

For the selective policy, configure the following values instead:

{
  "node": {
    ...
    "policy": "block",
    "scope": "all"
  }
}

Next, we configure the node’s external address with a subdomain/domain that points to the server and for which we’re later on going to create an SSL certificate:

{
  "node": {
    ...
    "externalAddresses": ["seed.domain.com:8776"]
  }
}

Now, we create the rc.d files and adjust the permissions:

seed# cat /etc/rc.d/radicle
#!/bin/ksh

daemon="/usr/local/bin/rad"
daemon_flags="node start -- --listen 0.0.0.0:8776"
daemon_user="_seed"

. /etc/rc.d/rc.subr

rc_cmd $1

seed# chmod 555 /etc/rc.d/radicle

seed# cat /etc/rc.d/radiclehttpd
#!/bin/ksh

daemon="/usr/local/bin/radicle-httpd"
daemon_flags="--listen 127.0.0.1:8080"
daemon_user="_seed"

. /etc/rc.d/rc.subr

pexp="$daemon"

rc_bg=YES

rc_cmd $1

seed# chmod 555 /etc/rc.d/radiclehttpd

Make sure to enable and start the services:

seed# rcctl enable radicle
seed# rcctl start radicle

seed# rcctl enable radiclehttpd
seed# rcctl start radiclehttpd

We can test that radicle-httpd is working now:

seed# curl http://127.0.0.1:8080/api/v1

SSL

To get an SSL certificate (from Let’s Encrypt) we’ll use OpenBSD’s acme-client and set up its configuration, as well as a cron job that will make sure our certificate gets prolonged automatically.

First, create the file /etc/acme-client.conf with the following content:

authority letsencrypt {
  api url "https://acme-v02.api.letsencrypt.org/directory"
  account key "/etc/ssl/private/letsencrypt.key"
}
domain seed.domain.com {
  domain key "/etc/ssl/private/seed.domain.com.key"
  domain certificate "/etc/ssl/seed.domain.com.crt"
  domain full chain certificate "/etc/ssl/seed.domain.com.pem"
  sign with letsencrypt
}

Make sure to replace seed.domain.com with the (sub)domain you pointed to your cloud instance. Next, we’ll need to configure httpd to run on port 80 and provide access to the ACME challenge. Create the file /etc/httpd.conf with the following content:

server "seed.domain.com" {
  listen on * port 80
  root "/htdocs/seed.domain.com"
  location "/.well-known/acme-challenge/*" {
    root "/acme"
    request strip 2
  }
}

Additionally, create the folder /var/www/htdocs/seed.domain.com/ and place an empty index.html into it.

Let’s enable httpd:

znc# rcctl enable httpd
znc# rcctl restart httpd

We can now try to issue our SSL certificate by running the following command:

znc# acme-client -v seed.domain.com

Your newly issued SSL certificate should be available under /etc/ssl/seed.domain.com.{crt,pem} and /etc/ssl/private/seed.domain.com.key.

Next, run crontab -e. This will open the current crontab for editing. Add the following line at the very end of the file:

0       0       *       *       *       acme-client -v seed.domain.com && rcctl 
restart relayd

This will make sure our SSL certificate won’t expire.

relayd

Next we set up relayd as a reverse proxy for the Radicle HTTP backend. relayd takes care of handling SSL termination. We’re going to use the following configuration under /etc/relayd.conf:

table <radicle-default-host> { 127.0.0.1 }

http protocol radicle-https {
  match request header append "X-Real-IP" value "$REMOTE_ADDR"
  match request header append "Host" value "$HOST"
  match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
  match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"
  match request path "/*" forward to <radicle-default-host>

  tcp { nodelay, sack, backlog 128 }

  tls keypair seed.domain.com
  tls { no tlsv1.0, ciphers HIGH }
}

relay radicle-https-relay {
  listen on egress port 443 tls
  protocol radicle-https
  forward to <radicle-default-host> port 8080
}

In /etc/rc.conf.local change relayd_flags to `` (empty). Then launch relayd:

seed# rcctl enable relayd
seed# rcctl start relayd

We can now verify that the API (radicle-httpd) is available over the internet:

your-computer$ curl https://seed.domain.com/api/v1

In addition, we can also verify that the Radicle web app can access the node by opening the following link in our web browser:

https://app.radicle.xyz/nodes/seed.domain.com/

And that’s about it. If you have chosen the selective policy for your node, you might now go ahead and start picking repositories that you want to seed using the rad seed <uri> command, e.g.:

seed% /usr/local/bin/rad seed rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5

To remove a seeded repository, use the /usr/local/bin/rad unseed <uri> command. Make sure to run these commands as the _seed user.

Hint: Make sure to always specify the full path (/usr/local/bin/rad), otherwise you end up calling OpenBSD’s router advertisement daemon (/usr/sbin/rad). You can also adjust your PATH for /usr/local/bin to take precedence over /usr/sbin/.

If you’d like to collaborate with me on Radicle, check out the projects on my seed and feel free to sync with it:

DELETED

(see update at the top)