Build Your Own Decentralized Dropbox with Syncthing

Not only are cloud storage services getting expensive over time, they’re also complete black-boxes in terms of data security and client implementations. Apart from Dropbox’ interesting past, and the company obviously being an attractive target for various actors, its client implementation isn’t even that great to begin with. While the desktop client might seem convenient, oftentimes it’s being worse than McAfee AntiVirus in terms of performance impact – which is one reason why a whole ecosystem of third party clients came to life.

With many people using Dropbox and the likes primarily to synchronize their files between two or more devices – since especially Dropbox is way too expensive to be used as just an archive/cloud backup and there are more suitable services like Wasabi for that – it begs the question, why a cloud service would be needed in first place? Instead of sending all your files up into the cloud and then downloading them back onto a second device, how about we simply connect the two devices with each other and let them exchange the files directly? Welcome to the world of peer-to-peer!

Peer-to-peer (or short, p2p) is a decentralized communication model where each participant (peer) has equal status and is capable of initiating communication with any other peer in the network. In the context of file synchronization, a p2p system allows devices to directly exchange data with each other, eliminating the need for a central server to coordinate the synchronization process.

Traditional file synchronization services, such as Dropbox, typically rely on a centralized server. When you update a file on one device, it is uploaded to the server, and then all other connected devices download the updated file from the server. Peer-to-peer file synchronization, on the other hand, enables devices to sync directly with each other, providing a more private, secure, and efficient way of managing files.

Syncthing is an open-source, peer-to-peer file synchronization application that allows you to securely sync files between devices without relying on a centralized server. It ensures privacy, security, and complete control over your data. We’ll be using Syncthing in this guide to set up our own decentralized Dropbox.

How does Syncthing work?

When you launch Syncthing on a device, it uses a unique identifier (Device ID) to distinguish itself in the network. Devices can discover each other using a decentralized discovery mechanism without relying on a central server. Once devices discover each other, they establish direct connections for communication. These connections are secured using TLS, ensuring the confidentiality and integrity of data exchanged between devices.

As a user, you define folders that you want to synchronize over Syncthing to other peers. Each folder can have its own synchronization settings and can be shared with different peers.

For synchronizing the files, Syncthing employs a block-level synchronization mechanism. Instead of transferring entire files, it breaks them down into smaller blocks. Only the modified blocks are transferred, reducing bandwidth usage and speeding up synchronization. Additionally, Syncthing uses an event-driven model to detect changes in files and trigger synchronization accordingly. When a file is modified, added, or deleted, Syncthing detects the change and updates the corresponding blocks on connected devices.

In cases where conflicts arise (e.g., changes made on multiple devices simultaneously), Syncthing provides conflict resolution mechanisms to ensure data integrity.

Installing Syncthing

Note: Apart from macOS, either of the commands below should be ran as root user.

Ubuntu/Debian

apt-get update
apt-get install syncthing

Fedora

dnf install syncthing

Arch

pacman -S syncthing

Gentoo

emerge -a net-p2p/syncthing

OpenBSD

pkg_add syncthing

macOS

brew install syncthing

Windows

Oh boy. Alright. First, visit the official Syncthing website at https://syncthing.net and click on the Download section. Select the appropriate version for Windows. This will typically be a 64-bit or 32-bit version, depending on your system.

Once the download is complete, locate the installer file (usually named something like syncthing-windows-amd64-vX.XX.X.exe) and double-click it to run it. In the installation wizard, choose your preferred language and click Next. Jeez, are we really going through all of this now? Ouff. Choose the components you want to install. The default settings are usually sufficient for most users. Click Next. Select the destination folder where you want to install Syncthing. Click Next. Choose whether you want to create a Start Menu folder for Syncthing. Click – you guessed it – Next. Decide whether you want to create a desktop icon for Syncthing. Click Next one last time. Review your choices and click “Install” to begin the installation process. While you’re at reviewing choices, review some life choices as well and ask yourself why you’re using Windows to begin with. Once the installation is complete, click Finish.

After installation, Syncthing should launch automatically. If not, you can find it in the Start Menu or on your desktop.

Setting up Syncthing

• Launch Syncthing on each device.
• Access the Syncthing web interface by navigating to http://localhost:8384 in your web browser.
• On the first device, click on Add Remote Device and enter the Device ID of the second device. This ID can be found in the web interface under the Actions menu.
• Configure folder synchronization:
  • Click on Add Folder.
  • Set the Folder Path to the desired directory (e.g., /home/user/cloud).
  • Add the Device ID of the remote device.
• Repeat the process on the second device, ensuring that both devices have each other’s Device IDs.

Let’s set up an example configuration with five devices: A Linux laptop, a Linux workstation, an OpenBSD server, an Android smartphone, and a macOS laptop.

Folder Structure

We’re going to create a folder named cloud in the $HOME of all computer devices, that is going to contain every synchronized folder. The overall structure is going to look roughly like this:

cloud
├── documents
│   ├── business
│   └── personal
├── downloads
├── ebooks
├── library
│   ├── keepass
│   ├── gnupg
│   └── ssh
└── photos
    ├── iphoto
    ├── smartphone
    └── wallpapers

Syncthing Configuration

It is important to note that file permissions are not supposed to be synced with Android devices. Hence, the Ignore Permissions option should be turned on for every share that’ll be synced with Android. However, in most cases, the option can be turned on for any share, unless the data depends on specific permissions.

Laptop

The laptop syncs the folder business in documents, the downloads folder, the folders keepass, gnupg and ssh in library and the smartphone and wallpapers folders in photos.

For that we add each of these folders as individual shares to Syncthing, ideally prefixing their names with the parent folder, e.g. documents-business for the documents/business/ folder. As folder path we specify ~/cloud/documents/business.

Workstation

The workstation syncs the folder personal in documents, the downloads folder, the folders keepass, gnupg and ssh in library and the smartphone and wallpapers folders in photos.

For that we add each of these folders as individual shares to Syncthing, ideally prefixing their names with the parent folder, e.g. documents-personal for the documents/personal/ folder. As folder path we specify ~/cloud/documents/personal.

Server

The server syncs all folders since we are using it as a backup for all the data. Again, due to the peer-to-peer nature of Syncthing we don’t need a server for sync to work. In our case, the server is just another participant that will take care that all data is redundant and hence synced at least on two different devices.

For that we add each folder as individual share to Syncthing, ideally prefixing the name with the parent folder, e.g. documents-personal for the documents/personal/ folder. As folder path we specify ~/cloud/documents/personal.

Smartphone

The Android smartphone syncs the downloads and the ebooks folders, the folder gnupg in library and the smartphone and wallpapers folders in photos.

The photos/smartphone/ folder is supposed to be configured as a Send Only folder type on the smartphone, and the local target should be your DCIM camera folder.

The downloads/ folder should target the default Android Download folder. You will have to enter the folder path manually, since the file picker won’t let you select it. Just enter ~/Download as folder path.

The ebooks, gnupg and wallpapers folder can be placed anywhere on the Android filesystem. However, in order to keep everything more or less identical, it’s advisable to create a folder cloud and place them into the same structure that is being used on all other devices.

MacBook

The MacBook syncs the folder personal in documents, the downloads folder, the folders gnupg and ssh in library and the iphoto and wallpapers folders in photos.

For that we add each of these folders as individual shares to Syncthing, ideally prefixing their names with the parent folder, e.g. documents-personal for the documents/personal/ folder. As folder path we specify ~/cloud/documents/personal.

The End Result

cloud
├── documents
│   ├── business    (server, laptop)
│   └── personal    (server, workstation, MacBook)
├── downloads       (server, laptop, workstation, smartphone, MacBook)
├── ebooks          (server, smartphone)
├── library
│   ├── keepass     (server, laptop, workstation)
│   ├── gnupg       (server, laptop, workstation, smartphone, MacBook)
│   └── ssh         (server, laptop, MacBook)
└── photos
    ├── iphoto      (server, MacBook)
    ├── smartphone  (server, laptop, workstation, smartphone [send only])
    └── wallpapers  (server, laptop, workstation, smartphone, MacBook)

“Why can’t we just sync the whole cloud folder as a single share on Syncthing?”, you might ask. We do this in order to have our own poor man’s selective sync – a feature that’s not quite supported by Syncthing. While Syncthing has a filter for ignoring files that could be used to have something like selective sync, I found it too cumbersome to use on a day to day basis and hence decided to split my cloud folder into individual shares.

With Syncthing, you’ve now created your decentralized file synchronization system, connecting multiple devices seamlessly. This not only provides a Dropbox-like experience but also empowers you with control over your data, ensuring privacy and security in the cloud. Syncthing’s simplicity and cross-platform support make it an excellent choice for those seeking a reliable and decentralized file synchronization solution.

Bonus Round: iOS

As you might have found out by now, Syncthing is not available for iOS devices. However, not all hope is lost. There is a proprietary, closed-source app that does pretty much what Syncthing does – that is p2p file syncing – and that is available on iOS: Resilio Sync. Since you own an iOS device, I’m assuming that you’re also owning at least one macOS or Windows device alongside of it, on which you can install Resilio Sync. In Resilio, you can then sync the very same folder that you shared via Syncthing to your iOS devices, using the Resilio Sync iOS app. This way, you could include your iOS devices into the peer-to-peer group, and you would only need one device that acts as sort of a bridge between Syncthing and Resilio, on which both, the Syncthing desktop app, as well as the Resilio Sync desktop app are running.

It’s a hacky solution, but it works.

Update: A reader (neropol) has pointed out that there is in fact an iOS client for Syncthing, published under the name Möbius Sync. It appears to be free, but it apparently has in-app purchases that relate to its sync functionality. I have not tested this app, however, according to neropol it has been working great for his use case.