Offsite backup with Unraid, Tailscale, Backrest/Restic and Tailvault (Updated)

nguyenquyhy Leave a comment

Update: After further testing, I no longer recommend this approach due to many limitations listed in the last section.

Restic and its companion UI Backrest already allow you to back up to many different locations. However, getting it setup to back up from one Unraid NAS to another might not be straight forward. Here I will go into details on the steps I have used to to make it work.

The goals of this setup are:

  1. Sender will use Backrest while receiver will use Tailvault app in Unraid.
  2. Both apps will have Tailscale integration enabled, and connectivity will be via Tailscale.

Tailvault is a very simple Alpine container that enables SFTP. However, Restic doesn’t support SFTP with password, hence the need for additional setup for key pairs between the 2 containers.

  1. Generate a keypair with PuTTYgen, choose EdDSA and Ed25519
    • After the pair is generated, click Conversions >Export OpenSSH Key and named it id_ed25519
    • The public key can be copied from the box “Public key for pasting into OpenSSH authorized_keys file:”
  2. Install TailVault app in the Community Store with the following modifications:
    • Repository: nguyenquyhy/tailvault:latest
    • Tailscale Hostname: change to something more specific if you are going to have multiple tailvault in your tailnet.
    • Remove SFTP_PASS
    • Add SFTP_PUB_KEY.
      • Config Type: Variable
      • Name: SFTP_PUB_KEY
      • Key: SFTP_PUB_KEY
      • Value: Use the public key in step 1
  3. Open TailVault container log and login to Tailscale using the link in log
  4. Open appdata/TailVault and download ssh_host_ed25519_key.pub
  5. Install Backrest app in the Community Store with Tailscale enabled
  6. Open Backrest container log and login to Tailscale using the link in log
  7. Create known_hosts file on your computer with the following content:
    • <tailvault domain from Tailscale> <content from ssh_host_ed25519_key.pub in step 3>
    • e.g. tailvault.abc.ts.net ssh-ed25519 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA root@aaaaaaaaa
  8. Upload id_ed25519 from step 1 and known_hosts from step 5 to appdata/backrest
  9. Select id_ed25519, click Permissions, change Group and Other to No Access
  10. Edit backrest app in Unraid’s Docker and add the following:
    • Path:
      • Name: Private Key
      • Container Path: /root/.ssh/id_ed25519
      • Host Path: /mnt/user/appdata/backrest/id_ed25519
    • Path:
      • Name: Known Host
      • Container Path: /root/.ssh/known_hosts
      • Host Path: /mnt/user/appdata/backrest/known_hosts
  11. Open backrest URL
  12. Add repo. For Repository URL, use sftp:<username set in step 2>@<tailvault domain from Tailscale>:/backups

References

Guide from backrest: Using SSH (SFTP) Remotes with Docker Compose · Backrest

Alternatives

  1. Backrest is very flexible and the recent UI update makes it look much nicer. However, the lack of concurrency support (for both same repository and different repositories) is a deal breaker for me. I have been moving to Zerobyte for the following features:
    • Concurrent backup: This is very important when there is a very large backup running for hours or days.
    • Allow inputing SFTP private key and host public key directly in the UI, for each repository: step 7, 8, 9 and 10 can be skipped.
  2. SFTP is very slow comparing to the alternative of Restic Rest Server. From my benchmark tests on a Tailscale direct connection of 200ms latency:
    • Initialize a repository: 5s with Rest server and 25s with SFTP
    • Listing snapshots (single snapshot in total): 1s with Rest server and 5s with SFTP
    • Backing up 50GB of data to HDD: 10-11m with Rest server and 15-45m with SFTP

Dev Drive and Dotnet Tools

nguyenquyhy 1 Comment

I recently tried out Dev Drive of Windows 11, where a ReFS partition can be created for your code and packages. It went smoothly and most of my projects were built successfully until it came to my Monogame project.

For your context, Monogame is a game engine written in .NET and allows you to write your game entirely in C#. It has a content pipeline to convert your audio and images assets to certain format, and the pipeline is written as local dotnet tools.

Turns out dotnet tools stop working after I moved Nuget package cache folder to the Dev Drive (with NUGET_PACKAGES environment variable). I tried to trigger the tool manually, but the error message is confusing an unhelpful:

❯ dotnet mgcb
Run "dotnet tool restore" to make the "mgcb" command available.
❯ dotnet tool restore
Skipping NuGet package signature verification.
Tool 'dotnet-mgcb' (version '3.8.1.303') was restored. Available commands: mgcb
❯ dotnet mgcb
Run "dotnet tool restore" to make the "mgcb" command available.

I tried to check my Dev Drive and I can see dotnet-mgcb had been restored correctly. Looking a bit further, I realized that .NET is still trying to look for the tool in the old path in %userprofile%\.nuget (even after computer restart).

I might have guessed the issue by now: there must be a cache somewhere pointing to the old paths. Turned out local dotnet tools cache the package information (including the path) in %userprofile%\.dotnet\toolResolverCache.

The fix is obvious now:

  1. Empty %userprofile%\.dotnet\toolResolverCache
  2. Run dotnet tool restore again and double check toolResolverCache to ensure the files are recreated.

I think ideally, since dotnet tool restore can already get the package to the new cache folder in Dev Drive, it should also try to update this toolResolverCache folder automatically instead of requiring a manual wipe like this.

New country and a new PC

nguyenquyhy Leave a comment

It has been a while since I last published a new post here. I still have quite a number of drafts lining up, but has never managed to come around to finish them. Probably I should restart writing with some update on my life and some interesting things I learned recently.

A couple months ago I moved to a new country halfway around the world with a new job (or actually same old job in a different company), and I took this chance to learn to build a new PC. I have had a bit of experience with PC hardware before, but mainly from replacing part from a complete build, rather than building from an empty case, so this would be my first time having to work with a new case and mainboard.

Read More

Javascript Module Loaders

nguyenquyhy Leave a comment

Javascript has a long history with so many changes over the years. One area that has so much improvements yet might still be so confusing is its support for modularization.

This blog post discusses a part of this area, specifically on how to your Javascript source files can be loaded into your application. This blog post also focuses on Javascript on browser instead of other native environment such as NodeJS.

Read More