skip to main content

Deploy to Linux VPS with SourceHut Builds

In this post, I will explore the deployment of a static blog from a SourceHut git repository to a Linux VPS. These instructions are specific to the Lektor static site generator (ssg), but they could easily apply to other static sites generators with minor modifications. This post assumes some familiarity with bash, nginx and sysadmin things.

Create a SourceHut account first. Then create a repository for a blog and push local changes to it. This repository must contain a Lektor blog.

E.g. https://git.sr.ht/~username/myblog

SourceHut requires a build manifest .build.yml to build, test and deploy code. Before such a manifest is created, we can test our build scripts on the SourceHut Builds UI.

Add build manifest

Go to SourceHut Builds at https://builds.sr.ht and click "Submit Manifest". Start with the smallest possible definition that has a single task 'greet'. This is simply a bash command to echo a hello.

image: debian/testing
tasks:
  - greet: |
      echo hello

Place this in the big textbox and click Submit. This will boot a Debian testing image and run the single task 'greet'. The output is shown in two sections. One for the setup of the image and the other for the sole task 'greet'.

Build output:

setup
[#3338383] 2022/08/29 03:44:56 Booting image debian/testing (default) on port 22708
[#3338383] 2022/08/29 03:44:57 Waiting for guest to settle
[#3338383] 2022/08/29 03:45:04 Sending tasks
[#3338383] 2022/08/29 03:45:04 Sending build environment
[#3338383] 2022/08/29 03:45:04 Running task greet
greet
+ echo hello
+ hello
Build complete: success 2 seconds ago (took 8 seconds)

Great. The build ran for 8 seconds. Two things to notice from the build output.

  1. After the Debian image is booted, first thing that happens is sending of tasks.
  2. The build job page reloaded once before it showed the greet step. This is facilitated by a tiny JavaScript snippet on the page. It uses the good old meta tag to do this.

Offtopic: This page reload snippet and some necessary JavaScript for payments are the only two pieces of JavaScript I found on the entire SourceHut suite yet.

It is now established how a build definition or a manifest is created for testing purposes. Given the primary goal, the next steps are broken down like this:

Setup build tasks

  • Install Lektor
  • Build static blog
  • Deploy to VPS

Before setting each of these three tasks up, update the build manifest as follows.

image: debian/testing
tasks:
  - install-lektor: |
      echo 'installing Lektor'
  - build-static-blog: |
      echo 'building static blog'
  - deploy-blog-to-server: |
      echo 'deploying blog to server'

Submit this manifest simply by clicking on Edit and resubmit.

Build output:

setup
[#834118] 2022/08/29 14:34:18 Booting image debian/testing (default) on port 22836
[#834118] 2022/08/29 14:34:19 Waiting for guest to settle
[#834118] 2022/08/29 14:34:26 Sending tasks
[#834118] 2022/08/29 14:34:26 Sending build environment
[#834118] 2022/08/29 14:34:27 Running task install-lektor
[#834118] 2022/08/29 14:34:27 Running task build-static-blog
[#834118] 2022/08/29 14:34:27 Running task deploy-blog-to-server
install-lektor
+ echo 'installing Lektor'
installing Lektor
build-static-blog
+ echo 'building static blog'
building static blog
deploy-blog-to-server
+ echo 'deploying blog to server'
deploying blog to server

Build complete: success 2 seconds ago (took 8 seconds)

Again, all tasks are sent to the Debian image as per the setup section.

Install-Lektor task

Lektor is not packaged in Debian. Put this command in the install task.

sudo pip install lektor

pip is a package installer for Python. So it is a build dependency for Lektor. Build dependencies go into another section called packages.

image: debian/testing
packages:
  - pip
tasks:
  - install-lektor: |
      sudo pip install lektor

Submit this manifest.

Build output:

setup
[#834374] 2022/08/29 21:34:28 Booting image debian/testing (default) on port 22095
.
[#834374] 2022/08/29 21:34:38 Sending tasks
.
[#834374] 2022/08/29 21:34:39 Installing packages
Reading package lists...
Building dependency tree...
.
Preconfiguring packages ...
Fetched 20.1 MB in 4s (4798 kB/s)
.
.
Setting up python3-minimal (3.10.6-1) ...
.
Setting up python3-pip (22.2+dfsg-1) ...

Full setup output is here

install-lektor
+ sudo pip install lektor
Collecting lektor
Collecting Jinja2>=3.0
Collecting Werkzeug<3
Collecting Flask
Collecting MarkupSafe>=2.0
. .
.
Building wheels for collected packages: inifile
Successfully built inifile
Installing collected packages: . . . .
Successfully installed . . . .

Full install-lektor output is here

Build complete: success 2 seconds ago (took 41 seconds)

Build-static-blog task

Next, building the Lektor blog is straightforward. There is a lektor build sub command with minimal arguments that does this.

lektor build --output-path public -f minify:html,css,js

This command builds all the markdown (.lr files in case of Lektor) files and outputs static HTML files into the public directory. The flag -f is a build flag that allows plugins to customize the pre and post build events. In this case, minify is a plugin that minifies† static files of type .html, .css and .js.

† Minification is a process of removing all unnecessary characters from static files without changing its functionality.

A git repository that holds the blog (or a site) is required. Specify that git repository that was created earlier in a sources section. This repository is automatically cloned in the current working directory ahead of the tasks.

image: debian/testing
packages:
  - pip
sources:
  - https://git.sr.ht/~username/myblog
tasks:
  - install-lektor: |
      sudo pip install lektor
  - build-lektor-blog: |
      lektor build --output-path public -f minify:html,css,js

Submit this manifest.

Build output:

setup
. .
.
[#834383] 2022/08/29 22:01:31 Cloning repositories
Cloning into 'myblog'...
+ cd myblog
+ git submodule update --init
install-lektor
skipped output in this section for brevity
build-lektor-blog
+ lektor build --output-path public -f minify:html,css,js
Usage: lektor build [OPTIONS]
Try 'lektor build --help' for help.
 
Error: Could not automatically discover project.  A Lektor project must exist in the working directory or any of the parent directories.
Build complete: failed 3 seconds ago (took 34 seconds) 

The build-lektor-blog has failed. The error says that Lektor couldn't find the lektor repository. The setup task has new lines that highlight that the git repository is indeed cloned. It then moves (cd) into the myblog directory and does a git submodule update --init. So why did the build-lektor-blog fail?

SourceHut build tasks seem to do a popd after a task finishes successfully. Subsequent tasks need to cd into the cloned repository directory. It is a simple fix.

image: debian/testing
packages:
  - pip
sources:
  - https://git.sr.ht/~username/myblog
tasks:
  - install-lektor: |
      sudo pip install lektor
  - build-lektor-blog: |
      cd myblog # ADD THIS
      lektor build --output-path public -f minify:html,css,js

Submit this manifest.

Build output:

setup
install-lektor
build-lektor-blog
+ cd myblog
+ lektor build --output-path public -f minify:html,css,js
Collecting lektor-minify==1.2
Collecting lektor-atom==0.4.0
Started build
U index.html
U run-websites-locally-with-caddy/index.html
U scoop-installer/index.html
U fresh-debian-testing-setup-with-bspwm/index.html
U 404.html
U sitemap.xml
U tags/index.html
U about/index.html
U changelog/index.html
U atom.xml
U tags/dotnet/index.html
U tags/blog/index.html
U tags/sourcehut/index.html
U tags/webapp/index.html
U static/simple.css
Finished build in 5.89 sec

Full build-lektor-blog output is here.

Build complete: success 4 seconds ago (took 41 seconds)

The public directory now has all the static HTML files.

Deploy-to-server task

The final step is to deploy the public directory into a Linux server. Login to the Linux server:

ssh my-linx-server

Create a separate user to handle the deployment of the static blog. The -m flag creates a home directory for this user.

$ sudo useradd -m deploy

Give this user permission to update the /var/www directory. Add the user deploy to a group called www-data This involves a few steps.

$ sudo usermod -aG www-data deploy

Verify that the user is added the group:

$ grep ^www-data /etc/group

Change the group ownership of the /var/www to the www-data group. This is so that file and directory modifications are limited to users from a single group:

$ sudo chgrp www-data /var/www/

Add read, write and execute permissions to the group owner:

$ sudo chmod g+rwx /var/www

Make the deploy user the owner for the specific directory under /var/www.

$ sudo chown deploy:www-data /var/www/myblog/ -R

This assumes that the myblog directory was created as part of the nginx server setup and configuration. If it wasn't earlier, then create the myblog directory manually and then configure for nginx later as needed.

With this, the deployment user is created and configured with the right permissions. Now, log in as the deploy user to generate an SSH key. The cd at the end is to go to the home directory of this user.

$ sudo su deploy && cd
$ pwd
/home/deploy

Generate ssh key. Note that the newer Ed25519 scheme is not yet supported in SourceHut Builds. Leave the passphrase as empty.

$ ssh-keygen

Append the generated public key into the authorized_keys file. This file maintains a list of authorized public keys that are allowed access into this server. When a client tries to authenticate with ssh into this server, it verifies its public key against this file.

$ cat .ssh/id_rsa.pub >> .ssh/authorized_keys

Display the generated private key. Copy this manually to the clipboard.

$ cat .ssh/id_rsa

This public and private key combination is our authentication mechanism from SourceHut Builds into the Linux server.

To enable this from the SourceHut Builds side, go to the secrets management dashboard https://builds.sr.ht/secrets and paste the private key into the Secret textbox. Select the SSH Key as the secret type and add this secret. Creating this secret will generate a UUID. This UUID is used directly in the builds to identify a particular secret.

Add this UUID to our build manifest.

image: debian/testing
secrets:
- c9394238-41de-4b43-a2f9-4609df0d7896
packages:
  - pip
sources:
  - https://git.sr.ht/~username/myblog
tasks:
  - install-lektor: |
      sudo pip install lektor
  - build-lektor-blog: |
      cd myblog
      lektor build --output-path public -f minify:html,css,js

The directory and the permissions are ready. Using a copy command is the final step. The rsync command is a standard for this purpose.

$ rsync -r public/* deploy@mylinuxserver-hostname:/var/www/myblog/ -e "ssh -o StrictHostKeyChecking=no"

This command recursively copies all content from the public directory into /var/www/myblog/ directory of the specified hostname as the user deploy using the ssh command. I suggest trying with rsync locally first to understand potential issues. Also, rsync is not available in Debian/testing image by default. Add it as a package.

image: debian/testing
secrets:
- c9394238-41de-4b43-a2f9-4609df0d7896
packages:
  - pip
  - rsync
sources:
  - https://git.sr.ht/~username/myblog
tasks:
  - install-lektor: |
      sudo pip install lektor
  - build-lektor-blog: |
      cd myblog
      lektor build --output-path public -f minify:html,css,js     
  - deploy-to-server
      cd myblog
      rsync -r public/* deploy@mylinuxserver-hostname:/var/www/myblog/ -e "ssh -o StrictHostKeyChecking=no"

Now, submit the manifest one last time.

Build output:

setup
[#832598] 2022/08/27 09:09:25 Booting image debian/testing (default) on port 22184
[#832598] 2022/08/27 09:09:26 Waiting for guest to settle
[#832598] 2022/08/27 09:09:34 Sending tasks
[#832598] 2022/08/27 09:09:34 Sending build environment
[#832598] 2022/08/27 09:09:35 Sending secrets
[#832598] 2022/08/27 09:09:35 Resolving secret b5487610-ce34-4bd3-b869-bfb009904a23
[#832598] 2022/08/27 09:09:35 Installing packages
install-lektor
+ sudo pip install lektor
. .
build-lektor-blog
+ cd blog
+ lektor build --output-path public -f minify:html,css,js
. .
deploy-to-server
+ rsync -r myblog/public/2011-in-review myblog/public/2012-in-review myblog/public/2016-in-review myblog/public/2017-in-review myblog/public/2018-in-review myblog/public/2019-in-review myblog/public/2020 myblog/public/2021 myblog/public/404.html myblog/public/404.html.gz myblog/public/a-crunchbang-experience myblog/public/a-crunchbang-experience-a-year-later myblog/public/a-plain-js-table-of-contents-generator myblog/public/about myblog/public/align-pagination-links-using-flexbox  deploy@myblog:/var/www/myblog/ -e 'ssh -o StrictHostKeyChecking=no'
Warning: Permanently added '[myblog]' to the list of known hosts.

If the deploy-to-server task is successful, ssh into the server and verify that all blog content is available at /var/www/myblog directory.

Commonly static blog users prefer to deploy to the server each time a new post/article is pushed to the server. To enable this, copy the build manifest from the most recent successful build and create a build manifest file .build.yml with these contents at the root of the blog's git repository.

Now, each git push to the blog's repository results in a new build automatically. The output from the push message displays the link to the build as an added convenience.