Deploy to Linux VPS with SourceHut Builds
In this post, I will explore the deployment of a static blog from a SourceHut git repository to a Linux VPS. These instructions are specific to the Lektor static site generator (ssg), but they could easily apply to other static sites generators with minor modifications. This post assumes some familiarity with bash, nginx and sysadmin things.
Create a SourceHut account first. Then create a repository for a blog and push local changes to it. This repository must contain a Lektor blog.
SourceHut requires a build manifest
.build.yml to build, test and deploy code. Before such a manifest is created, we can test our build scripts on the SourceHut Builds UI.
Add build manifest
Go to SourceHut Builds at https://builds.sr.ht and click "Submit Manifest". Start with the smallest possible definition that has a single task 'greet'. This is simply a bash command to echo a hello.
image: debian/testing tasks: - greet: | echo hello
Place this in the big textbox and click Submit. This will boot a Debian testing image and run the single task 'greet'. The output is shown in two sections. One for the setup of the image and the other for the sole task 'greet'.
[#3338383] 2022/08/29 03:44:56 Booting image debian/testing (default) on port 22708 [#3338383] 2022/08/29 03:44:57 Waiting for guest to settle [#3338383] 2022/08/29 03:45:04 Sending tasks [#3338383] 2022/08/29 03:45:04 Sending build environment [#3338383] 2022/08/29 03:45:04 Running task greet
+ echo hello + hello
Build complete: success 2 seconds ago (took 8 seconds)
Great. The build ran for 8 seconds. Two things to notice from the build output.
- After the Debian image is booted, first thing that happens is sending of tasks.
It is now established how a build definition or a manifest is created for testing purposes. Given the primary goal, the next steps are broken down like this:
Setup build tasks
- Install Lektor
- Build static blog
- Deploy to VPS
Before setting each of these three tasks up, update the build manifest as follows.
image: debian/testing tasks: - install-lektor: | echo 'installing Lektor' - build-static-blog: | echo 'building static blog' - deploy-blog-to-server: | echo 'deploying blog to server'
Submit this manifest simply by clicking on Edit and resubmit.
[#834118] 2022/08/29 14:34:18 Booting image debian/testing (default) on port 22836 [#834118] 2022/08/29 14:34:19 Waiting for guest to settle [#834118] 2022/08/29 14:34:26 Sending tasks [#834118] 2022/08/29 14:34:26 Sending build environment [#834118] 2022/08/29 14:34:27 Running task install-lektor [#834118] 2022/08/29 14:34:27 Running task build-static-blog [#834118] 2022/08/29 14:34:27 Running task deploy-blog-to-server
+ echo 'installing Lektor' installing Lektor
+ echo 'building static blog' building static blog
+ echo 'deploying blog to server' deploying blog to server
Build complete: success 2 seconds ago (took 8 seconds)
Again, all tasks are sent to the Debian image as per the
Lektor is not packaged in Debian. Put this command in the install task.
sudo pip install lektor
pip is a package installer for Python. So it is a build dependency for Lektor. Build dependencies go into another section called
image: debian/testing packages: - pip tasks: - install-lektor: | sudo pip install lektor
Submit this manifest.
[#834374] 2022/08/29 21:34:28 Booting image debian/testing (default) on port 22095 . [#834374] 2022/08/29 21:34:38 Sending tasks . [#834374] 2022/08/29 21:34:39 Installing packages Reading package lists... Building dependency tree... . Preconfiguring packages ... Fetched 20.1 MB in 4s (4798 kB/s) . . Setting up python3-minimal (3.10.6-1) ... . Setting up python3-pip (22.2+dfsg-1) ...
Full setup output is here
+ sudo pip install lektor Collecting lektor Collecting Jinja2>=3.0 Collecting Werkzeug<3 Collecting Flask Collecting MarkupSafe>=2.0 . . . Building wheels for collected packages: inifile Successfully built inifile Installing collected packages: . . . . Successfully installed . . . .
Full install-lektor output is here
Build complete: success 2 seconds ago (took 41 seconds)
Next, building the Lektor blog is straightforward. There is a
lektor build sub command with minimal arguments that does this.
lektor build --output-path public -f minify:html,css,js
This command builds all the markdown (.lr files in case of Lektor) files and outputs static HTML files into the public directory. The flag
-f is a build flag that allows plugins to customize the pre and post build events. In this case, minify is a plugin that minifies† static files of type .html, .css and .js.
† Minification is a process of removing all unnecessary characters from static files without changing its functionality.
A git repository that holds the blog (or a site) is required. Specify that git repository that was created earlier in a
sources section. This repository is automatically cloned in the current working directory ahead of the tasks.
image: debian/testing packages: - pip sources: - https://git.sr.ht/~username/myblog tasks: - install-lektor: | sudo pip install lektor - build-lektor-blog: | lektor build --output-path public -f minify:html,css,js
Submit this manifest.
. . . [#834383] 2022/08/29 22:01:31 Cloning repositories Cloning into 'myblog'... + cd myblog + git submodule update --init
skipped output in this section for brevity
+ lektor build --output-path public -f minify:html,css,js Usage: lektor build [OPTIONS] Try 'lektor build --help' for help. Error: Could not automatically discover project. A Lektor project must exist in the working directory or any of the parent directories.
Build complete: failed 3 seconds ago (took 34 seconds)
build-lektor-blog has failed. The error says that Lektor couldn't find the lektor repository. The
setup task has new lines that highlight that the git repository is indeed cloned. It then moves (cd) into the
myblog directory and does a
git submodule update --init. So why did the
SourceHut build tasks seem to do a
popd after a task finishes successfully. Subsequent tasks need to cd into the cloned repository directory. It is a simple fix.
image: debian/testing packages: - pip sources: - https://git.sr.ht/~username/myblog tasks: - install-lektor: | sudo pip install lektor - build-lektor-blog: | cd myblog # ADD THIS lektor build --output-path public -f minify:html,css,js
Submit this manifest.
+ cd myblog + lektor build --output-path public -f minify:html,css,js Collecting lektor-minify==1.2 Collecting lektor-atom==0.4.0 Started build U index.html U run-websites-locally-with-caddy/index.html U scoop-installer/index.html U fresh-debian-testing-setup-with-bspwm/index.html U 404.html U sitemap.xml U tags/index.html U about/index.html U changelog/index.html U atom.xml U tags/dotnet/index.html U tags/blog/index.html U tags/sourcehut/index.html U tags/webapp/index.html U static/simple.css Finished build in 5.89 sec
Full build-lektor-blog output is here.
Build complete: success 4 seconds ago (took 41 seconds)
The public directory now has all the static HTML files.
The final step is to deploy the public directory into a Linux server. Login to the Linux server:
Create a separate user to handle the deployment of the static blog. The
-m flag creates a home directory for this user.
$ sudo useradd -m deploy
Give this user permission to update the /var/www directory. Add the user deploy to a group called
www-data This involves a few steps.
$ sudo usermod -aG www-data deploy
Verify that the user is added the group:
$ grep ^www-data /etc/group
Change the group ownership of the
/var/www to the
www-data group. This is so that file and directory modifications are limited to users from a single group:
$ sudo chgrp www-data /var/www/
Add read, write and execute permissions to the group owner:
$ sudo chmod g+rwx /var/www
Make the deploy user the owner for the specific directory under
$ sudo chown deploy:www-data /var/www/myblog/ -R
This assumes that the
myblog directory was created as part of the nginx server setup and configuration. If it wasn't earlier, then create the
myblog directory manually and then configure for nginx later as needed.
With this, the deployment user is created and configured with the right permissions. Now, log in as the deploy user to generate an SSH key. The
cd at the end is to go to the home directory of this user.
$ sudo su deploy && cd $ pwd /home/deploy
Generate ssh key. Note that the newer Ed25519 scheme is not yet supported in SourceHut Builds. Leave the passphrase as empty.
Append the generated public key into the
authorized_keys file. This file maintains a list of authorized public keys that are allowed access into this server. When a client tries to authenticate with ssh into this server, it verifies its public key against this file.
$ cat .ssh/id_rsa.pub >> .ssh/authorized_keys
Display the generated private key. Copy this manually to the clipboard.
$ cat .ssh/id_rsa
This public and private key combination is our authentication mechanism from SourceHut Builds into the Linux server.
To enable this from the SourceHut Builds side, go to the secrets management dashboard https://builds.sr.ht/secrets and paste the private key into the Secret textbox. Select the SSH Key as the secret type and add this secret. Creating this secret will generate a UUID. This UUID is used directly in the builds to identify a particular secret.
Add this UUID to our build manifest.
image: debian/testing secrets: - c9394238-41de-4b43-a2f9-4609df0d7896 packages: - pip sources: - https://git.sr.ht/~username/myblog tasks: - install-lektor: | sudo pip install lektor - build-lektor-blog: | cd myblog lektor build --output-path public -f minify:html,css,js
The directory and the permissions are ready. Using a copy command is the final step. The
rsync command is a standard for this purpose.
$ rsync -r public/* deploy@mylinuxserver-hostname:/var/www/myblog/ -e "ssh -o StrictHostKeyChecking=no"
This command recursively copies all content from the public directory into
/var/www/myblog/ directory of the specified hostname as the user deploy using the ssh command. I suggest trying with rsync locally first to understand potential issues. Also,
rsync is not available in Debian/testing image by default. Add it as a package.
image: debian/testing secrets: - c9394238-41de-4b43-a2f9-4609df0d7896 packages: - pip - rsync sources: - https://git.sr.ht/~username/myblog tasks: - install-lektor: | sudo pip install lektor - build-lektor-blog: | cd myblog lektor build --output-path public -f minify:html,css,js - deploy-to-server cd myblog rsync -r public/* deploy@mylinuxserver-hostname:/var/www/myblog/ -e "ssh -o StrictHostKeyChecking=no"
Now, submit the manifest one last time.
[#832598] 2022/08/27 09:09:25 Booting image debian/testing (default) on port 22184 [#832598] 2022/08/27 09:09:26 Waiting for guest to settle [#832598] 2022/08/27 09:09:34 Sending tasks [#832598] 2022/08/27 09:09:34 Sending build environment [#832598] 2022/08/27 09:09:35 Sending secrets [#832598] 2022/08/27 09:09:35 Resolving secret b5487610-ce34-4bd3-b869-bfb009904a23 [#832598] 2022/08/27 09:09:35 Installing packages
+ sudo pip install lektor . .
+ cd blog + lektor build --output-path public -f minify:html,css,js . .
+ rsync -r myblog/public/2011-in-review myblog/public/2012-in-review myblog/public/2016-in-review myblog/public/2017-in-review myblog/public/2018-in-review myblog/public/2019-in-review myblog/public/2020 myblog/public/2021 myblog/public/404.html myblog/public/404.html.gz myblog/public/a-crunchbang-experience myblog/public/a-crunchbang-experience-a-year-later myblog/public/a-plain-js-table-of-contents-generator myblog/public/about myblog/public/align-pagination-links-using-flexbox deploy@myblog:/var/www/myblog/ -e 'ssh -o StrictHostKeyChecking=no' Warning: Permanently added '[myblog]' to the list of known hosts.
If the deploy-to-server task is successful, ssh into the server and verify that all blog content is available at
Commonly static blog users prefer to deploy to the server each time a new post/article is pushed to the server. To enable this, copy the build manifest from the most recent successful build and create a build manifest file
.build.yml with these contents at the root of the blog's git repository.
git push to the blog's repository results in a new build automatically. The output from the push message displays the link to the build as an added convenience.