eik_containers.mdwn (4292B)
1 Containers have some efficiency advantages but the way they are typically set up can use a lot of space on disk and eat up a lot of ram with duplicate services. Standard Debian or CentOS containers easily require 200-600MB of space each. 2 3 Docker and other popular container management systems are oriented towards more _enterprise_ scale operations where containers are frequently set up and torn down with automated cloud orchestration tools. 4 5 If we want to have a more PMC take on containers, what we need probably looks more like a few tiny linux matryoshka dolls. Alpine Linux is based on musl (instead of libc), Busybox (instead of GNU-derived toolchain) and OpenRC (instead of systemD) and gives us a bare-bones but very utilitarian base container image in just 16MB. We can, with a little bit of manual effort get this to run in a systemd-nspawn container. 6 7 ### Build a new alpine container 8 9 While it's possible to use the standard alpine tools to build a system image, somebody has made a [little script](https://github.com/quantum5/alpine-nspawn-install) that deals with a few small annoyances for us. The script is installed in /usr/sbin so you can just: 10 11 `sudo alpine-nspawn-install -d /var/lib/machines/my_pmc_container -p alpine_pkg1 -p some_other_package` 12 13 __TIP:__ systemd-nspawn images are basically just a normal filesystem tree under /var/lib/machines/my_pmc_container. So, in theory, you can do a lot of interaction with the conainer by just having a directory that is writeable from the containing host. 14 15 ### Run the container manually and set a root password 16 17 For the first spin-up, we should run this manually to set private users/groups and some basic stuff. 18 19 `sudo systemd-nspawn -D /var/lib/machines/my_pmc_container -M my_pmc_container --private-users=pick --private-users-ownership=chown` 20 21 Now you're basically chrooted inside the container (but not really properly booted) so set up a few things: 22 23 ``` 24 hostname > /etc/hostname 25 passwd 26 ``` 27 28 You can also create a user or edit any config files that you need for services that will run in the container. When you're done just exit out. 29 30 __NOTE:__ if you need to run the container manually again in the future, to set up networking or tweak some settings, use this command since we already have the private users set up and don't need to chown a second time: 31 32 `sudo systemd-nspawn -D /var/lib/machines/my_pmc_container -M my_pmc_container --private-users=yes` 33 34 ### Set up networking 35 36 This is the one thing that is slightly fiddly with alpine since it does not magically tie up with systemd on the containing host. We are using a private bridge running on the containing host to route our traffic to the internet. These IP addresses are assigned manually so we should be careful to keep track and not double up by accident. We will use the /etc/hosts file on the containing host as our authoritative record as well as provide us with convenient aliases to proxy or connect via SSH. 37 38 Edit /etc/hosts on eik and add a new entry for the container below the last entry with 192.168.0.X: 39 40 ``` 41 127.0.0.1 localhost 42 193.170.194.218eik 43 192.168.0.10 snac 44 192.168.0.11 borf 45 192.168.0.12 my_pmc_container 46 ``` 47 48 Now create an nspawn config file on eik in /etc/systemd/nspawn/my_pmc_container.nspawn 49 50 ``` 51 [Exec] 52 PrivateUsers=yes 53 KillSignal=SIGTERM 54 [Network] 55 Bridge=bridge0 56 ``` 57 58 Now, go inside the container and edit the /etc/network/interfaces file inside the container: 59 60 ``` 61 auto host0 62 iface host0 inet static 63 address 192.168.0.12/24 64 gateway 192.168.0.1 65 ``` 66 67 68 ### Start and Enable the container 69 70 On eik: 71 `sudo machinectl start my_pmc_container` 72 73 To enable so that the container always comes up when eik boots: 74 `sudo machinectl enable my_pmc_container` 75 76 ### Connect via ssh 77 78 Note, you may need to tweak the ssh configs on the container to allow root logins or use whatever keys setup you prefer. 79 80 `ssh my_user@my_pmc_container` 81 82 ### Optionally - set up an nginx proxy for public service 83 84 prolly best to use proxy_pass my_pmc_container 85 86 #### References: 87 88 1. Much of this document was based on [this blog post](https://quantum5.ca/2025/03/22/whirlwind-tour-of-systemd-nspawn-containers/) 89 2. The arch wiki (as ever) has some very good [general docs](https://wiki.archlinux.org/title/Systemd-nspawn) about systemd-nspawn