Apparently, BusyBox's `cp` does NOT copy SELinux contexts when the `-a`
argument is specified. This differs from GNU coreutils's `cp`, and
explains why the files copied from the rootfs image to the persistent
storage volume were not being labelled correctly. The `-c` argument is
required.
Now that files are labelled correctly when they are copied, the step to
run `restorecon` is no longer necessary.
In effort to support different builds of Aimee OS using the same
scripts, without necessarily having to fork this repository, the build
system now supports a `CONFIGDIR` setting. When this variable is set,
files defining the target environment, such as the lists of packages to
install, the kernel configuration, the Portage configuration, etc. are
found in the path it specifes.
The reference build, for the Home Assistant Yellow board, is configured
in the `yellow` directory. To build it, run:
```sh
CONFIGDIR=yellow ./vm-build.sh
```
We're going to want the ability for processes to have unique categories,
to enforce separation of container processes. Gentoo's SELinux policy
supports both Multi-Category Security and Multi-Level Security modes,
although the latter does not seem to work out of the box.
*systemd-tmpfiles* can create btrfs subvolumes with the `v` entry type.
Using this mechanism instead of the `init-storage` script will allow for
greater flexibility when adding other subvolumes later.
Unfortunately, the default configuration for *systemd-tmpfiles* already
includes an entry for `/var/log` with the `d` (directory) type. Since
individual entries cannot be overridden, we need to modify this entry.
The `factory-reset` command provides a way to completely wipe the data
partition, thus erasing any local configuration and state. The command
itself simply enables a special systemd service unit that is activated
during the shutdown process. This unit runs a script, after all
filesystems, except rootfs, have been unmmounted. It then erases the
signature of the filesystem on the data partition, so it will appear
blank the next time the system boots. This will trigger the
`init-storage` process, to create a new filesystem on the partition.
Gentoo uses GNU awk by default, but since we are using Busybox for the
rest of the userspace utilities, it makes sense to use awk from Busybox
as well.
Some *tmpfiles.d(5)* entries specify paths in the immutable root
filesystem. These need to be created at build time to prevent
*systemd-tmpfiles-setup.service* from failing at runtime.
This script can be used to rebuild a binary package in the SYSROOT and
reinstall it in the destination root.
```sh
./rebuild-pkg sec-policy/selinux-aimee-os
make -W /tmp/build/.built O=/tmp/build IMAGESDIR=${PWD}/images
```
The *aimee-os* SELinux policy module provides rules that are specific to
our custom commands and system configuration. These rules are not
suitable for including in the upstream policy, so we include them in a
separate package rather than patches to the base policy.
Currently, the policy module includes rules to allow the `init-storage`
and `system-update` programs to work. It also includes rules to allow
SSH host keys to be stored in `/var/lib/ssh` instead of `/etc/ssh`,
since our `/etc` is immutable.
There's no particular reason why the directory used as the temporary
mount point for the data volume needs to be random. Using a static
name, on the other hand, makes it easier for the SELinux policy to
apply the correct type transition and ensure the directory is labelled
correctly.
Using `tar` to copy files and directories from the overlay directory to
the destination root preserves their timestamps. This is not really
desirable, particularly for directories, because it may result in the
destination paths appearing older than the build. This is especially
problematic for `/usr`, since its timestamps are important for systemd
units that use `ConditionNeedsUpdate` settings.
To ensure the timestamps are set correctly, we now use `rsync` to copy
the overlay, with the `-O` (`--omit-dir-times`) argument, to avoid
changing the timestamps of directories. Additionally, we explicitly
update the timestamp of `/usr` to ensure that every new build triggers
the "needs update" condition.
If any file in the `overlay` directory changes, the `build-rootfs.sh`
script needs to be re-run in order to copy the changes into the
destination root and regenerate the SquashFS image.
Instead of copying the Portage configuration files to `/etc/portage` and
`/usr/${target}/etc/portage`, the build scripts now use the
configuration directories from the source directory. This avoids issues
with changes (especially removal of files) getting propagated to the
actual configuration paths.
For some reason, when OverlayFS is mounted at `/etc/ssh`, SELinux
prevents access both `sshd` and `ssh-keygen` access to the files there.
The AVC denials indicate that (some part of) the process is running in
the `mount_t` domain, which is not allowed to read or write `sshd_key_t`
files.
To work around this issue, without granting `mount_t` overly-permissive
access, we now configure the SSH daemon to read host keys from the
persistent data volume directly, instead of "tricking" it with
OverlayFS. The `ssh-keygen` tool does not read the `HostKey` options
from `sshd_config`, though, so it has to be explicitly instructed to
create keys in this alternate location. By using a systemd template
unit with `ConditionPathExists`, we avoid regnerating the keys on every
boot, since the `ssh-keygen` command is only run if the file does not
already exist.
Enabling SELinux on the target system needs build-time and run-time
configuration changes for ther kernel and userspace. Additionally,
SELinux requires a policy that defines allowed operations. Gentoo
provides a reasonable baseline for all of these changes, but some
modifications are required.
First and foremost, the Gentoo SELinux policy is missing several
necessary rules for systemd-based systems. Notably, services that use
alternate namespaces will fail to start because the base policy does not
allow systemd components the necessary privileges, so these rules have
to be added. Similarly, `systemd-journald` needs additional privileges
in order to be able to capture all metadata for processes generating
syslog messages. Finally, additional rules are necessary in order to
allow systemd to create files and directories prior to launching
servies.
Besides patching the policy, we also do some hackery to avoid shipping
the Python runtime in SELinux-enabled builds. Several SELinux-related
packages, including *libselinux* and *policycoreutils* have dependencies
on Python modules for some of their functionality. Unfortunately, the
Python build system does NOT properly cross-compile native extension
modules, so this functionality is not available on the target system.
Fortunately, none of the features provided by these modules are actually
needed at runtime, so we can safely ignore them and thus omit the entire
Python runtime and all Python programs from the final image.
It is important to note that it is impossible to build an
SELinux-enabled image on a host that is itself SELinux-enabled.
Operations such as changing file labels are checked against the SELinux
policy in the running kernel, and may be denied if the target policy
differs significantly from the running policy. The `setfiles` command
fails, for example, when run on a Fedora host. As such, building an
SELinux-enabled system should be done in a virtual machine using a
kernel that does not have a loaded SELinux policy. The `ocivm` script
can be used to create a suitable runtime from a container image.
The Portage packages that need to be built and/or installed are now
specified in the `build.packages` and `install.packages` files,
respectively. Similarly, packages to be installed on the host system
are specified in `host-tools.packages`. Finally, the
`installonly.packages` file contains a list of packages that are
installed in the destination root, but not built in the sysroot
beforehand.
This allows `make` to better track when the package sets change. It
will also make it easier to maintain different sets for different
variants in the future.
This script uses the `ocivm` tool to launch a QEMU micro VM to build
the operating system. This is necessary to produce an SELinux-enabled
system, since container runtimes interfere with the SELinux policy
build and filesystem labeling processes.
Since we have to build *sys-libs/libcap* with the default Portage
configuration in order to avoid the circular dependency with PAM,
our configuration for binary package builds is not yet in place. We
need to explicitly specify where to put the built packages and enable
multi-instance packages.
By default, `tar` copies file ownership UID/GID. This works fine when
the build is running in a rootless container, since the source UID/GID
numbers are mapped to 0/0 inside the container. In other scenarios,
though, such as building in a microvm with the source directory on a
shared filesystem, the original numbers are preserved. We need to
explicitly state that the files must be owned by root.
When running inside a QEMU microvm with the source directory shared
via 9pfs, the kernel build process fails
> Error: Could not mmap file: vmlinux
Thus, we need to run the build in a path on a local filesystem. To
support this, the Makefile now supports an `O` option, and all the build
scripts have been adjusted to make use of it as needed.
Since building in a local filesystem would ultimately discard the final
artifacts when the VM terminates, we need yet a different location for
the files we want to keep. The `IMAGESDIR` option can be used to
specify this path. This path can be on a shared filesystem, thus
saving the artifacts outside the microvm.
Several packages end up with circular dependencies, depending on which
Portage profile is selected. The default profiles have a circular
dependency between *sys-libs/pam* and *sys-libs/libcap*. Systemd and
SELinux profiles have even more issues.
We can break the circular dependencies by explicitly building *libcap*
with`USE=-pam` first, which happens to be the default configuration
generated by `crossdev`. Then, we need to switch to a more complete
profile in order to build *glibc* and *util-linux*. At this point, the
build root should be complete enough to build anything without circular
dependencies.
There's really no sense in creating a writable copy of the whole `/etc`
hierarchy at `/run/etc/rw`. Instead, let's just mount overlays at the
paths we want to make writable (which for now is only `/etc/ssh`).
In a "merged-usr" system, `/lib` is a symlink to `/usr/lib`. When
installing *sys-apps/systemd*, Portage checks to ensure this is the
case. If this happens after `make modules_install` is run, `/lib` is
a directory, which causes the installation to fail. To avoid this, we
need to explicitly install the modules into `/usr/lib` so that the
symlink can be created later.
Building the OS is now as simple as running `make` on a Gentoo system.
Interestingly, when `make` is executed as a (grand)child process of
another `make` process, it always prints an `Entering directory ...`
message. This breaks the `make kernelversion` command, by adding
extraneous text to the output.
The *ldconfig.service* fails because `/etc` is not writable and thus
`/etc/ld.so.cache` cannot be generated.
The files specified in the `provision.d` *tmpfiles.d(5)* configuration
are unnecessary, and many of them cannot be created at runtime because
the root filesystem is immutable.
Dustin