The *websites/pyrocufflink.net* role configures the public web server to
host *pyrocufflink.net*. This site has two functions:
* It redirects `/` to http://dustin.hatch.name/
* It proxies user home directories (i.e. /~dustin/) to the file server
The `ServerName` directive needs to be set inside the default SSL vhost,
as this property does not get inherited from the global configuration,
and it is needs to be set in order for SNI to work correctly.
By default, per-user directories (i.e. `/~username/`) are disabled in
Fedora's configuration of Apache. This commit introduces a new variable,
`apache_userdir`, which can be used to enable this feature. It should be
set to a string other than *disabled*, which is the path under users'
home directories that will be served, if it is accessible. Normally, the
value would be `public_html`.
To support multiple websites with separate Let's Encrypt certificates,
the *certbot* role needs to be applied as a dependency of each
individual website role. This will allow each application to specify a
different value for `certbot_domains`.
The default systemd unit configuration for *certbot-renew.service* runs
the `certbot renew …` command as root. This can cause permissions
issues, since this Ansible role expects the *certbot* user to be able to
access all configuration, data, and log files. As such, this commit adds
a systemd unit extension for *certbot-renew.service* to run the command
as *certbot*.
Moving the route definitions to global scope, and defining an address
pool, will allow other clients besides *dhatch-d4b* to connect to and
use the OpenVPN tunnel service. This may be useful in situations where
IPsec is blocked
The VPN capability of the UniFi Security Gateway is extremely limited.
It does not support road-warrior IPsec/IKEv2 configuration, and its
OpenVPN configuration is inflexible. As with DHCP, the best solution is
to simply move service to another machine.
To that end, I created a new VM, *vpn0.pyrocufflink.blue*, to host both
strongSwan and OpenVPN. For this to work, the necessary TCP/UDP ports
need to be forwarded, of course, and all of the remote subnets need
static routes on the gateway, specifying this machine as the next hop.
Additionally, ICMP redirects need to be disabled, to prevent confusing
the routing tables of devices on the same subnet as the VPN gateway.
The routes to FireMon networks are now defined using the
`firemon_networks` Ansible variable. The global `iroute` and
client-specific `route` options are generated from the CIDR blocks
specified in this list.
The *aria2* role installs the *aria2* download manager and sets it up to
run as a system service with RPC enabled. It also sets up the web UI,
though that must be installed manually from an archive, for now.
For hosts that do not have any TSIG keys, the `named_keys` variable
still must be defined (to an empty iterable) in order for the template
to expand properly.
To avoid having a single point of failure, a second recursive DNS server
is necessary. This will be useful in cases where the VM hosts must both
be taken offline, but Internet access is still required.
The new server, *dns1.pyrocufflink.blue*, has all the same zones defined
as the original. It forwards the *pyrocufflink.blue* zone and
corresponding reverse zones to the domain controllers, and acts as a
slave for the *pyrocufflink.red* zone.
*smtp1.pyrocufflink.blue* is a VM that will replace
*smtp0.pyrocufflink.blue*, a Raspberry Pi.
I decided that there is little use in having the availability guarantee of
a discreet machine for the SMTP relay. The only system that would NEED
to send mail if the VM host fails is Zabbix, which operates as its own
relay anyway. As such, the main relay can be a VM, and the Raspberry Pi
can be repurposed as a recursive DNS server.
The *koji-web* role installs and configures the Koji Web GUI front-end
for Koji. It requires Apache and mod_wsgi. A client certificate is
required for authentication to the hub, and must be placed in the
host-specific subdirectory of `certs/koji`.
The *koji-client* role is a generic role that can be used to configure
the Koji client library/`koji` CLI tool. By default, it manages the
default configuration at `/etc/koji`, but by using the
`koji_client_dir`, `koji_client_user`, and `koji_client_id` variables,
it can be used to configure per-user client configuration as well.
The *kojira* role sets up the Koji repository agent to manage
repository metadata for build tags. It runs as a daemon, usually on the
same machine as the Koji hub. A client certificate is required for
authentication, and must be supplied by placing it in the
`certs/koji/{{ inventory_hostname }}` directory.
The *koji-gc* role sets up the Koji garbage collector utility to run
periodically. It uses cron for scheduling. A client certificate is
required for authentication, and must be supplied by placing it in the
`certs/koji/{{ inventory_hostname }}` directory.
The minimal Fedora installation does not include a cron implementation.
The *cronie* role can be applied to hosts installed in this way to
ensure that cron is available for task scheduling.
The *burp-client* role installs and configures a BURP client. It should
support RHEL/CentOS/Fedora and Gentoo.
To manage the client password and other server-mandated configuration,
the role uses Ansible's delegation feature to generate a configuration
file in the "clientconfdir" on the BURP server.
An hourly cron task is scheduled that runs `burp -a t` every hour. This
allows the server to configure backup timebands and intervals.
The *burp-server* role installs and configures a BURP server. It is
adapted from a previous iteration, and should support CentOS/RHEL/Fedora
and Gentoo, as well as both BURP 1.x and 2.x (depending on which version
gets installed by the system package manager).
To manage the certificate authority, the *burp-server* role uses the
`burp_ca` command. This has the advantage of not requiring any external
certificate management, but effectively binds the CA to a specific
machine.
The value of the `shlib_directory` is dependent the system architecture.
Specifically, x86_64 machines use `/usr/lib64/postfix`, while everything
else uses `/usr/lib/postfix`. This role was originally deployed on a
Raspberry Pi, so the original path was correct. Attempting to deploy it
on an x86_64 machine revealed the error.
This commit adds a new task that loads a variables file based on the
architecture. Each option defines an `arch_libdir` variable, which can
be expanded in the `postfix_shlib_directory` variable as needed.
*myala.pyrocufflink.jazz* no longer hosts any public-facing websites,
and is in fact shut down. To prevent HAproxy from failing to start
because it cannot resolve the name, this backend needs to be removed.
The *fileserver* role configures Samba as a file sharing server. It uses
the *samba* role to handle cross-distribution installation of Samba
itself, and is focused primarily on configuring shared folders.
This commit adds an *after* ordering dependency on the network device
unit to the *wait-global-address@.service* template unit. Without this
dependency, the service will wait forever for a global address if the
device does not exist. With the dependency, though, if the device does
not appear within the default timeout, the wait service will never
start, causing all dependent services to fail, but allowing the boot
process to continue.
The *websites/darkchestofwonders.us* role prepares a machine to host
http://darkchestofwonders.us/. The website itself is published via rsync
by Jenkins.
The *websites/dustin.hatch.name* role configures a server to host
http://dustin.hatch.name/. The role only applies basic configuration;
the actual website application is published by Jenkins.
The Apache service needs to be reloaded after the *certbot* role
configures it to serve the `/.well-known/acme-challenge` path, so that
the changes can take effect before the `certbot` command is run to
request the certificate(s).
If another role that depends on the *apache* role accidentally creates
an invalid configuration, it will be impossible to correct it by
subsequent invocations of its playbook. This is because the *apache*
role always tries to start the service, which will fail if the
configuration is invalid, thus aborting the playbook. With this early
abort, there is no way for later tasks to correct the error.
Playbooks that include the *apache* role should have a task that is
executed after all the roles have been applied to ensure the service is
running.
The *dch-storage-net* role configures a machine to connect the storage
network and mount shared folders from the storage appliance.
The `wait-global-address.sh` script and corresponding
*wait-global-address@.service* systemd unit template are necessary to
ensure that the storage network is actually available before attempting
to mount the shared volumes. This is particularly important at boot,
since `dhcpcd` does not implement any kind of signaling that can be used
by *network-online.target*, so the network is considered "online" as
soon as the `dhcpcd` process has started. This typically results in
"network unreachable" errors.
The *net-ifaces* role manages a script that creates virtual network
interfaces, such as bridge, bond, and VLAN, that `dhcpcd`/`dhclient`
alone cannot. This provides a lightweight alternative to
*systemd-networkd* and *NetworkMangager*.
Though the default for the `fqdn` value is listed as `both` in
*dhcpcd.conf(5)*, the current behavior of `dhcpcd` suggests that it may
actually be `none`. Without explicitly setting `fqdn both`, the value of
the kernel node name is sent as-is in the *hostname* option (12). If the
node name is set to the FQDN, then dynamic DNS gets broken, since the
DHCP server always appends its domain name to the provided hostname.
Setting `fqdn both` causes `dhcpcd` to send the FQDN in the *FQDN*
option (81), which the DHCP server interprets correctly.
Using a list to specify the values for the `allowinterfaces` and
`denyinterfaces` parameters in `dhcpcd.conf` makes the configuration
policy cleaner and more type-safe.
Today I realized that `dhcpcd` has been logging several hundred thousand
of these messages every second:
libudev: received NULL device
This was causing both `dhcpcd` and `systemd-journald` to consume 100%
CPU.
I am not entirely sure what a "device management" module is in the
context of `dhcpcd`, but it does not seem to be required. Setting the
`nodev` option in `dhcpcd.conf` suppresses the messages, and seems to
have no effect on the operation of the daemon.
Traffic from the management network is not allowed except for specific
services. NTP is required of course, for time synchronization with the
pyrocufflink.blue domain controllers. RADIUS is necessary for WiFi
authentication, which is also handled by the DCs.
The `ifconfig` global directive specifies the IP address added to the
tunnel interface device, not the network. The `push route` directives
need to include this address to correctly send route information to
clients.
The *dch-openvpn-server* role installs and configures OpenVPN and
stunnel to provide both native OpenVPN service as well as
OpenVPN-over-TLS. The latter uses stunnel, listening on TCP port 9876,
to allow better firewall traversal and TCP port sharing via reverse
proxy.
The `apache_server_tokens` variable can now be set, which controls the
value of the `ServerTokens` directive. If the variable is set, the
`ServerTokens` directive will be added to the `00-servername.conf` file.
The `samba_interfaces` variable can now be defined to populate the
`interfaces` global configuration parameter in `smb.conf`. This
parameter controls the interfaces or addresses to which the Samba server
binds, and also the IP addresses that are registered in DNS.
The *certbot* role now supports copying the data for an existing Let's
Encrypt account to the managed node using an archive. If an archive
named for the inventory hostname (typically the FQDN) of the managed
node is found in the `accounts` directory under the `files` directory of
the *certbot* role, it will be copied to the managed node and extracted
at `/var/lib/letsencrypt/accounts`. This takes the place of running
`certbot register` to sign up for a new account.
The *install* tag is applied to any task that installs a package.
The *user* tag is applied to any task that creates an OS user or group.
The *group* tag is applied to any task that creates an OS user group.
For machines that do not use firewalld, the *zabbix-agent* role will now
skip attempting to open the Zabbix agent port using the `firewalld`
module. The `host_uses_firewalld` variable controls this behavior.
The *certbot* role installs and configures the `certbot` ACME client. It
adjusts the default configuration to allow the tool to run as an
unprivileged user, and then configures Apache to work with the *webroot*
plugin. It registers for an account and requests a certificate for the
domains specified by the `certbot_domains` Ansible variable. Finally, it
enables the *certbot-renew.timer* systemd unit to schedule automatic
renewal of all Let's Encrypt certificates.
The *dch-proxy* role sets up HAProxy to provide a revers proxy for all
public-facing web services on the Pyrocufflink network. It uses the TLS
Server Name Indication (SNI) extension to determine the proper backend
server based on the name requested by the client.
For now, only Gitea is configured; the name *git.pyrocufflink.blue* is
proxied to *git0.pyrocufflink.blue*. All other names are proxied to
Myala.
The *haproxy* installs HAproxy and sets up basic configuration for it.
It configures the systemd unit to launch the service with the `-f
/etc/haproxy` arguments, which will cause it to load all files from the
`/etc/haproxy` directory, instead of just `/etc/haproxy/haproxy.cfg`.
This will allow other roles to add frontend and backend configuration by
adding additional files to this directory.
The *sshd* role can be used to configure the OpenSSH daemon. It supports
configuring a few options globally, as well as a limited set of options
in `Match` blocks (e.g. per-user/group configuration).
The `trustca` role can be used to add CA certificates to the system
trust store. It requires a variable, `ca`, to be defined, referring to
the name of a file containing a CA certificate to install.
The `gitea_ssh_domain` and `gitea_http_domain` variables can be used to
configure the host portion of the URLs for cloning Git repositories over
SSH and HTTPS, respectively. By default, both values are the FQDN of the
machine hosting Gitea.
The *gitea* role installs Gitea using the system package manager and
configures Apache as a reverse proxy for it.
The configuration file requires a number of "secret" values that need to
be unique. These must be specified as Ansible variables:
* `gitea_internal_token`
* `gitea_secret_key`
* `gitea_lfs_jwt_secret`
The `gitea generate` command can be used to create these values.
Normally, Gitea expects to run its own setup tool to generate the
configuration file and create the administrative user. Since the
configuration file is generated from the template instead, no
administrative user is created automatically. Luckily, the `gitea`
command includes a tool to create users, so the administrator can be
created manually, e.g.:
sudo -u gitea gitea admin create-user -c /etc/gitea/app.ini \
--admin
--name giteadmin \
--password giteadmin \
--email giteadmin@example.org
In order to enable LDAPS/STARTTLS support in Samba, the `tls enabled`
option must be set to `yes` and the `tls keyfile` and `tls certfile`
options must be set to the path of the private key and certificate
files, respectively, that Samba will use. The `samba_tls_enabled`,
`samba_tls_keyfile`, and `samb_tls_certfile` Ansible variables can be
used to control these values.
The `socket options` directive does not need to be specified in
`smb.conf`. I think I copied it from an example many years ago, and
never bothered to remove it. It is definitely not required, most likely
not helping performance at all, and most likely hindering it.
This commit adjusts the firewall and networking configuration on dc0 to
host the Pyrocufflink remote access IPsec VPN locally instead of
forwarding it to the internal VPN server.
The *dch-vpn-server* role configures strongSwan to act as an IPsec
responder for `vpn.pyrocufflink.net` and provide an IKEv2/IPsec VPN for
remote access clients, as well as the reverse VPN to FireMon.
The *strongwan* role is intended to be used as a dependency of other
roles that use strongSwan for IPsec configuration. It deploys some basic
configuration and configures the *strongswan* service, but does not
configure any connections, secrets, etc.
Using `state=absent` with the `file` module in a `with_items` loop to
delete the "default" module and site configuration files and the example
certificates is incredibly slow. Especially on the Raspberry Pi, it can
take several minutes to apply this role, even when there are no changes
to make. Using the `command` module and running `rm` to remove these
files, while not as idempotent, is significantly faster. The main
drawback is that each item in the list is not checked, so new items to
remove have to be added to the end of the list instead of in
alphabetical order.
The *freeradius* role is used to install and configure FreeRADIUS. The
configuration system for it is extremely complicated, with dozens of
files in several directories. The default configuration has a plethora
of options enabled that are not needed in most cases, so they are
disabled here. Since the initial (and perhaps only) use case I have for
RADIUS is WiFi authentication via certificates, only the EAP-TLS
mechanism is enabled currently.
The *postfix* role installs and configures the Postfix MTA. It currently
supports a number of modes, including direct transfer and relay. Relay
mode supports STARTTLS security and PLAIN authentication.
Since the location of the configuration drop-in directory can vary by
distribution, it is important to expand the `zbx_agent_config_dir`
variable in the `Include` parameter.
The *zabbix-agent* role installs the Zabbix monitoring agent on the
managed node, and sets it up to communicate with the Zabbix server
specified by the `zabbix_server` variable. This role "should" be
compatible with most distributions; it has been tested with Fedora and
Gentoo.
The *zabbix-server* role deploys the Zabbix server database, daemon, and
web interface. It requires the *apache* role to configure Apache HTTPD
to serve the web UI.
The *apache* role installs and configures the Apache HTTPD server and
its *mod_ssl* module. It currently only works on Fedora/RHEL-based
distributions.
The `ad` identity mapper backend is apparently the only one that can
use shell, home directory, etc. attributes from the directory now (as of
Samba 4.6).
The *ssh-hostkeys* role is used to manage the global SSH host key
database. This file is consulted by the `ssh` command when verifying
remote host keys on first connect. If the host key is found here, it is
copied to the user's host key database file without prompting for
verification.
The *jenkins-slave* role prepares a host to have the Jenkins slave
agent deployed on it. Deploying the agent itself is done by the Jenkins
master, through the web UI.
The service principal name added to `/etc/krb5.keytab` had a trailing
`}` character because of a typo in the Ansible task. This resulted in
GSSAPI authentication failing because server processes could not find
the host key in the key table.
This commit introduces a new role, *hostname*, that is used by the
`hostname.yml` playbook to set the hostname. It also writes
`/etc/hosts` using a template.
It is occasionally necessary to advertise multiple prefixes on the same
interface, particularly when those prefixes are not on-link. The *radvd*
role thus now expects each item in `radvd_interfaces` list to have a
`prefixes` property, which itself is a list of prefixes to advertise.
Prefixes can specify properties such as `on_link`, `autonomous`,
`preferred_lifetime`, etc.
Marking packets matching port-forwarding rules, and then allowing
traffic carrying that mark did not seem to work well. Often, packets
seemed to get dropped for no apparent reason, and outside connections to
NAT'd services was sometimes slow as a result. Explicitly listing every
destination host/port in the `forward` table seems to resolve this
issue.
The *filter* table is responsible for deciding which packets will be
accepted and which will be rejected. It has three chains, which classify
packets according to whether they are destined for the local machine
(input), passing through this machine (forward) or originating from the
local machine (output).
The *dch-gw* role now configures all three chains in this table. For
now, it defines basic rules, mostly based on TCP/UDP destination port:
* Traffic destined for a service hosted by the local machine (DNS, DHCP,
SSH), is allowed if it does not come from the Internet
* Traffic passing through the machine is allowed if:
* It is passing between internal networks
* It is destined for a host on the FireMon network (VPN)
* It was NATed to in internal host (marked 323)
* It is destined for the Internet
* Only DHCP, HTTP, and DNS are allowed to originate from the local
machine
This configuration requires an `internet_iface` variable, which
indicates the name of the network interface connected to the Internet
directly.
`dhcpcd` needs to start after the `network` service has started, as the
latter creates the interfaces to which the former needs to delegate IPv6
prefixes.
The *nftables* role handles installation and basic configuration of the
userspace components for nftables.
Note that this role currently only works on Fedora, and requires
*nftables* 0.8 or later for wildcard includes.
The *networking* service, which is actually a legacy init script, is
provided by the *initscripts* package on RHEL and its derivatives. This
service needs to be running in order for the configuration generated by
the *rhel-network* role to be applied to the managed node.
The `network.yml` playbook is used to configure the network interfaces
on a managed node. Currently, it only supports the Red Hat configuration
style (i.e. `/etc/sysconfig/network-scripts/ifcfg-*` files).
The *samba-dc* role now configures `winbindd` on domain controllers to
support identity mapping on the local machine. This will allow domain
users to log into the domain controller itself, e.g. via SSH.
The Fedora packaging of *samba4* still has some warts. Specifically, it
does not have a proper SELinux policy, so some work-arounds need to be
put into place in order for confined processes to communicate with
winbind.
The *samba* role provides general configuration for Samba. Other roles
will provide configuration for specific features such as Active
Directory membership, file shares, etc.
The *system-auth* role deploys PAM configuration for system-wide user
authentication. It is specifically focused on Active Directory
authentication using Samba/Winbind.
The *nsswitch* role can be used to configure the name service switch on
glibc-based distributions, including Gentoo, Fedora, and CentOS. It is
specifically focused on Active Directory authentication via
Samba/Winbind.
Only *master* zones need zone files pre-populated, as the other types of
zones are populated by data named receives from queries and transfers.
Other types of zones require other options, however, to be usable. This
commit introduces minimal support for specifying *slave*, *forward*, and
*stub* zones.
Items in the `allow_update` property can use the address match list
syntax to specify arbitrary restrictions, including TSIG key names.
There is really no need for a special case for key names.
To support signing of updates, TSIG keys can be defined using the
`named_keys` variable. This variable takes a list of objects with the
following properties:
* `name`: The name of the key
* `algorithm`: The signature algorithm (default: `hmac-md5`)
* `secret`: The base64-encoded key material
The *named* role now supports generating configuration for authoritative
DNS zones and DNSSEC keys. Zones are defined by populating the
`named_zones` variable with a list of objects describing the zone. Zone
properties can include:
* `name`: The DNS domain name
* `type`: The zone type, defaults to `master`
* `allow_update`: A list of hosts/networks or DNSSEC key names (which
must be specified as an object with a `key` property)
* `update_policy`: A list of BIND update policy statements
* `ttl`: The default (minimum) TTL for the zone
* `origin`: The authoritative name server for the zone
* `refresh`, `retry`, `expire`: Record cache timeout values
* `default_records`: A list of default records, defined as objects with
the following properties:
* `name`: The RR name
* `type`: The RR type (default: `A`)
* `value`: The RR value
Zone files will be created in `/var/named/dynamic`. Existing zone files
will **not** be overwritten; management of zone records is done using
`nsupdate` or similar.
This commit adjusts the tasks in the *samba-dc* role to use a
conditional include to restrict tasks relating to the BIND9_DLZ plugin
only to hosts that are configured to use it.
The *bind-utils* package contains `dig` and `nsupdate`, which are used
to query and manage DNS records.
The *cyrus-sasl-gssapi* package contains the GSSAPI plugin for
SASL-aware applications, including `ldapsearch`.
The *ldb-tools* package contains `ldbsearch` and other tools for
directly using Samba database files.
The `/var/lib/samba/bind-dns` directory contains files that are
hard-linked to files in the `/var/lib/samba/private` directory. All
paths for a file must have the same context, or `restorecon` will
effectively "toggle" the labels each time it is run.
Evidently, some files in `/var/lib/samba` match multiple file context
rules. Thus, when running `restorecon` against the entire
`/var/lib/samba` directory, files in the `bind-dns` subdirectory may end
up with the wrong label. To work around this issue, `restorecon` is now
run only on that subdirectory to ensure the correct labels are applied.
This is likely to cause problems when a full filesystem relabel is
scheduled.
The `named` daemon does not seem to pick up all changes to the
configuration file during a graceful reload. To avoid strange states,
the daemon is now fully restarted after the configuration file is
regenerated.
The `allow-update` block in `named.conf` enumerates the hosts/networks
that are allowed to issue dynamic DNS updates. This is required in
Active Directory and other environments where clients and/or DHCP
servers create DNS records automatically.
By default, the block is omitted from the generated configuration file.
The `named_allow_update` variable can be set to a list of patterns (e.g.
CIDR blocks, ACL names, etc.) to populate it.
The *samba-dc* role now supports joining an existing Active Directory
domain as an additional domain controller. The `samba_is_first_dc` variable
controls whether the machine will be provisioned with a new domain (when
true) or added to an existing domain (when false).
Joining an existing domain naturally requires credentials of a user with
permission to add a new DC, the `samba_dc_join_username` and
`samba_dc_join_password` variables can be used to specify them.
Alternatively, if these variables are not defined, then the process will
attempt to use Kerberos credentials. This would require playbooks to
make a ticket-granting-ticket available somehow, such as by executing
`kinit` prior to applying the *samba-dc* role.
The *named* role configures the BIND DNS server on managed nodes. It
writes `/etc/named.conf`, using a template that supports most of the
commonly-used options. The configuration can be augmented by other
templates, etc. by specifying file paths in the `named_options_include`
or `named_global_include` variables, both of which are lists.
The *samba-dc* role installs Samba on the managed node and configures it
as an Active Directory Domain controller. A custom module,
`samba_domain` handles the provisioning using `samba-tool domain
provision` in an idempotent way.
The *kerberos* role configures the MIT Kerberos library. Specifically,
it creates `/etc/krb5.conf` and populates it with some basic default
options. It also creates the `/etc/krb5.conf.d` directory, into which
other roles can write additional configuration files.
The *base* role performs the basic tasks needed to manage a node using
Ansible. Specifically, it installs the necessary packages for
manipulating SELinux policy.
Dustin