-
Notifications
You must be signed in to change notification settings - Fork 13
TinySSH server
sd-tinyssh enables SSH access during the initramfs
phase of the
Linux boot process by means of TinySSH (a minimalistic
SSH server). The install hook sd-tinyssh takes care of providing the
initramfs image with all binaries and configuration files required by TinySSH
and enabling the listening TCP port 22 (or some other). Every incoming
connection request triggers a separate TinySSH process that handles one SSH
session.
There are two vital prerequisites that must be met:
-
sd-network from package mkinitcpio-systemd-extras must also be activated and configured in
/etc/mkinitcpio.conf. -
Package tinyssh must have been installed so that TinySSH's binaries can be copied into the initramfs.
It has to be mentioned that TinySSH is a minimal SSH server. It therefore lacks several features that you may know from OpenSSH. See the FAQ on TinySSH's web site. Notably TinySSH only supports keys of type ED25519.
Like every SSH server also TinySSH in the initramfs must be able to present a
server key (aka host key). With TinySSH this requires two separate files,
ed25519.pk (public key) and .ed25519.sk (secret key). These files are
copied from $SD_TINYSSH_KEYDIR (default /etc/tinyssh/sshkeydir).
It is highly recommended to provide the secret key in encrypted form
(.ed25519.sk.enc), created with systemd-creds. (See section
Server keys and security considerations
below for details.)
In case $SD_TINYSSH_KEYDIR (or /etc/tinyssh/sshkeydir) does not exist or
does not contain the files ed25519.pk and .ed25519.sk[.enc] the hook
sd-tinyssh tries to convert the OpenSSH key /etc/ssh/ssh_host_ed25519_key
on-the-fly. This requires that python has also been installed (which is the
case for most systems). In this case you will see a warning about possible
security risks (see below).
If none of the two options produces a valid pair of server keys the install
hook sd-tinyssh aborts with an error.
The hook sd-tinyssh also copies all ED25519 public keys of root (from
$SD_TINYSSH_AUTHORIZED_KEYS, default /root/.ssh/authorized_keys) to the
initramfs image (and ensures that at least one ED25519 public key exists). This
allows to log in with SSH as root.
The hook sd-tinyssh enlarges the initramfs image by approximately 110KiB,
that's around 62KiB for the zstd compressed archive.
Caution
Probably the most common use case for this hook is to remotely unlock an
encrypted partition (or device) before it can be mounted by systemd. Mind
that by default systemd only waits 90 seconds for the unlocked partition (or
device). After that systemd will give up and enter some emergency mode. You
are then effectively locked out! To avoid this painful situation define the
kernel parameter rootflags=x-systemd.device-timeout=0 in the configuration
of your boot loader. With this setting systemd will wait forever. See
systemd.mount
for details.
sd-tinyssh produces a warning when the server key for TinySSH has been
converted on-the-fly from the OpenSSH server key
/etc/ssh/ssh_host_ed25519_key:
Reusing the host keys from the regular operating environment poses a security risk. See https://github.com/wolegis/mkinitcpio-systemd-extras/wiki/TinySSH-server for details.
This warning is issued under the assumption that you use OpenSSH for the regular operating environment. Effectively, with the conversion a copy of your OpenSSH server key has been copied to the initramfs image. It is trivially easy to extract the server key from the initramfs image located on the unencrypted boot partition. This may allow a man in the middle to impersonate your server and thus intercept the traffic between you and your SSH server. The attack will go unnoticed since from the client's perspective the server key was as expected.
The best you can do is to go with an encrypted TinySSH secret server key. The second best option is to use a completely separate pair of server keys (not derived from the 'regular' server key).
Note
The following options and commands assume that you keep your TinySSH server
key pair for the initramfs image in /etc/tinyssh/sshkeydir. It is perfectly
fine to use some other directory. But when you do so you have to specify this
directory with SD_TINYSSH_KEYDIR in mkinitcpio.conf.
The basic idea of this option is to store the secret server key .ed25519.sk
in encrypted form in the initramfs image. systemd will decrypt this key only
when a new SSH session is initiated and provide it to the tinyssh process on
an ephemeral filesystem. Thus the plaintext secret key will never touch
persistent storage.
Important
This option requires that a TPM2 compliant module is present in your system to encrypt and later decrypt the secret server key. Nowadays, this is almost always true for Intel or AMD based desktop systems and also for many dedicated servers. However, this is not true for many virtual servers and ARM based systems.
sd-tinyssh expects the encrypted secret server key as .ed25519.sk.enc in
/etc/tinyssh/sshkeydir. When this file is present it is copied into the
initramfs image instead of .ed25519.sk. The systemd server unit for tinyssh
is set up in a way that the encrypted secret server key is decrypted on-the-fly
during startup of each tinyssh service. (See
Credentials
in systemd's documentation for details.)
-
Check if your system has a TPM2 chip:
systemd-analyze has-tpm2The tool needs to report yes. Otherwise encrypting and decrypting the secret server key will not work.
-
Convert your OpenSSH ED25519 key:
tinyssh-convert /etc/tinyssh/sshkeydir </etc/ssh/ssh_host_ed25519_keyLeave the resulting public key
ed25519.pkas it is. You still need it, but since it is a public key no encryption is required. In the next steps we will deal with the secret key.ed25519.sk. -
Encrypt the secret server key:
systemd-creds encrypt --with-key=tpm2 --name=.ed25519.sk \ /etc/tinyssh/sshkeydir/.ed25519.sk \ /etc/tinyssh/sshkeydir/.ed25519.sk.encThis encrypts the secret TinySSH key with a key hidden in the TPM. The secret TinySSH key is thus bound to this very system. In other words: Always encrypt and decrypt on the same system!
As desired, you will end up with an encrypted key in your initramfs image. However, anyone who can boot ANY operating system on the machine (e.g. from a USB stick) will be able to decrypt the secret key. It is therefore recommended to additionally bind against at least PCR0 (firmware) and PCR7 (secure boot state):
systemd-creds encrypt --with-key=tpm2 --tpm2-pcrs=0+7 --name=.ed25519.sk \ /etc/tinyssh/sshkeydir/.ed25519.sk \ /etc/tinyssh/sshkeydir/.ed25519.sk.encMind that after a firmware update or a secure boot policy change the key must be encrypted again.
-
(Optional) Make a backup of the plaintext secret key
.ed25519.skand delete it. This step is less important when/etc/tinyssh/sshkeydiris on an encrypted filesystem. -
Recreate your initramfs image with
mkinitcpio -PThe hook
sd-tinysshwill recognize the encrypted server key by its name.ed25519.sk.encand setup everything so that this encrypted key will be used.
In case you cannot go with an encrypted secret server key, the second best option is to use a different server key for your initramfs image. This way at least the server key of your regular operating environment cannot leak.
Create the separate key pair with:
tinysshd-makekey /etc/tinyssh/sshkeydir
As nearly always in IT the added security comes with a loss in usability. Depending on whether you connect to your server during initramfs phase (e.g. to unlock your encrypted root partition) or during regular operation the server will present different server keys - and your SSH client will complain.
One possible solution to deal with that is to configure separate Host
sections in $HOME/.ssh/config on the client side, e.g.:
Host myserver
Hostname myserver.mydoma.in
IdentityFile ....
IdentitiesOnly yes
CheckHostIP yes
User root
Host myserver-init
Hostname myserver.mydoma.in
IdentityFile ....
IdentitiesOnly yes
CheckHostIP no
User root
HostKeyAlias myserver-init
This way you have to use ssh myserver-init to connect to TinySSH during
initramfs phase and ssh myserver to connect to the same machine while running
in regular operating environment.
There may be situations where you have good reasons not to go with any of the two options mentioned above. To at least silence the security warning you just have to avoid the on-the-fly conversion of your OpenSSH server key. Do the conversion yourself:
tinyssh-convert /etc/tinyssh/sshkeydir </etc/ssh/ssh_host_ed25519_key
Add sd-tinyssh to the array HOOKS in /etc/mkinitcpio.conf. The entry must
be positioned somewhere after (right of) systemd. Apart from that the
concrete position is irrelevant.
There are five configuration variables affecting the behavior of the install
hook or the TinySSH server. These can be specified somewhere in
/etc/mkinitcpio.conf (or in some file in /etc/mkinitcpio.conf.d):
-
SD_TINYSSH_COMMAND: With the hooksd-tinysshyou can log into the Linux system during the initramfs phase as root. By default you get a shell (busybox dash) and can perform interactive tasks. This variable defines a shell command that is used as argument ofsh -c(i.e. it may contain blanks, double quotes and all kinds of special characters that are interpreted by the shell , but see note below). The resulting command is executed instead ofshafter login. Set this variable somewhere in/etc/mkinitcpio.conf. The SSH session terminates as soon as this command terminates.A good usage example is to set
SD_TINYSSH_COMMAND="systemd-tty-ask-password-agent --query"This allows to unlock LUKS encrypted devices - but nothing more.
Note
Due to the way how the command specified by SD_TINYSSH_COMMAND is passed to
the tinyssh executable usage of single quotes is highly problematic. It's
best to avoid single quotes all together - unless you know exactly what you
are doing. Refer to systemd's documentation regarding command
line
and
quoting
for all the details. After having run mkinitcpio check the resulting
command with
zstdcat /boot/initramfs-linux.img | \
cpio --extract --to-stdout etc/systemd/system/[email protected]/override.conf-
SD_TINYSSH_SCRIPT: This variable is somewhat similar toSD_TINYSSH_COMMAND. It defines the name of a file containing a shell script (a sequence of shell commands). This script is executed after login and before the user gets to the interactive shell. Set this variable somewhere in/etc/mkinitcpio.conf.For example the script file specified by
SD_TINYSSH_SCRIPTmay containsystemd-tty-ask-password-agent --query. This allows to unlock LUKS encrypted devices (similar to the above example withSD_TINYSSH_COMMAND), but withCtrl-Cthe user can escape to an interactive shell.SD_TINYSSH_COMMANDtakes precedence overSD_TINYSSH_SCRIPT. -
SD_TINYSSH_PORT: With this configuration variable it is possible to change the port opened for TinySSH. Default is 22.This may be used to circumvent simple firewall rules, e.g. by using port 443 instead of 22.
-
SD_TINYSSH_KEYDIR: Specifies a directory with a pair of server keys in TinySSH's native format (i.e.ed25519.pkand.ed25519.skor.ed25519.sk.enc) to be copied into the initramfs image. Default is/etc/tinyssh/sshkeydir. -
SD_TINYSSH_AUTHORIZED_KEYS: Set this configuration variable to specify some other source of the keys fileauthorized_keysused for root in the initramfs.