Preparing ubuntu for web-server, gaming and streaming. Including auto install with yaml config and encrypted partition

Preparing ubuntu for web-server, gaming and streaming. Including auto install with yaml config and encrypted partition

For your baremetal server hosting multiple services—a one-page website, a Counter-Strike game server, video hosting, and video streaming—designing your disk partitions with encryption requires balancing security, performance, and management ease. Here are best practices to map your disk partitions with encryption for such a setup:

  1. Separate Key Partitions for Isolation and Performance

    • /boot (unencrypted): Small partition (512MB–1GB) on an unencrypted partition. It holds bootloader and kernel files that must be accessible before unlocking encrypted volumes.
    • Encrypted root (/) partition: Holds the operating system and core software.
    • Encrypted data partitions, separately for major service data:
      • Web server data (e.g., /var/www or a dedicated mount point)
      • Game server files (Counter-Strike server files and logs)
      • Video storage (videos for hosting and streaming)
      • Logs and cache (optionally, isolate logs on a separate partition to avoid filling critical volumes)

  2. Use LUKS Full Disk / Partition Encryption

    • Encrypt each major partition handling sensitive or important data using LUKS for strong encryption.
    • The /boot partition is typically left unencrypted because the bootloader needs to access it before unlocking encrypted volumes.
    • Use strong passphrases and consider key management solutions (e.g., network-bound disk encryption, TPM integration, or clevis/tang) if automation or centralized management is desired.

  3. Partitioning and Filesystem Layout Recommendations

    • Use Logical Volume Manager (LVM) on top of your encrypted partitions. This provides flexibility to resize or manage partitions without downtime.
    • Filesystem choices can vary per use case:
      • For video storage, consider XFS or ext4, tuned for large files.
      • For game server and web files, ext4 or XFS are reliable.
    • Assign dedicated volumes for video storage to avoid interference with OS or game server performance.

  4. Performance and Stability Tips

    • Separate partitions prevent logs, cache, or video writes from consuming all disk space and affecting system stability.
    • Consider faster storage or SSDs for high I/O workloads like video streaming and game servers.
    • Align partitions properly for your disk type (modern tools like parted handle this automatically).

  5. Backup and Recovery

    • Keep encrypted backups of key volumes.
    • Securely store encryption keys/passphrases separately from the server.
    • Plan for recovery procedures that involve unlocking and restoring encrypted partitions.

  6. Security Best Practices

    • Keep the OS and software updated.
    • Secure network access with firewalls and VPN.
    • Limit access and service exposure.
    • Use HTTPS and encrypted transport for video streaming.
    • Encrypt storage data at rest with LUKS.
    • Store encryption keys securely and consider multi-factor unlocking mechanisms.

  7. Example Partition Layout (sizes depend on your disk size)

    /dev/sda1  -  /boot            - 500MB  - unencrypted
/dev/sda2  -  cryptroot (LUKS) - rest of disk
    └─ inside cryptroot (LVM)
       ├─ root (/)            - 20-50GB
       ├─ /var/www            - 10-20GB (web files)
       ├─ /home/game          - 20-50GB (Counter-Strike files)
       ├─ /media/videos       - remainder (video storage)
       └─ /var/log            - 5-10GB (logs)

This layout isolates critical system files from fluctuating data like videos and logs, all secured by encryption except the essential /boot.

In summary, partition your disk with separate encrypted volumes for OS, services, and data. Use LUKS for encryption with strong keys, LVM for flexibility, and keep /boot unencrypted. Prioritize performance for video storage and game files via suitable filesystems and dedicated partitions. Secure keys and plan backups meticulously. This approach balances security, scalability, and performance for your multi-service baremetal server environment.

Here is an example of an Ubuntu 24.04 automated installation YAML (autoinstall.yaml) snippet that demonstrates how to configure encrypted partitions separated for logs, web server, video hosting/streaming, and Counter-Strike server data. It uses LUKS encryption for each data partition and logical volumes (LVM) for flexibility:

autoinstall:
  version: 1

  identity:
    hostname: myserver
    username: myuser
    password: ${ENCRYPTED_PASSWORD}

  locale: en_US.UTF-8
  keyboard:
    layout: us

  storage:
    layout:
      name: lvm
    config:
      - type: disk
        id: disk0
        match:
          ssd: true
        wipe: superblock-recursive
        preserve: false
        ptable: gpt
        grub_device: true
        preserve: false
        name: disk0
        children:
          - type: partition
            id: boot-partition
            size: 512M
            flag: boot
          - type: partition
            id: luks-root-partition
            size: 50G
          - type: partition
            id: luks-web-partition
            size: 20G
          - type: partition
            id: luks-game-partition
            size: 30G
          - type: partition
            id: luks-video-partition
            size: 100G
          - type: partition
            id: luks-logs-partition
            size: 10G

      # Unencrypted /boot filesystem
      - type: format
        fstype: ext4
        volume: boot-partition
        mount:
          point: /boot
      
      # Encrypted root partition (OS)
      - type: luks
        id: luks-root
        volume: luks-root-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
          # passphrase: will be prompted on boot
        children:
          - type: lvm_volgroup
            id: vg-root
            name: vg-root
            children:
              - type: lvm_logicalvolume
                id: lv-root
                name: lv-root
                size: 100%FREE
                format: ext4
                mount:
                  point: /
      
      # Encrypted web server partition (mounted at /var/www)
      - type: luks
        id: luks-web
        volume: luks-web-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
        children:
          - type: format
            fstype: ext4
            mount:
              point: /var/www

      # Encrypted game server files (e.g., Counter-Strike, mounted at /home/game)
      - type: luks
        id: luks-game
        volume: luks-game-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
        children:
          - type: format
            fstype: ext4
            mount:
              point: /home/game

      # Encrypted video hosting and streaming data (mounted at /media/videos)
      - type: luks
        id: luks-video
        volume: luks-video-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
        children:
          - type: format
            fstype: xfs
            mount:
              point: /media/videos

      # Encrypted logs partition, mounted at /var/log
      - type: luks
        id: luks-logs
        volume: luks-logs-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
        children:
          - type: format
            fstype: ext4
            mount:
              point: /var/log

  ssh:
    install-server: true
    allow-pw: true

  packages:
    - curl
    - vim
    - htop

  # Optional: Running post-install scripts if needed to finish configuration
  late-commands:
    - curtin in-target --target=/target -- systemctl enable ssh.service

Key points in the above example:

  • /boot is a small unencrypted partition with ext4.
  • The root filesystem is fully encrypted with LUKS on a 50GB partition with an LVM inside for flexibility.
  • Separate encrypted LUKS partitions for:
    • Web server files (/var/www)
    • Game server files (/home/game)
    • Video hosting/streaming storage (/media/videos), using XFS for large file handling.
    • Logs (/var/log)
  • Each encrypted partition uses LUKS with AES-XTS encryption and SHA256 hashing.
  • Logical volumes to ease root resizing and management.
  • Passwords must be provided at boot to unlock these encrypted partitions.
  • SSH server installation enabled for remote access.

This YAML is a template and may need adjustment for your actual disk device name, sizes, and passphrases.

Here are customized yaml with different passphrase for each partition:

Here is a customized example of an Ubuntu 24.04 automated installation YAML (autoinstall.yaml) that includes separate encrypted partitions for your services (logs, web server, video hosting/streaming, Counter-Strike server) with LUKS encryption and incorporates passphrase management automation.

Explanation:

  • Separate encrypted LUKS partitions for your important data directories.
  • The root filesystem is encrypted and deployed via LVM for flexibility.
  • The /boot partition remains unencrypted as required.
  • Passphrase for full disk encryption is provided in cleartext in the YAML for automated unlocking during install.
  • This example assumes a single disk /dev/sda. Adjust disk names, sizes, and passphrases according to your hardware.
  • You can tailor user identity and SSH access as needed.

Customized autoinstall.yaml example:

autoinstall:
  version: 1

  identity:
    hostname: myserver
    username: myuser
    password: ${ENCRYPTED_PASSWORD}  # hashed password for user account

  locale: en_US.UTF-8
  keyboard:
    layout: us

  storage:
    layout:
      name: lvm
    config:
      - type: disk
        id: disk0
        match:
          name: "sda"
        ptable: gpt
        wipe: superblock-recursive
        grub_device: true
        children:
          - type: partition
            id: boot-partition
            size: 512M
            flag: boot
          - type: partition
            id: luks-root-partition
            size: 50G
          - type: partition
            id: luks-web-partition
            size: 20G
          - type: partition
            id: luks-game-partition
            size: 30G
          - type: partition
            id: luks-video-partition
            size: 100G
          - type: partition
            id: luks-logs-partition
            size: 10G

      - type: format
        fstype: ext4
        volume: boot-partition
        mount:
          point: /boot
      
      - type: luks
        id: luks-root
        volume: luks-root-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
          passphrase: ${ENCRYPTED_PASSWORD_ROOT}       # passphrase for root LUKS partition
        children:
          - type: lvm_volgroup
            id: vg-root
            name: vg-root
            children:
              - type: lvm_logicalvolume
                id: lv-root
                name: lv-root
                size: 100%FREE
                format: ext4
                mount:
                  point: /
      
      - type: luks
        id: luks-web
        volume: luks-web-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
          passphrase: ${ENCRYPTED_PASSWORD_WEB}        # passphrase for web partition
        children:
          - type: format
            fstype: ext4
            mount:
              point: /var/www

      - type: luks
        id: luks-game
        volume: luks-game-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
          passphrase: ${ENCRYPTED_PASSWORD_GAME}      # passphrase for game server partition
        children:
          - type: format
            fstype: ext4
            mount:
              point: /home/game

      - type: luks
        id: luks-video
        volume: luks-video-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
          passphrase: ${ENCRYPTED_PASSWORD_VIDEO}      # passphrase for video storage
        children:
          - type: format
            fstype: xfs
            mount:
              point: /media/videos

      - type: luks
        id: luks-logs
        volume: luks-logs-partition
        encryption:
          cipher: aes-xts-plain64
          key-size: 512
          hash: sha256
          passphrase: ${ENCRYPTED_PASSWORD_LOGS}      # passphrase for logs partition
        children:
          - type: format
            fstype: ext4
            mount:
              point: /var/log

  ssh:
    install-server: true
    allow-pw: true

  packages:
    - curl
    - vim
    - htop

  late-commands:
    - curtin in-target --target=/target -- systemctl enable ssh.service

Notes on passphrase management automation:

  • If the passphrases for each LUKS partition are defined in plain text under the encryption.passphrase field to enable automated unlocking during installation.
  • Security note: Including passphrases in plain text in the YAML poses a potential security risk if this file is exposed. Consider generating or encrypting the autoinstall.yaml securely and controlling access strictly.
  • To further automate unlocking at boot after installation, consider:
    • Using a keyfile stored on an isolated secure partition.
    • Integrating with TPM or network-bound disk encryption if supported.
  • For environments requiring more advanced key management, post-install scripts or configuration management tools can deploy and configure the unlocking mechanism.

To automate unlocking encrypted LUKS partitions at boot after installation, there are several methods to consider for secure and convenient key management. Here’s an overview of the approaches you mentioned:

1. Unlocking with a Keyfile Stored on an Isolated Secure Partition

  • Create a dedicated small partition (e.g., unencrypted or lightly protected) that stores a keyfile.
  • Generate a random keyfile during installation or beforehand: 
    dd if=/dev/urandom of=/secure_keyfile bs=4096 count=1
chmod 0400 /secure_keyfile
  • Add the keyfile to each LUKS volume so that it can unlock the volume: 
    sudo cryptsetup luksAddKey /dev/sdXn /secure_keyfile
  • Configure /etc/crypttab to use the keyfile for unlocking partitions at boot: 
    luks-root UUID= /secure_keyfile luks
luks-web UUID= /secure_keyfile luks
# and so on for each encrypted volume
  • Mount the partition containing the keyfile early during boot (for example, in /etc/fstab).

Security notes:

  • The partition with the keyfile should be protected with strict permissions or physically isolated.
  • If the keyfile partition is compromised, it undermines encryption security.

2. Integrating with TPM (Trusted Platform Module)

  • TPM can securely store encryption keys or keyfiles tied to the machine’s hardware state, so the volume is automatically unlocked if the system is in a trusted state.
  • Use tools like Clevis and Tang or systemd-cryptenroll (introduced in recent systemd versions) for TPM integration: 
    sudo systemd-cryptenroll --tpm2-device=auto /dev/sdXn
  • The TPM chips stores the decryption key securely; unlocking happens automatically during boot if TPM PCR values match, protecting from tampering.
  • TPM integration supports sealed keys and is ideal for unattended boot while retaining hardware-rooted security.

3. Network-Bound Disk Encryption (NBDE)

  • NBDE allows unlocking encrypted disks at boot only when the host can communicate securely with a network server (key server).
  • Typically implemented using Clevis (client-side) and Tang (server-side):
    • The Clevis client on the Ubuntu server integrates with initramfs to fetch the key during early boot.
    • The Tang server provides the decryption key over the network after policy-based attestation.
  • Set up example with Clevis: 
    sudo clevis luks bind -d /dev/sdXn tang '{"url":"http://tang_server_url"}'
  • Boots unattended only if network is available and authorization succeeds.

Putting it all together in Ubuntu 24.04 automated install:

  • You can embed custom scripts or config files in the late-commands section of your autoinstall YAML to:
    • Generate/store encryption keys.
    • Add keys to LUKS volumes (luksAddKey).
    • Install and configure Clevis and TPM tools.
    • Edit /etc/crypttab and rebuild initramfs accordingly.

This allows fully automated, secure unlocking of encrypted volumes without manual passphrase entry, with fallback options for security depending on your environment.

For creating your own fully functioning autoinstall.yaml for Ubuntu 24.04 with encrypted partition layouts, refer to the official Ubuntu autoinstall configuration reference manual and community guides on LUKS partition encryption in autoinstall.

[1] https://www.youtube.com/watch?v=ibvxiybT96M [2] https://blog.local-optimum.net/getting-started-with-autoinstall-on-ubuntu-desktop-24-04-lts-147a1defb2de [3] https://coastipc.com/tech-support/ubuntu-server-24-04-lts-yaml-installation-guide [4] https://www.baeldung.com/linux/ubuntu-autoinstall-no-screen-keyboard [5] https://c-nergy.be/blog/?p=20051 [6] https://nsg.cc/post/2024/autoinstall/ [7] https://linuxconfig.org/how-to-write-and-perform-ubuntu-unattended-installations-with-autoinstall [8] https://github.com/Kikyo-chan/Autoinstall-Ubuntu24.04-LTS-Server-and-Desktop [9] https://canonical-subiquity.readthedocs-hosted.com/en/latest/reference/autoinstall-reference.html [10] https://www.tencentcloud.com/techpedia/116710 [11] https://infotechys.com/full-disk-encryption-on-rhel-9/ [12] https://www.vdocipher.com/blog/aws-s3-video-streaming/ [13] https://www.linkedin.com/pulse/secure-your-bare-metal-server-like-pro-redswitches-39lcf [14] https://documentation.suse.com/pt-br/sles/15-SP4/html/SLES-all/cha-security-cryptofs.html [15] https://docs.redhat.com/en/documentation/openshift_container_platform/4.9/html/security_and_compliance/network-bound-disk-encryption-nbde [16] https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-111.pdf [17] https://lowendtalk.com/discussion/191877/best-full-disk-encryption-option-for-lowend-dedicated-server [18] https://documentation.suse.com/fr-fr/sles/12-SP5/html/SLES-all/cha-security-cryptofs.html [19] https://www.reddit.com/r/archlinux/comments/1aihaep/how_important_is_disk_encryption/ [20] https://www.reddit.com/r/selfhosted/comments/yu4ctw/any_self_hosting_solution_for_secure_video/ [21] https://xebia.com/blog/cloud-data-security-encryption-at-rest-best-practices/ [22] https://linuxsecurity.com/features/authoritative-guide-on-linux-disk-encryption [23] https://steamcommunity.com/discussions/forum/10/144512942752009777/ [24] https://appkodes.com/blog/best-video-streaming-servers/ [25] https://wafatech.sa/blog/windows-server/windows-security/unlocking-security-a-comprehensive-guide-to-encryption-in-windows-server/ [26] https://wafatech.sa/blog/linux/linux-security/securing-sensitive-files-on-linux-servers-a-guide-to-file-encryption/ [27] https://support.kaspersky.com/kes-for-windows/12.8/130984 [28] https://blog.ishosting.com/en/how-to-create-video-streaming-server [29] https://www.precisely.com/blog/data-security/data-encryption-101-guide-best-practices [30] https://nsg.cc/post/2024/autoinstall/ [31] https://www.reddit.com/r/Ubuntu/comments/1i3ljvf/ubuntu_autoinstall_for_server_seeking_examples/ [32] https://www.youtube.com/watch?v=ibvxiybT96M [33] https://linuxconfig.org/how-to-write-and-perform-ubuntu-unattended-installations-with-autoinstall [34] https://canonical-subiquity.readthedocs-hosted.com/en/latest/reference/autoinstall-reference.html [35] https://cosimomatteini.com/blog/automate-ubuntu-desktop-installation/ [36] https://discourse.ubuntu.com/t/subiquity-24-04-1-has-been-released-to-the-stable-channel/44493 [37] https://github.com/canonical/autoinstall-desktop