diff options
| author |   Eric Hameleers <alien@slackware.com> | 2023-09-10 17:14:47 +0200 |
|---|---|---|
| committer | Eric Hameleers <alien@slackware.com> | 2023-09-10 17:14:47 +0200 |
| commit | 672a9e4e7f160a65d1bdae49a65b734a2627a5f1 (patch) | |
| tree | 3d57adf2cf43db2e55d0bf25cb4cb4a62a46b070 | |
| parent | fad898a736db76ea3159b668a5979a8efc8ef80a (diff) | |
| download | liveslak-672a9e4e7f160a65d1bdae49a65b734a2627a5f1.tar.gz liveslak-672a9e4e7f160a65d1bdae49a65b734a2627a5f1.tar.xz | |
Several enhancements to LUKS containers and their support scripts
To name the most important:
- The filesystem inside the containers is no longer limited to ext4 and can
now be btrfs, ext2, ext4, f2fs, jfs or xfs. The scripts 'iso2usb.sh' and
'isocomp.sh' now support this filesystem choice via a new '-F' switch.
- The 'upslak.sh' script can now extend the size of your home and persistence
container files, in case you run out of storage there.
- The internal functions used in 'iso2usb.sh' 'isocomp.sh' and 'upslak.sh'
are mostly unified now, and I intend to split all functions off into a
separate file which makes maintenance as well as adding new functionaliy
easier.
- The 'upslak.sh' script will now create a 'kernelmodules' squashfs module
and copies it to liveslak's 'addons' directory after updating the boot
kernel on the USB stick and the kernel-modules inside the initrd image.
This way, you no longer end up with a broken Live OS if you forgot to
install a Slackware kernel-modules package inside the persistent Live OS
prior to running 'upslak.sh' with the '-k' and '-m' parameters.
In addition, the README.txt was updated with these enhancements so that
the liveslak documentation is again completely up to date.
| -rw-r--r-- | README.txt | 83 | ||||
| -rw-r--r-- | iso2usb.sh | 281 | ||||
| -rw-r--r-- | isocomp.sh | 252 | ||||
| -rw-r--r-- | liveinit.tpl | 8 | ||||
| -rwxr-xr-x | make_slackware_live.sh | 2 | ||||
| -rw-r--r-- | pkglists/min.lst | 6 | ||||
| -rw-r--r-- | upslak.sh | 526 |
7 files changed, 923 insertions, 235 deletions
@@ -166,23 +166,40 @@ This script, called 'iso2usb.sh', accepts the following parameters: <code> -f|--force Ignore most warnings (except the back-out). -h|--help This help. -i|--infile <filename> Full path to the ISO image file. + -l|--lukshome <name> Custom path to the containerfile for your LUKS + encrypted /home (slhome by default). -o|--outdev <filename> The device name of your USB drive. - -p|--persistence <name> Custom name of the 'persistence' directory/file. - If it does not exist yet, create it manually. + -p|--persistence <name> Custom path to the 'persistence' directory + or containerfile (persistence by default). -r|--refresh Refresh the USB stick with the ISO content. No formatting, do not touch user content. -s|--scan Scan for insertion of new USB device instead of providing a devicename (using option '-o'). -u|--unattended Do not ask any questions. -v|--verbose Show verbose messages. - -w|--wait<number> Add <number> seconds wait time to initialize USB. + -w|--wait <number> Add <number> seconds wait time to initialize USB. + -y|--layout <x,x,x,x> Specify partition layout and sizes (in MB). + Default values: '1,100,-1,' for 3 partitions, + the '-1' value for partition 3 meaning + 'use all remaining space', + and an empty 4th value means 'do not reserve + free space for a custom 4th partition'. -C|--cryptpersistfile size|perc Use a LUKS-encrypted 'persistence' file instead of a directory (for use on FAT filesystem). Format for size/percentage is the same as for the '-c' parameter. + -F|--filesystem <fs> Specify filesystem to create when formatting + devices/containers. Defaults to 'ext4', + Choices are btrfs,ext2,ext4,f2fs,jfs,xfs. + Note that the linux partition will always be + formatted as 'ext4' because extlinux is used + as the BIOS bootloader. -P|--persistfile Use an unencrypted 'persistence' file instead of a directory (for use on FAT filesystem). + Persistent data will not be migrated + when switching from directory to container file. + </code> Examples: @@ -192,8 +209,10 @@ Examples: # ./iso2usb.sh -i slackware64-live-current.iso -o /dev/sdX -c 750M -w 15 * Create a USB Live with an encrypted /home (allocating 30% of the stick's free space for /home) and where the persistent data will be stored in a container file instead of a directory: # ./iso2usb.sh -i slackware64-live-current.iso -o /dev/sdX -c 30% -P - * Create a USB Live with both the /home and the persistent data encrypted (the persistence filesystem will be 300 MB in size): - # ./iso2usb.sh -i slackware64-live-current.iso -o /dev/sdX -c 30% -C 300M + * Create a USB Live with both the /home and the persistent data encrypted (the persistence filesystem will be 300 MB in size) using a btrfs filesystem: + # ./iso2usb.sh -i slackware64-live-current.iso -o /dev/sdX -F btrfs -c 30% -C 300M + * Create a 32bit USB Live but use a custom partition layout: create a 1 MB BIOS boot partition and a 200 MB EFI partition, add a 4th un-used $ GB partition at the end, and allocate all remaining disk space to the main Linux partition: + # iso2usb.sh -i slackware-live-current.iso -o /dev/sdX -y 1,200,-1,4096 * Refresh the system modules on a USB Live using a Live ISO as the source. Let the script scan for insertion of a USB stick instead of specifying the device name on the commandline. Note that the addons and optional modules will not be touched by this action: # ./iso2usb.sh -i slackware64-live-current.iso -r -s @@ -227,11 +246,12 @@ The script is called 'isocomp.sh', and it accepts the following parameters: <cod file to be created in the filesystem (filename extension must be '.icc'!). -x|--extend <fullpath> Full path to existing (encrypted) container - file that you want to extend in size - (filename needs to end in '.icc'!). + file that you want to extend in size. Limitations: - - container needs to be LUKS encrypted, and - - internal filesystem needs to be ext{2,3,4}. + - container needs to be LUKS encrypted. + - filename needs to end in '.icc'. + Supported filesystems inside container: + - btrfs,ext2,ext4,f2fs,jfs,xfs. -L|--lcsize <size|perc> Size of LUKS encrypted /home ; value is the requested size of the container in kB, MB, GB, or as a percentage of free space @@ -252,7 +272,7 @@ Some examples of what the script can do, are given when you run the script with * Create a 1GB encrypted persistence container: # ./isocomp.sh -p /run/media/<user>/Ventoy/myfiles/persistence.icc -P 1G - * Create a 4GB encrypted home: + * Create a 4GB encrypted home with btrfs filesystem: # ./isocomp.sh -l /run/media/<user>/Ventoy/somedir/lukscontainers.icc -L 4000M -i /run/media/<user>/Ventoy/slackware64-live-current.iso * Increase the size of that encrypted home container with another 2GB: # ./isocomp.sh -x /run/media/<user>/Ventoy/somedir/lukscontainers.icc -X 2G -i /run/media/<user>/Ventoy/slackware64-live-current.iso @@ -306,6 +326,7 @@ Before making any modifications, the script will show you a prompt at which poin This script, called 'upslak.sh', accepts the following parameters: <code> -b|--nobackup Do not try to backup original kernel and modules. -d|--devices List removable devices on this computer. + -e|--examples Show some common usage examples. -h|--help This help. -i|--init <filename> Replacement init script. -k|--kernel <filename> The kernel file (or package). @@ -318,6 +339,26 @@ This script, called 'upslak.sh', accepts the following parameters: <code> providing a devicename (using option '-o'). -v|--verbose Show verbose messages. -w|--wait<number> Add <number> seconds wait time to initialize USB. + -x|--extend <fullpath> Full path (either in your filesystem or else + relative to the USB partition root) + to an existing (encrypted) container file, + whose size you want to extend. + Limitations: + - container needs to be LUKS encrypted. + - filename extension needs to be '.img'. + Supported filesystems inside container: + - btrfs,ext2,ext4,f2fs,jfs,xfs. + -N|--nolivemods Don't create an addon live module containing + the new kernelmodules. Normally you *will* need + this addon module, *unless* you have already + installed these kernel-modules in the Live OS. + FYI: the kernel and module upgrade applies only + to the USB boot kernel and its initrd. + -X|--extendsize <size|perc> Extend size of existing container; value + is the requested extension of the container + in kB, MB, GB, or as percentage of free space + (integer numbers only). + Examples: '-X 125M', '-X 2G', '-X 20%'. </code> Examples: @@ -328,7 +369,10 @@ Examples: * Restore the previous kernel and modules after a failed update, and let the script scan your computer for the insertion of your USB stick: # ./upslak.sh -s -r * Replace the Live init script with the latest template taken from the git repository: - # ./upslak.sh -o /dev/sdX -i liveslak/liveinit.tpl + # wget https://git.liveslak.org/liveslak/plain/liveinit.tpl + # ./upslak.sh -o /dev/sdX -i liveinit.tpl + * Extend the size of the pre-existing LUKS container for your homedirectory with 3 GB, and let the script scan for the insertion of your USB stick: + # ./upslak.sh -s -x /slhome.img -X 3G ==== PXE booting the Live OS ==== @@ -658,7 +702,7 @@ The "iso2usb.sh" script wipes and re-partitions the USB stick unless the "-r" or * Second partition: a 100 MB VFAT partition containing the kernel, initrd and all the other stuff required by syslinux and grub2 to boot Slackware Live Edition. * Third partition: a Linux partition taking up all of the remaining space. It contains the actual liveslak modules, the persistent live storage and optionally your encrypted homedirectory. You can use the remainder of this Linux ext4 filesystem's free space to store anything you like. -Note that this script is the only supported method of transfering the liveslak ISO content to a USB stick and make that USB stick into a persistent live OS. Several 3rd party tools (like multibootusb, rufus, unetbootin) that claim to be able to mix several Live OS'es on a single USB stick and make them all work in a multi-boot setup, are not currently supporting liveslak. +Note that this script extracts the ISO contents to transform a USB stick into into a persistent live OS. This is a destructive process, erasing all previously available content on that stick. Several 3rd party tools (like multibootusb, rufus, unetbootin) that claim to be able to mix several Live OS'es on a single USB stick and make them all work in a multi-boot setup, are not currently supporting liveslak. Ventoy on the other hand, is fully supported by liveslak and therefore your best bet if you don't want to wipe your data off your USB stick. As a bonus, the ''isocomp.sh'' script is able to add persistence to a liveslak ISO on a Ventoy boot disk. == Mounting a filesystem in an encrypted container == @@ -713,7 +757,7 @@ If the container is used for an encrypted /home, the script will copy the existi == Extending the size of an existing container file == -The 'isocomp.sh' script is able to extend your encrypted containers if you are running out of space on their enclosed filesystems. It does this by appending random bytes to the end of the file, unlocking and mounting the filesystem inside, and then resizing that filesystem so it grows to the new size of the container. Note that only containers with an internal ''ext4'' filesystem are supported. +The 'isocomp.sh' script is able to extend your encrypted containers if you are running out of space on their enclosed filesystems. It does this by appending random bytes to the end of the file, unlocking and mounting the filesystem inside, and then resizing that filesystem so it grows to the new size of the container. === makemod === @@ -792,9 +836,13 @@ Depending on the parameters passed to the script, it will then perform one or mo You can provide a new kernel and its modules in two ways. The '-k' option accepts a kernel image file or else a Slackware package contaning a kernel. The '-m' option accepts a directory tree of modules below "/lib/modules/, or else a Slackware package containing kernel modules. If there is sufficient space on the Linux and EFI partitions, the script will make a backup of the current kernel and modules by renaming the kernel and the module directory with a ".prev" suffix. Sufficient space means that at least 10 MB of free space must remain on the partition(s) after making the backup and installing the new kernel plus modules. If space is an issue, you can skip making a backup by providing the '-b' parameter to the script (a possibly unsafe choice). +Note that these new kernel-modules will be added to the initrd image (they are needed when booting the new kernel). In order for the Live OS to keep working with the new kernel however, these new kernel-modules must also be made available to the Live OS. The script achieves this by creating a '''kernelmodules''' squashfs module and copying the module into the '''addons''' directory of the liveslak installation on your USB stick. When the Live OS boots, the kernelmodules will then automatically be merged into the live filesystem. +It is possible to skip the creation of this squashfs module via the '-N' switch to the script. + == Restore backed-up kernel and modules == If a backup was made of kernel and modules, the upslak.sh script is able to restore these using the '-r' option, thereby removing the replacements. This comes in handy when the replacement kernel turns out to be non-functional. +Note that restoring the old kernel and its modules will leave an orphaned squashfs kernelmodule in liveslak's '''addons''' directory. You can safely delete that file. == Add network support modules == @@ -808,6 +856,10 @@ Similar to the functionality of the "iso2usb.sh" script, the "upslak.sh" script The init script inside the initrd image is the core of liveslak. The init script prepares the Live filesystem and configures several run-time OS parameters. If you have made modifications to this init script you can easily replace the default init script with your own script using the '-i' option. The "upslak.sh" script is smart enough to recognize a iveslak template as input. The ".tpl" extension of some liveslak files means that these are templates. They are not usable as-is, because they contain placeholder strings like "@VERSION@" or "@DISTRO@" that first need to be replaced with real values. The "upslak.sh" script will take care of these substitutions. +== Extend the size of an existing LUKS container == + +Your Slackware Live USB stick will probably have two LUKS containers: one for your ''/home'' directory and the other to store persistent data. If space is running out inside such a container, you can use the '-x' and '-X' parameters to the script to indicate the relevant container and provide a size increase for it. Your data inside the container is safe; the filesystem inside it will be extended and this does not touch existing file data. + == Wrap persistence data into a new squashfs module == Persistence data will accumulate over time on the USB stick. That is perfectly OK, and you can wipe it on boot if that is needed. But sometimes you want to capture the packages you installed into the persistent storage, and create a new squashfs module out of them. The "upslak.sh" script is able to move your persistence data into a new squashfs module using the '-p' option. The new module will be created in the "/liveslak/addons/" directory so that it will be loaded into the Live OS everytime your USB Live boots up. After creating the new module, the persistence store will be re-initialized (i.e. its content will be erased on the next boot). The new module's name is time-stamped (/liveslak/addons/0099-slackware__customchanges-yyyymmddHHMMSS.sxz where yyyymmddHHMMSS is the timestamp) so that this action can be repeated as many times as you want. @@ -879,7 +931,8 @@ The script's parameters are: DAW (Digital Audio Workstation), XFCE (basic XFCE, stripped), KTOWN (ktown Plasma5 replacement), MATE (Gnome2 fork replaces KDE), CINNAMON (fork of Gnome3 Shell - replaces KDE), DLACK (Gnome3 replaces KDE). + replaces KDE), DLACK (Gnome3 replaces KDE), + STUDIOWARE (Multimedia Studio). -e Use ISO boot-load-size of 32 for computers where the ISO won't boot otherwise (default: 4). -f Forced re-generation of all squashfs modules, @@ -894,7 +947,7 @@ The script's parameters are: -v Show debug/error output. -z version Define your Slackware version (default: current). -C Add RAM-based Console OS to boot menu. - -G Generate ISO file from existing directory tree + -G Generate ISO file from existing directory tree. -H hostname Hostname of the Live OS (default: darkstar). -M Add multilib (x86_64 only). -O outfile Custom filename for the ISO. @@ -32,11 +32,20 @@ FORCE=0 # The default layout of the USB stick is: # partition 1 (1MB), # partition 2 (100 MB) -# partition 3 (claim all free space - specified as 0 MB). +# partition 3 (claim all free space - specified as -1 MB). # The script allows for an amount of free space to be left at the end -# (partition 4, unused by liveslak) in case you need this: +# (partition 4, un-used by liveslak) in case you need this: DEF_LAYOUT="1,100,-1," +# The extension for containerfiles accompanying an ISO is '.icc', +# whereas the persistent USB stick created with iso2usb.sh uses '.img'. +DEFEXT=".img" +CNTEXT="${DEFEXT}" + +# Default filesystem for devices/containers: +DEF_FS="ext4" +FSYS="${DEF_FS}" + # By default, we use 'slhome.img' as the name of the LUKS home containerfile. DEF_SLHOME="slhome" SLHOME="${DEF_SLHOME}" @@ -97,9 +106,7 @@ else fi # Initialize more variables: -CNTBASE="" CNTDEV="" -CNTFILE="" HLUKSSIZE="" LUKSHOME="" LODEV="" @@ -111,6 +118,11 @@ CNTMNT="" USBMNT="" US2MNT="" +# Minimim free space (in MB) we want to have left in any partition +# after we are done. +# The default value can be changed from the environment: +MINFREE=${MINFREE:-10} + # Compressor used on the initrd ("gzip" or "xz --check=crc32"); # Note that the kernel's XZ decompressor does not understand CRC64: COMPR="xz --check=crc32" @@ -120,7 +132,7 @@ COMPR="xz --check=crc32" # # Clean up in case of failure: -cleanup() { +function cleanup() { # Clean up by unmounting our loopmounts, deleting tempfiles: echo "--- Cleaning up the staging area..." # During cleanup, do not abort due to non-zero exit code: @@ -130,7 +142,7 @@ cleanup() { # In case of failure, only the most recent device should still be open: if mount |grep -q ${CNTDEV} ; then umount -f ${CNTDEV} - cryptsetup luksClose $(basename ${CNTBASE}) + cryptsetup luksClose $(basename ${CNTDEV}) losetup -d ${LODEV} fi fi @@ -140,10 +152,10 @@ cleanup() { [ -n "${US2MNT}" ] && ( umount -f ${US2MNT} 2>/dev/null; rmdir $US2MNT 2>/dev/null ) [ -n "${IMGDIR}" ] && ( rm -rf $IMGDIR ) set -e -} +} # End of cleanup() trap 'echo "*** $0 FAILED at line $LINENO ***"; cleanup; exit 1' ERR INT TERM -showhelp() { +function showhelp() { cat <<EOT # # Purpose: to transfer the content of Slackware's Live ISO image @@ -186,6 +198,12 @@ cat <<EOT # of a directory (for use on FAT filesystem) # Format for size/percentage is the same # as for the '-c' parameter. +# -F|--filesystem <fs> Specify filesystem to create when formatting +# devices/containers. Defaults to '${DEF_FS}', +# Choices are $(createfs). +# Note that the linux partition will always be +# formatted as 'ext4' because extlinux is used +# as the BIOS bootloader. # -P|--persistfile Use a 'persistence' container file instead of # a directory (for use on FAT filesystem). # Persistent data will not be migrated @@ -198,30 +216,76 @@ cat <<EOT # $(basename $0) -i slackware-live-current.iso -o /dev/sdX -y 1,200,-1,4096 # EOT -} +} # End of showhelp() + +# Create a filesystem on a partition with optional label: +function createfs () { + MYDEV="${1}" + MYFS="${2:-'ext4'}" + MYLABEL="${3}" + + if [ -z "${MYDEV}" ]; then + # Without arguments given, reply with list of supported fs'es: + echo "btrfs,ext2,ext4,f2fs,jfs,xfs" + return + fi + + if [ -n "${MYLABEL}" ]; then + case "${MYFS}" in + fs2s) MYLABEL="-l ${MYLABEL}" ;; + *) MYLABEL="-L ${MYLABEL}" ;; + esac + fi + + case "${MYFS}" in + btrfs) mkfs.btrfs -f -d single -m single ${MYLABEL} ${MYDEV} + ;; + ext2) mkfs.ext2 -F -F ${MYLABEL} ${MYDEV} + # Tune the ext2 filesystem: + tune2fs -m 0 -c 0 -i 0 ${MYDEV} + ;; + ext4) mkfs.ext4 -F -F ${MYLABEL} ${MYDEV} + # Tune the ext4 filesystem: + tune2fs -m 0 -c 0 -i 0 ${MYDEV} + ;; + f2fs) mkfs.f2fs ${MYLABEL} -f ${MYDEV} + ;; + jfs) mkfs.jfs -q ${MYDEV} + ;; + xfs) mkfs.xfs -f ${MYDEV} + ;; + *) echo "*** Unsupported filesystem '${MYFS}'!"; exit 1 + ;; + esac +} # End of createfs() # Uncompress the initrd based on the compression algorithm used: -uncompressfs () { - if $(file "${1}" | grep -qi ": gzip"); then - gzip -cd "${1}" - elif $(file "${1}" | grep -qi ": XZ"); then - xz -cd "${1}" - elif $(file "${1}" | grep -qi ": lzip"); then - lzip -cd "${1}" +function uncompressfs () { + local IMGFILE="$1" + # Content is streamed to STDOUT: + if $(file "${IMGFILE}" | grep -qi ": gzip"); then + gzip -cd "${IMGFILE}" + elif $(file "${IMGFILE}" | grep -qi ": XZ"); then + xz -cd "${IMGFILE}" + elif $(file "${IMGFILE}" | grep -qi ": LZMA"); then + lzma -cd "${IMGFILE}" + elif $(file "${IMGFILE}" | grep -qi ": lzip"); then + lzip -cd "${IMGFILE}" fi -} +} # End of uncompressfs() # Scan for insertion of a USB device: -scan_devices() { +function scan_devices() { + local MYSCANWAIT="${1}" local BD # Inotifywatch does not trigger on symlink creation, # so we can not watch /sys/block/ - BD=$(inotifywait -q -t ${SCANWAIT} -e create /dev 2>/dev/null |cut -d' ' -f3) + BD=$(inotifywait -q -t ${MYSCANWAIT} -e create /dev 2>/dev/null |cut -d' ' -f3) echo ${BD} } # End of scan_devices() # Show a list of removable devices detected on this computer: -show_devices() { +function show_devices() { local MYDATA="${*}" if [ -z "${MYDATA}" ]; then MYDATA="$(ls --indicator-style=none /sys/block/ |grep -Ev '(ram|loop|dm-)')" @@ -237,16 +301,16 @@ show_devices() { } # End of show_devices() # Read configuration data from old initrd: -read_initrd() { +function read_initrd() { IMGFILE="$1" OLDPERSISTENCE=$(uncompressfs ${IMGFILE} |cpio -i --to-stdout init |grep "^PERSISTENCE" |cut -d '"' -f2 2>/dev/null) OLDWAIT=$(uncompressfs ${IMGFILE} |cpio -i --to-stdout wait-for-root 2>/dev/null) OLDLUKS=$(uncompressfs ${IMGFILE} |cpio -i --to-stdout luksdev 2>/dev/null) -} +} # End of read_initrd() # Add longer USB WAIT to the initrd: -update_initrd() { +function update_initrd() { IMGFILE="$1" # USB boot medium needs a few seconds boot delay else the overlay will fail. @@ -314,93 +378,133 @@ update_initrd() { rm -rf $IMGDIR/* } # End of update_initrd() +# Determine size of a mounted partition (in MB): +function get_part_mb_size() { + local MYPART="${1}" + local MYSIZE + MYSIZE=$(df -P -BM ${MYPART} |tail -n -1 |tr -s '\t' ' ' |cut -d' ' -f2) + echo "${MYSIZE%M}" +} # End of get_part_mb_size() + +# Determine free space of a mounted partition (in MB): +function get_part_mb_free() { + local MYPART="${1}" + local MYSIZE + MYSIZE=$(df -P -BM ${MYPART} |tail -n -1 |tr -s '\t' ' ' |cut -d' ' -f4) + echo "${MYSIZE%M}" +} # End of get_part_mb_free() + +# Determine requested container size in MB (allow for '%|k|K|m|M|g|G' suffix): +function cont_mb() { + # Uses global variables: PARTFREE + local MYSIZE="$1" + case "${MYSIZE: -1}" in + "%") MYSIZE="$(( $PARTFREE * ${MYSIZE%\%} / 100 ))" ;; + "k") MYSIZE="$(( ${MYSIZE%k} / 1024 ))" ;; + "K") MYSIZE="$(( ${MYSIZE%K} / 1024 ))" ;; + "m") MYSIZE="${MYSIZE%m}" ;; + "M") MYSIZE="${MYSIZE%M}" ;; + "g") MYSIZE="$(( ${MYSIZE%g} * 1024 ))" ;; + "G") MYSIZE="$(( ${MYSIZE%G} * 1024 ))" ;; + *) MYSIZE=-1 ;; + esac + echo "$MYSIZE" +} # End of cont_mb() + # Create a container file in the empty space of the partition -create_container() { - CNTPART=$1 - CNTSIZE=$2 - CNTBASE=$3 - CNTENCR=$4 # 'none' or 'luks' - CNTUSED=$5 # '/home' or 'persistence' +function create_container() { + local CNTPART=$1 # partition containing the ISO + local CNTSIZE=$2 # size of the container file to create + local CNTFILE=$3 # ${CNTEXT} filename with full path + local CNTENCR=$4 # 'none' or 'luks' + local CNTUSED=$5 # '/home' or 'persistence' + local MYMAP + local MYMNT + + # If containerfile extension is missing, add it now: + if [ "${CNTFILE%${CNTEXT}}" == "${CNTFILE}" ]; then + CNTFILE="${CNTFILE}${CNTEXT}" + fi # Create a container file or re-use previously created one: - if [ -f $USBMNT/${CNTBASE}.img ]; then - CNTFILE="${CNTBASE}.img" - CNTSIZE=$(( $(du -sk $USBMNT/${CNTFILE} |tr '\t' ' ' |cut -f1 -d' ') / 1024 )) - echo "--- Keeping existing '${CNTFILE}' (size ${CNTSIZE} MB)." + if [ -f ${CNTFILE} ]; then + # Where are we mounted? + MYMNT=$(cd "$(dirname "${CNTFILE}")" ; df --output=target . |tail -1) + CNTSIZE=$(( $(du -sk ${CNTFILE} |tr '\t' ' ' |cut -f1 -d' ') / 1024 )) + echo "--- Keeping existing '${CNTFILE#${MYMNT}}' (size ${CNTSIZE} MB)." return fi # Determine size of the target partition (in MB), and the free space: - PARTSIZE=$(df -P -BM ${CNTPART} |tail -1 |tr -s '\t' ' ' |cut -d' ' -f2) - PARTSIZE=${PARTSIZE%M} - PARTFREE=$(df -P -BM ${CNTPART} |tail -1 |tr -s '\t' ' ' |cut -d' ' -f4) - PARTFREE=${PARTFREE%M} + PARTSIZE=$(get_part_mb_size ${CNTPART}) + PARTFREE=$(get_part_mb_free ${CNTPART}) - if [ $PARTFREE -lt 10 ]; then - echo "*** Free space on USB partition is less than 10 MB;" + if [ $PARTFREE -lt ${MINFREE} ]; then + echo "*** Free space on USB partition is less than ${MINFREE} MB;" echo "*** Not creating a container file!" + cleanup exit 1 fi - # Determine requested container size (allow for '%|k|K|m|M|g|G' suffix): - case "${CNTSIZE: -1}" in - "%") CNTSIZE="$(( $PARTFREE * ${CNTSIZE%\%} / 100 ))" ;; - "k") CNTSIZE="$(( ${CNTSIZE%k} / 1024 ))" ;; - "K") CNTSIZE="$(( ${CNTSIZE%K} / 1024 ))" ;; - "m") CNTSIZE="${CNTSIZE%m}" ;; - "M") CNTSIZE="${CNTSIZE%M}" ;; - "g") CNTSIZE="$(( ${CNTSIZE%g} * 1024 ))" ;; - "G") CNTSIZE="$(( ${CNTSIZE%G} * 1024 ))" ;; - *) ;; - esac + # Determine requested container size in MB (allow for '%|k|K|m|M|g|G' suffix): + CNTSIZE=$(cont_mb ${CNTSIZE}) if [ $CNTSIZE -le 0 ]; then echo "*** Container size must be larger than ZERO!" echo "*** Check your '-c' commandline parameter." + cleanup exit 1 elif [ $CNTSIZE -ge $PARTFREE ]; then echo "*** Not enough free space for container file!" echo "*** Check your '-c' commandline parameter." + cleanup exit 1 fi echo "--- Creating ${CNTSIZE} MB container file using 'dd if=/dev/urandom', patience please..." - mkdir -p $USBMNT/$(dirname "${CNTBASE}") - CNTFILE="${CNTBASE}.img" - # Create a sparse file (not allocating any space yet): - dd of=$USBMNT/${CNTFILE} bs=1M count=0 seek=$CNTSIZE + mkdir -p $(dirname "${CNTFILE}") + if [ $? ]; then + # Create a sparse file (not allocating any space yet): + dd of=${CNTFILE} bs=1M count=0 seek=$CNTSIZE 2>/dev/null + else + echo "*** Failed to create directory for the container file!" + cleanup + exit 1 + fi # Setup a loopback device that we can use with cryptsetup: LODEV=$(losetup -f) - losetup $LODEV $USBMNT/${CNTFILE} + losetup $LODEV ${CNTFILE} + MYMAP=$(basename ${CNTFILE} ${CNTEXT}) if [ "${CNTENCR}" = "luks" ]; then # Format the loop device with LUKS: - echo "--- Encrypting the container file with LUKS; enter 'YES' and a passphrase..." + echo "--- Encrypting the container file with LUKS via '${LODEV}'" + echo "--- This takes SOME time, please be patient..." + echo "--- enter 'YES' and a passphrase:" until cryptsetup -y luksFormat $LODEV ; do echo ">>> Did you type two different passphrases?" read -p ">>> Press [ENTER] to try again or Ctrl-C to abort ..." REPLY done # Unlock the LUKS encrypted container: echo "--- Unlocking the LUKS container requires your passphrase again..." - until cryptsetup luksOpen $LODEV $(basename ${CNTBASE}) ; do + until cryptsetup luksOpen $LODEV ${MYMAP} ; do echo ">>> Did you type an incorrect passphrases?" read -p ">>> Press [ENTER] to try again or Ctrl-C to abort ..." REPLY done - CNTDEV=/dev/mapper/$(basename ${CNTBASE}) + CNTDEV=/dev/mapper/${MYMAP} # Now we allocate blocks for the LUKS device. We write encrypted zeroes, # so that the file looks randomly filled from the outside. # Take care not to write more bytes than the internal size of the container: + echo "--- Writing ${CNTSIZE} MB of random data to encrypted container; takes LONG time..." CNTIS=$(( $(lsblk -b -n -o SIZE $(readlink -f ${CNTDEV})) / 512)) - dd if=/dev/zero of=${CNTDEV} bs=512 count=${CNTIS} || true + dd if=/dev/zero of=${CNTDEV} bs=512 count=${CNTIS} status=progress || true else - CNTDEV=$LODEV # Un-encrypted container files remain sparse. + CNTDEV=$LODEV fi # Format the now available block device with a linux fs: - mkfs.ext4 ${CNTDEV} - # Tune the ext4 filesystem: - tune2fs -m 0 -c 0 -i 0 ${CNTDEV} + createfs ${CNTDEV} ${FSYS} if [ "${CNTUSED}" != "persistence" ]; then # Create a mount point for the unlocked container: @@ -414,7 +518,7 @@ create_container() { fi # Copy the original /home (or whatever mount) content into the container: echo "--- Copying '${CNTUSED}' from LiveOS to container..." - HOMESRC=$(find ${USBMNT} -name "0099-slackware_zzzconf*" |tail -1) + HOMESRC=$(find ${ISOMNT} -name "0099-slackware_zzzconf*" |tail -1) mount ${CNTDEV} ${CNTMNT} unsquashfs -n -d ${CNTMNT}/temp ${HOMESRC} ${CNTUSED} mv ${CNTMNT}/temp/${CNTUSED}/* ${CNTMNT}/ @@ -424,10 +528,9 @@ create_container() { # Don't forget to clean up after ourselves: if [ "${CNTENCR}" = "luks" ]; then - cryptsetup luksClose $(basename ${CNTBASE}) + cryptsetup luksClose ${MYMAP} fi losetup -d ${LODEV} || true - } # End of create_container() { # @@ -506,6 +609,10 @@ while [ ! -z "$1" ]; do PERSISTTYPE="file" shift 2 ;; + -F|--filesystem) + FSYS="$2" + shift 2 + ;; -P|--persistfile) PERSISTTYPE="file" shift @@ -533,7 +640,7 @@ fi if [ -z "$TARGET" ]; then if [ $SCAN -eq 1 ]; then echo "-- Waiting ${SCANWAIT} seconds for a USB stick to be inserted..." - TARGET=$(scan_devices) + TARGET=$(scan_devices ${SCANWAIT}) if [ -z "$TARGET" ]; then echo "*** No new USB device detected during $SCANWAIT seconds scan." exit 1 @@ -554,6 +661,16 @@ if [ $FORCE -eq 0 -a ! -f "$SLISO" ]; then exit 1 fi +if [ "${HLUKSSIZE%.*}" != "${HLUKSSIZE}" ] ; then + echo "*** Integer value required in '-c $HLUKSSIZE' !" + exit 1 +fi + +if [ "${PLUKSSIZE%.*}" != "${PLUKSSIZE}" ] ; then + echo "*** Integer value required in '-C $PLUKSSIZE' !" + exit 1 +fi + if [ $FORCE -eq 0 ]; then if [ ! -e /sys/block/$(basename $TARGET) ]; then echo "*** Not a block device: '$TARGET' !" @@ -566,6 +683,9 @@ if [ $FORCE -eq 0 ]; then fi fi +# Add required filesystem tools: +REQTOOLS="${REQTOOLS} mkfs.${FSYS}" + # Are all the required not-so-common add-on tools present? PROG_MISSING="" for PROGN in ${REQTOOLS} ; do @@ -710,14 +830,15 @@ if [ $REFRESH -eq 0 ]; then if mount |grep -qw ${TARGETP3} ; then umount ${TARGETP3} || true fi + # We use extlinux to boot the stick, so other filesystems are not accepted: + createfs ${TARGETP3} ext4 "${LIVELABEL}" # http://www.syslinux.org/wiki/index.php?title=Filesystem - # As of Syslinux 6.03, "pure 64-bits" compression/encryption is not supported. + # As of Syslinux 6.03, "pure 64-bits" compression/encryption is unsupported. # Modern mke2fs creates file systems with the metadata_csum and 64bit # features enabled by default. # Explicitly disable 64bit feature in the mke2fs command with '-O ^64bit'; # otherwise, the syslinux bootloader (>= 6.03) will fail. # Note: older 32bit OS-es will trip over the '^64bit' feature so be gentle. - mkfs.ext4 -F -F -L "${LIVELABEL}" ${TARGETP3} UNWANTED_FEAT="" if tune2fs -O ^64bit ${TARGETP3} 1>/dev/null 2>/dev/null ; then UNWANTED_FEAT="^64bit," @@ -829,9 +950,13 @@ if [ -n "$VERSION" ]; then fi if [ -n "${HLUKSSIZE}" ]; then - # Create LUKS container file for /home: - create_container ${TARGETP3} ${HLUKSSIZE} ${SLHOME} luks /home - LUKSHOME=${CNTFILE} + # If file extension is missing in the containername, add it now: + if [ "${SLHOME%${CNTEXT}}" == "${SLHOME}" ]; then + SLHOME="${SLHOME}${CNTEXT}" + fi + # Create LUKS container file for /home ; + LUKSHOME="${SLHOME}" + create_container ${TARGETP3} ${HLUKSSIZE} "${USBMNT}/${LUKSHOME}" luks /home fi # Update the initrd with regard to USB wait time, persistence and LUKS. @@ -847,15 +972,15 @@ if [ $REFRESH -eq 1 ]; then # The user specified a nonstandard persistence, so move the old one first; # hide any errors if it did not *yet* exist: mkdir -p ${USBMNT}/$(dirname ${PERSISTENCE}) - mv ${USBMNT}/${OLDPERSISTENCE}.img ${USBMNT}/${PERSISTENCE}.img 2>/dev/null + mv ${USBMNT}/${OLDPERSISTENCE}${CNTEXT} ${USBMNT}/${PERSISTENCE}${CNTEXT} 2>/dev/null mv ${USBMNT}/${OLDPERSISTENCE} ${USBMNT}/${PERSISTENCE} 2>/dev/null fi - if [ -f ${USBMNT}/${PERSISTENCE}.img ]; then + if [ -f ${USBMNT}/${PERSISTENCE}${CNTEXT} ]; then # If a persistence container exists, we re-use it: PERSISTTYPE="file" - if cryptsetup isLuks ${USBMNT}/${PERSISTENCE}.img ; then + if cryptsetup isLuks ${USBMNT}/${PERSISTENCE}${CNTEXT} ; then # If the persistence file is LUKS encrypted we need to record its size: - PLUKSSIZE=$(( $(du -sk $USBMNT/${PERSISTENCE}.img |tr '\t' ' ' |cut -f1 -d' ') / 1024 )) + PLUKSSIZE=$(( $(du -sk $USBMNT/${PERSISTENCE}${CNTEXT} |tr '\t' ' ' |cut -f1 -d' ') / 1024 )) fi elif [ -d ${USBMNT}/${PERSISTENCE} -a "${PERSISTTYPE}" = "file" ]; then # A persistence directory exists but the user wants a container now; @@ -876,10 +1001,10 @@ elif [ "${PERSISTTYPE}" = "file" ]; then # Note: the word "persistence" below is a keyword for create_container: if [ -z "${PLUKSSIZE}" ]; then # Un-encrypted container: - create_container ${TARGETP3} 90% ${PERSISTENCE} none persistence + create_container ${TARGETP3} 90% ${USBMNT}/${PERSISTENCE} none persistence else # LUKS-encrypted container: - create_container ${TARGETP3} ${PLUKSSIZE} ${PERSISTENCE} luks persistence + create_container ${TARGETP3} ${PLUKSSIZE} ${USBMNT}/${PERSISTENCE} luks persistence fi else echo "*** Unknown persistence type '${PERSISTTYPE}'!" @@ -50,6 +50,10 @@ VERSION="" DEFEXT=".icc" CNTEXT="${DEFEXT}" +# Default filesystem for devices/containers: +DEF_FS="ext4" +FSYS="${DEF_FS}" + # Default mount point for a LUKS container if not specified: DEFMNT="/home" LUKSMNT="" @@ -71,8 +75,10 @@ LIVESLAKROOT="" # Define ahead of time, so that cleanup knows about them: IMGDIR="" ISOMNT="" +CNTDEV="" CNTMNT="" EXTENSION="" +LODEV="" PERSISTENCE="" # Minimim free space (in MB) we want to have left in any partition @@ -85,14 +91,14 @@ MINFREE=${MINFREE:-10} COMPR="xz --check=crc32" # These tools are required by the script, we will check for their existence: -REQTOOLS="cpio cryptsetup fsck gzip isoinfo lsblk resize2fs unsquashfs xz zstd" +REQTOOLS="cpio cryptsetup fsck gzip isoinfo lsblk unsquashfs xz zstd" # # -- function definitions -- # # Clean up in case of failure: -cleanup() { +function cleanup() { # Clean up by unmounting our loopmounts, deleting tempfiles: echo "--- Cleaning up the staging area..." # During cleanup, do not abort due to non-zero exit code: @@ -103,7 +109,7 @@ cleanup() { # In case of failure, only most recent LUKS mapped device is still open: if mount | grep -q ${CNTDEV} ; then umount -f ${CNTDEV} - cryptsetup luksClose $(basename ${CNTFILE} ${CNTEXT}) + cryptsetup luksClose $(basename ${CNTDEV}) losetup -d ${LODEV} fi fi @@ -116,7 +122,7 @@ cleanup() { trap 'echo "*** $0 FAILED at line $LINENO ***"; cleanup; exit 1' ERR INT TERM # Show the help text for this script: -showhelp() { +function showhelp() { cat <<EOT # # Purpose: enhance the functionality when booting a Slackware Live ISO file. @@ -146,10 +152,14 @@ cat <<EOT # (filename extension must be '${CNTEXT}'!). # -x|--extend <fullpath> Full path to existing (encrypted) container # file that you want to extend in size -# (filename needs to end in '${CNTEXT}'!). # Limitations: -# - container needs to be LUKS encrypted, and -# - internal filesystem needs to be ext{2,3,4}. +# - container needs to be LUKS encrypted. +# - filename extension needs to be '${CNTEXT}'. +# Supported filesystems inside container: +# - $(resizefs). +# -F|--filesystem <fs> Specify filesystem to create when formatting +# devices/containers. Defaults to '${DEF_FS}', +# Choices are $(createfs). # -L|--lcsize <size|perc> Size of LUKS encrypted /home ; value is the # requested size of the container in kB, MB, GB, # or as a percentage of free space @@ -170,7 +180,7 @@ EOT } # End of showhelp() # Show some common usage examples: -showexamples() { +function showexamples() { cat <<EOT # # Some common usage examples for $(basename $0) @@ -182,8 +192,8 @@ cat <<EOT # Create a 1GB encrypted persistence container: # ./$(basename $0) -p /run/media/<user>/Ventoy/myfiles/persistence.icc -P 1G # -# Create a 4GB encrypted home: -# ./$(basename $0) -l /run/media/<user>/Ventoy/somedir/lukscontainers.icc -L 4000M -i /run/media/<user>/Ventoy/slackware64-live-current.iso +# Create a 4GB encrypted home with btrfs filesystem: +# ./$(basename $0) -l /run/media/<user>/Ventoy/somedir/lukscontainers.icc -L 4000M -F btrfs -i /run/media/<user>/Ventoy/slackware64-live-current.iso # # Increase the size of that encrypted home container with another 2GB: # ./$(basename $0) -x /run/media/<user>/Ventoy/somedir/lukscontainers.icc -X 2G -i /run/media/<user>/Ventoy/slackware64-live-current.iso @@ -197,8 +207,110 @@ cat <<EOT EOT } # End of showexamples() +# Create a filesystem on a partition with optional label: +function createfs () { + MYDEV="${1}" + MYFS="${2:-'ext4'}" + MYLABEL="${3}" + + if [ -n "${MYLABEL}" ]; then + case "${MYFS}" in + fs2s) MYLABEL="-l ${MYLABEL}" ;; + *) MYLABEL="-L ${MYLABEL}" ;; + esac + fi + + if [ -z "${MYDEV}" ]; then + # Without arguments given, reply with list of supported fs'es: + echo "btrfs,ext2,ext4,f2fs,jfs,xfs" + return + fi + case "${MYFS}" in + btrfs) mkfs.btrfs -f -d single -m single ${MYLABEL} ${MYDEV} + ;; + ext2) mkfs.ext2 -F -F ${MYLABEL} ${MYDEV} + # Tune the ext2 filesystem: + tune2fs -m 0 -c 0 -i 0 ${MYDEV} + ;; + ext4) mkfs.ext4 -F -F ${MYLABEL} ${MYDEV} + # Tune the ext4 filesystem: + tune2fs -m 0 -c 0 -i 0 ${MYDEV} + ;; + f2fs) mkfs.f2fs ${MYLABEL} -f ${MYDEV} + ;; + jfs) mkfs.jfs -q ${MYDEV} + ;; + xfs) mkfs.xfs -f ${MYDEV} + ;; + *) echo "*** Unsupported filesystem '${MYFS}'!" + cleanup + exit 1 + ;; + esac +} # End of createfs() + +# Resize the filesystem on a block device: +function resizefs() { + local MYDEV="${1}" + + if [ -z "${MYDEV}" ]; then + # Without arguments given, reply with list of supported fs'es: + echo "btrfs,ext2,ext4,f2fs,jfs,xfs" + return + fi + + # Determine the current filesystem for the block device: + local MYFS=$(lsblk -n -o FSTYPE ${MYDEV}) + if [ -z "${MYFS}" ]; then + echo "*** Failed to resize filesystem on device '${MYDEV}'!" + echo "*** No filesystem found." + cleanup + exit 1 + fi + + local TMPMNT=$(mktemp -d -p /tmp -t alienres.XXXXXX) + if [ ! -d $TMPMNT ]; then + echo "*** Failed to create temporary mount for the filesystem resize!" + cleanup + exit 1 + else + chmod 711 ${TMPMNT} + fi + + # Mount the block device prior to the resize + # (btrfs, jfs and xfs do not support offline resize): + mount -o rw -t ${MYFS} ${MYDEV} ${TMPMNT} + + # Resize the filesystem to occupy the full new device capacity: + case "${MYFS}" in + btrfs) btrfs filesystem resize max ${TMPMNT} + ;; + ext*) resize2fs ${MYDEV} + ;; + f2fs) resize.f2fs ${MYDEV} + ;; + jfs) mount -o remount,resize,rw ${TMPMNT} + ;; + xfs) xfs_growfs -d ${TMPMNT} + ;; + *) echo "*** Unsupported filesystem '${MYFS}'!"; exit 1 + ;; + esac + + if [ ! $? ]; then + echo "*** Failed to resize '${MYFS}'filesystem on device '${MYDEV}'!" + cleanup + exit 1 + else + # Un-mount the device again: + sync + umount ${TMPMNT} + rmdir ${TMPMNT} + fi +} # End of checkfs() + # Uncompress the initrd based on the compression algorithm used: -uncompressfs () { +function uncompressfs () { if $(file "${1}" | grep -qi ": gzip"); then gzip -cd "${1}" elif $(file "${1}" | grep -qi ": XZ"); then @@ -206,19 +318,21 @@ uncompressfs () { fi } # End of uncompressfs() -# Read configuration data from the initrd inside the ISO: -read_initrd() { +# Read configuration data from the initrd inside the ISO, +# after it has been extracted into a directory: +function read_initrddir() { local IMGDIR="$1" + local INITVARS="$2" cd ${IMGDIR} # Read the values of liveslak template variables in the init script: - for TEMPLATEVAR in DISTRO LIVEMAIN MARKER MEDIALABEL ; do + for TEMPLATEVAR in ${INITVARS} ; do eval $(grep "^ *${TEMPLATEVAR}=" ./init |head -1) done -} # End read_initrd() +} # End of read_initrddir() # Extract the initrd: -extract_initrd() { +function extract_initrd() { local IMGFILE="$1" local IMGDIR=$(mktemp -d -p /tmp -t alienimg.XXXXXX) if [ ! -d $IMGDIR ]; then @@ -236,21 +350,21 @@ extract_initrd() { } # End of extract_initrd() # Determine size of a mounted partition (in MB): -get_part_mb_size() { +function get_part_mb_size() { local MYSIZE MYSIZE=$(df -P -BM ${1} |tail -n -1 |tr -s '\t' ' ' |cut -d' ' -f2) echo "${MYSIZE%M}" } # End of get_part_mb_size() # Determine free space of a mounted partition (in MB): -get_part_mb_free() { +function get_part_mb_free() { local MYSIZE MYSIZE=$(df -P -BM ${1} |tail -n -1 |tr -s '\t' ' ' |cut -d' ' -f4) echo "${MYSIZE%M}" } # End of get_part_mb_free() # Determine requested container size in MB (allow for '%|k|K|m|M|g|G' suffix): -cont_mb() { +function cont_mb() { local MYSIZE="$1" case "${MYSIZE: -1}" in "%") MYSIZE="$(( $PARTFREE * ${MYSIZE%\%} / 100 ))" ;; @@ -266,7 +380,7 @@ cont_mb() { } # End of cont_mb() # Expand existing encrypted container file: -expand_container() { +function expand_container() { local MYPART="$1" # disk partition local MYINC="$2" # requested increase ('%|k|K|m|M|g|G' suffix) local MYFILE="$3" # full path to ${CNTEXT} containerfile @@ -289,29 +403,47 @@ expand_container() { # Append random bytes to the end of the container file: dd if=/dev/urandom of=${MYFILE} bs=1M count=${MYINC} oflag=append conv=notrunc 2>/dev/null - # Unlock the LUKS encrypted container: - MYMAP=$(basename ${MYFILE} ${CNTEXT}) - echo "--- Unlocking the LUKS container requires your passphrase..." - until cryptsetup luksOpen ${MYFILE} ${MYMAP} ; do - echo ">>> Did you type an incorrect passphrases?" - read -p ">>> Press [ENTER] to try again or Ctrl-C to abort ..." REPLY - done + # Setup a loopback device that we can use with or without cryptsetup: + LODEV=$(losetup -f) + losetup $LODEV ${MYFILE} + + if cryptsetup isLuks ${LODEV} ; then + # Unlock LUKS encrypted container first: + MYMAP=$(basename ${MYFILE} ${CNTEXT}) + CNTDEV=/dev/mapper/${MYMAP} + echo "--- Unlocking the LUKS container requires your passphrase..." + until cryptsetup luksOpen ${LODEV} ${MYMAP} ; do + echo ">>> Did you type an incorrect passphrases?" + read -p ">>> Press [ENTER] to try again or Ctrl-C to abort ..." REPLY + done + else + # The loopmounted block device for the un-encrypted container: + CNTDEV=${LODEV} + fi # Run fsck so the filesystem is clean before we resize it: - fsck -fvy /dev/mapper/${MYMAP} + fsck -fvy ${CNTDEV} # Resize the filesystem to occupy the full new size: - resize2fs /dev/mapper/${MYMAP} + resizefs ${CNTDEV} # Just to be safe: - fsck -fvy /dev/mapper/${MYMAP} + fsck -fvy ${CNTDEV} + + # Don't forget to clean up after ourselves: + if cryptsetup isLuks ${LODEV} ; then + cryptsetup luksClose ${MYMAP} + fi + losetup -d ${LODEV} || true + } # End of expand_container() # Create container file in the empty space of the partition -create_container() { - CNTPART=$1 # partition containing the ISO - CNTSIZE=$2 # size of the container file to create - CNTFILE=$3 # ${CNTEXT} filename with full path - CNTENCR=$4 # 'none' or 'luks' - CNTUSED=$5 # 'persistence', '/home' or custom mountpoint +function create_container() { + local CNTPART=$1 # partition containing the ISO + local CNTSIZE=$2 # size of the container file to create + local CNTFILE=$3 # ${CNTEXT} filename with full path + local CNTENCR=$4 # 'none' or 'luks' + local CNTUSED=$5 # 'persistence', '/home' or custom mountpoint + local MYMAP # Create a container file or re-use previously created one: if [ -f ${CNTFILE} ]; then @@ -346,14 +478,21 @@ create_container() { exit 1 fi - echo "--- Creating ${CNTSIZE} MB container file using 'dd if=/dev/urandom', patience please..." + echo "--- Creating ${CNTSIZE} MB container file '$(basename =${CNTFILE})' using 'dd if=/dev/urandom', patience please..." mkdir -p $(dirname "${CNTFILE}") - # Create a sparse file (not allocating any space yet): - dd of=${CNTFILE} bs=1M count=0 seek=$CNTSIZE 2>/dev/null + if [ $? ]; then + # Create a sparse file (not allocating any space yet): + dd of=${CNTFILE} bs=1M count=0 seek=$CNTSIZE 2>/dev/null + else + echo "*** Failed to create directory for the container file!" + cleanup + exit 1 + fi # Setup a loopback device that we can use with cryptsetup: LODEV=$(losetup -f) losetup $LODEV ${CNTFILE} + MYMAP=$(basename ${CNTFILE} ${CNTEXT}) if [ "${CNTENCR}" = "luks" ]; then # Format the loop device with LUKS: echo "--- Encrypting the container file with LUKS; takes SOME time..." @@ -364,26 +503,24 @@ create_container() { done # Unlock the LUKS encrypted container: echo "--- Unlocking the LUKS container requires your passphrase again..." - until cryptsetup luksOpen $LODEV $(basename ${CNTFILE} ${CNTEXT}) ; do + until cryptsetup luksOpen $LODEV ${MYMAP} ; do echo ">>> Did you type an incorrect passphrases?" read -p ">>> Press [ENTER] to try again or Ctrl-C to abort ..." REPLY done - CNTDEV=/dev/mapper/$(basename ${CNTFILE} ${CNTEXT}) + CNTDEV=/dev/mapper/${MYMAP} # Now we allocate blocks for the LUKS device. We write encrypted zeroes, # so that the file looks randomly filled from the outside. # Take care not to write more bytes than the internal size of the container: - echo "--- Writing random data to encrypted container; takes LONG time..." + echo "--- Writing ${CNTSIZE} MB of random data to encrypted container; takes LONG time..." CNTIS=$(( $(lsblk -b -n -o SIZE $(readlink -f ${CNTDEV})) / 512)) - dd if=/dev/zero of=${CNTDEV} bs=512 count=${CNTIS} 2>/dev/null || true + dd if=/dev/zero of=${CNTDEV} bs=512 count=${CNTIS} status=progress || true else - CNTDEV=$LODEV # Un-encrypted container files remain sparse. + CNTDEV=$LODEV fi # Format the now available block device with a linux fs: - mkfs.ext4 ${CNTDEV} - # Tune the ext4 filesystem: - tune2fs -m 0 -c 0 -i 0 ${CNTDEV} + createfs ${CNTDEV} ${FSYS} if [ "${CNTUSED}" == "${DEFMNT}" ]; then # Copy the original /home content into the container. @@ -415,7 +552,7 @@ create_container() { } # End of create_container() -read_config() { +function read_isoconfig() { local MYISO="${1}" # Read ISO customization from the .cfg file if it exists: if [ -f "${MYISO%.iso}.cfg" ]; then @@ -429,9 +566,9 @@ read_config() { fi done fi -} # End of read_config() +} # End of read_isoconfig() -write_config() { +function write_isoconfig() { local MYISO="${1}" # Write updated customization into the ISO .cfg: echo "# Liveslak ISO configuration file for ${VERSION}" > ${MYISO%.iso}.cfg 2>/dev/null @@ -448,7 +585,7 @@ write_config() { fi done fi -} # End of write_config() +} # End of write_isoconfig() # # -- end of function definitions -- @@ -500,6 +637,10 @@ while [ ! -z "$1" ]; do EXTENSION="$(cd "$(dirname "$2")"; pwd)/$(basename "$2")" shift 2 ;; + -F|--filesystem) + FSYS="$2" + shift 2 + ;; -L|--lcsize) LUKSSIZE="$2" shift 2 @@ -528,6 +669,9 @@ if [ "$(id -u)" != "0" ]; then exit 1 fi +# Add required filesystem tools: +REQTOOLS="${REQTOOLS} mkfs.${FSYS}" + # Are all the required tools present? PROG_MISSING="" for PROGN in ${REQTOOLS} ; do @@ -670,10 +814,10 @@ fi # Collect data from the USB initrd: IMGDIR=$(extract_initrd ${ISOMNT}/boot/initrd.img) -read_initrd ${IMGDIR} +read_initrddir ${IMGDIR} "DISTRO LIVEMAIN MARKER MEDIALABEL" # Collect customization parameters for the ISO: -read_config ${SLISO} +read_isoconfig ${SLISO} # Determine where in LUKSVOL the /home is defined. # The LUKSVOL value looks like: @@ -763,7 +907,7 @@ if [ ${#CONTAINERS[@]} -gt 0 ]; then fi # Write customization parameters for the ISO to disk: -write_config ${SLISO} +write_isoconfig ${SLISO} # Write ISO version to the liveslak rootdir if that exists: if [ -d "${USBMNT}/${LIVESLAKROOT}" ]; then diff --git a/liveinit.tpl b/liveinit.tpl index 44d6c1d..0fc46e2 100644 --- a/liveinit.tpl +++ b/liveinit.tpl @@ -1463,7 +1463,7 @@ if [ "$RESCUE" = "" ]; then # Let Slackware mount the unlocked container: luksfs=$(blkid /dev/mapper/$luksnam |rev |cut -d'"' -f2 |rev) - if ! grep -q /dev/mapper/$luksnam /mnt/overlay/etc/fstab ; then + if ! grep -q "^/dev/mapper/$luksnam" /mnt/overlay/etc/fstab ; then echo "/dev/mapper/$luksnam $luksmnt $luksfs defaults 1 1" >> /mnt/overlay/etc/fstab fi # On shutdown, ensure that the container gets locked again: @@ -1807,7 +1807,7 @@ EOT done if [ $RUN_DEPMOD -eq 1 ]; then # This costs a few seconds in additional boot-up time unfortunately: - echo "${MARKER}: Additional kernel module(s) found... need a bit" + echo "${MARKER}: Additional kernel module(s) found... need a bit" chroot /mnt/overlay /sbin/depmod -a fi unset RUN_DEPMOD @@ -1859,8 +1859,8 @@ fi /sbin/udevadm control --exit unset ERR -umount /proc -umount /sys +umount /proc 2>/dev/null +umount /sys 2>/dev/null umount /run 2>/dev/null echo "${MARKER}: Slackware Live system is ready." diff --git a/make_slackware_live.sh b/make_slackware_live.sh index 52bfb16..d1e860e 100755 --- a/make_slackware_live.sh +++ b/make_slackware_live.sh @@ -255,7 +255,7 @@ TESTINGLIST_DAW="" # List of kernel modules required for a live medium to boot properly; # Lots of HID modules added to support keyboard input for LUKS password entry; # Virtio modules added to experiment with liveslak in a VM. -KMODS=${KMODS:-"squashfs:overlay:loop:efivarfs:xhci-pci:ohci-pci:ehci-pci:xhci-hcd:uhci-hcd:ehci-hcd:mmc-core:mmc-block:sdhci:sdhci-pci:sdhci-acpi:rtsx_pci:rtsx_pci_sdmmc:usb-storage:uas:hid:usbhid:i2c-hid:hid-generic:hid-apple:hid-cherry:hid-logitech:hid-logitech-dj:hid-logitech-hidpp:hid-lenovo:hid-microsoft:hid_multitouch:jbd:mbcache:ext3:ext4:isofs:fat:nls_cp437:nls_iso8859-1:msdos:vfat:exfat:ntfs:virtio_ring:virtio:virtio_blk:virtio_balloon:virtio_pci:virtio_pci_modern_dev:virtio_net"} +KMODS=${KMODS:-"squashfs:overlay:loop:efivarfs:xhci-pci:ohci-pci:ehci-pci:xhci-hcd:uhci-hcd:ehci-hcd:mmc-core:mmc-block:sdhci:sdhci-pci:sdhci-acpi:rtsx_pci:rtsx_pci_sdmmc:usb-storage:uas:hid:usbhid:i2c-hid:hid-generic:hid-apple:hid-cherry:hid-logitech:hid-logitech-dj:hid-logitech-hidpp:hid-lenovo:hid-microsoft:hid_multitouch:jbd:mbcache:ext3:ext4:zstd_compress:lz4hc_compress:lz4_compress:btrfs:f2fs:jfs:xfs:isofs:fat:nls_cp437:nls_iso8859-1:msdos:vfat:exfat:ntfs:virtio_ring:virtio:virtio_blk:virtio_balloon:virtio_pci:virtio_pci_modern_dev:virtio_net"} # Network kernel modules to include for NFS root support: NETMODS="kernel/drivers/net kernel/drivers/virtio" diff --git a/pkglists/min.lst b/pkglists/min.lst index a042446..745ef93 100644 --- a/pkglists/min.lst +++ b/pkglists/min.lst @@ -11,6 +11,7 @@ binutils # Add 'bind' at the end because it does a chroot: #bind bsd-games +btrfs-progs bzip2 ca-certificates coreutils @@ -34,6 +35,7 @@ elvis etc eudev exfatprogs +f2fs-tools file findutils floppy @@ -55,11 +57,13 @@ haveged hostname idnkit infozip +inih inotify-tools iproute2 iptables iputils jemalloc +jfsutils kbd kernel-generic kernel-firmware @@ -146,12 +150,14 @@ tar terminus-font traceroute usbutils +userspace-rcu utempter util-linux wget which whois zstd +xfsprogs xxHash xz # Bind needs to run at the end: @@ -62,15 +62,28 @@ UPKERNEL=0 # Do not change usb wait time by default: WAIT=-1 +# Not extending any container by default: +EXTENSION="" + # --------------------------------------------------------------------------- # END possible tasks to be executed by the script: # --------------------------------------------------------------------------- +# The extension for containerfiles accompanying an ISO is '.icc', +# whereas the persistent USB stick created with iso2usb.sh uses '.img'. +DEFEXT=".img" +CNTEXT="${DEFEXT}" + +# Default filesystem for devices/containers: +DEF_FS="ext4" +FSYS="${DEF_FS}" + # Determine whether the USB stick has a supported kernel configuration # i.e. one active and optionally one backup kernel plus mmodules: SUPPORTED=1 # Values obtained from the init script on the USB: +CORE2RAMMODS="" DEF_KBD="" DEF_LOCALE="" DEF_TZ="" @@ -81,7 +94,6 @@ LIVEUID="" MARKER="" MEDIALABEL="" PERSISTENCE="" -CORE2RAMMODS="" SQ_EXT_AVAIL="" VERSION="" @@ -89,9 +101,15 @@ VERSION="" KBACKUP=1 # Does the initrd contain an old kernel that we can restore? -# The 'read_initrd' routing may set this to '0': +# The 'read_initrddir' routine may set this to '0': KRESTORE=1 +# By default we create an addon live module for the new kernel modules, +# otherwise the Live OS will be broken after reboot. +# User can skip this if they already installed the kernel-modules package +# in the Live OS earlier: +NOLIVEMODS=0 + # Timeout when scanning for inserted USB device, 30 seconds by default, # but this default can be changed from outside the script: SCANWAIT=${SCANWAIT:-30} @@ -109,8 +127,6 @@ MINFREE=${MINFREE:-10} # Variables to store content from an initrd we are going to refresh: OLDKERNELSIZE="" -OLDKMODDIRSIZE="" -OLDKVER="" OLDWAIT="" # Record the version of the new kernel: @@ -121,10 +137,22 @@ IMGDIR="" KERDIR="" USBMNT="" EFIMNT="" +CNTDEV="" +LODEV="" + +# Empty initialization: +INCSIZE="" +PARTFREE="" +PARTSIZE="" # These tools are required by the script, we will check for their existence: REQTOOLS="cpio gdisk inotifywait lsblk strings xz" +# Minimim free space (in MB) we want to have left in any partition +# after we are done. +# The default value can be changed from the environment: +MINFREE=${MINFREE:-10} + # Compressor used on the initrd ("gzip" or "xz --check=crc32"); # Note that the kernel's XZ decompressor does not understand CRC64: COMPR="xz --check=crc32" @@ -133,7 +161,7 @@ COMPR="xz --check=crc32" # List of kernel modules required for a live medium to boot properly; # Lots of HID modules added to support keyboard input for LUKS password entry; # Virtio modules added to experiment with liveslak in a VM. -KMODS=${KMODS:-"squashfs:overlay:loop:efivarfs:xhci-pci:ohci-pci:ehci-pci:xhci-hcd:uhci-hcd:ehci-hcd:mmc-core:mmc-block:sdhci:sdhci-pci:sdhci-acpi:rtsx_pci:rtsx_pci_sdmmc:usb-storage:uas:hid:usbhid:i2c-hid:hid-generic:hid-apple:hid-cherry:hid-logitech:hid-logitech-dj:hid-logitech-hidpp:hid-lenovo:hid-microsoft:hid_multitouch:jbd:mbcache:ext3:ext4:isofs:fat:nls_cp437:nls_iso8859-1:msdos:vfat:exfat:ntfs:virtio_ring:virtio:virtio_blk:virtio_balloon:virtio_pci:virtio_pci_modern_dev:virtio_net"} +KMODS=${KMODS:-"squashfs:overlay:loop:efivarfs:xhci-pci:ohci-pci:ehci-pci:xhci-hcd:uhci-hcd:ehci-hcd:mmc-core:mmc-block:sdhci:sdhci-pci:sdhci-acpi:rtsx_pci:rtsx_pci_sdmmc:usb-storage:uas:hid:usbhid:i2c-hid:hid-generic:hid-apple:hid-cherry:hid-logitech:hid-logitech-dj:hid-logitech-hidpp:hid-lenovo:hid-microsoft:hid_multitouch:jbd:mbcache:ext3:ext4:zstd_compress:lz4hc_compress:lz4_compress:btrfs:f2fs:jfs:xfs:isofs:fat:nls_cp437:nls_iso8859-1:msdos:vfat:exfat:ntfs:virtio_ring:virtio:virtio_blk:virtio_balloon:virtio_pci:virtio_pci_modern_dev:virtio_net"} # Network kernel modules to include for NFS root support: NETMODS="kernel/drivers/net kernel/drivers/virtio" @@ -147,22 +175,32 @@ NETEXCL="appletalk arcnet bonding can dummy.ko hamradio hippi ifb.ko irda macvla # # Clean up in case of failure: -cleanup() { +function cleanup() { # Clean up by unmounting our loopmounts, deleting tempfiles: echo "--- Cleaning up the staging area..." # During cleanup, do not abort due to non-zero exit code: set +e sync + + if [ -n "$CNTDEV" ]; then + # In case of failure, only most recent LUKS mapped device is still open: + if mount | grep -q ${CNTDEV} ; then + umount -f ${CNTDEV} + cryptsetup luksClose $(basename ${CNTDEV}) + losetup -d ${LODEV} + fi + fi + # No longer needed: [ -n "${IMGDIR}" ] && ( rm -rf $IMGDIR ) [ -n "${KERDIR}" ] && ( rm -rf $KERDIR ) if [ -n "${USBMNT}" ]; then if mount |grep -qw ${USBMNT} ; then umount ${USBMNT} ; fi - rm -rf $USBMNT + rmdir $USBMNT fi if [ -n "${EFIMNT}" ]; then if mount |grep -qw ${EFIMNT} ; then umount ${EFIMNT} ; fi - rm -rf $EFIMNT + rmdir $EFIMNT fi set -e } # End of cleanup() @@ -170,7 +208,7 @@ cleanup() { trap 'echo "*** $0 FAILED at line $LINENO ***"; cleanup; exit 1' ERR INT TERM # Show the help text for this script: -showhelp() { +function showhelp() { cat <<EOT # # Purpose: to update the content of a Slackware Live USB stick. @@ -178,6 +216,7 @@ cat <<EOT # $(basename $0) accepts the following parameters: # -b|--nobackup Do not try to backup original kernel and modules. # -d|--devices List removable devices on this computer. +# -e|--examples Show some common usage examples. # -h|--help This help. # -i|--init <filename> Replacement init script. # -k|--kernel <filename> The kernel file (or package). @@ -190,27 +229,71 @@ cat <<EOT # providing a devicename (using option '-o'). # -v|--verbose Show verbose messages. # -w|--wait<number> Add <number> seconds wait time to initialize USB. -# -# Note that if you upgrade the kernel and modules using $(basename $0), -# this will apply only to the USB boot kernel and its initrd! -# Before running $(basename $0) to upgrade the kernel & modules, -# you first need to upgrade the kernel-generic and kernel-modules packages -# in the Live OS using installpkg/upgradepkg or slackpkg. +# -x|--extend <fullpath> Full path (either in your filesystem or else +# relative to the USB partition root) +# to an existing (encrypted) container file, +# whose size you want to extend. +# Limitations: +# - container needs to be LUKS encrypted. +# - filename extension needs to be '${CNTEXT}'. +# Supported filesystems inside container: +# - $(resizefs). +# -N|--nolivemods Don't create an addon live module containing +# the new kernelmodules. Normally you *will* need +# this addon module, *unless* you have already +# installed these kernel-modules in the Live OS. +# FYI: the kernel and module upgrade applies only +# to the USB boot kernel and its initrd. +# -X|--extendsize <size|perc> Extend size of existing container; value +# is the requested extension of the container +# in kB, MB, GB, or as percentage of free space +# (integer numbers only). +# Examples: '-X 125M', '-X 2G', '-X 20%'. # EOT } # End of showhelp() +function showexamples() { +cat <<EOT +# +# Some common usage examples for $(basename $0) +# --------------------------------------------------------------------------- +# +# Get a listing of all available removable devices on the computer: +# ./$(basename $0) -d +# +# Updating kernel and modules, providing two packages as input and assuming +# that the USB stick is known as /dev/sdX: +# ./$(basename $0) -o /dev/sdX -m kernel-modules-4.19.0-x86_64-1.txz -k kernel-generic-4.19.0-x86_64-1.txz +# +# Restore the previous kernel and modules after a failed update, +# and let the script scan your computer for the insertion of your USB stick: +# ./$(basename $0) -s -r +# +# Replace the Live init script with the latest template taken from +# the liveslak git repository: +# wget https://git.liveslak.org/liveslak/plain/liveinit.tpl +# ./$(basename $0) -o /dev/sdX -i liveinit.tpl +# +# Extend the size of the pre-existing LUKS container for your homedirectory +# with 3 GB, and let the script scan for the insertion of your USB stick: +# ./$(basename $0) -s -x /slhome.img -X 3G +# +EOT +} # End of showexamples() + # Scan for insertion of a USB device: -scan_devices() { +function scan_devices() { + local MYSCANWAIT="${1}" local BD # Inotifywatch does not trigger on symlink creation, # so we can not watch /sys/block/ - BD=$(inotifywait -q -t ${SCANWAIT} -e create /dev 2>/dev/null |cut -d' ' -f3) + BD=$(inotifywait -q -t ${MYSCANWAIT} -e create /dev 2>/dev/null |cut -d' ' -f3) echo ${BD} } # End of scan_devices() # Show a list of removable devices detected on this computer: -show_devices() { +function show_devices() { local MYDATA="${*}" if [ -z "${MYDATA}" ]; then MYDATA="$(ls --indicator-style=none /sys/block/ |grep -Ev '(ram|loop|dm-)')" @@ -224,8 +307,110 @@ show_devices() { echo "#" } # End of show_devices() +# Determine size of a mounted partition (in MB): +function get_part_mb_size() { + local MYPART="${1}" + local MYSIZE + MYSIZE=$(df -P -BM ${MYPART} |tail -n -1 |tr -s '\t' ' ' |cut -d' ' -f2) + echo "${MYSIZE%M}" +} # End of get_part_mb_size() + +# Determine free space of a mounted partition (in MB): +function get_part_mb_free() { + local MYPART="${1}" + local MYSIZE + MYSIZE=$(df -P -BM ${MYPART} |tail -n -1 |tr -s '\t' ' ' |cut -d' ' -f4) + echo "${MYSIZE%M}" +} # End of get_part_mb_free() + +# Determine requested container size in MB (allow for '%|k|K|m|M|g|G' suffix). +# Note: sizes need to be integer values! Bash arithmetics don't work for floats. +function cont_mb() { + # Uses global variables: PARTFREE + local MYSIZE="$1" + case "${MYSIZE: -1}" in + "%") MYSIZE="$(( $PARTFREE * ${MYSIZE%\%} / 100 ))" ;; + "k") MYSIZE="$(( ${MYSIZE%k} / 1024 ))" ;; + "K") MYSIZE="$(( ${MYSIZE%K} / 1024 ))" ;; + "m") MYSIZE="${MYSIZE%m}" ;; + "M") MYSIZE="${MYSIZE%M}" ;; + "g") MYSIZE="$(( ${MYSIZE%g} * 1024 ))" ;; + "G") MYSIZE="$(( ${MYSIZE%G} * 1024 ))" ;; + *) MYSIZE=-1 ;; + esac + echo "$MYSIZE" +} # End of cont_mb() + +# Expand existing encrypted container file: +function expand_container() { + # Uses external function: cleanup + # Uses global variables: CNTEXT, MINFREE + # Sets global variables: CNTDEV, LODEV, PARTFREE, PARTSIZE + local MYPART="$1" # disk partition + local MYINC="$2" # requested increase ('%|k|K|m|M|g|G' suffix) + local MYFILE="$3" # full path to ${CNTEXT} containerfile + local MYMAP="" # Name of the device-mapped file + local CNTIS="" # Stores size of the container + + # Determine requested container increase in MB: + MYINC=$(cont_mb ${MYINC}) + + # Determine size of the target partition (in MB), and the free space: + PARTSIZE=$(get_part_mb_size ${MYPART}) + PARTFREE=$(get_part_mb_free ${MYPART}) + + if [ $PARTFREE -lt $(( ${MYINC} + ${MINFREE} )) ]; then + echo "*** Free space on USB partition after file-resizing would be less than ${MINFREE} MB;" + echo "*** Not resizing the container file!" + cleanup + exit 1 + fi + + if ! file ${MYFILE} |grep -q 'LUKS' ; then + echo "*** No LUKS container: '${MYFILE}'" + cleanup + exit 1 + else + echo "--- Expanding '$(basename ${MYFILE})' on '${MYPART}' with ${MYINC} MB..." + fi + + # Append random bytes to the end of the container file: + dd if=/dev/urandom of=${MYFILE} bs=1M count=${MYINC} oflag=append conv=notrunc 2>/dev/null + + # Setup a loopback device that we can use with or without cryptsetup: + LODEV=$(losetup -f) + losetup ${LODEV} ${MYFILE} + + if cryptsetup isLuks ${LODEV} ; then + # Unlock LUKS encrypted container first: + MYMAP=$(basename ${MYFILE} ${CNTEXT}) + CNTDEV=/dev/mapper/${MYMAP} + echo "--- Unlocking the LUKS container requires your passphrase..." + until cryptsetup luksOpen ${LODEV} ${MYMAP} ; do + echo ">>> Did you type an incorrect passphrases?" + read -p ">>> Press [ENTER] to try again or Ctrl-C to abort ..." REPLY + done + else + # Use the loopmounted block device for the un-encrypted container: + CNTDEV=${LODEV} + fi + + # Run fsck so the filesystem is clean before we resize it: + fsck -fvy ${CNTDEV} + # Resize the filesystem to occupy the full new size: + resizefs ${CNTDEV} + # Just to be safe: + fsck -fvy ${CNTDEV} + + # Don't forget to clean up after ourselves: + if cryptsetup isLuks ${LODEV} ; then + cryptsetup luksClose ${MYMAP} + fi + losetup -d ${LODEV} || true +} # End of expand_container() + # Uncompress the initrd based on the compression algorithm used: -uncompressfs () { +function uncompressfs () { if $(file "${1}" | grep -qi ": gzip"); then gzip -cd "${1}" elif $(file "${1}" | grep -qi ": XZ"); then @@ -233,10 +418,76 @@ uncompressfs () { fi } # End of uncompressfs () +# Resize the filesystem on a block device: +function resizefs() { + # Uses external function: cleanup + local MYDEV="${1}" + local MYFS + local TMPMNT + + if [ -z "${MYDEV}" ]; then + # Without arguments given, reply with list of supported fs'es: + echo "btrfs,ext2,ext4,f2fs,jfs,xfs" + return + fi + + # Determine the current filesystem for the block device: + MYFS=$(lsblk -n -o FSTYPE ${MYDEV}) + if [ -z "${MYFS}" ]; then + echo "*** Failed to resize filesystem on device '${MYDEV}'!" + echo "*** No filesystem found." + cleanup + exit 1 + fi + + TMPMNT=$(mktemp -d -p ${TMP:=/tmp} -t alienres.XXXXXX) + if [ ! -d $TMPMNT ]; then + echo "*** Failed to create temporary mount for the filesystem resize!" + cleanup + exit 1 + else + chmod 711 ${TMPMNT} + fi + + # Mount the block device prior to the resize + # (btrfs, jfs and xfs do not support offline resize): + mount -o rw -t ${MYFS} ${MYDEV} ${TMPMNT} + + # Resize the filesystem to occupy the full new device capacity: + case "${MYFS}" in + btrfs) btrfs filesystem resize max ${TMPMNT} + ;; + ext*) resize2fs ${MYDEV} + ;; + f2fs) resize.f2fs ${MYDEV} + ;; + jfs) mount -o remount,resize,rw ${TMPMNT} + ;; + xfs) xfs_growfs -d ${TMPMNT} + ;; + *) echo "*** Unsupported filesystem '${MYFS}'!" + cleanup + exit 1 + ;; + esac + + if [ ! $? ]; then + echo "*** Failed to resize '${MYFS}'filesystem on device '${MYDEV}'!" + cleanup + exit 1 + else + # Un-mount the device again: + sync + umount ${TMPMNT} + rmdir ${TMPMNT} + fi +} # End of resizefs() + + # Collect the kernel modules we need for the liveslak initrd. # When calling this function, the old module tree must already # have been renamed to ${OLDKVER}.prev -collect_kmods() { +function collect_kmods() { local IMGDIR="$1" # Borrow (and mangle) code from Slackware's mkinitrd @@ -369,9 +620,15 @@ collect_kmods() { fi } # End of collect_kmods () -# Read configuration data from old initrd: -read_initrd() { +# Read configuration data from old initrd, +# after it has been extracted into a directory: +function read_initrddir() { local IMGDIR="$1" + local INITVARS="$2" + local OLDKVER + local OLDMODDIR + local OLDKMODDIRSIZE + local PREVMODDIR cd ${IMGDIR} @@ -379,7 +636,7 @@ read_initrd() { OLDWAIT=$(cat ./wait-for-root) # Read the values of liveslak template variables in the init script: - for TEMPLATEVAR in DEF_KBD DEF_LOCALE DEF_TZ DISTRO LIVE_HOSTNAME LIVEMAIN LIVEUID MARKER MEDIALABEL PERSISTENCE CORE2RAMMODS SQ_EXT_AVAIL VERSION ; do + for TEMPLATEVAR in ${INITVARS} ; do eval $(grep "^ *${TEMPLATEVAR}=" ./init |head -1) done @@ -411,23 +668,36 @@ read_initrd() { fi fi fi -} # End read_initrd() - -# Extract the initrd: -extract_initrd() { - local IMGFILE="$1" +} # End read_initrddir() + +# Extract the initrd into a new directory and report the dirname back: +function extract_initrd() { + local MYIMGFILE="$1" + local MYIMGDIR=$(mktemp -d -p ${TMP:=/tmp} -t alienimg.XXXXXX) + if [ ! -d $MYIMGDIR ]; then + echo "*** Failed to create temporary extraction directory for the initrd!" + cleanup + exit 1 + else + chmod 711 $MYIMGDIR + fi - cd ${IMGDIR} - uncompressfs ${IMGFILE} \ - | cpio -i -d -m -H newc + cd ${MYIMGDIR} + uncompressfs ${MYIMGFILE} 2>/dev/null \ + | cpio -i -d -m -H newc 2>/dev/null + echo "$MYIMGDIR" } # End of extract_initrd() # Modify the extracted initrd and re-pack it: -update_initrd() { - local IMGFILE="$1" +function update_initrd() { + local MYIMGFILE="$1" + local MYIMGDIR="$2" local NEED_RECOMP=0 + local NEWMODDIR + local OLDMODDIR + local OLDKVER - cd ${IMGDIR} + cd ${MYIMGDIR} if [ ${WAIT} -ge 0 ]; then if [ $WAIT != $OLDWAIT ]; then echo "--- Updating 'waitforroot' time from '$OLDWAIT' to '$WAIT'" @@ -438,24 +708,23 @@ update_initrd() { if [ $UPKERNEL -eq 1 ]; then OLDMODDIR=$(find ./lib/modules -type d -mindepth 1 -maxdepth 1 |grep -v .prev) + OLDKVER=$(strings $(find ${OLDMODDIR}/kernel/ -name "*.ko*" |head -1) |grep ^vermagic |cut -d= -f2 |cut -d' ' -f1) rm -rf ./lib/modules/*.prev if [ $KBACKUP -eq 1 ]; then # We make one backup: - if [ $VERBOSE -eq 1 ]; then - echo "--- Making backup of kernel modules" - fi + echo "--- Making backup of kernel modules (${OLDKVER}) in initrd" mv -i ${OLDMODDIR} ${OLDMODDIR}.prev else - echo "--- No room for backing up old kernel modules" + echo "--- No room for backing up old kernel modules in initrd" rm -rf ${OLDMODDIR} fi # Add modules for the new kernel: - echo "--- Adding new kernel modules" - collect_kmods ${IMGDIR} + echo "--- Adding new kernel modules (${KVER}) to initrd" + collect_kmods ${MYIMGDIR} NEED_RECOMP=1 elif [ $RESTORE -eq 1 -a $KRESTORE -eq 1 ]; then # Restore previous kernel module tree. - # The 'read_initrd' routine will already have checked that we have + # The 'read_initrddir' routine will already have checked that we have # one active and one .prev modules tree: OLDMODDIR=$(find ./lib/modules -type d -mindepth 1 -maxdepth 1 |grep .prev || true) NEWMODDIR=$(find ./lib/modules -type d -mindepth 1 -maxdepth 1 |grep -v .prev) @@ -466,6 +735,11 @@ update_initrd() { fi if [ -n "${LIVEINIT}" ]; then + if ! file "${LIVEINIT}" |grep -q 'shell script' ; then + echo "*** Not a shell script: "${LIVEINIT}"!" + cleanup + exit 1 + fi echo "--- Replacing live init script" cp ./init ./init.prev if grep -q "@LIVEMAIN@" ${LIVEINIT} ; then @@ -480,19 +754,21 @@ update_initrd() { if [ ${NEED_RECOMP} -eq 1 ]; then echo "--- Compressing the initrd image again" - chmod 0755 ${IMGDIR} - find . |cpio -o -H newc |$COMPR > ${IMGFILE} + chmod 0755 ${MYIMGDIR} + find . |cpio -o -H newc |$COMPR > ${MYIMGFILE} fi - cd - 1>/dev/null # End of 'cd ${IMGDIR}' + cd - 1>/dev/null # End of 'cd ${MYIMGDIR}' } # End of update_initrd() # Accept either a kernelimage or a packagename, # and return the path to a kernelimage: -getpath_kernelimg () { +function getpath_kernelimg () { local MYDATA="${*}" - [ -z "${MYDATA}" ] && echo "" - if [ -n "$(file "${MYDATA}" |grep -E 'x86 boot (executable|sector)')" ]; then + if [ -z "${MYDATA}" ]; then + echo "" + return + elif [ -n "$(file "${MYDATA}" |grep -E 'x86 boot (executable|sector)')" ]; then # We have a kernel image: echo "${MYDATA}" else @@ -505,38 +781,48 @@ getpath_kernelimg () { # Accept either a directory containing module tree, or a packagename, # and return the path to a module tree: -getpath_kernelmods () { +function getpath_kernelmods () { local MYDATA="${*}" - [ -z "${MYDATA}" ] && echo "" + local MYKVER - if [ -d "${MYDATA}" ]; then + if [ -z "${MYDATA}" ]; then + echo "" + return + elif [ -d "${MYDATA}" ]; then # We have directory, assume it contains the kernel modules: - echo "${MYDATA}" + MYKVER=$(strings $(find ${MYDATA}/kernel/ -name "*.ko*" |head -1) |grep ^vermagic |cut -d= -f2 |cut -d' ' -f1) + if [ -z "${MYKVER}" ]; then + echo "*** Could not determine new kernel version from module directory!" + cleanup + exit 1 + fi + mkdir -p ${KERDIR}/lib/modules/${MYKVER} + rsync -a ${MYDATA}/ ${KERDIR}/lib/modules/${MYKVER}/ else # We assume a Slackware package: # Extract the kernel modules from the package and return the path: tar -C ${KERDIR} -xf ${MYDATA} lib/modules - cd ${KERDIR}/lib/modules/* - pwd fi + cd ${KERDIR}/lib/modules/* + pwd } # End of getpath_kernelmods # Determine size of a mounted partition (in MB): -get_part_mb_size() { +function get_part_mb_size() { local MYSIZE MYSIZE=$(df -P -BM ${1} |tail -1 |tr -s '\t' ' ' |cut -d' ' -f2) echo "${MYSIZE%M}" } # End of get_part_mb_size # Determine free space of a mounted partition (in MB): -get_part_mb_free() { +function get_part_mb_free() { local MYSIZE MYSIZE=$(df -P -BM ${1} |tail -1 |tr -s '\t' ' ' |cut -d' ' -f4) echo "${MYSIZE%M}" } # End of get_part_mb_free -parse_template() { - # Parse a liveslak template file and substitute the placeholders. +# Parse a liveslak template file and substitute the placeholders. +function parse_template() { local INFILE="$1" local OUTFILE="$2" @@ -587,6 +873,10 @@ while [ ! -z "$1" ]; do show_devices exit ;; + -e|--examples) + showexamples + exit + ;; -h|--help) showhelp exit @@ -631,6 +921,18 @@ while [ ! -z "$1" ]; do WAIT="$2" shift 2 ;; + -N|--nolivemods) + NOLIVEMODS=1 + shift + ;; + -x|--extend) + EXTENSION="$2" + shift 2 + ;; + -X|--extendsize) + INCSIZE="$2" + shift 2 + ;; *) echo "*** Unknown parameter '$1'!" exit 1 @@ -651,8 +953,8 @@ fi # Either provide a block device, or else scan for a block device: if [ -z "$TARGET" ]; then if [ $SCAN -eq 1 ]; then - echo "-- Waiting ${SCANWAIT} seconds for a USB stick to be inserted..." - TARGET=$(scan_devices) + echo "--- Waiting ${SCANWAIT} seconds for a USB stick to be inserted..." + TARGET=$(scan_devices ${SCANWAIT}) if [ -z "$TARGET" ]; then echo "*** No new USB device detected during $SCANWAIT seconds scan." exit 1 @@ -661,6 +963,7 @@ if [ -z "$TARGET" ]; then fi else echo "*** You must specify the Live USB devicename (option '-o')!" + echo "*** Alternatively, let the script scan for insertion (option '-s')!" exit 1 fi elif [ $SCAN -eq 1 ]; then @@ -712,7 +1015,6 @@ else KVER=$(strings $(find ${KMODDIR}/kernel/ -name "*.ko*" |head -1) |grep ^vermagic |cut -d= -f2 |cut -d' ' -f1) if [ -z "${KVER}" ]; then echo "*** Could not determine kernel version from the module directory" - echo "*** (querying module kernel/fs/overlayfs/overlay.ko)!" cleanup exit 1 fi @@ -726,7 +1028,16 @@ if [ -n "${LIVEINIT}" -a ! -f "${LIVEINIT}" ]; then exit 1 fi -if [ $CHANGES2SXZ -eq 1 ]; then +if [ -n "${EXTENSION}" ]; then + if [ -z "${INCSIZE}" ]; then + echo "*** LUKS container '${EXTENSION}' defined but no extension size provided!" + echo "*** Not extending encrypted ${EXTENSION}, please use '-X' parameter." + cleanup + exit 1 + fi +fi + +if [ $CHANGES2SXZ -eq 1 ] || [ $UPKERNEL -eq 1 ]; then # We need to create a module, so add squashfs to the required tools: REQTOOLS="${REQTOOLS} mksquashfs" fi @@ -739,9 +1050,9 @@ for PROGN in ${REQTOOLS} ; do fi done if [ ! -z "$PROG_MISSING" ] ; then - echo "-- Required program(s) not found in search path '$PATH'!" + echo "--- Required program(s) not found in search path '$PATH'!" echo -e ${PROG_MISSING} - echo "-- Exiting." + echo "--- Exiting." cleanup exit 1 fi @@ -766,7 +1077,7 @@ echo q |gdisk -l $TARGET 2>/dev/null | \ # If the user just used the scan option (-s) and did not select a task, # we will exit the script gracefully now: -if [[ $WAIT -lt 0 && $UPKERNEL -ne 1 && $RESTORE -ne 1 && $NETSUPPORT -ne 1 && $LIVEINIT = "" && $CHANGES2SXZ -ne 1 ]]; then +if [[ $WAIT -lt 0 && $UPKERNEL -ne 1 && $RESTORE -ne 1 && $NETSUPPORT -ne 1 && $LIVEINIT = "" && $CHANGES2SXZ -ne 1 && $EXTENSION = "" ]]; then cleanup exit 0 else @@ -786,15 +1097,19 @@ TARGETP1=$(fdisk -l $TARGET |grep ^$TARGET |cut -d' ' -f1 |grep -E '[^0-9]1$') TARGETP2=$(fdisk -l $TARGET |grep ^$TARGET |cut -d' ' -f1 |grep -E '[^0-9]2$') TARGETP3=$(fdisk -l $TARGET |grep ^$TARGET |cut -d' ' -f1 |grep -E '[^0-9]3$') -# Create a temporary extraction directory for the initrd: -mkdir -p /mnt -IMGDIR=$(mktemp -d -p /mnt -t alienimg.XXXXXX) -if [ ! -d $IMGDIR ]; then - echo "*** Failed to create temporary extraction directory for the initrd!" - cleanup - exit 1 +# Normalize filepath: +if [ -f "${EXTENSION}" ]; then + # Container is an actual file, so where are we mounted? + EXTPART=$(cd "$(dirname "${EXTENSION}")" ; df --output=source . |tail -1) + EXTMNT=$(cd "$(dirname "${EXTENSION}")" ; df --output=target . |tail -1) + if [ "${EXTPART}" == "${TARGETP3}" ]; then + # User already mounted the USB linux partition; remove mountpoint: + EXTENSION="${EXTENSION#$EXTMNT}" + fi +elif [ "$(dirname ${EXTENSION})" == "." ]; then + # Containerfile was provided without leading slash, add one: + EXTENSION="/${EXTENSION}" fi -chmod 711 $IMGDIR # Create temporary mount point for the USB device: mkdir -p /mnt @@ -832,13 +1147,27 @@ EFIPFREE=$(get_part_mb_free ${EFIMNT}) # Record the Slackware Live version: OLDVERSION="$(cat ${USBMNT}/.isoversion 2>/dev/null)" -echo "-- The medium '${TARGET}' contains '${OLDVERSION}'" +echo "--- The medium '${TARGET}' contains '${OLDVERSION}'" + +# Try a write to the partition: +if touch ${USBMNT}/.rwtest 2>/dev/null && rm ${USBMNT}/.rwtest 2>/dev/null +then + echo "--- The partition '${TARGETP3}' is writable." +else + echo "--- Trying to remount readonly partition '${TARGETP3}' as writable..." + mount -o remount,rw ${USBMNT} + if [ $? -ne 0 ]; then + echo "*** Failed to remount '${TARGETP3}' writable, unable to continue!" + cleanup + exit 1 + fi +fi # Find out if the USB contains an EFI bootloader and use it: if [ ! -f ${EFIMNT}/EFI/BOOT/boot*.efi ]; then EFIBOOT=0 - echo "-- Note: UEFI boot file 'bootx64.efi' or 'bootia32.efi' not found on ISO." - echo "-- UEFI boot will not be supported" + echo "--- Note: UEFI boot file 'bootx64.efi' or 'bootia32.efi' not found on ISO." + echo "--- UEFI boot will not be supported" else EFIBOOT=1 fi @@ -853,10 +1182,10 @@ fi OLDKERNELSIZE=$(du -sm "${KIMG}" |tr '\t' ' ' |cut -d' ' -f1) # Collect data from the USB initrd: -extract_initrd ${USBMNT}/boot/initrd.img -read_initrd ${IMGDIR} +IMGDIR="$( extract_initrd ${USBMNT}/boot/initrd.img )" +read_initrddir ${IMGDIR} "DEF_KBD DEF_LOCALE DEF_TZ DISTRO LIVE_HOSTNAME LIVEMAIN LIVEUID MARKER MEDIALABEL PERSISTENCE CORE2RAMMODS SQ_EXT_AVAIL VERSION" -# The read_initrd routine will set SUPPORTED to '0' +# The read_initrddir routine will set SUPPORTED to '0' # if it finds a non-standard configuration for kernel & modules: if [ $KBACKUP -eq 1 ]; then if [ $SUPPORTED -ne 1 ]; then @@ -883,14 +1212,27 @@ if [ $KBACKUP -eq 1 ]; then fi # Update the initrd with regard to USB wait time, liveinit, kernel: -update_initrd ${USBMNT}/boot/initrd.img +update_initrd ${USBMNT}/boot/initrd.img ${IMGDIR} + +# Add the new modules as a squashfs module: +if [ $UPKERNEL -eq 1 ] && [ $NOLIVEMODS -eq 0 ]; then + LIVE_MOD_SYS=$(dirname $(find ${USBMNT} -name "0099-${DISTRO}_zzzconf*.sxz" |head -1)) + LIVE_MOD_ADD=$(dirname ${LIVE_MOD_SYS})/addons + MODNAME="0100-${DISTRO}_kernelmodules_${KVER}.sxz" + echo "--- Creating kernelmodules addon live module '${MODNAME}'" + rm -f ${LIVE_MOD_ADD}/${MODNAME} + mksquashfs ${KERDIR} ${LIVE_MOD_ADD}/${MODNAME} -e boot -noappend -comp xz -b 1M + unset LIVE_MOD_SYS LIVE_MOD_ADD MODNAME +fi # Take care of the kernel in the Linux partition: if [ $UPKERNEL -eq 1 ]; then if [ $KBACKUP -eq 1 ]; then # We always make one backup with the suffix ".prev": if [ $VERBOSE -eq 1 ]; then - echo "-- Backing up ${KIMG} to ${USBMNT}/boot/$(basename \"${KIMG}\").prev" + echo "--- Backing up ${KIMG} to ${USBMNT}/boot/$(basename \"${KIMG}\").prev" + else + echo "--- Backing up old kernel" fi mv "${KIMG}" ${USBMNT}/boot/$(basename "${KIMG}").prev else @@ -898,15 +1240,21 @@ if [ $UPKERNEL -eq 1 ]; then fi # And we name our new kernel exactly as the old one: if [ $VERBOSE -eq 1 ]; then - echo "-- Copying \"${KERNEL}\" to ${USBMNT}/boot/$(basename \"${KIMG}\")" + echo "--- Copying \"${KERNEL}\" to ${USBMNT}/boot/$(basename \"${KIMG}\")" + else + echo "--- Adding new kernel" fi cp "${KERNEL}" ${USBMNT}/boot/$(basename "${KIMG}") elif [ $RESTORE -eq 1 -a $KRESTORE -eq 1 ]; then if [ $VERBOSE -eq 1 ]; then - echo "-- Restoring ${USBMNT}/boot/$(basename \"${KIMG}\").prev to ${KIMG}" + echo "--- Restoring ${USBMNT}/boot/$(basename \"${KIMG}\").prev to ${KIMG}" + else + echo "--- Restoring old kernel" fi + OLDKVER=$(file "${KIMG}" |sed 's/^.*\(version [^ ]* \).*$/\1/' |cut -d' ' -f2) rm -f "${KIMG}" mv ${USBMNT}/boot/$(basename "${KIMG}").prev "${KIMG}" + echo "--- You may remove obsolete 'addons/0100-${DISTRO}_kernelmodules_${OLDKVER}.sxz' module" fi if [ $EFIBOOT -eq 1 ]; then @@ -936,12 +1284,24 @@ if [ $CHANGES2SXZ -eq 1 ]; then LIVE_MOD_SYS=$(dirname $(find ${USBMNT} -name "0099-${DISTRO}_zzzconf*.sxz" |head -1)) LIVE_MOD_ADD=$(dirname ${LIVE_MOD_SYS})/addons MODNAME="0100-${DISTRO}_customchanges-$(date +%Y%m%d%H%M%S).sxz" - echo "-- Moving current persistence data into addons module '${MODNAME}'" + echo "--- Moving current persistence data into addons module '${MODNAME}'" mksquashfs /mnt/live/changes ${LIVE_MOD_ADD}/${MODNAME} -noappend -comp xz -b 1M -e .wipe fi fi fi +# Should we extend the size of a container? +if [ -n "${EXTENSION}" ]; then + if [ "$(basename ${EXTENSION} ${CNTEXT})" == "$(basename ${EXTENSION})" ]; + then + echo "*** File '${EXTENSION}' does not have an '${CNTEXT}' extension!" + cleanup + exit 1 + fi + # Expand existing container file: + expand_container ${TARGETP3} ${INCSIZE} ${USBMNT}/${EXTENSION} +fi + # Unmount/remove stuff: cleanup |