From 59d33e1c570bb222616d043e6c9c6bd81e3c9bf7 Mon Sep 17 00:00:00 2001 From: a-vogel Date: Wed, 10 Apr 2024 23:58:31 +0200 Subject: [PATCH] LaunchDaemon, rm old items, PATH, more changes --- README.md | 37 ++--- .../LaunchDaemons/de.wycomco.misty.plist | 25 ++++ .../Shared/Mist/skel/localized_startos.txt | 4 +- .../Users/Shared/Mist/skel/override.txt | 0 .../Users/Shared/Mist/skel/pre_inserts_x86_64 | 4 +- misty/payload/usr/local/wycomco/misty | 127 +++++++++++++----- misty/scripts/postinstall | 37 +++-- 7 files changed, 168 insertions(+), 66 deletions(-) create mode 100644 misty/payload/Library/LaunchDaemons/de.wycomco.misty.plist mode change 100755 => 100644 misty/payload/Users/Shared/Mist/skel/override.txt diff --git a/README.md b/README.md index 6286d9c..ce99991 100644 --- a/README.md +++ b/README.md @@ -5,35 +5,35 @@ This script checks for availability of new releases of macOS, starting from macO ## Goals of this script If a new update for any major version is found, it will import into munki repo, creating the following installers: -- Apple Silicon: `stage_os_installer` and preloader for the installer -- Intel: `startosinstall` with precache key set to true +- Apple Silicon: *stage_os_installer* and preloader for the installer +- Intel: *startosinstall* with precache key set to true - Both architectures: Place installer in /Applications directory -The item’s `name` keys will not include a major version. Scoping is done using `board_id` and `device_id` in the plists as `installable_condition`. By using this logic, only the latest available macOS upgrade will be offered to the individual client. +The item’s *name* keys will not include a major version. Scoping is done using *board_id* and *device_id* in the plists as *installable_condition*. By using this logic, only the latest available macOS upgrade will be offered to the individual client. -`misty` is not intended for updating clients (minor updates within a major macOS version). There are other solutions available for that task. +*misty* is not intended for updating clients (minor updates within a major macOS version). There are other solutions available for that task. ## Running the script -- The script requires root privileges. -- The script does not have any options. You need to specify them in the override files. -- Currently, after installation a restart is required. After that, you can call it using `misty` or via full path `usr/local/wycomco/misty`. +The script does not have any command line options. You specify them in the override files. ### First run: customization -The script will create a `usr/` subfolder in `/Users/Shared/Mist/`, which is already present by installing mist-cli. An override file will be created that you should customize to suite your needs. The script will also ask if you require localizations. If you do, localization templates for the relevant plist files will be copied to the `usr/` subfolder that you should also adjust to your language(s). +Please run the script using `sudo misty`. The script will then create a `usr/` subfolder in `/Users/Shared/Mist/`, that is already present after installing mist-cli. An override file will be created that you should customize to suite your needs. The script will also ask if you require localizations. If you do, localization templates for the relevant plist files will be copied to the `usr/` subfolder that you should also adjust to your language(s). -You can also place a script called `postinstall.sh` into your usr folder that will get executed after each run. Make sure to make it executable. +You can also place a script called `postinstall.sh` into your usr folder that will be executed after each run. Make sure to make it executable. + +> **Caution**: During the first run, the LaunchDaemon (see below) will be enabled if not running yet. Please make sure you customize your settings before 10:00 pm local time after the first run. ### Subsequent runs -After you have edited the files in the `usr/` subdirectory, you can start another run of `misty`. It will then download the current full installers and create the respective plist files. +During installation, a LaunchDaemon will be created that runs the script at 22:00 every night. Please change the file `/Library/LaunchDaemons/de.wycomco.misty.plist` if required. Beware that the LaunchDaemon will be reset after each update. Except for the first run you do not have to run the script manually, but of course you can. -The next run of `misty` compares the full versions of each major version in the `Logs/` subdirectory. If no new updates are available, the script will terminate without downloading or packaging any item. You may want to create a LaunchAgent that will run `misty` every day. It will automatically create new munki items each time a new full installer is available. +Each run of *misty* compares the full versions of each major version in the `Logs/` subdirectory. If no new updates are available, the script will terminate without downloading or packaging any item. ## Folder structure -As mentioned above, `mist-cli` is required for running this script, since the search for and download of full installers is done using this tool. `mist-cli` creates a folder `Mist` inside `/Users/Shared`. We are using this folder. Inside that folder, we have several subfolders: +As mentioned above, *mist-cli* is required for running this script, since the search for and download of full installers is done using this tool. *mist-cli* creates a folder `Mist` inside `/Users/Shared`. We are using this folder. Inside that folder, we have several subfolders: * `skel/`: These are the template files. Do not edit files in here. `misty` may not work properly if the wrong files are edited, and updates may overwrite some files, too. * `usr/`: This is your place to edit files to suit your needs. @@ -41,19 +41,20 @@ As mentioned above, `mist-cli` is required for running this script, since the se During the packaging process, the installer for the respective major version will be present inside the `/Users/Shared/Mist` folder. It will be deleted after all plists have been created. -The script `misty` itself is located in `/usr/local/wycomo`. +The script *misty* itself is located in `/usr/local/wycomo`. ## System Requirements This script was tested with macOS 14 Sonoma. It should work with prior macOS versions, but this is has not been tested. -You should at least have 60 GB of free disk space available during each run. +You should at least have 60 GB of free disk space available during first run. ## To do This is a pre-release. It is working, but we have some tasks on our to do list: -- mist-cli changes the release date, although version and build number remain the same. We need to extract only the build number for comparison. As long as this is not done, `rm_previous_files` will only list the files, but not delete them. Also, all files are being listed, not only the previous versions. -- Check for space left. We need to check the space on the munki repo, but more importantly, the space on the system disk. After each run involving an installation, files are written to `/private/tmp/msu-target*/` that cannot be deleted by root. If not enough space is available, the resulting installer .app will not be complete, resulting in unusable plists and payloads. More testing needs to be done. -- LaunchAgent or LaunchDaemon. If the munki repo is not a local storage, we need a LaunchAgent to ensure that the user running this script can access the repo. On the other hand, a LaunchDaemon will run also if no user is logged in. -- Add `/usr/local/wycomco` to logged in user's PATH +- Testing in different environments. +- Check for space available. We need to check the space on the munki repo, but more importantly, the space on the system disk. If not enough space is available, the resulting installer .app will not be complete, resulting in unusable plists and payloads being offered to clients. There exists a check with hard-coded values, but more testing needs to be done if the values are appropriate. +- Proper redirection of echo messages, depending on interactive run or launchd job. +- Harmonization of variable names. +- Readability of code in general. diff --git a/misty/payload/Library/LaunchDaemons/de.wycomco.misty.plist b/misty/payload/Library/LaunchDaemons/de.wycomco.misty.plist new file mode 100644 index 0000000..df97944 --- /dev/null +++ b/misty/payload/Library/LaunchDaemons/de.wycomco.misty.plist @@ -0,0 +1,25 @@ + + + + + Label + de.wycomco.misty + ProgramArguments + + /bin/zsh + /usr/local/wycomco/misty + + StartCalendarInterval + + Hour + 22 + Minute + 0 + + StandardOutPath + /var/log/misty.log + StandardErrorPath + /var/log/misty_error.log + + + diff --git a/misty/payload/Users/Shared/Mist/skel/localized_startos.txt b/misty/payload/Users/Shared/Mist/skel/localized_startos.txt index cd5233f..900c045 100644 --- a/misty/payload/Users/Shared/Mist/skel/localized_startos.txt +++ b/misty/payload/Users/Shared/Mist/skel/localized_startos.txt @@ -9,9 +9,9 @@ preupgrade_alert alert_detail - Die Installation startet erst, wenn der Benutzer durch den Installer abgemeldet wird. + Die Installation muss ein zweites Mal durch Auswählen von "Aktualisieren" angestoßen werden. -Die Installation muss ein zweites Mal durch Auswählen von "Aktualisieren" angestoßen werden. +Die Installation startet erst, wenn der Benutzer durch den Installer abgemeldet wird. alert_title Anleitung zum macOS-Upgrade cancel_label diff --git a/misty/payload/Users/Shared/Mist/skel/override.txt b/misty/payload/Users/Shared/Mist/skel/override.txt old mode 100755 new mode 100644 diff --git a/misty/payload/Users/Shared/Mist/skel/pre_inserts_x86_64 b/misty/payload/Users/Shared/Mist/skel/pre_inserts_x86_64 index 170d8c9..daedf1f 100644 --- a/misty/payload/Users/Shared/Mist/skel/pre_inserts_x86_64 +++ b/misty/payload/Users/Shared/Mist/skel/pre_inserts_x86_64 @@ -3,9 +3,9 @@ preupgrade_alert alert_detail - The installation will not start until the user is logged out by the installer. + The installation must be triggered a second time by selecting "Upgrade". -The installation must be triggered a second time by selecting "Upgrade". +The installation will not start until the user is logged out by the installer. alert_title Instructions for macOS upgrade cancel_label diff --git a/misty/payload/usr/local/wycomco/misty b/misty/payload/usr/local/wycomco/misty index 2c8dbed..befefd2 100755 --- a/misty/payload/usr/local/wycomco/misty +++ b/misty/payload/usr/local/wycomco/misty @@ -1,13 +1,33 @@ #!/bin/zsh -e -# This script checks for availability of new releases of macOS, starting from macOS Monterey, -# using mist-cli (https://github.com/ninxsoft/mist-cli) and imports into munki. +# This script checks for availability of new releases of macOS, starting +# from macOS Monterey, using mist-cli (https://github.com/ninxsoft/mist-cli) +# and imports into munki. +# +# Copyright 2024 by wycomco. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . ################################## # Global definitions, first run # ################################## +export PATH="/usr/bin:/usr/local/bin:/usr/local/munki:$PATH" + +repo_required=14 # Size required on repo for single installer in GB +diskspace_required=30 # Size required on boot disk for single installer in GB Base_Path="/Users/Shared/Mist" LogPath="$Base_Path/Logs" Template="$Base_Path/skel" @@ -17,8 +37,8 @@ if [[ ! -d "$Base_Path/usr" ]]; then chmod 777 "$Base_Path/usr" fi if [[ ! -f "$Base_Path/usr/override.txt" ]]; then + currentUser=$(echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }') cp "$Template/override.txt" $Base_Path/usr - chmod 666 "$Base_Path/usr/override.txt" while true; do echo echo "Do you want to use localized strings in the resulting plists? (y/n)" @@ -26,19 +46,27 @@ if [[ ! -f "$Base_Path/usr/override.txt" ]]; then case $yn in [Yy]* ) cp "$Template"/localized_arm.txt "$Template"/localized_deploy.txt "$Template"/localized_stage_os.txt "$Template"/localized_startos.txt "$Base_Path"/usr - chmod 666 "$Base_Path"/usr/localized_*.txt sed -i '' 's/localization="no"/localization="yes"/g' "$Base_Path"/usr/override.txt - echo "Please adjust the files override.txt and any localized_*.txt in $Base_Path/usr to your needs and start the script again." + echo "Please adjust the file override.txt and any localized_*.txt files in $Base_Path/usr to your needs now." + echo "A finder window will open in 5 seconds." + sleep 5 + su -l "$currentUser" -c "open $Base_Path/usr" exit 0 ;; [Nn]* ) - echo "Please adjust the file $Base_Path/usr/override.txt to your needs and start the script again." + echo "Please adjust the file $Base_Path/usr/override.txt to your needs now." + echo "A finder window will open in 5 seconds." + sleep 5 + su -l "$currentUser" -c "open $Base_Path/usr" exit 0;; * ) echo "Please answer using [y] or [n].";; esac done + if ! launchctl list | grep -q "de.wycomco.misty"; then + launchctl load /Library/LaunchDaemons/de.wycomco.misty.plist + fi fi -# Definitions from config file in /Users/Shared/Mist/usr +# Load definitions from config file in /Users/Shared/Mist/usr source "$Base_Path"/usr/override.txt echo "RepoPath: $RepoPath" > /dev/null echo "munki_path: $munki_path" > /dev/null @@ -59,25 +87,25 @@ pkgsinfodir="$RepoPath/pkgsinfo/$munki_path" # Check environment # ################################## -mist_cmd=$(which mist) +mist_check=$(which mist) if [ $? -ne 0 ]; then - echo "Error: mist not found. Please install mist-cli or put it into the PATH." + echo "Error: mist not found. Please install mist-cli or put it into the PATH." >&2 exit 1 fi munkiimport_check=$(which munkiimport) if [ $? -ne 0 ]; then - echo "Error: munkiimport not found. Please check your munki installation." + echo "Error: munkiimport not found. Please check your munki installation." >&2 exit 1 fi if [[ ! -d "$LogPath" ]]; then mkdir -p "$LogPath" fi if [[ ! -d "$Template" || ! $(ls -A "$Template") ]]; then - echo "Error: The directory '$Template' either does not exist or is empty." + echo "Error: The directory '$Template' either does not exist or is empty." >&2 exit 1 fi if [[ ! -d "$RepoPath" ]]; then - echo "Error: Repository not accessible, please check." + echo "Error: Repository not accessible, please check." >&2 exit 1 fi if [[ $EUID -ne 0 ]]; then @@ -99,6 +127,32 @@ fi # Functions # ################################## +check_space() { + local repo_space bootdisk_space repo_disk bootdisk_disk total_required_space + repo_space=$(df -k "$RepoPath" | awk 'NR==2{print int($4 / 1024 / 1024)}') + if (( repo_space < repo_required )); then + echo "Less than $repo_required GB space on repo left. Please provide more space." >&2 + exit 1 + fi + + bootdisk=$(df -P "$Base_Path" | awk 'NR==2{print $1}') + bootdisk_space=$(df -k "$bootdisk" | awk 'NR==2{print int($4 / 1024 / 1024)}') + if (( bootdisk_space < diskspace_required )); then + echo "Less than $diskspace_required GB space on boot volume left. Please provide more space." >&2 + exit 1 + fi + + repo_disk=$(df -P "$RepoPath" | awk 'NR==2{print $1}') + bootdisk_disk=$(df -P "$Base_Path" | awk 'NR==2{print $1}') + if [[ "$repo_disk" == "$bootdisk_disk" ]]; then + total_required_space=$((repo_required + diskspace_required)) + if (( repo_space + bootdisk_space < total_required_space )); then + echo "Less than $total_required_space GB on the boot disk holding both repo and boot volume. Please provide more space." >&2 + exit 1 + fi + fi +} + rm_previous_files() { local fqos_retired=() @@ -114,8 +168,7 @@ rm_previous_files() { # Add the filename to fqos_retired array fqos_retired+=("$filename") fi - done - + done # Loop through unique major versions for major_version in $(echo "${fqos_retired[@]}" | cut -d. -f1 | sort -u); do # Filter files based on major version @@ -123,14 +176,13 @@ rm_previous_files() { # Sort files based on minor and patch version numbers in descending order major_files_sorted=($(printf '%s\n' "${major_files[@]}" | sort -rV)) # Keep only the first element (highest minor and patch version) - major_files_sorted=("${(@)major_files_sorted:1}") - + major_files_sorted=("${(@)major_files_sorted:1}") # Loop through major_files_sorted array and remove corresponding files for file in "${major_files_sorted[@]}"; do - ls -alh "${pkgsdir}"/*"${file}".* - ls -alh "${pkgsinfodir}"/*"${file}".* - ls -alh "${pkgsinfodir}"/arm64/*"${file}".* - ls -alh "${pkgsinfodir}"/x86_64/*"${file}".* + rm -f "${pkgsdir}"/*"${file}".dmg + rm -f "${pkgsinfodir}"/*"${file}".plist + rm -f "${pkgsinfodir}"/arm64/*"${file}".plist + rm -f "${pkgsinfodir}"/x86_64/*"${file}".plist echo "Removed version(s) ${file} from pkgs and pkgsinfo directories" done done @@ -145,8 +197,13 @@ extract_macos_version() { } download_macos() { - # Reduce output, but maintain single steps for log file or interactive run (progress) - $mist_cmd download installer $os_major application --application-name "Install macOS $os_nice.app" --force | grep '\[ [1-9][0-9]* \/ [1-9][0-9]* \]' + if [[ -t 1 ]]; then + # Reduce output, but output progress of single steps for interactive run + mist download installer $os_major application --application-name "Install macOS $os_nice.app" --force | grep '\[ [1-9][0-9]* \/ [1-9][0-9]* \]' + else + # If invoked by launchd, only print error messages to StandardErrorPath + mist download installer $os_major application --application-name "Install macOS $os_nice.app" --force > /dev/null 2>> /var/log/misty_error.log + fi } munkiimport_stage_os() { @@ -277,31 +334,31 @@ munkiimport_startos() { ################################## # We want to check each major version seperately -$mist_cmd list installer 14 --latest | grep GB > "$LogPath"/tmp_state_14.txt -$mist_cmd list installer 13 --latest | grep GB > "$LogPath"/tmp_state_13.txt -$mist_cmd list installer 12 --latest | grep GB > "$LogPath"/tmp_state_12.txt +mist list installer 14 --latest | grep GB > "$LogPath"/tmp_state_14.txt +mist list installer 13 --latest | grep GB > "$LogPath"/tmp_state_13.txt +mist list installer 12 --latest | grep GB > "$LogPath"/tmp_state_12.txt # Remove color codes resulting from grep and clean up rm_color_codes "$LogPath"/tmp_state_14.txt > "$LogPath"/current_state_14.txt rm_color_codes "$LogPath"/tmp_state_13.txt > "$LogPath"/current_state_13.txt rm_color_codes "$LogPath"/tmp_state_12.txt > "$LogPath"/current_state_12.txt -/bin/rm -f "$LogPath"/tmp_state_14.txt "$LogPath"/tmp_state_13.txt "$LogPath"/tmp_state_12.txt +/bin/rm -f "$LogPath"/tmp_state_*.txt -# Compare the content with the previous state or create initial state if it doesn't exist +# Compare the content with the previous state if present, ignoring last two columns (date and compatibliity) if [ -f "$LogPath"/previous_state_14.txt ]; then - if ! cmp -s "$LogPath"/current_state_14.txt "$LogPath"/previous_state_14.txt; then + if ! awk '{ for(i=2;i<=NF-4;i++) printf "%s ", $i; print "" }' "$LogPath"/current_state_14.txt | cmp -s - <(awk '{ for(i=2;i<=NF-4;i++) printf "%s ", $i; print "" }' "$LogPath"/previous_state_14.txt); then macos_14=$(extract_macos_version Sonoma "$LogPath"/current_state_14.txt) fi fi if [ -f "$LogPath"/previous_state_13.txt ]; then - if ! cmp -s "$LogPath"/current_state_13.txt "$LogPath"/previous_state_13.txt; then + if ! awk '{ for(i=2;i<=NF-4;i++) printf "%s ", $i; print "" }' "$LogPath"/current_state_13.txt | cmp -s - <(awk '{ for(i=2;i<=NF-4;i++) printf "%s ", $i; print "" }' "$LogPath"/previous_state_13.txt); then macos_13=$(extract_macos_version Ventura "$LogPath"/current_state_13.txt) fi fi if [ -f "$LogPath"/previous_state_12.txt ]; then - if ! cmp -s "$LogPath"/current_state_12.txt "$LogPath"/previous_state_12.txt; then + if ! awk '{ for(i=2;i<=NF-4;i++) printf "%s ", $i; print "" }' "$LogPath"/current_state_12.txt | cmp -s - <(awk '{ for(i=2;i<=NF-4;i++) printf "%s ", $i; print "" }' "$LogPath"/previous_state_12.txt); then macos_12=$(extract_macos_version Monterey "$LogPath"/current_state_12.txt) fi fi @@ -319,6 +376,7 @@ if [ -n "$macos_14" ]; then os_mini="10.13" os_munki="sonoma" os_nice="Sonoma" + check_space rm_previous_files echo "Downloading installer for macOS $macos_14" download_macos @@ -337,6 +395,7 @@ if [ -n "$macos_13" ]; then os_mini="10.12" os_munki="ventura" os_nice="Ventura" + check_space rm_previous_files echo "Downloading installer for macOS $macos_13" download_macos @@ -355,6 +414,7 @@ if [ -n "$macos_12" ]; then os_mini="10.9" os_munki="monterey" os_nice="Monterey" + check_space rm_previous_files echo "Downloading installer for macOS $macos_12" download_macos @@ -380,6 +440,9 @@ if [[ ( -n "$macos_12" || -n "$macos_13" || -n "$macos_14" ) && -f "$Base_Path"/ "$Base_Path"/usr/postinstall.sh fi -makecatalogs "$RepoPath" | grep 'warning' +# Run makecatalogs only if new package(s) were created +if [[ ( -n "$macos_12" || -n "$macos_13" || -n "$macos_14" ) ]]; then + makecatalogs "$RepoPath" | grep 'warning' +fi -exit 0 \ No newline at end of file +exit 0 diff --git a/misty/scripts/postinstall b/misty/scripts/postinstall index daf27d2..e7eb75a 100755 --- a/misty/scripts/postinstall +++ b/misty/scripts/postinstall @@ -1,19 +1,32 @@ #!/bin/bash -MISTY_DIR="/usr/local/wycomco" +currentUser=$(echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }') +homeDir=$(dscl . -read /Users/"$currentUser" NFSHomeDirectory | awk '{print $2}') +mistyDir="/usr/local/wycomco" -if [[ ":$PATH:" != *":$MISTY_DIR:"* ]]; then - if [ -f "$HOME/.bash_profile" ]; then - echo 'export PATH="/usr/local/wycomco:$PATH"' >> "$HOME/.bash_profile" - elif [ -f "$HOME/.bashrc" ]; then - echo 'export PATH="/usr/local/wycomco:$PATH"' >> "$HOME/.bashrc" - elif [ -f "$HOME/.zshrc" ]; then - echo 'export PATH="/usr/local/wycomco:$PATH"' >> "$HOME/.zshrc" - else - osascript -e 'display alert "PATH variable could not be set" message "Unable to find appropriate shell configuration file. Please add '$MISTY_DIR' to your PATH manually to run misty."' +if [[ ":$PATH:" != *":$mistyDir:"* ]]; then + if [ -f "$homeDir/.bashrc" ]; then + if ! grep -q ":$mistyDir" "$homeDir/.bashrc"; then + sed -i '' -e '/^export PATH=/ s|$|:'"$mistyDir"'|' "$homeDir/.bashrc" + echo "Added '$mistyDir' to your PATH in .bashrc." + fi + fi + if [ -f "$homeDir/.bash_profile" ]; then + if ! grep -q ":$mistyDir" "$homeDir/.bash_profile"; then + sed -i '' -e '/^export PATH=/ s|$|:'"$mistyDir"'|' "$homeDir/.bash_profile" + echo "Added '$mistyDir' to your PATH in .bash_profile." + fi + fi + if [ -f "$homeDir/.zshrc" ]; then + if ! grep -q ":$mistyDir" "$homeDir/.zshrc"; then + sed -i '' -e '/^export PATH=/ s|$|:'"$mistyDir"'|' "$homeDir/.zshrc" + echo "Added '$mistyDir' to your PATH in .zshrc." + fi + fi + if [ ! -f "$homeDir/.bashrc" ] && [ ! -f "$homeDir/.bash_profile" ] && [ ! -f "$homeDir/.zshrc" ]; then + osascript -e 'display alert "PATH variable could not be set" message "Unable to find .bashrc, .bash_profile, or .zshrc. Please add '$mistyDir' to your PATH manually to run misty."' exit 1 fi - echo "Added '$MISTY_DIR' to your PATH." else - echo "'$MISTY_DIR' is already in your PATH." + echo "'$mistyDir' is already in your PATH." fi