- Added transaction support

- New resource modeling, with new RESTful endpoints
- Some misc refactoring
- Additional specifications for missing code coverage

Author: jbornemann

README.md

@@ -25,7 +25,7 @@ v2.x - CQ 5.6
 
 Active development is on the "master" branch, which is currently the 4.x version line. Security patches and the like are sometimes back-ported to prior versions.
 
-Of course pull-requests are happily accepted for those that would like to submit things like support for AEM 6.0, back-porting features for AEM 5.6, etc.
+Of course pull-requests are happily accepted for those that would like to submit things like back-porting features for AEM 5.6, etc.
 
 # Runtime Dependencies
 
@@ -63,18 +63,38 @@ _For both Windows and Mac : To verify that installation was successful, `protoc
 
 # Running Grabbit #
 
-[This] (grabbit.sh) shell script can be used to initiate grabbit jobs for a given Grabbit configuration and to check the status of those jobs.
+[This] (grabbit.sh) shell script can be used to initiate new Grabbit jobs, or monitor existing jobs. 
 
-1. Create a configuration file with the path(s) you wish to copy (see "Config Format" section below, you'll also set the server side URL and credentials in this step)
-2. Execute the grabbit.sh script (which comes with the codebase, by default lives in the root of the project)
-3. Enter the full address for the client instance including port (details for the server instance should be in the config fil you create in step 1)
-4. Enter Username for the client instance
-5. Enter the Password for the client instance
-6. Enter the path to the Grabbit config you created in step. Example of how it will look : !["Grabbit configuration enter example"](assets/GrabbitConfig.png)
-7. Once the Grabbit content sync is kicked off, you will get a confirmation of the kicked off jobs like : !["Grabbit confirmation example"](assets/GrabbitComplete.png)
+##Getting Started With grabbit.sh
 
+- Run grabbit.sh
+- Enter connection details to your Grabbit "client" server (The server you wish to pull content into)
+ 
+ !["Grabbit connection example"](assets/grabbitConnection.png)
+ 
+##Creating a New Job Request
+
+From the main screen, enter "n" for a new request. Enter an absolute path (from the machine you are running grabbit.sh) to the Grabbit configuration file you wish to use for creating a new request.
+See the "Configuration" section for details on how to create these configuration files. 
+
+ !["New Job"](assets/newJob.png)
+ 
+After the job request is sent, and started you should see a confirmation screen with the job IDs of all the jobs started, as well as their transaction ID
+
+ !["Job Confirmation"](assets/jobKickedOff.png)
+ 
+##Monitoring
 
-### Config Format
+From the main screen, enter "m" to monitor an existing job, or jobs. You will be prompted. If you would like to monitor a group of jobs by their transaction, enter "t". If you would like to monitor a specific jobs, enter "j".
+ 
+ !["Monitor Jobs"](assets/monitor.png)
+ 
+See "Monitoring / Validating the Content Sync" for information on evaluating the returned information.
+ 
+
+### Configuration
+
+Configuration can be developed in both YAML, and JSON formats.
 
 A `JSON` configuration file of following format is used to configure Grabbit.
 
@@ -115,7 +135,7 @@ A `JSON` configuration file of following format is used to configure Grabbit.
     ]
 }
 ```
-You can also use `YAML` file format to configure Grabbit. The corresponding YAML configuration for the JSON above will look something like -
+The corresponding YAML configuration for the JSON above will look something like:
 ```yaml
 # Client Type: author
 
@@ -142,7 +162,7 @@ pathConfigurations :
     path : /content/someContent
   -
     path : /content/someOtherContent
-    excludePaths: [ someOtherContent/someExcludeContent ]
+    excludePaths: [ someExcludeContent ]
   -
     path : /content/dam/someDamContent
     excludePaths :
@@ -209,11 +229,10 @@ Invalid:
 
 # Monitoring / Validating the Content Sync #
 
-You can validate / monitor sync by going to the following URI for each job on your Grabbit Client: 
-
-`/grabbit/job/<jobId>.json`
+You may choose to use grabbit.sh to monitor your sync, or you can validate/monitor your sync by going to the following URIs: 
 
-This will give you the status of a particular job. It has the following format -
+`/grabbit/job/<jobId>`
+`/grabbit/job/<transactionId>`
 
 A job status is has the following format : 
 
@@ -229,7 +248,8 @@ A job status is has the following format :
        "jobExecutionId": "JobExecutionId",
        "path": "currentPath",
        "startTime": "TimeStamp",
-       "timeTaken": "TimeInMilliSeconds"
+       "timeTaken": "TimeInMilliSeconds",
+       "transactionId": "transactionId"
    }
 ```
 
@@ -240,9 +260,10 @@ Couple of points worth noting here:
 
 If `exitCode` returns as `UNKNOWN`, that means the job is still running and you should check for its status again.
 
+
 __Sample of a real Grabbit Job status__
 
-![alt text](assets/ExampleGrabbitStatus.jpg)
+![alt text](assets/jobStatus.png)
 
 Two loggers are predefined for Grabbit. One for Grabbit Server and the other for Grabbit Client. 
 They are [batch-server.log](grabbit/src/main/content/SLING-INF/content/apps/grabbit/config/org.apache.sling.commons.log.LogManager.factory.config-com.twcable.grabbit.server.batch.xml) and [batch-client.log](grabbit/src/main/content/SLING-INF/content/apps/grabbit/config/org.apache.sling.commons.log.LogManager.factory.config-com.twcable.grabbit.client.batch.xml) respectively.

assets/ExampleGrabbitStatus.jpg

@@ -15,7 +15,78 @@ SET_RED=$(tput setaf 1)
 
 #### Constants
 
-GRABBIT_URL="/grabbit/job"
+GRABBIT_JOB="/grabbit/job"
+GRABBIT_TRANSACTION="/grabbit/transaction"
+
+
+rm -f /tmp/grabbit
+rm -f /tmp/grabbit_headers
+
+function newGrabbitRequest() {
+    echo
+    echo "Enter path to your Grabbit Configuration file"
+    read configpath
+    clear
+    echo $SET_YELLOW
+    echo "Processing....."
+    echo $SET_NO_COLOR
+    curl -s -f -X PUT --data-binary "@$configpath" -u $username:$password -D /tmp/grabbit_headers $client$GRABBIT_JOB > /tmp/grabbit
+    RSP_CODE=$?
+    if [ $RSP_CODE -ne 0 ]; then
+        clear
+        echo $SET_RED
+        echo "~~~~~~~~~Failure~~~~~~~~~~~~~"
+        echo
+        echo "Something went wrong when processing the given config."
+        echo
+        echo "cURL, the transfer library used by this shell received an error code of $RSP_CODE. See https://curl.haxx.se/libcurl/c/libcurl-errors.html for debugging."
+        echo "If this doesn't help, please check the client log for more details."
+        exit 1
+    fi
+    clear
+    echo "~~~~~~~~~~~Jobs kicked off ~~~~~~~~~~~~~"
+    echo
+    echo "Transaction ID: ${SET_BLUE}$(cat /tmp/grabbit_headers | grep GRABBIT_TRANSACTION_ID | sed -e 's/GRABBIT_TRANSACTION_ID: //')${SET_NO_COLOR}"
+    echo "Job IDs: ${SET_GREEN}$(cat /tmp/grabbit |  sed -e 's/\]//' -e 's/\[//' -e 's/,/  /g')${SET_NO_COLOR}"
+    echo
+    echo "~~~~~Client configuration : Client : $client Config: $configpath ~~~~~"
+}
+
+function monitorStatus {
+    while true; do
+        echo
+        echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+        echo "Enter \"j\" for job status, \"t\" for transaction status, or \"q\" to quit"
+        read selection
+        if [ "$selection" == "q" ]; then
+            exit 0;
+        elif [ "$selection" == "j" ]; then
+            while true; do
+                echo "Enter jobID to access status of a job, \"all\" to access all jobs, or \"b\" to go back"
+                read selection
+                if [ "$selection" == "b" ]; then
+                   break
+                fi
+                statusJson=`curl -s -u $username:$password --request GET $client$GRABBIT_JOB"/"$selection.json`
+                echo
+                echo "Status for JobId [$selection] is :"
+                echo "$statusJson" | python -m json.tool
+            done
+        elif [ "$selection" == "t" ]; then
+            echo "Enter transactionID to access transaction status, or \"b\" to go back"
+            read selection
+            if [ "$selection" == "b" ]; then
+                break
+            fi
+            statusJson=`curl -s -u $username:$password --request GET $client$GRABBIT_TRANSACTION"/"$selection.json`
+            echo
+            echo "Status for transaction [$selection] is :"
+            echo "$statusJson" | python -m json.tool
+        else
+            echo "Invalid selection"
+        fi
+    done
+}
 
 clear
 echo $SET_BLUE
@@ -26,51 +97,23 @@ echo $SET_NO_COLOR
 
 echo "Enter Client [http(s)://server:port] to initiate Grabbit request "
 read client
-
 echo
 echo "Client Username :"
-read -s username 
+read -s username
 echo
 echo "Client Password :"
-read -s password 
+read -s password
 
-echo
-echo "Enter path to your Grabbit Configuration file"
-read configpath
-
-rm -f /tmp/grabbit
-# Kick off Grabbit jobs
-clear
-echo $SET_YELLOW
-echo "Processing....."
-echo $SET_NO_COLOR
-curl -s -f -X PUT --data-binary "@$configpath" -u $username:$password $client$GRABBIT_URL > /tmp/grabbit
-if [ $? -eq 22 ]; then
-    clear
-    echo $SET_RED
-    echo "~~~~~~~~~Failure~~~~~~~~~~~~~"
-    echo   
-    echo "Something went wrong when processing the given config. Please check the client log for more details."
-    echo    
-    exit 1
-fi
-clear
-echo "~~~~~~~~~~~Jobs kicked off ~~~~~~~~~~~~~"
-echo 
-echo "Job IDs: ${SET_GREEN}$(cat /tmp/grabbit |  sed -e 's/\]//' -e 's/\[//' -e 's/,/  /g')${SET_NO_COLOR}"
-echo 
-echo "~~~~~Client configuration : Client : $client Config: $configpath ~~~~~"
-# Monitor Job Status
 while true; do
-	echo	
-	echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-	read -p "Enter Job ID or \"all\" to get job status or enter 'q' to quit: " id
-	if [ "$id" == "q" ]; then
-		exit 0;
-	fi
-	statusJson=`curl -s -u $username:$password --request GET $client$GRABBIT_URL"/"$id.json`
-	echo
-	echo "Status for JobId [$id] is :"
-	echo "$statusJson" | python -m json.tool 
-
+    echo "Enter \"n\" for new job request, \"m\" to monitor, or \"q\" to quit"
+    read selection
+    if [ "$selection" == "n" ]; then
+        newGrabbitRequest
+    elif [ "$selection" == "m" ]; then
+        monitorStatus
+    elif [ "$selection" == "q" ]; then
+        exit 0;
+    else
+        echo "Invalid selection"
+    fi
 done