Monday, December 18, 2017

#1 2017-09-26 04:01:08 pm

misterfriendly
Member
From:: Massachusetts, USA
Registered: 2007-10-16
Posts: 111

stumped by 'do shell script' non-zero status

Here's what should be a simple script to compare the contents of two student enrollment lists (which are .csv files), so that we can see who has added or dropped courses.

When I run it, the line with the 'diff' command throws an error.
error "The command exited with a non-zero status." number 1

When I run the exact same command in the Terminal, it works fine.
Stranger still, the diff command actually completes because I can see the file. I could just stick that line inside a try block but I'd rather figure out what's wrong!

It's not ultimately important to capture the results of diff to a file -- I'd rather have Applescript parse the results and package the 'adds' and 'drops' into two separate text files. But first I have to figure out why this error is happening.

Applescript:

tell application "Finder"
   set oldFile to (choose file with prompt "Choose a .CSV file containing the original list of enrolled students")
   set a to quoted form of (POSIX path of oldFile)
   set newFile to (choose file with prompt "Choose a .CSV file containing the new list of added students")
   set b to quoted form of (POSIX path of newFile)
end tell
set {t1, t2, t3} to {"/private/tmp/tmp1.txt", "/private/tmp/tmp2.txt", "/private/tmp/tmp3.txt"}
set theShells to {}

set end of theShells to "touch " & t1 & " " & t2 & " " & t3

-- ## sort lines 2 thru end, by last name

set end of theShells to "/usr/bin/tail -n +2 " & a & " | /usr/bin/sort --field-separator=',' --key=3 > " & t1
set end of theShells to "/usr/bin/tail -n +2 " & b & " | /usr/bin/sort --field-separator=',' --key=3 > " & t2
set end of theShells to "/usr/bin/diff --strip-trailing-cr " & t1 & " " & t2 & " > " & t3

repeat with thisShell in theShells
   do shell script thisShell
end repeat


Filed under: shell, diff, bash, unix

Offline

 

#2 2017-09-26 05:34:55 pm

DJ Bazzie Wazzie
Member
From:: the Netherlands
Registered: 2004-10-20
Posts: 2727
Website

Re: stumped by 'do shell script' non-zero status

The misconception is that there is an error. In the shell programs returns 0 on success or other value when otherwise. "otherwise" doesn't have to mean an error however an do shell script assumes that there is an error when another value than 0 is returned. To catch this problem you should have something like this:

Applescript:

try
   do shell script thisShell
on error errmsg number errnr
   if errnr = 1 then
       -- files are not equal; no error
   else if errnr > 1 then
       -- error occured
   else
       -- we should never be here
   end if
end try

Or you can catch the problem in the shell and use a subshell to return 0 when there there is a difference.

Last edited by DJ Bazzie Wazzie (2017-09-26 05:35:19 pm)

Offline

 

#3 2017-09-27 09:57:45 am

misterfriendly
Member
From:: Massachusetts, USA
Registered: 2007-10-16
Posts: 111

Re: stumped by 'do shell script' non-zero status

That didn't work; same result. And there is clearly no error when I run the exact same command directly in the Terminal. diff returns a detailed list of the differences between two files, it doesn't just tell you they're the same or different.

However, I thought I'd try adding "exit 0" to the shell command and it worked:

Applescript:

set end of theShells to "/usr/bin/diff --strip-trailing-cr " & t1 & " " & t2 & " > " & t3 & " ; exit 0 ; "

This causes the shell command to return nothing to AppleScript, but that's OK because I'm writing the diff output to a temp file "t3" which I can then read in Applescript for further processing.

Thanks for pointing me in the right direction!

Offline

 

#4 2017-09-27 10:55:10 am

DJ Bazzie Wazzie
Member
From:: the Netherlands
Registered: 2004-10-20
Posts: 2727
Website

Re: stumped by 'do shell script' non-zero status

misterfriendly wrote:

That didn't work; same result.



My mistake, it was a draft/concept not working code per se. Here a working example on how to catch errors properly with do shell script:

Applescript:

try
   do shell script ">&2 echo 'This message is send to stderr'; exit 44"
on error msg number nr
   if nr = 44 then
       display dialog msg
   else
       error msg number nr
   end if
end try

misterfriendly wrote:

diff returns a detailed list of the differences between two files, it doesn't just tell you they're the same or different.



Every command or program exits with number. That number is returned and do shell script determines based on that integer value if an error occurred or not. The "detailed list of the difference between two files" is data written to stdout by the program. stdout is read by do shell script and returned to AppleScript as a string, but the return value of diff itself is an integer

This is what the manual (info diff) of diff has to say about the return values:

info diff wrote:

An exit status of 0 means no differences were found, 1 means some
differences were found, and 2 means trouble.




misterfriendly wrote:

And there is clearly no error when I run the exact same command directly in the Terminal.



The terminal doesn't throw errors. It prints the output (from stdout and stderr) in a window, nothing more.

misterfriendly wrote:

However, I thought I'd try adding "exit 0" to the shell command and it worked:



Because the last command of your script returns 0. Just like i said your script will now return an zero value and therefore do shell script assumed everything worked well. Keep in mind that when something actually does go wrong you won't be noticed either.

You can also use the built-in command true which I use for grep with an or operator.

Applescript:

do shell script "echo 'hello world!' | grep q || true"

The above command won't throw an error in AppleScript for not finding character q in the hello world string. Because grep will return an non-zero value if it cannot find the regex in the string. So the command true will be executed when grep returns a non-zero value and do shell script won't throw an error. In bash 0 = true and non-zero is false, therefore we use the or (||) operator.


misterfriendly wrote:

Thanks for pointing me in the right direction!



You're welcome, hopefully the rest makes it all even more clear cool

Offline

 

#5 2017-09-27 04:45:13 pm

misterfriendly
Member
From:: Massachusetts, USA
Registered: 2007-10-16
Posts: 111

Re: stumped by 'do shell script' non-zero status

Part of the reason I was getting weird diff results was that one of the CSV files wasn't a CSV file! Even though it had the .csv file extension, it was a binary UTF-16 file that diff couldn't read. So if I'm really going to make this script usable by other people, it needs to test if the .csv files you feed it are properly formatted before anything else happens.

The script does work now with a real csv file, formatted like this:

Semester,Primary Section,Student Name,E-mail,ID
2017/FA,COMSC-226-01,"Collins, Bootsy",bcollins@funkschool.edu,12345678
(etc)

Applescript:

set oldFile to (choose file with prompt "Choose a .CSV file containing the original list of enrolled students")
set a to quoted form of (POSIX path of oldFile)
set newFile to (choose file with prompt "Choose a .CSV file containing the new list of added students")
set b to quoted form of (POSIX path of newFile)

--set up the variables & temp files
set {t1, t2, t3} to {"/private/tmp/tmp1.txt", "/private/tmp/tmp2.txt", "/private/tmp/tmp3.txt"}
set theShells to {}
set {addTxt, dropTxt, resultTxt} to {"", "", ""}

set end of theShells to "touch " & t1 & " " & t2 & " " & t3

-- ## sort lines 2 thru end, by last name
set end of theShells to "/usr/bin/tail -n +2 " & a & " | /usr/bin/sort --field-separator=',' --key=3 > " & t1
set end of theShells to "/usr/bin/tail -n +2 " & b & " | /usr/bin/sort --field-separator=',' --key=3 > " & t2
set end of theShells to "/usr/bin/diff --strip-trailing-cr " & t1 & " " & t2 & " > " & t3 & " ; exit 0 ; "

repeat with thisShell in theShells
   do shell script thisShell
end repeat

-- read the diff output and format it nicely
if size of (info for t3) > 0 then
   
   set diff to paragraphs of (read t3 as text)
   
   repeat with thisLine in diff
       if (count characters of (thisLine as text)) > 0 then
           if (character 1 of thisLine = "<") then
               set dropTxt to dropTxt & (characters 3 thru -1 of thisLine) & return
           else if (character 1 of thisLine = ">") then
               set addTxt to addTxt & (characters 3 thru -1 of thisLine) & return
           end if
       end if
   end repeat
   
   if (count paragraphs of dropTxt) = 0 then set dropTxt to "[None]" & return
   if (count paragraphs of addTxt) = 0 then set addTxt to "[None]"
   
   set resultTxt to "Comparing enrollment files :: " & (do shell script "date '+%c'") & return & "Old enrollment file: " & tab & a & return & "New enrollment file: " & tab & b & return & return & "The following enrollments have been dropped:" & return & dropTxt & return & "The following enrollments have been added:" & return & addTxt
   set reportPath to "~/Desktop/EnrollmentReport_" & (do shell script "date '+%a_%m-%d-%Y_%I.%M.%S_%p'") & ".txt"
   do shell script "echo " & resultTxt & " > " & reportPath -- dump output to report file
   do shell script "sed -i '' \"s/\\,/\\    /g\" " & reportPath -- replace commas with tabs
else
   activate
   display dialog "The files are identical."
   
end if

--cleanup time
do shell script "rm " & t1 & " " & t2 & " " & t3

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)