stumped by 'do shell script' non-zero status

Here’s what should be a simple script to compare the contents of two student enrollment lists (which are .csv files), so that we can see who has added or dropped courses.

When I run it, the line with the ‘diff’ command throws an error.
error “The command exited with a non-zero status.” number 1

When I run the exact same command in the Terminal, it works fine.
Stranger still, the diff command actually completes because I can see the file. I could just stick that line inside a try block but I’d rather figure out what’s wrong!

It’s not ultimately important to capture the results of diff to a file – I’d rather have Applescript parse the results and package the ‘adds’ and ‘drops’ into two separate text files. But first I have to figure out why this error is happening.

tell application "Finder"
	set oldFile to (choose file with prompt "Choose a .CSV file containing the original list of enrolled students")
	set a to quoted form of (POSIX path of oldFile)
	set newFile to (choose file with prompt "Choose a .CSV file containing the new list of added students")
	set b to quoted form of (POSIX path of newFile)
end tell
set {t1, t2, t3} to {"/private/tmp/tmp1.txt", "/private/tmp/tmp2.txt", "/private/tmp/tmp3.txt"}
set theShells to {}

set end of theShells to "touch " & t1 & " " & t2 & " " & t3

-- ## sort lines 2 thru end, by last name

set end of theShells to "/usr/bin/tail -n +2 " & a & " | /usr/bin/sort --field-separator=',' --key=3 > " & t1
set end of theShells to "/usr/bin/tail -n +2 " & b & " | /usr/bin/sort --field-separator=',' --key=3 > " & t2
set end of theShells to "/usr/bin/diff --strip-trailing-cr " & t1 & " " & t2 & " > " & t3

repeat with thisShell in theShells
	do shell script thisShell
end repeat

The misconception is that there is an error. In the shell programs returns 0 on success or other value when otherwise. “otherwise” doesn’t have to mean an error however an do shell script assumes that there is an error when another value than 0 is returned. To catch this problem you should have something like this:

try
	do shell script thisShell
on error errmsg number errnr
	if errnr = 1 then
		-- files are not equal; no error
	else if errnr > 1 then
		-- error occured
	else
		-- we should never be here
	end if
end try

Or you can catch the problem in the shell and use a subshell to return 0 when there there is a difference.

That didn’t work; same result. And there is clearly no error when I run the exact same command directly in the Terminal. diff returns a detailed list of the differences between two files, it doesn’t just tell you they’re the same or different.

However, I thought I’d try adding “exit 0” to the shell command and it worked:

set end of theShells to "/usr/bin/diff --strip-trailing-cr " & t1 & " " & t2 & " > " & t3 & " ; exit 0 ; "

This causes the shell command to return nothing to AppleScript, but that’s OK because I’m writing the diff output to a temp file “t3” which I can then read in Applescript for further processing.

Thanks for pointing me in the right direction!

My mistake, it was a draft/concept not working code per se. Here a working example on how to catch errors properly with do shell script:

try
	do shell script ">&2 echo 'This message is send to stderr'; exit 44"
on error msg number nr
	if nr = 44 then
		display dialog msg
	else
		error msg number nr
	end if
end try

Every command or program exits with number. That number is returned and do shell script determines based on that integer value if an error occurred or not. The “detailed list of the difference between two files” is data written to stdout by the program. stdout is read by do shell script and returned to AppleScript as a string, but the return value of diff itself is an integer

This is what the manual (info diff) of diff has to say about the return values:

The terminal doesn’t throw errors. It prints the output (from stdout and stderr) in a window, nothing more.

Because the last command of your script returns 0. Just like i said your script will now return an zero value and therefore do shell script assumed everything worked well. Keep in mind that when something actually does go wrong you won’t be noticed either.

You can also use the built-in command true which I use for grep with an or operator.

do shell script "echo 'hello world!' | grep q || true"

The above command won’t throw an error in AppleScript for not finding character q in the hello world string. Because grep will return an non-zero value if it cannot find the regex in the string. So the command true will be executed when grep returns a non-zero value and do shell script won’t throw an error. In bash 0 = true and non-zero is false, therefore we use the or (||) operator.

You’re welcome, hopefully the rest makes it all even more clear :cool:

Part of the reason I was getting weird diff results was that one of the CSV files wasn’t a CSV file! Even though it had the .csv file extension, it was a binary UTF-16 file that diff couldn’t read. So if I’m really going to make this script usable by other people, it needs to test if the .csv files you feed it are properly formatted before anything else happens.

The script does work now with a real csv file, formatted like this:

Semester,Primary Section,Student Name,E-mail,ID
2017/FA,COMSC-226-01,“Collins, Bootsy”,bcollins@funkschool.edu,12345678
(etc)

set oldFile to (choose file with prompt "Choose a .CSV file containing the original list of enrolled students")
set a to quoted form of (POSIX path of oldFile)
set newFile to (choose file with prompt "Choose a .CSV file containing the new list of added students")
set b to quoted form of (POSIX path of newFile)

--set up the variables & temp files
set {t1, t2, t3} to {"/private/tmp/tmp1.txt", "/private/tmp/tmp2.txt", "/private/tmp/tmp3.txt"}
set theShells to {}
set {addTxt, dropTxt, resultTxt} to {"", "", ""}

set end of theShells to "touch " & t1 & " " & t2 & " " & t3

-- ## sort lines 2 thru end, by last name
set end of theShells to "/usr/bin/tail -n +2 " & a & " | /usr/bin/sort --field-separator=',' --key=3 > " & t1
set end of theShells to "/usr/bin/tail -n +2 " & b & " | /usr/bin/sort --field-separator=',' --key=3 > " & t2
set end of theShells to "/usr/bin/diff --strip-trailing-cr " & t1 & " " & t2 & " > " & t3 & " ; exit 0 ; "

repeat with thisShell in theShells
	do shell script thisShell
end repeat

-- read the diff output and format it nicely
if size of (info for t3) > 0 then
	
	set diff to paragraphs of (read t3 as text)
	
	repeat with thisLine in diff
		if (count characters of (thisLine as text)) > 0 then
			if (character 1 of thisLine = "<") then
				set dropTxt to dropTxt & (characters 3 thru -1 of thisLine) & return
			else if (character 1 of thisLine = ">") then
				set addTxt to addTxt & (characters 3 thru -1 of thisLine) & return
			end if
		end if
	end repeat
	
	if (count paragraphs of dropTxt) = 0 then set dropTxt to "[None]" & return
	if (count paragraphs of addTxt) = 0 then set addTxt to "[None]"
	
	set resultTxt to "Comparing enrollment files :: " & (do shell script "date '+%c'") & return & "Old enrollment file: " & tab & a & return & "New enrollment file: " & tab & b & return & return & "The following enrollments have been dropped:" & return & dropTxt & return & "The following enrollments have been added:" & return & addTxt
	set reportPath to "~/Desktop/EnrollmentReport_" & (do shell script "date '+%a_%m-%d-%Y_%I.%M.%S_%p'") & ".txt"
	do shell script "echo " & resultTxt & " > " & reportPath -- dump output to report file
	do shell script "sed -i '' \"s/\\,/\\	/g\" " & reportPath -- replace commas with tabs
else
	activate
	display dialog "The files are identical."
	
end if

--cleanup time
do shell script "rm " & t1 & " " & t2 & " " & t3