rsync --exclude option (sucks?)

Hey All,

I’ve got a script (below) for using rsync to do a folder sync, one or two-way. I’ve included the option to exclude folder form the one-way sync, but there’s a problem with the --exclude option of rsync.

rsync’s --exclude is pattern-based, even if you’re providing it with a filepath. This wouldn’t be a problem, since each directory has a unique absolute file path, but --exclude only accepts relative file paths, relative to the source directory. So, if a user selects a folder in the source directory called “dir1” it will exclude “source/dir1/” from the sync. This is fine, but it will also exclude “source/dir2/dir1/”, which is not what I want.

Can anyone think of a way around this?

Here’s the script. Note that this will create a log in console entitled “BackupLog”

Also, I’m not entirely sure what error messages rsync might produce, so the logic for dealing with those is commented out. This lacks comments, but I don’t really need advise on the script, but rather with rsync.


(*
	Performs a backup of a source folder onto a destination folder,
	only backing up new or changed files. Folders can be excluded from
	the backup.
	
	All activity is logged to the "BackupLog" log in console
*)

on run
	-- See if terminal needs to be closed when finished
	set terminalWasRunning to appIsRunning("Terminal")
	
	tell application "Finder"
		activate
		
		set mirrorSync to ""
		
		display dialog "One-way backup or synchronize folders?" buttons {"Cancel", "Synchronize", "One-way"} default button "One-way" with icon note
		if button returned of result is "Cancel" then
			return 0
		else if button returned of result is "One-way" then
			set mirrorSync to button returned of (display dialog "Mirror destination to source? (delete files in destination that are not present in source)" buttons {"Cancel", "No", "Yes"} default button "Yes" with icon note)
			if mirrorSync is "Cancel" then return 0
			set oneWay to true
			set sourceMsg to "Select the folder to backup (source):"
			set destMsg to "Select the folder in which to place backup (destination):"
		else
			set oneWay to false
			set sourceMsg to "Select the first folder to be synced:"
			set destMsg to "Select the second folder to be synced:"
		end if
		
		repeat
			set sourceFolder to choose folder with prompt sourceMsg default location path to (home folder)
			set destFolder to choose folder with prompt destMsg default location path to (home folder)
			if (destFolder as text) is (sourceFolder as text) then
				display dialog "The two folders you selected are the same. Try again." buttons {"Cancel", "OK"} default button "OK" with icon stop
				if button returned of result is "Cancel" then return 0
			else
				exit repeat
			end if
		end repeat
		
		set excludeFolders to missing value
		if oneWay then
			set hasFolders to false
			try
				set sourceList to every item of sourceFolder
				repeat with anItem in sourceList
					if class of anItem is folder then
						set hasFolders to true
						exit repeat
					end if
				end repeat
			end try
		end if
		try
			if oneWay and hasFolders then set excludeFolders to choose folder with prompt "Select folder(s) that WILL NOT be backed up (Cancel if none):" default location sourceFolder with multiple selections allowed and invisibles
		end try
		
		if oneWay then
			set logString to (("Backing up " & sourceFolder as text) & " onto " & destFolder as text) & return & return & "Mirror sync: " & mirrorSync
		else
			set logString to (("Syncing " & sourceFolder as text) & " and " & destFolder as text)
		end if
		
		if excludeFolders is not missing value then
			set sourcePath to (POSIX path of sourceFolder) as text
			set exStr to ""
			repeat with i from 1 to count of excludeFolders
				set tmpParse to end of my parseLine((POSIX path of item i of excludeFolders) as text, sourcePath)
				set exStr to exStr & " --exclude " & quoted form of tmpParse
			end repeat
			set logString to logString & return & return & "Exluding folders:" & return
			repeat with i from 1 to count of excludeFolders
				set logString to logString & return & (item i of excludeFolders) as text
				set item i of excludeFolders to name of item i of excludeFolders
			end repeat
		else
			if oneWay then set logString to logString & return & return & "No folders excluded from backup"
		end if
		
		set sourceName to name of sourceFolder
		set destName to name of destFolder
	end tell
	
	if oneWay then
		set logString to logString & return & return & "Backup report:" & return & return
	else
		set logString to ((logString & return & return & "Part 1, " & sourceFolder as text) & " onto " & destFolder as text) & return & return
	end if
	
	set startTime to current date
	
	if mirrorSync is "Yes" then
		set cmdStr to "rsync -av --out-format='%t-- %i %f%L' --stats --delete --delete-during --delete-excluded"
	else
		set cmdStr to "rsync -av --out-format='%t-- %i %f%L' --stats"
	end if
	if excludeFolders is missing value then
		set cmdStr to cmdStr & space
		set scriptStr to cmdStr & quoted form of POSIX path of sourceFolder & space & quoted form of POSIX path of destFolder
	else
		set scriptStr to cmdStr & exStr & space & quoted form of POSIX path of sourceFolder & space & quoted form of POSIX path of destFolder
	end if
	
	tell application "Terminal"
		activate
		do script scriptStr
		delay 1
		repeat
			tell window 1
				if contents contains "total size is " then
					set ScriptFinished to true
					set scriptOutput to history
					exit repeat
				else
					--repeat with Msg in ErrorMsgs
					--	if contents contains ProgError of Msg then
					--		set ScriptError to true
					--		set Message to ErrorMsg of Msg
					--		exit repeat
					--	end if
					--end repeat
				end if
				--if ScriptError then exit repeat
			end tell
			delay 0.5
		end repeat
		close window 1
	end tell
	
	set hostStr to beginning of parseLine(host name of (system info), ".")
	
	set scriptOutputParsed to end of parseLine(scriptOutput, scriptStr)
	set scriptOutputParsed to beginning of parseLine(scriptOutputParsed, hostStr & ":")
	
	if not oneWay then
		set backupReport to logString & ((scriptOutputParsed & return & "Folder sync part 1 completed in " & timeElapsed(startTime) & return & return & "Starting sync part 2, " & destFolder as text) & " onto " & sourceFolder as text) & return & return
		logit(backupReport, "BackupLog")
		set startTime2 to current date
		set scriptStr to "rsync -av --out-format='%t-- %i %f%L' --stats " & quoted form of POSIX path of destFolder & space & quoted form of POSIX path of sourceFolder
		tell application "Terminal"
			activate
			do script scriptStr
			delay 1
			repeat
				tell window 1
					if contents contains "total size is " then
						set ScriptFinished to true
						set scriptOutput to history
						exit repeat
					else
						--repeat with Msg in ErrorMsgs
						--	if contents contains ProgError of Msg then
						--		set ScriptError to true
						--		set Message to ErrorMsg of Msg
						--		exit repeat
						--	end if
						--end repeat
					end if
					--if ScriptError then exit repeat
				end tell
				delay 0.5
			end repeat
			close window 1
		end tell
		
		set scriptOutputParsed to end of parseLine(scriptOutput, scriptStr)
		set scriptOutputParsed to beginning of parseLine(scriptOutputParsed, hostStr & ":")
		set backupReport to return & scriptOutputParsed & return & "Sync part 2 completed in " & timeElapsed(startTime2) & return & return & "Sync completed successfully in " & timeElapsed(startTime) & return & return
	else
		set backupReport to logString & scriptOutputParsed & return & "Backup Completed Successfully in " & timeElapsed(startTime) & return & return
	end if
	
	if not terminalWasRunning then tell application "Terminal" to quit
	
	--	set backupReport to return & return & "*** BACKUP FAILED ***" & return & return
	tell application "Finder"
		if not oneWay then update folder sourceFolder necessity yes
		update folder destFolder necessity yes
	end tell
	
	logit(backupReport, "BackupLog")
	
	tell application "Finder"
		activate
		display dialog "Manual folder sync operation completed" & return & ((current date) as text) & return & "Completed in " & (my timeElapsed(startTime) as text) buttons {"OK"} default button "OK" with icon note
	end tell
end run

to logit(log_string, log_file)
	-- This handler came from McUsr
	
	do shell script ¬
		"echo `date '+%Y-%m-%d %T: '`\"" & log_string & ¬
		"\" >> $HOME/Library/Logs/" & log_file & ".log"
end logit

on timeElapsed(startTime)
	-- takes start time and returns the elapsed time in hh:mm:ss format as text
	set currentTime to current date
	set elapsed to currentTime - startTime
	set elapsedHours to elapsed div 3600
	set elapsedMinutes to (elapsed - (elapsedHours * 3600)) div 60
	set elapsedSeconds to (elapsed - (elapsedHours * 3600) - (elapsedMinutes * 60)) div 1
	if elapsedMinutes < 10 then
		set elapsedMinutes to "0" & elapsedMinutes as text
	end if
	if elapsedSeconds < 10 then
		set elapsedSeconds to "0" & elapsedSeconds as text
	end if
	set elapsedTime to (elapsedHours & ":" & elapsedMinutes & ":" & elapsedSeconds) as text
	return elapsedTime
end timeElapsed

on parseLine(theLine, delimiter)
	-- This came from Nigel Garvey
	
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to {delimiter}
	set theTextItems to theLine's text items
	set AppleScript's text item delimiters to astid
	
	repeat with i from 1 to (count theTextItems)
		if (item i of theTextItems is "") then set item i of theTextItems to missing value
	end repeat
	
	return theTextItems's every text
end parseLine

on appIsRunning(appName)
	tell application "System Events" to (name of processes) contains appName
end appIsRunning

Some more research that I should’ve done before posting reveals the answer.

In my script you’ll notice that the --exclude option is used as “–exclude ‘path/to/dir’”. The missing leading “/” means that it is an unanchored name, so any directory called “dir” that’s in a directory structure called “path/to” will be excluded. A leading “/” will anchor it to the source directory, so “–exclude /path/to/dir” will ONLY exclude the single “/path/to/dir” directory.

There are several places on the web that claim that the missing piece is an equals sign, but “–exclude=‘path/to/dir’” is the same as “–exclude ‘path/to/dir’”. There is no functional difference between those two (I’m 99% sure).

To recap: if you want to exclude a directory “source/dir3/” but not exclude “source/dir1/dir3” then you use “–exclude ‘/dir3’” so that only the “dir3/” directly in the source directory will be excluded.

Thanks,

Tim