escape a space to shell script not working???

OK, I don’t get this - I’m trying to pass a variable to a shell script, and the variable has a space in it. I know that I need to escape spaces in the shell, so I fully understand that. Maybe I’m not getting something, but everywhere that I’ve read says that using ‘quoted form of’ will escape any spaces in the variable while it passes it to the shell, but it simply isn’t working, and I don’t undestand why. What am I missing? this should be very basic.

Here’s what I’m trying to do in it’s most simplistic form:


set backupDisk to "Test Me"
do shell script "echo " & quoted form of backupDisk

What I’m expecting to see (what the shell needs to see) is: echo Test\ Me

So how do I escape the space in the variable to pass it to the shell properly?

The ultimate objective here is to pass a file path (with spaces properly escaped) to a do shell script command: rsync -a --delete ~ /Volumes/" & backupDisk
…and have the script be flexible enough to accomodate whichever name the user types, spaces or not. the variable backupDisk is also referenced by finder and the script several times througout the script, so manually escaping the space by assigning “Test\ Me” to the backupDisk variable will screw up those parts of the script.

Help?

Hi,
quoted form is your friend. Try typing this in Terminal:
echo ‘Test Me’
you should find this works without any problem and that’s because you’ve protected the spaces by quoting the text/variable without needing to escape the spaces (echo Test\ Me).
Here are a couple of examples of how to use quoted form:

set thefile to quoted form of POSIX path of (choose file) -->the unix path of a file

set backupDisk to quoted form of text returned of (display dialog "enter file name" default answer "Test Me") --> quoted form of a variable
do shell script "ls /Volumes/" & backupDisk

I hope this helps!
Thanks,
Nik

You are doing it correctly, but your expectations are a bit off.

quoted form of takes a string value in AppleScript and produces a string that when evaluated by the shell (as a part of a command line) will produce a string value in the shell that is “identical” to the input AppleScript string value. TN2065 describes quoted form of like this: “The quoted form property gives the string in a form that is safe from further interpretation by the shell, no matter what its contents are.”.

The method that it uses to accomplish this is not specified (anywhere that I have seen), but every time I have bothered to check, it uses an algorithm that has the same overall effects as the following steps:

  1. Set the initial output string value to just one character: single quote.
  2. For each character, c, of the input string value (starting with the first, proceeding through the last):
    2.1) If c is not a single quote character, append c to the output string value.
    2.2) If c is a single quote character, append the three character sequence <single-quote, backslash, single quote, single quote> to the output string value.
  3. Append a single quote character to the output string value.

The effect is to wrap the string in a pair of single quotes and to take care to properly re-embed and single quotes in the original value. So, while in theory, the backslash method of escaping spaces could be used, it never is (in my experience). But it does not really matter whether quoted form of uses the “escape special characters” method, the “single quote most, escape single quotes, and concatenate them all” (the above algorithm), or some other method. The point is that the string value that the shell commands eventually sees is the same as the original string value inside AppleScript.

So, whether the shell is given with ‘Test Me’ or Test\ Me, it will be interpreted as a single shell “word” with a value of Test Me. In the case of your echo test, the echo command will receive one argument with the value of Test Me and then it will do its job of printing out that value. It is only the shell that ever sees the ‘quoted form of’ the original string value. The whole point of quoted form of is to get values past the shell and into the arguments of actual commands without the shell misinterpreting any “special characters” (like space, asterisk, backslash, single quote, double quote, semicolon, less than, greater than, pipe, etc.) that might be present in the initial string value inside AppleScript.

If you know a bit about shell syntax, then this example may be enlightening:

set s to "foo ; echo bar"
set rq to do shell script "echo " & quoted form of s
set r to do shell script "echo " & s
display dialog "Original: <<" & s & ">>" & return & ¬
	"With qfo: <<" & rq & ">>" & return & ¬
	"Without qfo: <<" & r & ">>" & return

What we see is that with quoted form of, echo gives us back the same string we started with. Without quoted form of, the shell sees an “unquoted” semicolon in there and stops to makes a second command with what comes after the semicolon. This type of example gives a hint of why it is so vitally important from a security perspective to use quoted form of with user-provided string values. If you do not use quoted form of and the users suspects this, she can give your program a string with a semicolon followed by some mischievous or malicious shell command. Usually the do shell script ends up running under the same UID as the AppleScript, but if, for example, the shell code is run with administrator privileges, then you have potentially just given admin-level access of any shell command to a non-admin user.

Assuming backupDisk is the name of an entry under /Volumes, quoted form of will do what you need: do shell script “rsync -a --delete ~ /Volumes/” & quoted form of backupDisk. Any spaces or other special character will make it to rsync intact without special interpretation by the shell. Along security/unpleasant-surprises lines, you also might want to make sure that the user did not type something like or the start volume’s name, since might let them damage the contents of the startup volume (if they have permission - e.g. they are an admin user).

Since you said you were planning on using the value in backupDisk with Finder and do shell script, you should be made aware of a detail that may cause a single string value not to work in both contexts.

Like the shell itself and other shell commands that deal with files, rsync expects what in AppleScript is called a POSIX path. The major characteristics of such a path is that the slash character is used to separate directories’ names from each other and from any final filename component (this is called “slash delimited”) and that the startup volume is just / while other mounted volumes are generally under /Volumes/. But Finder works with “HFS paths”. HFS paths are characterized by using the colon as delimiter between folder names and any final filename component as well as by always having the first component be the volume name (the startup volume is not given special priority in the namespace).

While it may seem that none of this should matter since you are only dealing with a single path component (under /Volumes/ in POSIX-land and the volume name in HFS-land), it does still matter because of how slash and colon are represented inside pathname components (i.e. an individual filename or directory/folder name). In POSIX paths, the slash is not allowed in pathname components. In HFS paths, a colon is not allowed inside pathname components. These are both reasonable since the restricted character is the pathname component delimiter. But colons are allowed in POSIX pathnames components and slashes are allowed in HFS pathname components. They are swapped when a pathname (or pathname component) is converted from one namespace to the other.

So, if in your example you have a volume with an HFS path name of Work w/Jane:, its POSIX equivalent is /Volumes/Work w:Jane/. Not only are colons swapped for slashes to convert the pathname component delimiters, but slashes are also swapped for colons inside pathname components. The same holds in the reserve but with the reasons swapped.

What this means is if the user types Work w/Jane to specify a volume name that they see on their desktop in Finder (which shows HFS-style pathname components), it will not work to just prefix /Volumes/ to it to get the corresponding POSIX path.

If none of the pathname components that you will be using will have a slash (HFS-side; colon POSIX-side), then maybe you can ignore this issue. But if you want to support such pathname components, you will need to do some conversion. POSIX path of is probably the best way to convert from HFS pathname strings (or many non-string pathname-like objects like aliases) to POSIX pathname strings (though there is a maybe-bug, at least on Tiger, if the HFS volume name start with a slash!). In the other direction, POSIX file is the way to take a POSIX path and turn it into an object that can be understood by most APIs that want HFS-paths. In my experience, it is best to apply these to full pathnames, not just pathname components.

wow, chrys - and blend - thanks for the useful info. I’m going to have to read this a couple more times, when my brain isn’t so tired!

using the echo command was probably not the best example, as entering Test Me and Test\ Me returns the same result, whereas not escaping a space in the rsync command will return a path not found error. But I see your point in protecting the input using quotes.

I have a feeling I might be at the point where it’s time to clean up my code a bit. In a nutshell, the relevant snippets:


set backupDisk to "DiskName"
set backupPath to ":Backup:Backup:" --would love to ditch the colons or have the user enter it as the more universally recognized HFS path
set a to POSIX path of backupPath

tell application "Finder"
	do shell script "diskutil mount " & last word of (do shell script "diskutil list | grep 'Apple_HFS '" & backupDisk)
	
	if folder backupPath of disk backupDisk exists then
	else
		do shell script "mkdir /Volumes/" & backupDisk & "/" & a
	end if
	
	get free space of disk backupDisk
	set y to free space of disk backupDisk
	
	get physical size of (folder backupPath of disk backupDisk)
	set z to physical size of (folder backupPath of disk backupDisk)
	
	--do some math
	
	do shell script "rsync -a --exclude=.*/ --delete ~ /Volumes/" & backupDisk
	
end tell

LOL - I just hit “cmd-K” after typing this! my brain’s fried…

I could probably move 80% of this to the outside of the tell block.

Without spaces in the backupDisk and backupPath variables, everything works fine. With spaces, I observed that the BackupDisk variable is not being passed to the shell with the spaces properly escaped, so rsync fails saying the destination path doesn’t exist. Haven’t had the mental energy to look at how the backupPath variable gets screwed up by spaces yet.

Not fully understanding the different ways which applescript, finder and the shell deals with paths has obviously hindered my ability to write solid code for this script. I’ll need a couple days to properly process what’s been said here, (doing this in my free time) but in the meantime, any additional feedback is appreciated. If it helps, I can post the entire script, but it’s a little bloated with logging steps.

OK, I think I absorbed most of what you said.

When I open that snippet and run it:


set backupDisk to "Disk name with spaces"
do shell script "rsync -an --delete ~ /Volumes/" & quoted form of backupDisk

(added the -n switch to do a dry run)

it passes this to the shell:
rsync -an --delete ~ /Volumes/‘disk name with spaces’
as it should, which errors out because the spaces need to be escaped.

What rsync needs is:
rsync -an --delete ~ /Volumes/disk\ name\ with\ spaces

I’m trying to figure out how to pass a variable with spaces in it to the shell with the spaces escaped, not preserved.

I think I get what quoted form of does, but quoted form of is obviously not what I’m looking for then?

Actually, after the shell does its interpretation, those command lines result in exactly the same string getting to rsync.

I have a small util I keep handy that just dumps out the arguments it receives, but unlike echo, it makes (more) obvious the actual values of each argument. I call it pargs (Print ARGS). Here is what is shows for those two command lines:

$ pargs rsync -an --delete ~ /Volumes/'disk name with spaces' arg 0: <</Users/username/cmd/pargs>> arg 1: <<rsync>> arg 2: <<-an>> arg 3: <<--delete>> arg 4: <</Users/username>> arg 5: <</Volumes/disk name with spaces>> $ pargs rsync -an --delete ~ /Volumes/disk\ name\ with\ spaces arg 0: <</Users/username/cmd/pargs>> arg 1: <<rsync>> arg 2: <<-an>> arg 3: <<--delete>> arg 4: <</Users/username>> arg 5: <</Volumes/disk name with spaces>>
Is there some doc or error message or something that is telling you that you need backslashes instead of single quoting?

chrys - to answer your question, yes, I was getting a ‘no such file or directory’ error output from rsync if I didn’t escape the spaces in the destination path. I believe I was getting the error both with and without using quoted form of.

After seeing the output from your commands, I tried again and wtf - now it’s working…
You’re right - rsync accepts the path without spaces escaped - IF I preserve those spaces with quotes. (or, for applescript, using quoted form of). If I don’t quote, it errors out unless I escape the spaces.


set backupdisk to "disk name with spaces"
do shell script "rsync -an --delete ~ /Volumes/" & quoted form of backupdisk

event log shows: tell current application
do shell script “rsync -an --delete ~ /Volumes/‘disk name with spaces’”
“”
end tell

with no error output from rsync!

So now I gotta go rewind my brain and try and figure out why this was giving me such a headache in the first place!

I think maybe I was getting a ‘no such file or directory’ error from rsync for another reason - maybe I mistyped my path …but several times??? (stranger things have happened) and then seeing by the event log that do shell script appears to be sending the command with spaces to the shell and just got myself confused. The only difference that I can see between what I WAS doing then and what I’m doing now is the -n switch for the dry run. Tomorrow I’ll give it a shot with only a few small files (instead of ~) without the dry run and see what happens, but I bet it all works fine as long as I use quoted form of properly. What a noob mistake.

I’m fried - going to bed…
thanks for your help - I’ll post back here what the problem was when I figure it out. At the least, I’ve just received a good lashing on how to use quoting… which is a good thing - I now clearly understand what it’s all about. :slight_smile: