Mail parsing and upload

Hey everyone,

So I don’t know a lick of AppleScript, and while I’d like to learn, right now I just want this script to work. Here’s the deal: Every day I receive a certain email from a certain mailing list. What I need to happen is the following: upon receiving the email, which would presumably be identifiable from a rule, Mail will…

  1. Export the text contents of the email to a file, plain text, on the disk.
  2. Upload the file to a certain location on a certain server; I’m popping open Transmit to do this, since I’m not very familiar with curl or the other command-line options.

So what’s the problem? Well, two problems – one, it doesn’t work; the file generates but doesn’t call Transmit. Two, the script only works when the email contains no double quotes, which is bad.

I can’t fix these problems because the only AppleScript I know is what I can puzzle out by the human-readable syntax :slight_smile: I had a friend write what I’ve got so far, but he doesn’t know enough about writing to disk to handle the quote issue – he’s already cheating by shell-scripting that bit and echoing the file out.

So, can anyone make some suggestions?

Script follows (the stuff in comments is where my friend was trying to fix the quotes; he gave up, but I’ll leave that in in case it helps anyone):


on perform_mail_action(info)
	tell application "Mail"
		set thedata to ""
		set selectedMessages to |SelectedMessages| of info
		set theRule to |Rule| of info
		repeat with eachMessage in selectedMessages
			--repeat with cha in content of eachMessage
			--if (cha = """) then
			--cha = "'"
			--end if
			--end repeat
			--set thedata to thedata & content of eachMessage
			do shell script "echo "" & content of eachMessage & "" >> ~/filepath"
			--write the content of eachMessage to file "~/filepath/index.txt"
		end repeat
	end tell
	--display dialog thedata
	--tell application "BBEdit 6.5"
	--set foo to make new document
	--set text of window of foo to thedata
	--replace """ using "'" searching in text of window foo
	--set thedata to the text of window of foo
	--	save window of foo to file ":Totally Sweet Ninja:Users:two:science"
	--close window foo
	--end tell
	--do shell script "echo "" & thedata & "" >>  ~/filepath/index.txt"
end perform_mail_action
tell application "Transmit"
	connect to "foo.org" as user "user" with password "pass" with initial path "/www/directory/"
	delete remote "index.txt"
	upload "~/index.txt"
end tell

Any help is appreciated,

  • Brandon

Hmm… no thoughts, then?

Does this much work? Once you set up a rule to run the script, it should generate a text file named “Message Text.txt” on the desktop. To test the rule, select a message that will trigger the rule and choose “Apply rules to selection” from the “Message” menu.

property destinationPath : path to desktop as text
property fileName : "Message Text.txt"

on perform_mail_action(Info)
	tell application "Mail"
		set selectedMessages to |SelectedMessages| of Info
		repeat with eachMessage in selectedMessages
			set content_ to content of eachMessage as string
			my write_to_file(content_, (destinationPath & fileName), false)
		end repeat
	end tell
end perform_mail_action

on write_to_file(this_data, target_file, append_data)
	try
		set the target_file to the target_file as text
		set the open_target_file to ¬
			open for access file target_file with write permission
		if append_data is false then ¬
			set eof of the open_target_file to 0
		write this_data to the open_target_file starting at eof
		close access the open_target_file
		return true
	on error
		try
			close access file target_file
		end try
		return false
	end try
end write_to_file

If this much works, we can work on the upload.

– Rob

The problem with the upload is simple.

The line in the original code:

do shell script "echo "" & content of eachMessage & "" >> ~/filepath"

writes the data to ~/filepath - that is, the file called ‘filepath’ in the user’s home directory.

Later on in the script, you try to tell Transmit to upload a completely different file:

upload "~/index.txt"

i.e. you’re trying to upload the file called ‘index.txt’ in the user’s home directory. Since this isn’t the file you created earlier, I’m not surprised the upload doesn’t appear to work.

Change the file writing code to Rob’s code, then change the Transmit code to reference the correct file to upload.

Rob: Thanks a lot – that seems to work great. File is generated, quotes are fine, all is good in the world. I had a small scare until I realized that I had to place that script rule BEFORE any other rules that move the affected email around – dunno why – but it’s kicking right now.

Camelot: Pretty sure that was just an error on my part when I edited out the actual information for internet postage :wink: The data is correct in the actual script. Anyway, that’s not the problem; Transmit isn’t being called at all, not connecting; filenames never have a chance to be a problem.

You’d think that a scripting language this simple, with readily-usable libraries, would be harder to mess up with a simple string of commands than this. But I guess sufficiently talented fools are unstoppable :slight_smile:

Thanks for the help so far.

Does the Transmit code work when tested outside of this script?

– Rob

Nope. I just suck. Not even a straightforward tell with a connect command will work. The error: “NSReceiversCantHandleCommandScriptError”.

If anyone doesn’t have Transmit, here’s the relevant library item:

connect: Connect to a specified server.
	connect  reference  -- the object for the command
		[as user  Unicode text]  -- The user name to use when connecting.
		[using port  Unicode text]  -- The network port to use when connecting.
		[with initial path  Unicode text]  -- The initial path to change to after connection is made.
		[with password  Unicode text]  -- The password to use when connecting.
		[with connection type  SFTP/FTP]  -- The connection type to use when connecting.
		to  Unicode text  -- The server address to connect to.
	Result:   boolean  -- the reply for the command

This should be very close to what you need for Transmit once you set the user defined variables.

-- begin user defined variables --
set myUserName to "username"
set myPass to "password"
set ftpServer to "foo.org" -- your ftp server
set initialPathOnFTPserver to "/www/directory/"
set fileToUpload to "/path/to/index.txt"
-- end user defined variables --

set tids to text item delimiters
try
	set text item delimiters to "/"
	set fileName to last text item of fileToUpload
	set text item delimiters to tids
on error
	set text item delimiters to tids
end try

tell application "Transmit"
	activate
	set newWindow to make new document at end
	tell newWindow
		connect to ftpServer as user myUserName with password myPass ¬
			with initial path initialPathOnFTPserver with connection type FTP
		delete remote item ((ftpServer & initialPathOnFTPserver & fileName) as text)
		upload item fileToUpload
		--	disconnect
	end tell
	--quit
end tell

– Rob

This is fast becoming the script that never dies. Rob and eveyone else, I want to thank you again for your endless patience and help with this.

Okay, here’s the deal now. I took your Transmit code and tweaked it until it worked and was lovely. So everything should be fine now, right? Well, no. This is extremely odd.

Basically the Transmit code works fine when I run it alone – put the code in Script Editor and click Run. Doesn’t matter whether it’s in a standalone script or thrown in with the Mail uploader script. It works.

But when I try to execute it from Mail as part of the whole ensemble, it just doesn’t happen. The script goes through the file parsing fine, as it always did; Transmit never makes even a whimper. It doesn’t connect, upload, or even (if it’s closed) launch. It does pretty much what you’d expect it to if you spelled the application name wrong, or some similar such nonsense that prevented it from even contacting the app.

In other words, I have half of a script (the file generator) that works only from Mail, as it’s meant to; and I have half of a script (the Transmit piece) that only works when run directly.

Attempts to work around this failed. I tried simply executing a shell script, at the end of the file generation script, that launched a separate script which held the Transmit code; didn’t happen. Tried to launch it directly with an AppleScript “open” command; same deal. Something hates me.

Has anyone heard of this kind of thing happening before? A script working from Script Editor, but not when called by another process?

Here’s a complete working script, tested successfully as a Mail rule. For now, try to limit changes to the parts flagged “NEED YOUR INFO”. It can be tweaked once we get it working for you.

-- Note: The local file defined in the next 2 lines can be deleted by the script after upload if desired.
property destinationPath : path to desktop as text -- Customize if needed
property fileName : "Message Test.txt" -- Customize if needed

on perform_mail_action(Info)
	tell application "Mail"
		set selectedMessages to |SelectedMessages| of Info
		repeat with eachMessage in selectedMessages
			set content_ to content of eachMessage as string
			my write_to_file(content_, (destinationPath & fileName), false)
			my upload_(destinationPath & fileName)
		end repeat
	end tell
end perform_mail_action

on upload_(local_file_path)
	-- begin user defined variables -- 
	set myUserName to "username" -- NEED YOUR INFO
	set myPass to "password" -- NEED YOUR INFO
	set ftpServer to "foo.org" -- your ftp server -- NEED YOUR INFO
	set initialPathOnFTPserver to "/www/directory/" -- NEED YOUR INFO
	-- end user defined variables -- 
	
	set fileToUpload to POSIX path of (local_file_path)
	set tids to text item delimiters
	try
		set text item delimiters to "/"
		set fileName to last text item of fileToUpload
		set text item delimiters to tids
	on error
		set text item delimiters to tids
	end try
	
	tell application "Transmit"
		activate
		set newWindow to make new document at end
		tell newWindow
			connect to ftpServer as user myUserName with password myPass ¬
				with initial path initialPathOnFTPserver with connection type FTP
			delete remote item ((ftpServer & initialPathOnFTPserver & fileName) as text)
			upload item fileToUpload
			--   disconnect 
		end tell
		--quit 
	end tell
end upload_

on write_to_file(this_data, target_file, append_data)
	try
		set the target_file to the target_file as text
		set the open_target_file to ¬
			open for access file target_file with write permission
		if append_data is false then ¬
			set eof of the open_target_file to 0
		write this_data to the open_target_file starting at eof
		close access the open_target_file
		return true
	on error
		try
			close access file target_file
		end try
		return false
	end try
end write_to_file

– Rob

Well, I’ve got a simple FTP upload handler that uses curl, but doesn’t require you to know how to use it. Perhaps it would be useful, if you don’t want to rely on Transmit.

I thought of suggesting curl too but I’m not too curly yet. It would likely be a more elegant solution and your handlers look great! Thanks for the reminder. :slight_smile:

– Rob (who is now becoming partly curly by adding Dan’s curl handlers to his collection) :stuck_out_tongue:

Woohoo!

Script tested, working, and pressed into service – fully functional so far.

Let me offer my sincerest thanks and appreciation to Rob for spending so much time with this. It’s definitely not something I could have done myself, certainly not as quickly as I’d have wanted. Really great stuff.

Good show, everyone. A real pleasure.

Much thanks,

  • Brandon

Glad it worked. :slight_smile:

Transmit is my favorite FTP client when I need to work manually with more than a couple of files. With that said, this task is a good candidate for curl since it involves a single file and visual feedback isn’t required (I assume). Krioni’s upload script works quite nicely and it’s very easy to use. It took all of about a minute to set up and it will overwrite the existing remote file, eliminating the need to delete it first.

– Rob

I’ve got to say that I love curl - I just wish I had time to learn all the things it can do. It can do FTP, TELNET, HTTP, HTTPS, and even other protocols like LDAP, GOPHER, DICT, and FILE. I’ve only used it for FTP and HTTP. It can also change its user-agent, for sites that require Internet Explorer for Windows. It can read and feed-back cookies, so that sites that set a cookie to allow you to connect can be accessed. It can submit forms (including file-upload forms), follow web page redirects, get the size of a file through FTP (instead of just downloading it), list FTP directories, use a proxy if you have to use one, and much more.

It makes life a lot easier when you want to script some interaction between a Mac (running OS X) and the Internet.

ok I’m an extreme newb, but that upload script is just what I need. However, I’m trying to set it up and well… I’m getting errors. Like "… doesn’t match the parameters {remoteURL, macFilePath, userName, userPasswd} for simpleFtpUpload.

Have I put in my own information in the wrong spot?

Sounds like it, but seeing is believing. Show us how you’re trying to call the handler, and I can say what is wrong. It needs 4 parameters, each of which is a string.

ok being a newbie at apple script, I would appreciate it if you could just show me where to put username, pwd and those variables into the script. I haven’t had time to do any research so I don’t know the correct conventions for using variables in apple scripts yet.

OK. I’ll do my best, but as I mentioned, it’s very helpful to those trying to help if you show what you are currently using. Much easier to tell you what to change (and also why) than to guess at what you want.

If your username is “bob” (no quotes), your password “peanuts”, your ftp server is at ftp.bobsserver.com, the directory-path you want to put the file in is public/images/ and the local file (on your Mac) is at “Macintosh HD:Pictures:bobspicture.jpg” then you would do the following:
(Note that the remoteURL parameter is a combination of the transfer protocol part “ftp://” the server address “ftp.bobsserver.com” and the path on that server “/public/images/”)


-- SAMPLE USAGE
simpleFtpUpload("ftp://ftp.bobsserver.com/public/images/", "Macintosh HD:Pictures:bobspicture.jpg", "bob", "peanuts")

on simpleFtpUpload(remoteURL, macFilePath, userName, userPasswd)
        -- version 1.2, Dan Shockley (http://www.danshockley.com)
        -- uses curl to do the upload
        -- remoteURL is the complete ftp url to the remote destination directory
        try
                if userName is "" then
                        set userName to "anonymous"
                        set userPasswd to "devnull@devnull.null"
                        -- no email address, change if desired
                end if
                
                -- get parentFolder
                if (macFilePath as string) ends with ":" then -- it is a folder itself
                        set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ":"}
                        set parentFolder to (text items 1 thru -3 of (macFilePath as string)) as string
                        set AppleScript's text item delimiters to od
                else -- it is just a file
                        set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ":"}
                        set parentFolder to (text items 1 thru -2 of (macFilePath as string)) as string
                        set AppleScript's text item delimiters to od
                end if
                
                set localPosixPath to quoted form of POSIX path of alias macFilePath
                
                set uploadFileName to do shell script "basename " & localPosixPath
                set uploadDirectory to quoted form of POSIX path of parentFolder
                
                -- curl --upload-file SOMEFILE ftp://SERVER.com --user MYUSER:THEPASSWD
                -- must be IN the directory of the file you want to upload
                set myCommand to "cd " & uploadDirectory & ";" & "curl --upload-file "
                set myCommand to myCommand & quoted form of uploadFileName
                set myCommand to myCommand & " " & remoteURL
                set myCommand to myCommand & " --user " & userName & ":" & userPasswd
                set myCommand to myCommand & " " & "--write-out " & "%{size_upload}"
                set myCommand to myCommand & " --quote -quit"
                
                -- output is the 'size_upload' result from curl
                
                set uploadResult to do shell script myCommand
                
                return uploadResult -- size uploaded in kilobytes
        on error errMsg number errNum
                error "simpleFtpUpload FAILED: " & errMsg number errNum
        end try
end simpleFtpUpload

Hey folks!

Just stumbled upon this thread during some autogooglatio. If Rob and Co. are still around, I thought you might be interested – the script you helped me with was used daily for two years (my last post in this thread to the current date) and is actually still used, on my website BerkeleyHigh.org. It served a critical purpose that wouldn’t have been answered without it, and until/unless the current site admin comes up with a better solution, it’ll continue to. It’s also been replicated into two clones that upload similar files to other sites for the local PTSA. Thanks a ton!