Out of memory error with big list

Thanks to those who helped me yesterday! Latest problem:

I have written a script whose purpose is to search through a parent folder and its sub-folders. When it finds a folder called �Photography� it looks inside it for .tif files and lists them. That is all it needs to do.

It works fine (though slow) if I start on a sub-folder, but it needs to be able to tackle the whole publication - which contains around 350 nested folders called �Photography�. This causes it to crash error message -108 (out of memory). I�ve set the memory of the app to 10000k, and anyway the same thing happens under OSX.

This is the script:

set theFolder to choose folder with prompt "Select the folder to check"
set FileList to {}
set newFileList to {}

tell application "Finder"
	try
		set folderList to (folders of entire contents of theFolder whose name is "Photography")
	on error errMsg number errNum
		display dialog errMsg & return & errNum
	end try
	repeat with currentFolder in folderList
		set FileList to FileList & (files of currentFolder whose name contains ".tif")
	end repeat
end tell

copy (choose file name) as string to dataFile
open for access file (dataFile) with write permission

repeat with theFile in FileList
	write ((theFile as alias) as text) & return & return to file (dataFile) starting at eof
end repeat
write ((length of FileList) as text) & " items found" to file (dataFile) starting at eof
close access file (dataFile)

My question: Is there some way of getting around this problem, or do I need to break the process down into smaller steps, i.e. avoiding the �folders of entire contents� command? Does anyone have a sample script of a loop that drills down through folders - I just can�t get my head around the concept!

Thanks for your help…
Emma Glaisher
Leeds, UK

Something like this might work:

property fileList : {}

set theFolder to choose folder with prompt "Select the folder to check"
copy (choose file name) as string to dataFile

traverse(theFolder)

set s to ""
repeat with itm in fileList
	set s to (s & itm as text) & return
end repeat
set s to s & (count of fileList) & " items found"
write s to dataFile

on traverse(f)
	tell application "Finder" to set lst to (folders of f whose name is "Photography")
	repeat with photoFolder in lst
		traverse(photoFolder)
	end repeat
	
	tell application "Finder" to set lst to (files of f whose name ends with ".tif")
	set fileList to fileList & lst
end traverse

Actually, I wasn’t aware of either “choose file name” or “entire contents of”, so I learned something here. I’m reading now, though, that “entire contents of” is unreliable.
Also, something that might speed things up a little is gathering all the file names into a string or list and then writing one time.
Hope that helps …
-Dan

thanks, that looks interesting. Scary self-referential procedure!

I found a workaround which was fine in OS 9 but fell down in OSX - I guess it’s the ‘entire contents’ that I need to get rid of. What I did was copy the top level of folder names into a list, then work through it doing the previous script to one folder at a time, which worked much better… until it met OSX.

I’ll take a good look at your neat-looking script soon.
Thanks

Just a minor adjustment, now that I glance at it. I say

copy (choose file name) as string to dataFile 

but that won’t work with

write s to dataFile 

down below, so I should have removed “as string” from the “copy…” line. (I’m assuming “choose file name” returns a file alias.) I know that’s not really what your question is, though…

Getting every something of ‘entire contents of …’ was rather dodgy before OS X. It was a Finder memory problem, so increasing the memory of the script application wouldn’t help. It was wasn’t so bad when setting, since the Finder didn’t physically have to gather all the info to return to the script.

I think dant’s script only traverses folders called “Photography”. I’d guess that what Emma actually needs is to traverse every folder in the hierarchy, but only to return “.tif” files from “Photography” folders.

main()

-- Main code in a handler to keep the variables local.
-- Avoids long lists of files being saved back into the script file.
on main()
	set theFolder to choose folder with prompt "Select the folder to check"
	
	set theAliases to traverse(theFolder)
	set theAliases to theAliases & (((count theAliases) as string) & " items found")
	
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to return
	set theText to theAliases as string
	set AppleScript's text item delimiters to astid
	
	set dataFile to (choose file name with prompt "Save the data file as?") -- (new file with prompt ...) before OS 9
	set fRef to (open for access dataFile with write permission)
	try
		set eof fRef to 0
		write theText to fRef
	end try
	close access fRef
end main

on traverse(f)
	tell application "Finder"
		if f's name is "Photography" then
			try
				set lst to (files of f whose name ends with ".tif") as alias list
			on error
				set lst to (first file of f whose name ends with ".tif") as alias as list
			end try
		else
			set lst to {}
		end if
		
		set subFolders to folders of f
		repeat with thisSubfolder in subFolders
			set lst to lst & my traverse(thisSubfolder)
		end repeat
	end tell
	
	return lst
end traverse

Nigel, thanks! I worked through Dant’s solution, and unfortunately it didn’t handle the fact that the Photography folders are nested 3 down.

I’ll try your script at work on Monday. Meanwhile I hard-wired the script to go down 3 levels, which removed the need for ‘entire contents’ (pity, it was SO useful!). It’s inelegant but I hope it will work - it’s ok on OS9 but still to be tested on OSX.

Yikes…that’s how I meant to write it, but looking back at the original post I now understand what was really being requested…sorry. I think Nigel’s version looks good…

  • Dan

Mine searches through the entire hierarchy on the assumption that “Photography” folders could turn up anywhere. If Emma’s hierarchy is regular enough that a script can be hard-wired to get to the required level, that could be the way to go. Knowing where to look makes the search potentially much faster. :slight_smile:

If all the subfolders of the root are known to have identical hierarchies, but the position of the “Photography” folders within them isn’t known at the time of writing the script, you could use recursion on the first subfolder to find its “Photography” sub-subfolder, deduce the path difference, and use that as “hard-wiring” with the other subfolders.

Yes Nigel, my photography folders are always at the same level, so I’ve written the script to work down to the level, and it works fine on OS9.

I’ve posted the next dilemma to the OSX forum…

thanks, all!