Running script multiple times. Each time longer, and longer.

Hi
I have 3 scripts that I am using to catalog files into Portfolio. I have to catalog ( 3 x 12 x 26 ) folders with these scripts. I run each script once on each folder. Each time I run the three scripts it seems to take longer and longer. I have a feeling that it might have something to do with applescript remembering the globals I’m setting and those taking up too much memory. Any ideas anyone?? I have 2 weeks to catalog some 10,000 files and this lag is killing me.

Thanks
Matt

HI Matt…

Can you give us some more details? What are you trying to catalog? What data are you collecting? Can you share the script you have so far?

… and yes, it does sound like some variable is sticking around when you re-run the script. The best work around to that is to set your variables to be empty at the start of your script. The quick ‘test’ is to make a minor change in your script, and save it again… that resets the variables.

Cheers,

Dave

Thanks for the response! Making a minor change (as you suggested) and rerunning the script seems to make it go a little faster. I think there is a bigger problem but I can’t put my finger on it.

Let me explain the scenario a little bit better. On our server we have 3 years worth of newspaper ads in QuarkXPress documents, PDF Files, EPS files, Tiff files and Jpeg files. These are seperated by year, by month and finally by the starting letter of the advertiser name.

Script 1 (LogMaker)
Traverses a directory(including its subdirectories) and finds out a bunch of information about every file in there and writes all that information to log files in csv style.

Script 2 (Quark Doc Fixer) - The slow one
This script is the troublemaker. It runs great the first time through but every time after that it gets slower, and slower to the point where it takes applscript 2-3 seconds per command to execute. This file loads approximately 10 additional scripts of varying length to perform its work (i’ve essentially written a bunch of handler libraries in seperate files). It loads these script libraries at the beginning of the script with a series of “load script file” commands. They look like this

Script 3 (Other Doc Fixer) - Kinda slow but throws errors much later than the second.
This script processes all those other files except for the Quark Document.

set path_findClient to the_desktop & “Handlers:TextEdit:findClient.scpt”
set path_logFiles to the_desktop & “Handlers:TextEdit:logFiles.scpt”
set path_readCSVLog to the_desktop & “Handlers:TextEdit:readCSVLog.scpt”
set path_delimitString to the_desktop & “Handlers:Strings:delimitString.scpt”
set path_findDate to the_desktop & “Handlers:Date:findDate.scpt”
set path_inkyIssues to the_desktop & “Handlers:Date:inkyIssues.scpt”
set path_getContainerDate to the_desktop & “Handlers:Finder:getContainerDate.scpt”
set path_pictureUsage to the_desktop & “Handlers:Quark:pictureUsage.scpt”
set path_searchPictures to the_desktop & “Handlers:Quark:searchPictures.scpt”
set path_saveImgProps to the_desktop & “Handlers:Quark:saveImgProps.scpt”
set path_catalogQuarkDoc to the_desktop & “Handlers:Quark:catalogQuarkDoc.scpt”
set path_catalogQuarkArt to the_desktop & “Handlers:Quark:catalogQuarkArt.scpt”
set path_pictureBoxes to the_desktop & “Handlers:Quark:pictureBoxes.scpt”

set find_client_obj to load script file path_findClient
set log_files_obj to load script file path_logFiles
set delim_str_obj to load script file path_delimitString
set find_date_obj to load script file path_findDate
set inky_issues_obj to load script file path_inkyIssues
set cont_date_obj to load script file path_getContainerDate
set pic_usage_obj to load script file path_pictureUsage
set search_pic_obj to load script file path_searchPictures
set save_props_obj to load script file path_saveImgProps
set cat_quark_obj to load script file path_catalogQuarkDoc
set cat_art_obj to load script file path_catalogQuarkArt
set pic_boxes_obj to load script file path_pictureBoxes

(i can hear the snickering now…)
The total lenght of these files is probably aroun d 3-4 thousand lines. This script (like I said) works great the first time I run it, but each subsequent time I run it on a new directory it gets slower…and slower…and slower. If I run the script enough times without saving it as a different file name I’ll eventually get the error “Applescript: There are too many files open”. I’m assuming this has something to do with AppleScript keeping its memory of all those scripts I’ve loaded sound of CS folks shuddering but I’m not quite sure how to reuse AppleScripts knowledge of all these script objects. Could I tell it to save them in a different file after it loads them the first time then point each subsequent run of the script to those stored objects?? [/b]

Hi Matt,

I’m don’t know much about libraries, but it seems to me that this would save time. You can just load the libraries on the first run with something like this:

property first_run : true
local dp

set dp to (path to desktop) as string
if first_run then
set S1 to (load script file (dp & “RunCounter”))
set first_run to false
beep 3
end if
S1’s IncrCount(7)

By declaring the variable dp as local, its value will not be saved.

Another thing is this. If you have properties and statements in the run handler of the loaded script, then you can have changes saved or not depending on how you run the script. For instance, if you use ‘run’, then properties will be saved in the loaded script:

property first_run : true
local dp

set dp to (path to desktop) as string
if first_run then
set S1 to (load script file (dp & “RunCounter”))
set first_run to false
beep 3
end if
–S1’s IncrCount(7)
run S1

Using ‘run script’ will not save changes in the loaded script’s property:

property first_run : true
local dp

set dp to (path to desktop) as string
if first_run then
set S1 to (load script file (dp & “RunCounter”))
set first_run to false
beep 3
end if
–S1’s IncrCount(7)
run script S1

I used this for the library script although it has statements in the run handler and a property just for the second part of this post.

property c : 0

on run
set c to IncrCount(c)
return c
end run

on IncrCount(cl)
set cl to cl + 1
display dialog cl
return cl
end IncrCount

gl,

Not at all. It’s AppleScript’s fault for not providing a proper library import mechanism.

No, ‘too many files open’ is a filesystem complaint - caused by [e.g.] opening files with ‘open for access’ and not calling ‘close access’ to close them again when done - and unconnected with in-memory script objects. So I really doubt it has anything to do with your use of libraries, and it’s probably not connected to your slowdown problem either.

The accumulative slowdown could be down to something like a global variable or property containing a list or string to which you’re appending/concatenating values - the bigger the list or string gets, the slower it becomes. It’s a common gotcha. This is where knowledge of basic algorithms and how to calculate efficiency becomes important. Furthermore, properties and globals will persist between runs until the script is recompiled, which would cause the slowdown to become worse with each run.

Therefore, look over your code for any suspicious globals/properties where something like this might be happening. Also, get yourself a timing osax and start profiling your code to narrow down the area where the performance problem occurs.

HTH