I have a simple script that parses lines of a text file and puts different parts of the lines of the text file into items of a list, then writes the list to disk. I always read the files from disk into a variable and then write the variable back to disk. Each item contains 10-50 characters of text.
It works fine up until the list gets to around 1530 items and the list file grows to around 40k. Then the list items get corrupt all at once.
Should I be declaring large variables up front to avoid this?
There’s a limit on the number of text items in a list. I don’t know exactly what the limit is, but I think it depends on the length of the text items also. There are several methods of overcomeing this limit.
One method is to read in a certain size of blocks of text or take blocks of paragraphs of the entire text. Are you getting paragraphs from the text and working with that? This is probably what you’re doing since you’re reading the entire text into a variable. I think the best method to use, depends on what you’re doing with the text, because there may be some shortcuts.
Hi kel, thanks for the help. I’m starting with a text file that consists of lines delimited by paragraph. Each line then contains items delimited by tabs. So I take a paragraph (line) at a time and then take each tab delimited item and send it to the “end” of the list.
So this sounds like what you’re thinking I’m doing?
What I’m doing with the list is using it to note the path of a quicktime movie on the computer (item 1) and then set the next 5 items in the list that are related to the movie such as timecode ins and outs and other things. Then the process repeats. So every 7th list item is the path of a quicktime movie and the 5 items after that are attributes of the movie. I access the list in another AppleScript program that loads the quicktime movies and does stuff with its related items. So the item describing the movie’s path can be quite long and the other items around 12 characters or so. And then it goes nuts after a certain point (the items get filled with wrong parts of the original text file and so on)
Should I be using something other than a list to keep these items together?
set f to choose file
set t to read f
set i to -1
set m_list to {}
try
repeat
set i to i + 1
set m_start to i * 6 + 1
set m to paragraphs m_start thru (m_start + 5) of t
set def_tid to AppleScript’s text item delimiters
set AppleScript’s text item delimiters to return
set s to m as string
set end of m_list to s
set AppleScript’s text item delimiters to def_tid
end repeat
on error _err
set AppleScript’s text item delimiters to def_tid
display dialog _err
end try
return m_list
This will get every six paragraphs as text and place it in the list. See if you get an error. The number of list items will be six times less. If you get an error, then you can increase the number of paragraphs read or use another method where you just read six paragraphs at a time from the file.
Thanks again, kel. Unfortunately, it seems a list is limited by both the number of items and length of each item. I’ve been experimenting and I can get more items with shorter items and vice-versa. Seems it’s always the same ratio that causes the list to get garbaged. BTW, i’ve been using as my check for garbage the “choose from list” command to see if the list’s contents are garbled.
It seems that Applescript creates the list in a variable as large as you want - as examined by looking at the actual text that got stored in a variable written to disk, but AppleScript is limited in how it reads and uses that data. At some point the reading “rolls over” and the list items get shifted from item to item.
So, it seems that AppleScript cannot parse its own list properly if it gets too big. Can anyone confirm that?
Yeah, it’s a limitation on the value of the variable, but you can’t declare a variable to have a greater limit. You need to break down the text and just work with blocks or read blocks of text at a time. Why are you giving up? You just need to modify your scripts to read say one movie’s info at a time. Just pass one movie to your other script to process. i don’t know exactly how you’re doing that.
Hi Kel, I just edited that last post after experimenting some more. It still seems that if I send larger blocks of text to each list item then I’ll end up with fewer items available it the list - because each item is larger. I don’t see a way out. I’ll look at the script you suggested, but I’m still stumped for now. Seems like I just want to store more data than the list function can handle.
You’re probably writing your script wrongly. Here’s an example of how I might store the info I needed to a text file:
– writes QT info to file
set f to choose folder
tell application “Finder”
try
set movie_list to (every file of f whose file type is “MooV”) as alias list
on error
set movie_list to (first file of f whose file type is “MooV”) as alias as list
end try
end tell
set file_spec to (“” & f & “MovieInfo.txt”) as file specification
set ref_num to (open for access file_spec with write permission)
try
repeat with m in movie_list
tell application “QuickTime Player”
launch
open m
set m_info to {m as string}
tell front movie
set more_info to {name, data rate, duration, rate, scale}
end tell
end tell
set m_info to m_info & more_info
set def_tid to AppleScript’s text item delimiters
set AppleScript’s text item delimiters to tab
set write_string to m_info as string
set AppleScript’s text item delimiters to def_tid
write write_string to ref_num
write return & return to ref_num
end repeat
close access ref_num
on error _err
close access ref_num
display dialog _err
return
end try
I just jammed it out, so maybe it could be more efficient. What it does is take a folder the user chooses and writes (or overwrites) the selected info for each movie in that folder to a file in that folder. I made it so that each write ends with two carriage returns. This wil help in our reading the file for each movie.
Here’s how I might send the text for each movie to another script:
script QT
on run p
set t to p as string
display dialog t
end run
end script
set f to (choose folder)
set read_file to (“” & f & “MovieInfo.txt”) as alias – error if file does not exist
set ref_num to (open for access read_file)
try
repeat
set temp_t to read ref_num before return
read ref_num until return – move the file marker to after the blank line
run script QT with parameters temp_t
end repeat
close access ref_num
on error
close access ref_num
end try
Here, I used a script object, but you could easily send the text to another compiled script. The compiled script should return the new settings. I didn’t use any changing of movie properties because I don’t have QT Pro.
Hi Kel, thanks for the virtuoso coding! Unfortunately I’m way too green to really understand what you’re up to. What I’ve been doing to create this list is simply using the "set end of [list] to[item of data] and repeating until done. Each movie represents a clip in a video editing application like avid or final cut. The items are: 1: path of movie 2:reel number 3:TC start 4:TC end 5:name of clip in editing application 6: blank (for visual clarity when using the choose from list command - don’t really need this item) The list I’m getting these values from is a text file from either program. I then use the list later to reference a particular movie to its TC and reel number and clip name. The first repeat loop checks the movie path item for matches to previous entries so a movie isn’t imported twice if it appears on multiple lists.
Here’s the script to transfer the data from an Avid bin list to my AppleScript list. I know it’s probably laughably written but it works, just not with lists that generate more then 1500 list items or so… I know I could save some space in the list but 1500 items is so under what I really need to make this useful that it doesn’t seem as if I can use AppleScript for this…
thanks again for your help,
mark
property match : false
property indexfile : "Macintosh HD:Users:markPWB:Movies:movindex"
set indx to read file indexfile as list
set noitems to count every item of indx
set logfile to (choose file) as text
set lg to read file logfile as string
set logitems to count every text item of lg
set avidpath to choose folder with prompt "WHERE ARE THE EXPORTED QT REFERENCE MOVIES?" as text
set text item delimiters to tab
set x to 2 -- set the first item of the log to be checked
repeat
set itno to 1 -- set the first list entry to check
if x > logitems then
exit repeat
end if
repeat
if itno > noitems then
set match to false
exit repeat
end if
if ((avidpath as text) & (text item x of lg) & ".mov") = (item itno of my indx) then
set match to true
exit repeat
end if
set itno to itno + 6
end repeat
if not match then
set end of indx to (avidpath as text) & (text item x of lg) & ".mov" as text
set x to x + 1
--reel name
set end of indx to (text item x of lg) as text
set x to x + 1
--tc in
set end of indx to (text item x of lg) as text
set x to x + 1
--tc out
set end of indx to (text item x of lg) as text
set end of indx to "from Avid" -- filler for FCP Name
set end of indx to "" --blank line
set x to x + 2
else
set x to x + 5
end if
end repeat
(*
this is commented so it doesn't write people's hard drives
open for access file indexfile with write permission
write indx as list to file indexfile
close access file indexfile
*)
I think you’ll find (at some point) that you may run out of room with the string also. If it’s working fine for you now, that’s great, but Kel had a pretty good idea to read “chunks” of the file.
If the movie info is based on name, you can configure your script to read just that line and give it a reference number where it appears in your original file and base individual operations off of that. It’s kinda like using your original file as a big "prefs’ file.