I’m having a bit of a problem over here. I’m writing a script that will filter a certain log-file, extract all relevant data over a certain period of time and show the found data ordered in excel. The script is working fine, but there is one small problem: It works fine, as long as I run it with only a part of the log-file (a part, as in select a certain amount of lines from the log-file, copy them to a new .txt and run the script with that .txt). When I try to run the script with the complete log-file, I get an out of memory error and the script crashes.
here is the relevant code in the script where the .txt log-file gets read in:
tell application "Finder"
set varLogFile to choose file with prompt "Select a log file" default location "Macintosh HD:Users:che_mobile:Desktop:Automate Retrospect loggings:workfiles:logs:" as alias without invisibles
set fileRef to (open for access file varLogFile as string)
set listTotalLog to read fileRef as string
close access varLogFile
end tell
The script seems to crash on the line:
set listTotalLog to read fileRef as string
I also have to tell you that the log-files I’m trying to read in vary from 13 to 14,5 MB in size (I know, quite a lot for a simple .txt file), and that I only need the last 10000-15000 lines of it for the script, not the entire content. (But it’s crashing because I try to read in the entire content, I don’t know of any other way)
So, is there anyone out there who might know a solution to my problem (I don’t want to be copy-pasting a new text file each time I run the script, so I really have to get it working with the whole log-file). I’ve been thinking about creating the copy-pasted text file via running a shell command, but I’m really not that familiar with those things
probably it is not necessary to use open /close
and you can also omit the Finder tell block
set varLogFile to choose file with prompt "Select a log file" default location ((path to desktop) as string) & "Automate Retrospect loggings:workfiles:logs:" as alias without invisibles
set listTotalLog to read file (varLogFile as string)
It seems that the script itself can’t hold that much text. With the excerpt from your script, I can read half of a 38 MB text file, but not all of it and not both halves in the same run. Since your log file lines aren’t likely to be very long, and you only need the last 15000 or so of them, you could try reading just the second half of the file “ or less, if necessary “ like this:
set varLogFile to (choose file with prompt "Select a log file" default location (((path to desktop as Unicode text) & "Automate Retrospect loggings:workfiles:logs:") as alias) without invisibles)
set fileRef to (open for access varLogFile)
try
set halfway to (get eof fileRef) div 2
set listTotalLog to (read fileRef from halfway)
on error msg
display dialog msg
end try
close access varLogFile
set listTotalLog to text from paragraph -15000 to -1 of listTotalLog
In your script, you have this line:
set listTotalLog to read fileRef as string
Just in case you didn’t know, the ‘as string’ in this case is a parameter of ‘read’, not an AppleScript coercion. It makes the ‘read’ command interpret the file’s contents as string data. This is the default behaviour, so there’s no need to include it if the data are already in string form. But if the file contains Unicode text, say, and you want a string result, you have to read the data ‘as Unicode text’ and then coerce the result to string.
set listTotalLog to (read fileRef as Unicode text) as string
Thanks for the replies, guys. They where really helpfull.
By testing with your suggestions, I found the real reason why the script would go “out of memory”. It was not because it could not read in the log-file, but it was because it could not add that much lines to a list (as was happening in the next part of my script).
So by adding a repeat loop to the “add to list”-part and letting it go from (count text items in log-file) - 5000 to (count text items in log-file), I get the last 5000 lines in the log-file (and that’s enough to do the analysis with) AND I got rid of the “out of memory”-error ^^
But, as Nigel Garvey wrote, there is an character limit on the amount of text you can read. I think this includes how much memory the whole script takes and is about (3210241024 bytes, just guessing). So, you can only read about 31MB. On the safe side, I would read 1MB blocks. Something like this:
-- this short script can read up to about
-- 32MB (32*2^20 or 32*1024*1024 bytes)
property block_size : 1024 * 1024 -- bytes
--
set f to choose file
set ref_num to open for access f
set num_bytes to get eof ref_num
set num_blocks to (num_bytes div block_size) + 1
set t to ""
set c to 0
try
repeat num_blocks times
set t to t & (read ref_num for block_size)
set c to c + 1
end repeat
close access ref_num
on error err_mess
close access ref_num
display dialog err_mess buttons {"OK"} default button "OK"
return c
end try
length of t
Note that the ‘for’ parameter in the ‘read’ command, can be used. ‘read’ will read to end of file if the amount is greater than the number of bytes left.
Edited: I edited the text concatenation and read f.
Edited: now, the script is only reading 15MB before it errors.
