Avoiding 'var is undefined' errors

Scripting Forums AppleScript | Mac OS X
,
#1

Hey,

I am using a small script I wrote to download huge sections of websites, but sometimes Safari can’t load the page fast enough to give me the source back (using the Safari Save script code I found here in code exchange). I added a delay of 10 seconds and don’t want to add more, cause for more than 1000 pages, that would become a huge delay. Also, only 1 in 100 or more pages times out.

When it doesn’t return the source to the ‘s’ variable, it always gives an error ‘variable s is not defined’ and the script blocks.

Is there a way to catch that ? I tried
if(s is not defined) then … and … if (s is undefined) then…
but I still get the undefined errors.

BTW, this is the Safari Save script, slighty adjusted:
tell application “Safari”
make new document at end of documents
set URL of document 1 to this_URL
end tell
delay 15

set f to "Hard Disk:" & page_name & ".html"
set my text item delimiters to {"http://"}
tell application "Safari" to set s to source of front document
open for access file f with write permission
write "" & s to file f
close access file f

Thanks in advance,

Thomas

#2

You can substitute this line with this one (within the Safari’s tell block):

repeat
     do JavaScript "document.readyState" in document 1
     if result is "complete" then exit repeat
end repeat

And you can also substitute all Safari stuff with shell’s “curl” command, which will allways wait for the entire loaded source…

set f to (path to desktop)
set page_name to "bbs.applescript.net"
do shell script "cd " & quoted form of POSIX path of f & "; curl 'http://bbs.applescript.net/' -o " & quoted form of (page_name & ".html")
#3

I tried using curl, but then the terminal was hanging after 20 to 30 URLs.
The delay is used to avoid looking like a Pagesucker app, which gets blocked by some of the sites I want to get pages from.

Thanks for the tips though !

I just fixed the undefined thing using a try… on error section. It quites Safari if the page isn’t loaded in 15 seconds and retries, with a 40 seconds delay instead of the 15. I do the quiting because I have seen browsers that just don’t load stuff if they don’t do it immediatly.

#4

This doesn’t seem to work in my Safari… It just flashes through all pages, without loading any. I just see 100 safari windows open and close, one after another… I also tried with a delay, like the example on Apple’s site, but it also didn’t work.

#5 
do JavaScript "document.readyState" in document 1

This returns for me “complete” when the page is entire loaded… Maybe you have not active javascript in your safari prefs?
Or maybe it returns an error because it is defined as a “missing value”, then “result” is undefined…
Perhaps you can try this:

repeat
     set x to "whatever"
     try
          set x to (do JavaScript "document.readyState" in document 1)
     end try
     if x is "complete" then exit repeat 
end repeat