getting source code from safari reliably

Scripting Forums AppleScript | Mac OS X
#1

I’m trying to periodically get the source code from a web url using Safari, and basically i’m running into two issues:

  • if the document hasn’t finished loading yet, i get no source, after that i see the window loading in safari, so i looks like i need a way to have the script keep checking if the source is available (loaded completely), and then get it from Safari

  • i periodically check the source of the page, leaving the window open, but when i do the above repeatedly, then instead of no source on the first load, i get the previously loaded source on each consecutive load, because at that time that is the available source, while the “refresh” is still in progress.

I looked in Safari’s dictionary, but there does not seem to be a reload command.

Does anyone know of a reliable way to get a page’s source? Or can recommend another browser that does this well?

Btw I just need the bare source from a url, nothing else, so if there are any shell commands that can do this without even needing a browser, i’d be happy to use them.

this is the bit of script i have now



set query_url to "http://some.site.com/index.php?i_need=this_source"

tell application "Safari"
		
		-- this opens a window or reloads the same url in the same window
		open location query_url

		set the_window to document 1
			
		-- get source
		-- this gives an error when it is not yet available
		try
			set query_url_source to source of the_window
		on error
			set query_url_source to "Not yet!"
		end try
		
	end tell
#2

If you just want to get the source of a website, I think you could use this:

do shell script "curl 'http://awebsite.com'"

hope it works,
ief2

#3

Hi,

cURL can do this


set query_url to "http://some.site.com/index.php?i_need=this_source"
set query_url_source to do shell script "/usr/bin/curl " & quoted form of query_url
#4

Thanks both, works as advertised!

What is the difference between using “/usr/bin/curl” or just “curl” as in the first example?

#5

I prefer to specify always the full path to the executable. Even If the file /private/etc/paths is missing or it does not contain /usr/bin, the script works in any case.