GUI Scripting

Scripting Forums AppleScript | Mac OS X
,
#1

Hi,

I’m trying to use UI Browser to create som Scripts to login to som different pages,

Info from UI Browser
“application “Safari”
standard window “Welcome to Gmail” (1)
group 3
scroll area 1
web area 1
group 21
text field 1”

“application “Safari”
standard window “Welcome to Gmail” (1)
group 3
scroll area 1
web area 1
group 23
secure text field 1”

“application “Safari”
standard window “Welcome to Gmail” (1)
group 3
scroll area 1
web area 1
group 26
button “Sign in” (1)”

I guess that this code should be inserted inte something like this

tell application “Safari”
activate
tell application “System Events”
tell process “Safari”
GUI CODE
end tell
end tell

But I wonder hos should the GUI) code be written as copy paste doesn’t work, also how can I typ for example adamsen87 in the script so that it becomes text field 1?

//Adam

#2

Adam,

Heres something that might help you get started. I had to search a little to get the “UI element” thing. This is part of what I used to get to the login section of my web mail account. I started off just getting the "properties of " various parts of the window in Safari (window 1 or window “Web Mail”. I was then able to use the info in the event log to get the text field of the password for the login. Of course, as many people point out, its probably not a good idea to have your password in your script (unless maybe you’re the only one that uses your computer and the script). The first script is what I used to get info from the Safari window. The second is what I used to input text into the text field. To get info about buttons try using "get properties of every button of " in a loop through the ui elements, groups and such.

Hope this helps some.

tell application "Safari"
	tell application "System Events"
		tell process "Safari"
			set x to count every group of UI element 1 of scroll area 1 of group 3 of window 1
			repeat with i from 1 to x
				get every UI element of group i of UI element 1 of scroll area 1 of group 3 of window 1
			end repeat
		end tell
	end tell
end tell

This is the script to input text into a text field.

tell application "Safari"
	tell application "System Events"
		tell process "Safari"
			set value of text field 1 of group 17 of UI element 1 of scroll area 1 of group 3 of window "Web Mail" to "Hey"
		end tell
	end tell
end tell

PreTech

#3

Thanks, it works on some pages but for example Gmail says that nothing has been written in it, is it possible to simulate an enter key to login?

Sorry if Iäm asking stupid questions:/

//Adam

PS. If anyone has got one that works with Gmail please post it

I’ve tried this but that maybe doesn’t work

tell button “Sign In” of group 3 of UI element 1 of scroll area 1 of group 3 of window “Welcome to Gmail” to perform action

#4

If you’re using Prefab’s UI Browser (highly recommended), once you locate a particular button, popup, etc. it will give you the reference via the “AppleScript” popup. It can also do keyboard references (like keyboard shortcut typing), as well as the complete tell block.

I don’t have any experience with gmail, or I’d do an example. But here’s a TextEdit UI script for an example, maybe give you some ideas:


tell application "TextEdit"
	tell application "System Events"
		--NOTE: All delays fine-tuned, do not remove or script acts very oddly
		
		tell process "TextEdit"
			-- move cursor to start of file
			keystroke (ASCII character 30) using {command down}
			
			--initiate Find
			keystroke "f" using {command down}
			delay small_delay
			keystroke search_string --enter what to find
			keystroke return --start Find
			delay small_delay
			keystroke "." using {command down} --close Find dialog
			
			--use right-arrow to move over to the field data (intentionally capture leading space)
			--(for my script I had a fixed-format text file so I could make many cursor-movement assumptions)
			keystroke (ASCII character 29) --undoes found selection
			keystroke (ASCII character 29)
			
			--highlight data (highlight to end of line, then unhighlight paragraph break)
			keystroke (ASCII character 29) using {command down, shift down}
			keystroke (ASCII character 28) using {shift down}
			
			--copy to clipboard
			keystroke "c" using {command down}
			delay small_delay
		end tell
	end tell
end tell

This is a handler for a solution I’m working on that accesses Find to hunt-up data and return it via the clipboard. Note the use of “delay.” I’ve slowly learned that many UI scripts fail because since the OS is literally clicking stuff, only alot faster than any human could, that the appropriate windows and dialogs are not always ready, or menus and menu bars refreshed. I used a variable for delay so I could fine-tune it to keep the delay to a minimum. You don’t want to know how long it took me to do that snippet above. :stuck_out_tongue:

#5

You can use this:

tell application "Safari"
	activate
	tell application "System Events"
		tell process "Safari"
			tell window "Untitled" to keystroke return -- "Untitled" is the name of the window. You could also use window 1.
		end tell
	end tell
end tell

PreTech

#6

Thanks both of you used something from all of you :slight_smile:

//Adam

If anyone else wants t use a script to log into Gmail


tell application "Safari"
	activate
	tell application "System Events"
		tell process "Safari"
			tell text field 1 of group 21 of UI element 1 of scroll area 1 of group 3 of window "Welcome to Gmail" to keystroke "user_name" -- Change to your User Name
			tell window "1" to keystroke tab
			tell text field 1 of group 23 of UI element 1 of scroll area 1 of group 3 of window "Welcome to Gmail" to keystroke "password" -- Change to your password
			tell window "1" to keystroke return
		end tell
	end tell
end tell

One more question :stuck_out_tongue: is their anyway to store the password in a cryptic way so that you can’t read yourself to it or is it just to right keystroke 0, keystroke 2, keystroke 0?

//Adam

#7

I’m not sure if storing an encrypted password is possible or not. Someone more versed will have to answer that question.:confused:

PreTech

#8

I could throw out a new question now then :stuck_out_tongue:

Is there any way to write so that a function waits to go until the homepage is finished loading


	set the URL of document 1 to "http://www.gmail.com/"
	delay 2
	my login_gmail()

For example on this one the dely should vary as the homepage takes a different amount of time to laod every time and needs to load completly for the next function to work.

It says Loading “Welcome to Gmail” shouldn’t it be possible to write something like

when Loading isn’t there continue or is that way to advanced for it’s quite simple and not really that important function

//Adam

#9

There are a couple of ways to wait for Safari:


tell application "Safari"
	open location "http://www.macosxhints.com/"
	repeat until (name of window 1 as text) is not "Untitled"
		delay 0.1
		beep
	end repeat
end tell

-- Alternatively, this

if my page_loaded(50) then display dialog "Done"

on page_loaded(timeout_value)
	delay 5
	tell application "Safari"
		repeat with i from 1 to the timeout_value
			if (do JavaScript "document.readyState" in document 1) is "complete" then return true
			delay 1
		end repeat
	end tell
	return false
end page_loaded
#10

I think that the one at the top seems to be the best one but the problem is that I want it to wait until it says

Welcome to Gmail and not Loading “Welcome to Gmail”.

something like does not contain Loading

//Adam

#11

If you have follow my post and remember the updated version to include ‘Presets’

Inside the tell block of outline 1 we ask for ‘class of UI elements’ and it give us ‘row’

If we then type this…
get static text of UI element of row 6
→ {{static text “Fast 1080p30” of UI element “Fast 1080p30” of row 6…

Now we could use: ‘tell UI element of row 6 inside a tell statement like this’

tell UI element of row 6
	get the name
	--> {"Fast 1080p30"}
end tell

It will output the string: → {“Fast 1080p30”}

Now we also could get every string name of row.

tell UI element of every row
        get the name
end tell

→ {{“General”}, {“Very Fast 1080p30”}, {“Very Fast 720p30”}, {“Very Fast 576p25”}, {“Very Fast 480p30”}, {“Fast 1080p30”}, {“Fast 720p30”}, {“Fast 576p25”}, {“Fast 480p30”}, {“HQ 1080p30 Surround”}, {“HQ 720p30 Surround”}, {“HQ 576p25 Surround”}, {“HQ 480p30 Surround”}, {“Super HQ 1080p30 Surround”}, {“Super HQ 720p30 Surround”}, {“Super HQ 576p25 Surround”}, {“Super HQ 480p30 Surround”}, {“Web”}, {“Devices”}, {“Android 1080p30”}, {“Android 720p30”}, {“Android 576p25”}, {“Android 480p30”}, {“Apple 2160p60 4K HEVC Surround”}, {“Apple 1080p60 Surround”}, {“Apple 1080p30 Surround”}, {“Apple 720p30 Surround”}, {“Apple 540p30 Surround”}, {“Apple 240p30”}, {“Chromecast 2160p60 4K HEVC Surround”}, {“Chromecast 1080p60 Surround”}, {“Chromecast 1080p30 Surround”}, {“Amazon Fire 2160p60 4K HEVC Surround”}, {“Amazon Fire 1080p30 Surround”}, {“Amazon Fire 720p30”}, {“Playstation 2160p60 4K Surround”}, {“Playstation 1080p30 Surround”}, {“Playstation 720p30”}, {“Playstation 540p30”}, {“Roku 2160p60 4K HEVC Surround”}, {“Roku 1080p30 Surround”}, {“Roku 720p30 Surround”}, {“Roku 576p25”}, {“Roku 480p30”}, {“Xbox 1080p30 Surround”}, {“Xbox Legacy 1080p30 Surround”}, {“Matroska”}, {“H.265 MKV 2160p60”}, {“H.265 MKV 1080p30”}, {“H.265 MKV 720p30”}, {“H.265 MKV 576p25”}, {“H.265 MKV 480p30”}, {“H.264 MKV 2160p60”}, {“H.264 MKV 1080p30”}, {“H.264 MKV 720p30”}, {“H.264 MKV 576p25”}, {“H.264 MKV 480p30”}, {“VP9 MKV 2160p60”}, {“VP9 MKV 1080p30”}, {“VP9 MKV 720p30”}, {“VP9 MKV 576p25”}, {“VP9 MKV 480p30”}, {“VP8 MKV 1080p30”}, {“VP8 MKV 720p30”}, {“VP8 MKV 576p25”}, {“VP8 MKV 480p30”}, {“Production”}, {missing value}, {missing value}}