Safari 6.0 AXShowMenu Defect

I have a problem with an AppleScript for Safari 6.0 that attempts to Peform Action AXShowMenu for an off-screen control. Here’s an example from the AppleStore home page http://store.apple.com/us


activate application "Safari"
tell application "System Events" to tell process "Safari"
	perform action "AXShowMenu" of static text "Education Store" of UI element "Education Store" of group 1 of list 2 of UI element 1 of scroll area 1 of group 1 of group 1 of group 4 of window "Official Apple Store - Buy iPad mini, iPad, iPhone 5, MacBook Pro with Retina display, Mac mini, and more. - Apple Store (U.S.)"
end tell

Open the AppleStore home page in safari and run the above script. It should open a pop-up menu for the Education Store control on the right of the window under more stores.

Now resize the AppleStore window so that it is so short that the Education Store control is below the bottom. Run the script again and it will not open the correct pop-up. That’s expected since the control is off-page.

Now scroll the window so the control is visible. Again run the script. When I do this the wrong pop-up is displayed as if the control was still off-screen. If i manually right click on the control I get the correct pop-up.

In versions of Safari before 5.0.5 the script worked correctly. That is any control visible after scrolling would work correctly. Since then I get this unwanted behavior. Can someone confirm this? I get it on both my Mac Pro and my MacBook Pro.

Does anyone know a fix or workaround?

Rich

Are you sure that GUI scripting may apply to an offscreen item ?

As far as I know, it’s designed to mimic manual user’s action and those apply only upon visible objects.

Yvan KOENIG (VALLAURIS, France) jeudi 27 décembre 2012 10:18:44

Yvan,

You asked if I was sure that GUI scripting may apply an offscreen item. No, just the opposite. I’m sure it doesn’t. However, with Safari 5.0.5 and earlier, if you scrolled the screen so that the offscreen control moved on-screen, then the AXShowMenu would work. Now scrolling doesn’t do any good. That’s the bug I think I’ve discovered. In my original message, seen the paragraph beginning “Now scroll the window so the control is visible.”

Rich

Hi.

Applications aren’t designed to be GUI Scripted. A change in a GUI structure is entirely for operational reasons and is only a “bug” if the wrong things then happen when a user clicks things on the screen. A script relying on GUI Scripting has to take its chances on the structure not changing or else (if possible) take active steps to deal with it if it does.

The question should be “How do I reliably get this menu to open?” (a different mindset from “Does anyone know a fix or workaround?”) or possibly “How do I script the effect of clicking such-and-such an item in the menu?”

Hello.

If the menu is on a webpage, then I think some javascript may be the work around or fix if you like. I am not sure if it will bring an element on screen, but I think it will manipulate the offscreen element in the DOM-tree and finally bring forward what you sought.

This solution uses the tab order to move and select the Ui Element. (Very simple usage of System Event’s UI scripting capabilities.

MacScripter / Button click action in Safari webpage.

This one goes in and manipulates the DOM - tree with Javascript. Look at post #11 and beyond.

MacScripter / Safari says it has no popup in a sheet which has…?

Hello

Small problem.


set wName to "Official Apple Store - Buy iPad mini, iPad, iPhone 5, MacBook Pro with Retina display, Mac mini, and more. - Apple Store (U.S.)"
tell application "Safari"
	activate
	open location "http://store.apple.com/us"
	repeat until exists window wName
		delay 0.2
	end repeat
end tell
tell application "System Events" to tell process "Safari"
	tell window wName
		class of UI elements
		{button, button, button, group, group, group, button, static text, tool bar}
	end tell # window
end tell

As you see, the script find three groups in the window.

I’m not really surprised that it can’t trigger « group 4 » as is coded in the original script

Yvan KOENIG (VALLAURIS, France) vendredi 28 décembre 2012 13:04:02

Nigel is right. I’m still upset because a GUI trick that used to work no longer works.

More important is the issue of properly framing the question. My real question is how to automatically invoke the “Download Linked File” function from a control. In some cases I can do this by opening the associated URL in Safari. In the remaining cases I’ve used this GUI trick.

Whoops! My bad. I always run Safari with the Bookmarks bar visible. That adds another group. In my original code I should have used group -1 instead of group 4.

Thanks for trying to help.

Rich

Maybe this simple code will do the trick :


set wName to "Official Apple Store - Buy iPad mini, iPad, iPhone 5, MacBook Pro with Retina display, Mac mini, and more. - Apple Store (U.S.)"
tell application "Safari"
	activate
	open location "http://store.apple.com/us"
	repeat until exists window wName
		delay 0.2
	end repeat
	open location "http://store.apple.com/us-hed/findyourschool?aid=AOS-US-Edu-NavLink"
end tell

Yvan KOENIG (VALLAURIS, France) vendredi 28 décembre 2012 18:40:12

A couple of ideas arising from this discussion. Both assume the use of Safari:

Return all the URLs for a given link text on http://store.apple.com/us:

set linkText to "Education Store"

set linkURLs to paragraphs of (do shell script "osascript -e 'tell app \"Safari\" to get source of document 1' | sed -En 's|&|\\&|g ; s|…|.|g ; /" & linkText & "/ { s|^[^<]+(<[^a][^>]*>)*<a href=\"([^\"]+).+|\\2| ; s|^/|http://store.apple.com/| ; /^http/ p ; }'")

Get all the static texts associated with the link text for GUI Scripting, regardless of page configuration:

set linkText to "Education Store"

tell application "Safari" to activate

set staticTextListText to (do shell script ("osascript -e 'tell application \"System Events\" to tell application process \"Safari\" to get entire contents of front window' -s s | grep -Eo 'static text \"" & linkText & "\"[^\"]+(\"[^\"]+\" of [^\"]+ )+\"System Events\"' | sed -n '1 h ; 2,$ H ; $ { g ; s/\\n/, /g ; s/.*/{&}/p ; }'"))

tell application "System Events" to set staticTextRefs to (run script ("tell app \"System Events\" to get " & staticTextListText))