Does anyone here have any code capable of retrieving ALL of the UIElements from an app window? I am aware that I can use…
set localUIElementList to entire contents
…but with some applications this is insufficient. Only currently rendered UIElements are returned. If you have to scroll down to reveal additional UIElements then ( some of ) those hidden UIElements are not returned.
An example of this is the “Wallpaper” pane of the System Settings app. This pane returns a variable number of UIElements depending on the scrolled position of the pane, the revealed status of the sub-groups etc.
Open System Settings/Wallpaper and run this several times while at different scrolling positions to demonstrate this. EDIT: In order to retrieve them all, also unfold all controls that read “Show All (##)” to reveal all the controls
tell application "System Events"
tell application process "System Settings"
activate
tell window 1
tell group 2 of splitter group 1 of group 1
set localUIElementList to entire contents
return length of localUIElementList
end tell
end tell
end tell
end tell
I can reveal the sub-groups by script. I can script the AXScrollToVisible to bring additional elements into retrievable range. I can gather the UIElements ‘entire contents’ after scrolling I get a new unique set, but I can’t compare them to the other already-listed UIElements with something like…
if localUIElementList does not contain curLocalUIElement then set the end of localUIElementList to curLocalUIElement
’
…because "“Can’t make «class asDB» id (application "System Events") of «class sgrp» 1 of «class sgrp» 2 of «class splg» 1 of «class sgrp» 1 of window "Wallpaper" of «class pcap» "System Settings" of application "System Events" into type vector.”
Which makes sense with UIElement references. But UIElements do not have any unique keys to filter by. Checking them against already listed UIELements will be tedious requiring comparing their positional hierarchies.
Any ideas?
Did you try reference to entire contents?
EDIT:
The entire contents is a reference .
Try
Get entire contents of …… as list
technomorph, Thanks for the suggestions. I have determined the issue really is just that the System Settings panes only know about currently displayed items ( and some unknown quantity of out-of-frame content. ) and I have found a method that can overcome this. I am aware of the old error dialog parsing trick. I use it in my System Settings library constructor. Here I don’t think it will help me.
The vast majority of the System Settings panes don’t exhibit this issue. Screen Savers and Wallpaper are the two exceptions due to the large amount of options available on those two panes. These two panes are among the last holdouts in my efforts to programmatically create handlers for all of the controls in System Settings.
I need to be able to programmatically scan each wallpaper category for available wallpapers. Once I have this info I can create handlers that will open the wallpaper pane, open the appropriate category, and then select the specified wallpaper.
If anyone else is interested, the following code will…
- collapse all categories “Show Less” in the wallpaper pane.
- Expand each category, one at a time, listing the category’s available wallpapers
- Return a record of wallpaper categories and their available wallpaper names.
Not the simplest process, but this discovery would only be required once per macOS update.
tell application "System Settings"
activate
repeat
try
reveal pane id "com.apple.Wallpaper-Settings.extension"
tell application "System Events"
tell application process "System Settings"
activate
if name of window 1 is "Wallpaper" then exit repeat
end tell
end tell
end try
delay 1
end repeat
end tell
tell application "System Events"
tell application process "System Settings"
set frontmost to true
tell window 1
tell scroll area 2 of group 1 of group 2 of splitter group 1 of group 1
set allWallPapersList to {}
set wallPaperCategoryList to {}
tell UI element 1 to perform action "AXScrollToTop"
UI element 1
set wallpaperCategoryCount to (length of (get (buttons of UI element 1))) --Count wallpaper categories "Show All" / "Show Less" buttons.
--Click all "Show Less" buttons to collapse all category groups.
repeat with wallpaperCategoryIndex from 1 to wallpaperCategoryCount
if value of attribute "AXIdentifier" of button wallpaperCategoryIndex of UI element 1 contains "Show Less" then
tell button wallpaperCategoryIndex of UI element 1
perform action "AXScrollToVisible"
perform action "AXPress"
end tell
end if
end repeat
--Click each wallpaper category one-at-a-time and gather their displayed buttons.
repeat with wallpaperCategoryIndex from 1 to 8 --There are 8 wallpaper categories.
tell button wallpaperCategoryIndex of UI element 1
set thisCategoryName to value of attribute "AXIdentifier"
set thisCategoryName to (words 3 thru -1 of thisCategoryName) as text
perform action "AXScrollToVisible"
perform action "AXPress"
end tell
--Get a list of buttons visible
set buttonsList to buttons of UI element 1
--Get first listing of the buttons / AXIdentifiers.
set buttonsAXIdentifiersList to {}
repeat with thisButton in buttonsList
set thisAXIdentifier to value of attribute "AXIdentifier" of thisButton
if thisAXIdentifier does not contain "Show All" and thisAXIdentifier does not contain "Show Less" then
set the end of buttonsAXIdentifiersList to thisAXIdentifier
end if
end repeat
--Get second listing of the buttons / AXIdentifiers after scrolling the window down to load all wallpapers in the category.
tell UI element 1 to perform action "AXScrollToBottom"
set secondButtonsList to buttons of UI element 1
if secondButtonsList is not buttonsList then --Buttons list changed due to scrolling. Combine the results.
set secondButtonsAXIdentifiersList to {}
repeat with thisSecondButton in secondButtonsList
set thisAXIdentifier to value of attribute "AXIdentifier" of thisSecondButton
if thisAXIdentifier does not contain "Show All" and thisAXIdentifier does not contain "Show Less" then
if buttonsAXIdentifiersList does not contain thisAXIdentifier then
set the end of buttonsAXIdentifiersList to thisAXIdentifier
end if
end if
end repeat
end if
--buttonsAXIdentifiersList
set the end of wallPaperCategoryList to {wallpaperCategory:thisCategoryName, wallpaperList:buttonsAXIdentifiersList}
--Scroll window up to ensure access to the "Show Less" button for the current category and the "Show All" button for the next category.
tell UI element 1 to perform action "AXScrollToTop"
--collapse the current wallpaper category
tell button wallpaperCategoryIndex of UI element 1
perform action "AXScrollToVisible"
perform action "AXPress"
end tell
end repeat
end tell
end tell
end tell
end tell
return wallPaperCategoryList
