Is there a way to automate date picker in app without Dictionary?

Hello to everyone,

The past couple of days I dug as deep as possible into AppleScript documentation and code examples. As a result I managed to automate the user interaction with a third party application (on Mac OSX) that doesn’t come with a Dictionary / Library. Text Field, button, menu are straight forward and with the help of AppleScript and commands like UI elements I can identify the hierarchy of GUI elements.

However, this 3rd party application has a date picker as shown in the attached image.
Whereas the orange triangle icons are of class “button”, I have no idea what the date boxes are.
Besides UI inspectors and attempts of reverse engineering with the use of Automator “watch me do” I also tried AppleScript commands to return all UI elements of the respective window (type dialog) as well as the attribute AXChildren etc to identify the datepicker’s GUI representation.
Result: nothing ! No UI element.

Using the app “Script Debugger” with the “UI elements” command in the context of the window that I am analysing and the date picker activated (I.e. opened), returns me the tree of UI elements and there values, and I can see the date picker’s orange triangle icons representing buttons. But that’s it. There is no other UI element representing the table of days nor an menu etc.

Screenshot 2023-01-03 at 09.47.41

Could you please enlighten me and guide me in the right direction?
I mean, if a user can click the dates in this date picker, there must be a representation, right?

Many thanks for your time and expertise.

It would be nice to know what app this is.
Is it available to play around with?

You can use Apple’s “Accessibility Inspector” app to see the UI elements and their hierarchy when the mouse rolls over each item.

The “Accessibility Inspector” is part of Xcode

Latest version require full Xcode install, whereas older version “4.1” could be copied over and works without Xcode being installed.

Thanks a lot for your quick reply @robertfern

Is it available to play around with ?

No, unfortunately it’s a confidential business app not authorised to reveal more than I show in the attached 2 screenshots.

“Accessibility Inspector” shows more or less the same as “” did for me before.

Screenshot #1 shows that the orange triangle icon is a button (it’s parent is the window)
Screenshot #2 shows when I hover one of the dates, e.g. Jan 10, 2023. The context being identified is the window itself (as you can see as well from the fact, that the parent is the application itself).

However, as a human user, you can click the date boxes.
Note: this is not a web application! It is an installed app.

and here is the 2nd screenshot

Having > 10 years experience in web development, where every element is represented in the DOM, I apparently lack knowledge to understand how in this app visual elements can exist without being represented in the UI hierarchy.

Can anybody with an OSX app development background or automation background explain me if elements can be hidden from “inspector / scripting eyes” by design?

It would seem that the application isn’t using cocoa UI objects in its drawing of the calendar grid. You will most likely have to calculate coordinates to send a click to.

1 Like

@Fredrik71 Many thanks for this precious hint. I didn’t know about Qt yet.