First off, if any of you know me, I’m not a fan of UI scripting, but I recognize there are valid use cases (as a last resort, when all else has failed and hell has frozen over )
One thing that comes up is sending keystrokes to non-frontmost applications. I know historically this has been a no-no in AppleScript - keystrokes always go to the frontmost app, which leads to shenanigans around context switching, etc., e.g.
tell application "Safari" to activate
tell application "System Events"
keystroke "a"
end tell
which gets into a whole mess of restoring the previous frontmost app, etc.
However, If you use Automator’s Watch Me Do action to record keystrokes, it includes an application target:
This target is tracked and honored, such that on playback, keystrokes go to the target application, ignoring the frontmost application, and avoiding the context switch of activating/restoring the target app.
In fact, this is so ingrained I can’t see any way to change it to a different app (or even frontmost), even if I wanted to.
So my question is this:
Was this level of application targeting added to AppleScript, and I’m just not aware of it?
Is Automator doing all the work under the hood?
Why can’t we do this in AppleScript? [rhetorical question ]
It looks like Watch Me Do uses a private UIRecording framework, which seems to hide most everything about itself. Since the user-facing event script doesn’t have the target in it, it must get the context from elsewhere, such as the workflow document. That has a tantalizing entry for userEvents, but it is also obfuscated pretty well, so I think “under the hood” or “behind the scenes” definitely covers it.
The action is surprisingly useful for getting UI object references, though.
If you have a reference to the property of any
UI element of any application.
You can just tell system events
then Perform the click action on the UIElement property reference
Or set its focus to true.
Then click at point x,y
… then Perform the click action on the UIElement property reference
Right, but that’s not what I was aiming for. I was trying to understand (exploit?) the mechanism that Automator uses to be able to target keystrokes to the non-frontmost application.
There are too many variables to be able to target specific UI elements in an arbitrary app when all I want to do it send a keystroke to the app and let it work out what to do with it. If Automator can do it, why not AppleScript?
If you have a reference to that you can do it in AppleScript.
I disagree.
at least, having ‘a reference to’ the app is not sufficient and does not replicate Automator’s action.
I know my application. I can determine the front most window in that application. I can even parse that window to find active elements, or I can change that window. It doesn’t help.
Because, what I can NOT do with AppleScript, even if I know every intimate detail of that application, is send that application a keystroke without activating the app first.
This is the contrast I’m trying to bridge.
Automator will send a keystroke to a specific app, regardless of whether that app is frontmost/active, and without any regard as to its state.
AppleScript will send a keystroke to the frontmost application only.
I am out on a knowledge ledge here but I was wondering if an NSSharingService/Picker could be used to communicate between frontmost and non-frontmost applications.
Take a look at the XAppProcess Library from Kurita Tetsuro. Using this Library you’ll be able to send Keyboard Shortcuts to background Applications with plain Vanilla AppleScript.
He has written a lot of great AppleScript Tools which are all available as Open Source on Github, too.
To be fair, it did sometimes initiate a new window in TextEdit, but not consistently, and since it crashed my Script Debugger the rest of the script was moot.
)
