Connect to my iPhone via UI clicking its Connect Button, assuming that my iPhone is locked.
View my iPhone Spotlight Menu Item
Manually keystroke an app’s name into Spotlight’s search field in Iphone Mirroring,
I want to input the name of an app into Spotlight’s search field in Mac’s Iphone Mirroring app. Can this be done via any of the following:
System Events?
Apple Shortcut?
Other hack?
# In this example, I am attempting to spotlight search the app "Podcasts".
tell application "System Events"
tell process "iPhone Mirroring"
tell first window
tell first group
set staticTexts to value of static texts
if staticTexts contains "Iphone in Use" then
set ConnectButoon to first button
tell ConnectButoon to perform action 1
end if
end tell
end tell
delay 2
click menu item "Spotlight" of menu 1 of menu bar item "View" of menu bar 1
delay 1
tell first window
tell first group
click
keystroke "Podcasts"
end tell
end tell
end tell
end tell
This works for me, but we may have different iPhone Mirroring settings.
(*
This script assumes that the iPhone's locked and that iPhone Mirroring requires the Mac user's password.
*)
tell application "iPhone Mirroring" to activate
tell application "System Events"
tell application process "iPhone Mirroring"
repeat until (it exists)
delay 0.2
end repeat
set frontmost to true
-- Password:
tell window 1
tell text field 1 of group 1
repeat until (it exists)
delay 0.2
end repeat
set value to "my_password" -- EDIT AS REQUIRED.
end tell
end tell
keystroke return
-- End password.
tell menu item "Spotlight" of menu "View" of menu bar 1
repeat until (enabled)
delay 0.2
end repeat
perform action "AXPress"
end tell
end tell
-- The Spotlight search window's group 1 apparently has no UI elements visible to System Events.
delay 0.5
keystroke "Podcasts"
delay 0.5
keystroke return
end tell
Nigel, your solution is amazing!
How did you detect that Sytem Events keystroke return would work?
Especially, in the abscence of application process “iPhone Mirroring”'s Spotlight’s search window’s group 1 lacking any UI elements?
It was largely a matter of trying it to see. System Events’s ‘keystroke’ command just simulates actual keystrokes. It doesn’t necessarily have to be applied to something that System Events itself can see in an application’s UI. Much of iPhone Mirroring’s window UI seems to be hidden from System Events. I’m not sure why. Possibly it’s for security reasons. But since hitting the Return key is the way to get to whatever it is whose name’s been typed into the Spotlight search field, it seemed worth trying to see if a System Events-generated return would work too.
When I have been trying to open an app, named VIP Access, that is only on my iPhone, the UI script intermittently fails.
On my iPhone I was able to create a simple Shortcut Using Shortcuts’s command “Open” VIP Access. Although this shortcuts executes properly when run from iPhone, it fails when running from the Mac. My thought is that Applescript cannot command an iphone Shortcut,
Is there a method to run a Shortcut from AppleScript so that it tells iphone to execute that command, in this case open a specific app on iPhone?
I’m afraid I’ve never used Shortcuts and don’t know much (ie. anything!) about it. (I wasn’t even able to view the contents of this shortcut of peavine’s on my Mac which has iPhone Mirroring. However, there was no problem on the older machine, so no doubt there’s some setting I don’t know about. )
The Shortcuts app on the Mac does have a scripting dictionary which allows shortcuts to be run from an AppleScript, but I imagine it can’t be used with shortcuts on an iPhone.
Scripting the UI window of iPhone Mirroring does not capture text from within its window. As such, I am looking for a way to capture the contents of its window and then OCR its contents to find a specificed text or number.
The shell command screencapture has an option to capture an image identifited by window id using option -l as in
screencapture -l
The iPhone Mirroring application errs when attempting to obtain a window id
tell application "System Events" to get id of window 1 of process "iPhone Mirroring"
=> System Events got an error: Can’t get id of window 1 of process "iPhone Mirroring".
What methods, in a shell script or otherwise, might allow Applescript to screen capture a small part of the screen where the iPhone Mirror window is located.
2, What methods will allow Applescript to OCR those contents to find a number or text?
Are there other methods to capture text or numbers from iPhone Mirroring?
