AppleScript philosophical question

I’m new to applescript, I’ve been reading about it, and playing with it for about a month now. Here’s my basic summary; frustrating. My project is to convert a program that runs in Windows using AutoIt ( AutoIt Scripting Language - AutoIt). The job is about 300 times harder than it was with AutoIt. Am I wrong here? Why can’t I just measure the location of the mouse using x/y coordinates and write a script dictating where each click falls? The GUI scripting is a nightmare, UI Element Inspector is neat, but lacks documentation. UI Browser is neat also, and a bit more useful, but the scripts generated never worked. The more I play with this, the more it seems AppleScript is not simple, easy or for the average mac user.

I want a simple solution to record my mouse’s actions, and generate a log which will run as a script. Is such an option available, and if not, should I hire some monkey to build it and sell it to everyone?


That’s not philosophical; that’s a fact!

You are right, what I meant to say was, “Why Applescript?”

I think UI Scripting is concibed to be used only when you can’t do the work using a more direct approach.
In OS 9 days, we had various utilities for this task, specially Sändi’s Additions, AutoType/AutoClick and the fabulous PreFab Player (and others)… But I think near to nobody used them for serious workflows. I mean, lots of folks used them, but for example in a serious business you allways hire someone to write some C for you.
Anyway, most of times there are different solutions for the same task. Perhaps if you explain what you need, someone will be able to help you.

If you’re trying to write some kind of “click” recorder, I think it could be done “easilly” using PreFab UI Actions. But I think as well we have such “macro” tools (not applescript-based)… Hmmm… A quick search:

Philosophically, AppleScript isn’t a GUI language. It’s purpose is to automate repetitive, complex, or time-consuming tasks by telling the application(s) concerned to carry out the necessary actions. It does this directly, through the application’s scripting interface (for want of a better expression), not through the user interface.

The success of this obviously depends on application developers having implemented AppleScriptability in their apps, and with a set of commands and classes that allow scripters to achieve what they want to do. Not all application developers do that, of course.

GUI scripting is a recent addition to the system application System Events and is an attempt to extend the reach of AppleScript a little to unscriptable applications and to unscriptable features of applications. It’s not part of AppleScript itself, but a facility offered by System Events. You can tell System Events (through its scripting interface) to kid another running application (through its user interface) that the user has clicked certain buttons, pressed certain keys, or selected certain menu items. It’s a nightmare because the user interface of each application is different and can vary from version to version.

That’s my answer to the philosophy question. I don’t personally know how to get clicked screen coordinates.

The real difficulty underlying the OP’s philosophical question is that he is trying to repeat something he did in WinXP on an OS X machine as if it was WinXP. The OP is far better off to discover how to make the Mac applications do his bidding than he is to attempt to copy what he did before even if the GUIs of the programs he is using look exactly the same.

[NOTE: This is not a criticism or comparison of either system - it is a bald statement of the fact that they are different, and the original poster will have greatest success if he goes with the flow]

The first link posted above claims to do it (not applescript-based), for example.

As for grabbing such info using AppleScript, we have some stuff: Extra Suites, XTool, Smile…

This is a sample script for Smile, which will grab mouse clicks:

set rec to {}

--> script will be stopped bringing Smile to the front,
--> then using the shortcut apple+dot
		if mouse button then
			set ml to mouse location
				if rec's item -1 is not ml then set rec's end to ml
			on error --> add first mouse location
				set rec's end to ml
			end try
		end if
	end repeat
end try

rec --> {{159, 40}, {172, 80}}

This list of clicks made a new window in Safari, then clicked my toolbar-favorite button for MacScripter.

You can use later this list to re-create the clicks using one of the mentioned tools: System Events, Extra Suites, XTool, etc. But obviously it would be much easier:

tell application "Safari" to make new document with properties {URL:""}

This is exactly what I’m looking for, thanks for the help.

While I believe XTool is not currently supported (and its click command seems to be broken), here’s a similar routine for Extra Suites:

To record mouse click positions:

display alert "Mouse Recorder" message ("Click \"OK\" to start recording the position of mouse clicks." as Unicode text) & ¬
	return & return & "To return the recorded results, press " & «data utxt2318» & ¬
	"-. (command-period) or the Esc key." buttons {"Cancel", "OK"} cancel button 1 default button 2

set locList to {}
tell application "Extra Suites" to repeat
		if ES mouse down then
			set locList's end to ES mouse location
			repeat while ES mouse down
				delay 0.1
			end repeat
		end if
	on error
		return locList
	end try
end repeat

To replay recorded mouse clicks:

set locList to {{662, 290}, {894, 278}, {827, 373}, {439, 355}} (* or whatever *)
tell application "Extra Suites" to repeat with mouseLoc in locList
	ES move mouse mouseLoc
	ES click mouse
	delay 0.3 (* modify as appropriate *)
end repeat

On the subject of philosophy, I’m perfectly happy to go along with that of Bill Gates:wink:

Yes, it’s partially broken (its double-click command, though, is sometimes the only tool available in certain impossible clicking tasks → ie, in some Java/Flash apps) and actually it only exists in the database. There is a chance, though, to resurrect it in a near future (fingers crossed :confused: )