Click and Drag without Additional Libraries

Not a question - just posting here in case it’s useful to someone.

I needed the ability to click and drag for UI scripting.

System Events only lets you click at a location.
Applescript Toolbox can be script-embedded, avoiding user installation, but can’t click and drag.
MouseTools and ClickClick should both do what I need, but need to be installed on each user’s computer.

So here’s a solution that just uses included Python:

on mouseDrag(xDown, yDown, xUp, yUp, delayTime)
-- delayTime because the drag may fail if the UI isn't fast enough without a delay. For what I do, .1 works.
	do shell script "
/usr/bin/python <<END

from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventCreate
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap
from Quartz.CoreGraphics import kCGEventLeftMouseDragged
import time

def mouseEvent(type, posx, posy):

          theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)

          CGEventPost(kCGHIDEventTap, theEvent)

def mousemove(posx,posy):

          mouseEvent(kCGEventMouseMoved, posx,posy);

def mousedrag(posx,posy):

          mouseEvent(kCGEventLeftMouseDragged, posx,posy);

def mousedown(posxdown,posydown):

          mouseEvent(kCGEventLeftMouseDown, posxdown,posydown);

def mouseup(posxup,posyup):

      mouseEvent(kCGEventLeftMouseUp, posxup,posyup);

ourEvent = CGEventCreate(None);

mousedown(" & xDown & "," & yDown & ");
time.sleep(" & (delayTime as text) & ");
mousedrag(" & xUp & "," & yUp & ");
time.sleep(" & (delayTime as text) & ");
mouseup(" & xUp & "," & yUp & ");

end mouseDrag

Just found:

use framework "Foundation"
set mouseLoc to current application's NSEvent's mouseLocation()

Which makes me suspect there’s a succinct ASObjC solution to replace the need for ClickClick, MouseTools, AppleScript Toolbox, or Python for all these mouse-related actions.

For now, I’m using System Event to click, ASObjC to get the mouse position, and Python to drag. Works… doesn’t feel very elegant.

  • Tom.

Just to further note for anyone finding this thread:

System Events clicking has proved unreliable for me
ASObjC’s “current application’s NSEvent’s mouseLocation()” returns the mouse coordinates from the bottom left, instead of the top left. Then I have to get the screen resolution and convert. Which means I also have to detect which display they’re on… and how’s Retina affect all that? All in all, I’m back to using Applescript Toolbox for getting the mouse location and basic clicks, and Python to drag.

There’s not. The CGEvent stuff is all C-based APIs. Although ASObjC can call C-based code, it can’t provide the arguments needed in this case.

That’s how they are calculated internally (mostly). AppleScript was based on the the old coordinate system used pre-OS X. But OS X was built on Display PostScript, and I’m sure you’re aware of its coordinate system.

it doesn’t. The values are in points, not pixels.