problems controlling mouse from script or macro program

I’m trying to control the mouse and running into the same problematic behavior whether I use AppleScript or Keyboard Maestro, which suggests there is something about OS X that affects both. I am hoping that the experts here might be able to offer suggestions about what could be wrong, and generally what might be the hope for the solution.

I want to automate certain mouse functions. One thing I want to do is to have a script (or Keyboard Maestro) simulate a left mouse button press – and hold – while I move the mouse. This will create a drag action. Then the script will release the mouse when the drag is done.

I will probably use keyboard “hotkeys” or shortcuts to initiate and terminate the drag, but there may be more than one way to do it. This is flexible.

Among other uses, I want to use this in Photoshop for painting.

I have tried doing this in Keyboard Maestro. I also tried using AppleScript together with the shell program “MouseTools”.

In both cases the behavior is the same. What happens in Photoshop (while painting) is this. When the script or KM does the “left button down” action, a spot of paint appears. This means that Photoshop has registers the left button press. Then I start to move the mouse. At this point the painting cursor moves. However, no line of paint appears. Then after moving the mouse some distance and I cause a “left button release” action to take place, suddenly a line of paint (forming a direct linear connection from the original mouse location to the final location) appears.

There is similar behavior when trying to simulate drags in the AppleScript editor. Normally when dragging over text in the editor, it is highlighted. When I use my script or KM to simulate a drag, a very similar thing happens. I do the following (1) cause the left button down and hold to take place, (2) move the mouse, (3) cause the “left button release” to take place. During step 2, no highlight appears. Then in step 3, suddenly all the text between the initial and final locations is highlighted.

Here is the AppleScript code I am using to test this

set mouseToolsPath to (path to home folder as text) & "bin:MouseTools"

delay 3 -- 3 seconds after starting the script to move the mouse where you want


do shell script quoted form of POSIX path of mouseToolsPath & " -leftClickNoRelease"

delay 3

do shell script quoted form of POSIX path of mouseToolsPath & " -releaseMouse"

Model: MacBook Pro
Browser: Safari 537.36
Operating System: Mac OS X (10.10)

If you have AppleScript Toolbox (AST) installed you can move the mouse while holding left mouse button down. Not sure how events are handled in PhotoShop but it could be worth a try.

-- Place the mouse to starting point
AST set mouse point location {100, 40}
-- move the mouse while holding mouse down (dragging). 
AST set mouse point location {400, 40} with holding mouse down

I you prefer not to install an osax but rather use a command line utility: I think Hank’s mousetools can do the job as well. Both the osax and Hank’s mousetools makes use of CoreGraphics.

I don’t think this addresses my issue.

I want the script to take care of holding the left button, but I’m going to move the physical mouse to determine the shape of the line of pain.