Applescript behind on functions

I disagree. I think Apple has made it pretty clear that it wants to discourage them, and I think several recent developments were/are aimed at providing an alternative. People are welcome to ignore the advice, of course, but it seems to me these sorts of situations have historically come to a natural logical conclusion.

So I’d rather see scarce developer resources put into something with more of a guaranteed future. The fact that it is so simple to make Objective-C code (or C code with a wrapper) available to AS these days also means it’s a lot less work.

That is the actual problem. Objective-C (and Cocoa with it) is an application programming language on application level, desktop level. Most things can be solved using AppleScriptObjC or faceless background applications. Unfortunate both comes with their flaws too. For instance I have a free 3rd party dyib whose functions I want to use in AppleScript. To call these functions using Objective-C is a much too fragile and long path for communications and will need me to code everything twice. While with a scripting addition I’m not only sitting on the front row, but also have full control over the communication between AppleScript and the 3rd party functions. Something that will be lost in any other situation.

I’m not saying scripting additions should be used all time or more than other solutions, in practice it should be the least used solution. Sometimes it’s obvious that the first choice would be writing a scripting addition. Why advising and forcing other developers to do it the clumsy way especially when someone would have enough experience in C to write one?

Apple still improves scripting additions in performance and in security, they’ve also indicated that in future releases they will improve it even more. It’s a note to scripting addition developers, not to AppleScripters but I think that is the opposite of discouraging. Of course discouraging in terms of saving lines of code in AppleScript was a no go since the purple book about scripting additions was published in 1994. Only write scripting additions when needed functions lacks in AppleScript.

One important thing is that the alternative provided is for people who write AppleScript(ObjC) while the audience for writing scripting additions are not. If you want to extend the AppleScript functionality with script libraries you will bump against exactly the same problems as with scripting addition and will work less pretty as an scripting addition. Technically loading script objects is not extending AppleScript’s functionality, just efficient way of re-using already written code. The possibilities with script libraries creates an opportunity to do the opposite of how Apple “protect” the scripting definitions from useless AppleScript syntax pollution. When you take a good look at it the alternative isn’t better, again it’s for another audience.

I’m no trying to convince or want you to agree with me (I know you hate C :wink: ). I’m only inviting you to convince me that scripting additions are a bad choice in any situation. From what I see scripting additions are still widely used also by ruby and python developers but the scripting addition itself (the compiled C code) is nothing more than a harmless dylib, something that will be in our OS for the next decades, a technique that dates back to the 60s. So the big flaw and hatred about scripting additions is the scripting definition that is needed to put those fancy AppleScript words into AppleEvents descriptors. Actually an issue you’ll have on any level in AppleScript that is required in order to translate AppleScript syntax into OSA visible or not.

That’s now what I’m saying, though. If they solve your problem, great. I don’t pretend to be able to make sense of how wrapping it in Objectiv-C is so hard given the need to wrap it in AS-style terminology anyway, but I’ll happily take your word for it. It was the general call: “I think 3rd party developers should create more scripting additions again” that I disagree with. And yes, my perspective is from AppleScript, which seems reasonable given where we’re discussing the issue.

Scripting additions have a long history of making it harder for developers to implement scripting because of terminology conflicts, and I think the priority should be on encouraging developers to make their apps scriptable.

Sometimes I do get over excited :rolleyes:

Scripting additions allow arbitrary code injection into other processes. They need to DIAF for that reason alone.

The right way to implement extensions in 10.9 and later is as AppleScript libraries. While AS’s built-in library loading system is mediocre and all the stupid SDEF-based keyword stuff makes writing libraries unnecessarily complicated and intimidating to ASers, it’s still a huge improvement on the OSAX system.

Personally, I can’t figure out why Apple’s OS X security team hasn’t brought the ax down on OSAXen years ago. 10.6 added a scout’s honor system, but loading arbitrary code at such levels simply shouldn’t be allowed at all. StandardAdditions’ GUI dialog commands should be put into System Events, the rest of the Standard Additions commands permanently embedded in AppleScript itself, and the few remaining third-party OSAXen given notice to switch to library-based implementations over the next year or so. The entire OSAX loading mechanism can then be officially deprecated, allowing it to be removed entirely in a few years’ time once developers and users have had sufficient time to make any [modest] updates required.

As to the thread starter’s original point: 10.9’s biggest shortcoming is that the AS team didn’t bother to develop a basic collections of general-purpose libraries to include in OS X as standard. Every other language on the planet comes with its own standard library of common math, text, list, date, and file system manipulation commands; thereby saving every single user having to reinvent every single one of these basic wheels themselves. Writing those core libraries is also the best opportunity for the library system’s designers to eat their own dogfood and get rid of any untasty bits before serving it up to everyone else.

Obviously, time’s a-getting tight w.r.t. 10.10, but the most sensible thing you all could do would be to rally everyone to file lots of requests for the AS team to develop a small standard library to include in the OS at the earliest possible opportunity. That will instantly make their whole library system useful to all AppleScripters, whereas right now it’s only of interest to a small number of more advanced users with the interest and ability to develop and install their own. If enough ASers go tell them that this is what’s needed right now more than anything, then hopefully they’ll listen and do something about it; if not for this year then definitely for next (10.11).

Looking further ahead, the other thing thing that’s really needed if AS libraries are ever to take off is some sort of central library repository, equivalent to Perl’s CPAN, Python’s PyPI, Ruby’s RubyGems, etc. While a built-in standard library is nice, the real value multiplier is making it trivially easy for the community to share their own libraries with each other. A central repository allows library authors to upload third-party libraries which everyone else can then download and install. It also enables a nice set of tools for uploading/installing/upgrading/uninstalling libraries at a click of a button, thereby making it easy for even complete novices to take advantage of that resource.

I think here the best thing to do is to see how much traction JavaScript for Automation gets in 10.10: if it does well, the growing JXA community will likely start creating their own online library repository, at which point the AS community could get involved as well. After all, AS and JXA both use .scpt[d] files as their standard format, so there’s no reason a repository for one OSA language couldn’t accommodate the other OSA language as well.

Loading a dynamic library is code injection :cool: The whole system and ever process (applications) to the kernel relies on code injection, so why is this an issue? On higher level (desktop) we have code signing for that, which is something that any scripting addition should have enabled and the standard scripting addition uses it as my own.

It inherits the same flaws but don’t share the same pros (both have their own pros), how would it be an improvement? Also it has a different audience and different goals and it isn’t a replacement for scripting additions. Like I said scripting libraries are designed to efficiently re-using code like routines, create wrappers to access indirectly the world of Objective-C when used by plain AppleScript scripts but it isn’t designed to add missing functions to AppleScript. Scripting additions are actually designed to add functionality to your AppleScript (and JavaScript in 10.10) and should never be used like script libraries.

They changed the loading mechanism as well, loading arbitrary code isn’t “possible” anymore. They made it static and predefined and the functionality to load OSAXHandlers on the fly will be history soon (partially already is). Mavericks already blocks loading OSAXHandlers on the fly. Maybe in next OS releases code signatures are required and will remove the loadingHandler functions in AE. That the connection of the function is defined in the info.plist is just as insecure as defining them in an sdef file (when making a cocoa application scriptable).

Again the sdef issues remains. There has to be a translation between AppleScript or JavaScript syntax and AppleEvents use of 32 and 64 bit integers (4 and 8 char human readable code).

There is a world of difference between a process that knowingly chooses to import a known extension into itself, and a process having an unknown extension forced into it by an unknown stranger. The latter makes an absolute mockery of sandboxing, XPC-based privilege separation and stability, and just basic Software Design 101.

No it doesn’t. AS’s library architecture has its own set of flaws that it invented all by its very self. But that’s beside the point: libraries can only be imported into the process that is running the AppleScript.

Derp. “Doesn’t add missing function[ality]” What do you think all libraries and all FFIs do? You seem very confused between ‘purpose’ and ‘implementation’. The fact that AS-based libraries with their ASOC bridge support are constructed differently to OSAXen which are written in pure C/ObjC is merely an implementation detail. They are both solving exactly the same problem: how to introduce novel new functionality into a language?

The only point where AS libraries and OSAXen do fundamentally differ is that OSAXen handlers communicate through Apple events, with all parameter and result data being serialized and deserialized as AEDescs, whereas library handlers are native AS so do not involve any bridging: the AS values you pass in are the AS values you get out. (Going over to ASOC from there does involve a degree of bridging, but even then AS tries to preserve original values rather than copy them.) The only arguable ‘benefit’ of OSAXen’s AE bridging is that it’s language-agnostic, so any language than can speak AEs can use OSAXen. OTOH, every other scripting language in the world already comes with a built-in library a billionty times better than what OSAX have to offer, so why would they even bother? And the other ‘benefit’, that it allows process A to diddle process B’s innards without its permission, isn’t a benefit at all, but a great big stinky hole in the middle of much-needed OS-level security.

What is this I don’t even.

You misunderstand. JXA libraries aren’t going to be interchangeable with AS libraries: they are two completely separate runtimes with their own incompatible type systems. (This isn’t to say you can’t call into one from the other, but you’ll need to use something like ‘do script’ or NSAppleScript to provide a bridge. But this is not the point.)

What I mean is that the library repository itself, along with any accompanying library management tools, can and should be language agnostic. The only thing it would require is for library uploaders to indicate if theirs is a JS or AS library, so that library downloaders can filter results for their language only, and install downloaded libraries to the appropriate location for that language.

The whole point of libraries is that it allows a small number of expert scripters to create libraries that can be easily used by everyone in the community, no matter how casual or novice they are. A large, vibrant library ecosystem provides a large amount of benefit to a large number of people at very low cost. Libraries make the language more useful to more people, which in turn draws more people into the language, which in turn fuels the creation of more libraries, and so on. Substitute ‘scriptable applications’ or ‘scripting additions’ for ‘libraries’, and it’s the same thing. See: Network Effect.

This is important. The AS community has already had a year in which they could’ve set up their own library repository, and absolutely nothing has happened. This is hardly surprising: the lack of a standard library in 10.9 means that the vast majority of ASers have yet to realize what the point of libraries is or what they could/should/would do for them as users. And it’s unlikely to happen either, as the AS community largely lacks the expertise and resources to develop and run such a system.

OTOH, the JS folks who come into JXA are likely to bring along some knowledge and experience and prior work that allows them to bootstrap their own library distribution infrastructure in a very short time. And when they do, the AS community would be absolute dopes not to steal all that lovely hard work by other people and use it to bootstrap their own AS library ecosystem completely for free.

Bear in mind, if JXA’s application scripting support turns out good, there’s going to be lots of bright-faced, wet-eared young JSers suddenly clamouring for advice and assistance on how to script this application or that. And nobody knows their way around application scripting in general, and the particular interfaces and vagaries of individual apps in particular, as AppleScripters do. So there are huge potential benefits to be reaped by both sides through mutual cooperation and assistance. Time will tell if that happens or not, but JXA is now the best, if not only, hope the AS world has of bringing large quantities of fresh blood into the world of Mac application scripting, and finally reversing the long, slow decline in the supply of new and improved scriptable apps - which is something that should concern us all, ASer, Pythonista, and soon-to-be JXAer alike.

Security 101, as long as you fit the right protocols to the right situations. For starters, the idea is nice but AppleScript is not an application while XPC services is to dispatch own code and run in a sandbox to improve stability and security. This is because when using a XPC service the executable is not linked to the application’s executable. But an event driven system is sitting in between handled by launchd. In worst case scenario the XPC service is not working, but can’t break down an application or framework. But anyway, if it would work would it be a good solution?

Well let’s look what Apple says when to use XPC services:

When we take a brief look at the AppleEvents that are given and returned by OSA are formed we can’t say osaxs are working with untrusted data. The data (both send and reply event) send to my OSAXHandler are vaiidated and formed into correct data, the data good. XPC services are therefore not needed.

Like internet plugins (they don’t use XPC services as well) the stability of the framework relies on how well written osaxen really are. But a well written OSAX is a pretty and direct solution.

The initial posts was asking why the standard addition doesn’t add search and replace functionality. According to papers from the AppleScript SDK developers a scripting addition should never be a replacement for a routine. Or as they say, don’t re-invent the wheel. String replacements can be done with AppleScript using text item delimiters, so having a command in the standard addition for it would only be against their own guidelines.

Nope, scripting languages, are in general not self expandable. They don’t have typedefs like C or possibility to create classes like Objective-C (script objects are not classes). However PHP and JavaScript, who are scripting languages as well, are even better in it than AppleScript.

My mistake on this one. When I saw the developer conference video about the JavaScript I thought they simply added a new scripting component next to the AppleScript scripting component. For the record, a new scripting component can make use of the same AppleEvent manager and event codes. But I haven’t really looked into the new JavaScript.

Tried to parse it in detail, I really did. Gave up. However, some general observations:

All credit for trying to educate yourself on all this stuff, but you don’t yet understand programming and software development nearly as well as you think you do. (Hey, we’ve all been there; doubtless there’s old posts of mine still floating around that’d turn me bright red today.) If you haven’t already got yourself a copy of Steve McConnell’s Code Complete, do so now. Also, bookmark TheDailyWTF as required reading as well.

In the real security world, nobody gives a flying crap how honest and reliable your app/OSAX/plugin/library/whatever claims to be. Total paranoia is the only measure that matters. All third-party code is untrusted code by definition, and must be treated as such. That’s why we have sandboxing in the first place: to limit the damage that can be done to the system as a whole should any one of those untrusted components run amok, either through incompetence or malice. Isolate each untrusted component by default, and open tightly controlled communication channels between them only where absolutely necessary. Externally injecting OSAXen directly into any other process on the system without so much as a by-your-leave utterly violates that. It should not be allowed, period.

Of course scripting language are expandable. They include abstraction features that allow more powerful, sophisticated behaviors to be composed from existing primitives, and any scripting language worth a damn has some kind of FFI (foreign function interface) for calling into a lower level language in order to add novel new behaviors when existing primitives alone are insufficient. Heck, even AppleScript now has a proper FFI in the form of ASOC, even if it is imperfectly implemented, inadequately documented (not counting Shane’s excellent third-party books on it), and poorly integrated into AS outside of Xcode-based projects.

AS and JXA are peer OSA components; however, the Open Scripting framework not enable them to access each others’ scripts without further explicit bridging (e.g. ‘run script’ is the simplest example, although a sufficiently sophisticated attachable app could provide interfaces for inter-script communication as well). But once you do that you lose a lot of the benefits of using libraries anyway, since all data has to be serialized and deserialized as Apple events, which means data cannot be shared, only copied, and everything is 10x slower due to the overhead.

Hi,

I think that the main thing the user wants is speed. Sometimes scripting additions is faster. But the library is fast also. Application additions are easier to make. You can’t modify scripting additions while you can modify your own library. What I’m trying to say is that there are certain things that mere mortals can do.

The hardest part about learning how to write library modules is reading the developer site’s documentation. But, after you get it then you can do many things. To whom it may concern :slight_smile:

gl,
kel

You know what’s missing, a tutorial on how to convert NSSomethings to ASObjC.

gl,
kel

Read documentation about (Discussed at 2012 WWDC) NSUserScriptTask which should be used by commerical use of AppleScript. No OSAX injection at all or system apple event handler table manipulation.

FYI: NSUserScriptTask is, unlike normal NSAppleScript object, NOT executed by the application but in a safe-mode by the system itself.

FWIW, I suspect that all it does change the default target of a script to some runner app. And because of a bug in 10.9, it therefore can’t call ASObjC-based script libraries.

I was just quoting Sal Soghoian and Chris Nebel what they said about NSUserScriptTask at the WWDC in 2012. They explicitly say that the script will run outside the sandbox and will be launched by the system with no restrictions. An running instance of AppleScript needs an running instance of an Application (to create/modify the system apple event dispatch table for the application) since Mac OS X. So yes, there must be a running process somewhere in the system that eventually will run your script.

UPDATE: I’ve created an project in xcode and fired an NSUserScriptTask using Objective-C to run an AppleScript. What happens is that a task/process is added to launchd (com.apple.foundation.UserScriptService). Then an XPC-service will launch the script using osascript command line utility.

FFS. I know what NSUserScriptTask does. And it has sweet FA to do with preventing Random Process A injecting Random OSAXen into Random Process B. Honestly, please stop. You don’t know what you are talking about.

[For benefit of others here: Traditionally, ‘attachable’ applications have used NSAppleScript or the Carbon OpenScripting API to run user-supplied scripts within the main application process. That approach is of very limited usefulness in sandboxed apps since the sandbox prevents those scripts sending Apple events to any other application. The sandbox-friendly solution is for those apps to use NSUserAppleScriptTask, which runs user-supplied scripts in a separate process outside of the sandbox, allowing them to talk to any application they like. (Caveat: NSUserAppleScriptTask is utterly lame and crappy in every single other respect.) None of which has any bearing on where OSAX commands can be sent. OSAX commands go to whatever process your script tells them to go to, except where 10.6+'s OSAX “security” system says otherwise, in which case they’re bounced back to the current process. And IIRC that system runs entirely on Boy Scout honor badges, making it worse than useless too.]

For benefit of others here: Hhas clearly can read a documentation but not quite see a solution to a bigger problem. A self targeted event can always load an OSAX without asking permission from the user. Resulting that running a untrusted script inside the application will give someone the opportunity to actually load and inject an OSAX into the application and call malicious code from the script. To avoid this the application can take the user defined script and run it by the system outside the application context using NSUserScriptTask, which eventually will use osascript command line utility. In order to inject the osax into the application, the user will be warned and needs to confirm this with administrator privileges before the script can be used. The reason why this security guard shows up is that the script needs to explicitly tell that he wants to load an OSAX into another process than itself. When the end user using the app ignores this OSAX request, the OSAX will not load and the event will be bounced back with error -10004. Using NSAppleScript to run a user defined script inside an application can easily load an unknown OSAX and use it from that script (read: it’s self targeting). Therefore it’s said at the WWDC 2012 that it’s wise to use for both sandboxed and non-sandboxed applications NSUserScriptTask (or one of it’s subclasses) when loading and executing untrusted scripts.

Anyway I wasn’t able to inject a self written OSAX in any kind of process other than itself, quite secure to me.

Bump as I haven’t forgotten about this, but I haven’t had a free moment to dig further into it yet. (Short version: BW may be right about OSX now blocking third-party osax commands to other processes, but if it does it doesn’t appear to be officially documented and I need to run further tests on older versions of OS X. Gods I hate Apple’s management of AppleScript at times.)

I just thought that it was different. How when AppleScript might need a function and Apple didn’t add it. In other scripting languages they try to add what you need. It must be more complicated than that with AppleScript.

gl,
kel

I totally agree!

I think it was added in Leopard’s security update 2008-005. But I’m not sure, I don’t have Leopard versions with and without the security update installed.