Deleting items within a repeat loop - is that safe?

I am adding scriptability to my Mac program and ran into a typical programming issue that I don’t know how AppleScript is supposed to deal with. I hope someone here can tell me how it’s supposed to work:

Consider the program having objects (elements) of class “person”. I like to delete every person whose name is “bob”. I am using this code:

repeat with p in persons
	tell p
		if name is "bob" then
			delete p
		end if
	end tell
end repeat

Now, when I run this loop, Applescript interates over my program’s Person objects by using their index. When my program is asked to delete an item, the item is selected by its index that was given at the start of the repeat loop. When my app deletes that item, the following items the program manages will get their indexes adjusted down by one.

But Applescript keeps using the original indexes, i.e. it does not adjust its index for the deleted items. Which makes the loop fail eventually because Applescript asks my program for indexes that are out of bounds after the deletions.

Now, how is this supposed to work correctly? I can imagine several things:

  1. It’s a known issue with Applescript and has to be worked around in the script, e.g. by iterating backwards or by using other means to reference the to-be-deleted items (such as first collecting all candidates into a list using references, then deleting them by their refs or IDs - not sure what of that should work).

  2. AppleScript expects my program to handle this somehow on its own.

  3. My program must not allow such objects be accessible by index at all, but only by ID. I hope that’s not the case as it would affect performance badly, I’m afraid.

Hi. Use get to solidify the references.

tell application "Contacts" to repeat with p in (get people)
	if p's name is "bob" then delete p
end repeat

Correct. But a scripter would instead expect to be able to say:

delete every person whose name is "bob"

That would still lead to the same problem for my program.

But the responses so far suggest that my deletion loop is not entirely unusual and is expected to work, right?

So the problem lies in the way the Cocoa Scripting engine talks to my program, making the wrong references. I will ask on StackOverflow for help, I guess that’s the better place for this kind of question.

It’s not so much an “issue” as a natural consequence of the way things work.

repeat with p in persons sets up a repeat in which p will be an index reference understood by the owner of persons to some element in that group. The set-up includes finding out how many such elements there are so that the repeat knows when to stop. During the repeat, the script goes through its processes and the owner of persons (if it’s not the script itself) goes through its processes. It’s a matter of managing the communications between them so that they tell each other the right things at the right times.

To the application, of course, everything’s a one-off. It’s only aware of its current state and of any individual commands it receives. It has no sense of the script’s progress, no memory what it’s just done, and no idea of what will come next:

INCOMING: How many ‘persons’ do you have?
APP: (Counts persons.) Three. (End of story.)
INCOMING: What’s the name of your first person?
APP: (Locates first person. Extracts name.) bill. (End of story.)
INCOMING: What’s the name of your second person?
APP: (Locates second person. Extracts name.) bob. (End of story.)
INCOMING: Delete your second person.
APP: (Locates second person. Deletes it. Performs housekeeping.) Done. (End of story.)
INCOMING: What’s the name of your third person?
APP: (Tries to locate third person. Finds only two.) Ungh?

Edit: The above dramatisation’s just to illustrate the application’s limited interest in what’s going on. The information exchange may actually be implemented differently in reality. Whatever the precise details, the problem’s still an indexing mismatch between the fixed, self-incrementing repeat and the shrinking data set to which it’s applied.

These are both effective strategies if you’re going to be deleting a number of items in an application. Iterating through them backwards means that deleting one of them won’t change the indices of the yet-to-be-checked items in front of it. Getting a list of references to the items and iterating through that instead guarantees a collection of fixed length and references in fixed slots. Items may be deleted in the application, but the references to them in the list remain unchanged. Obviously these references mustn’t be index references, but applications usually return id or name references to their items.

Shane’s mentioned a ‘whose’ filter, as in delete every person whose name is “bob”. This is great (if the application supports it) in that it saves the scripter having to think about a deletion strategy, there’s only one communication exchange between the script and the application, and the application can work its way through the items without having to go back to the beginning each time. However, it does place the onus of making decisions based on values on the application itself ” something it probably won’t originally have been designed to do. The applications I know are very slow thinkers in this respect and can take a very long time to complete such commands. The best strategy for speed is usually to get the application to dump all the necessary information to the script in bulk ” which is normally quite fast ” and to let the script do the thinking, communicating with the application again only if the application needs to change something:

tell application "My Application" to set {theNames, theIDs} to {name, id} of persons
repeat with i from 1 to (count theNames)
	set thisName to item i of theNames
	if (thisName is "bob") then
		set thisID to item i of theIDs
		tell application "My Application" to delete person id thisID
	end if
end repeat

There’s nothing wrong per se with index references to application objects. But they have to be treated with caution when objects are added or deleted and, as shown above, it can be more efficient to deal with a list of relevant values than with the objects individually.

Edit: I was forgetting repeat until .:

tell application "My Application"
	set personCount to (count persons)
	set i to 1
	repeat until (i > personCount)
		if (name of person i is "bob") then
			delete person i
			set personCount to personCount - 1
		else
			set i to i + 1
		end if
	end repeat
end tell

Nigel, are you saying that I cannot expect my example code to work? Because I am using the wrong way to address the objects to be deleted? But why isn’t AppleScript telling me so, then? You say AS is using indexed addressing there, but I am using objects in the loop, so I don’t even see the indexes. I could understand your argument if I used indexes but when I say “delete that object I give you”, I would expect that it’s clearly understood and that not some other object is wrongly identified.

So, this repeat loop with referencing objects and deleting them the way I showed in my script is actually a conceptual and fairly hidden issue with AppleScript that Apple never addressed and expects the users to figure out themselves?

Or is my example wrong in a way that it should not even work at all, meaning I’ve made mistakes defining the vocabulary?

BTW, could someone explain to me why this doesn’t compile:

repeat with x in cities
	delete x -- Error: Can't get item 1 of every city
end repeat

Whereas this does:

repeat with x in (get cities)
	delete x
end repeat

Should both work? (If so, then my code must have a bug).

I wonder if it has to do with the fact that this won’t run, either:

set thelist to {"a", "b", "c"}
repeat with x in thelist
	delete x -- Error: Can't make "a" into type specifier.
end repeat

How does one do that?

As Marc mentioned in the first answer you should always use the

repeat with anItem in (get items)

syntax, because the get statement captures the references.
Consider also that anItem does not represent item 1 of items directly but a reference to item 1 of {item_1, item_2, … item_z} which is a big difference.

Cocoa Scripting is supposed to handle the control flow automatically.

The implementation of the KVC methods is basically for performance reasons.

Hi Stefan (I’ve also asked this question on SO, BTW: http://stackoverflow.com/questions/36571009/cocoa-scripting-indexed-array-acess-kvc-and-deleting-elements-not-working-tog).

Are you saying that what I do should work? I am still confused about whose fault it is that my example doesn’t work right, i.e. that it keeps indexing the wrong items. Is that an inherent issue with AS, a bug in the scripting bridge or an issue with my own code?

To make it clear:

repeat with anItem in items
	delete anItem
end repeat

will throw an error, because the array runs out of bounds.

However

repeat with anItem in (get items)
	delete anItem
end repeat

deletes all items without error because the get statement captures the references.

That is normal AppleScript behavior for many, many years and does not affect how you implement your classes and properties in Cocoa Scripting.

I’ve made a test program which has a list of cities (stored in a NSArray, which Cocoa Scripting can directly access, without me using any special accessors for indexed access or removal). Even this simple program still shows the same issue.

	get name of every city
	repeat with x in (get cities)
		get name of x
		delete x
	end repeat
	get cities

And here’s the log:

tell application "AppleScriptable.debug"
	get name of every city
		--> {"London", "Liverpool", "Berlin", "Munich", "Frankfurt(M)", "Paris", "Rome"}
	get every city
		--> {city 1, city 2, city 3, city 4, city 5, city 6, city 7}
	get name of city 1
		--> "London"
	delete city 1
	get name of city 2
		--> "Berlin"
	delete city 2
	get name of city 3
		--> "Frankfurt(M)"
	delete city 3
	get name of city 4
		--> "Rome"
	delete city 4
	get name of city 5
		--> error number -1719 from city 5

What we see here is that the loop uses item numbers like Nigel explained, and that these item numbers get out of sync once items get deleted.

So, it seems that this is indeed an issue with AppleScript and not with the scripting engine or my code, right? But that contradicts what Stefan just wrote above. I will test this with Apple’s own example code (Sketch) after I’ve gotten some lunch.

BTW, curiously, “delete every city” does not have this issue.

Update: I think it just came to me.

I was confusing the access to the array with the access to the actual object. I must keep those separate. Currently, my code mixes both up and that’s causing the problem. So it is a bug in my code, which is a relief because I should be able to manage that.

But it also reveals an issue with the Cocoa Scripting bridge because it doesn’t handle this correctly when one stores the actual items in an NSArray (“allPersons”) that the bridge directly manipulates, because then the scenario I just showed in my previous post will occur.

Yes. In a repeat like repeat with p in persons, where a person is an object in an application, the value of p is an index reference into all the application’s personseg. item 1 of every person of application “My Application” ” not an individual reference to one of the persons. Since references are resolved each time they’re used, every person of application “My Application” will return a smaller set after each deletion.

Nigel, is there a way to write the script without using an item counter (the way DJ Bazzie Wazzie shows) to make repeat work with deletion?

I mean, can I reference the objects in a way that won’t use indexes? I know I can address items by ID, but how would the actual code look?

Only because I’m implementing scripting support doesn’t mean I’m good with AppleScript :slight_smile:

Alright, I’ve solved the problem and can happily say: It’s indeed been my code’s fault. After a simple fix which I’ve explained here (http://stackoverflow.com/a/36575728/43615), the repeat - delete loop works now as hoped. No need for cludges such as using a helper list or counting the indexes on your own!

Thanks for all your comments, it helped me understand some concepts better. It also shows that there’s still a lot of misunderstanding even amoung the more experienced AS users, it seems :stuck_out_tongue:

To explain, what was happening: The fact that the items were enumerated by their item numbers is not inherent behavior of AppleScript but actually controlled by my program. When Applescript asks my code to return the elements of “every person”, I do include a so-called object specifier that tells AS how to address the objects later again. I had used the default code that would return numeric index specifiers, which worked just fine for all this time before I added support for deletion. By now changing the object specifiers to identify the elements by a unique ID, the issue got resolved: Now each item is addressed by “person id X”, and since these IDs are managed by my code in a way that does not affect their order, this solves the issue.

So, any time you encounter an app specifying its elements by item number, yet supporting modification of the element set, may be making the same mistake and should therefore be considered a bug in that program as it will lead to the deletion dilemma expressed in this thread.