Script is SLOOOW running InDesign CS4

Hi

I’ve been trying to avoid doing this, but I’m kind of at my wits end. I’ve written a script that goes through an InDesign document and plants codes around elements that get lost on export to EPUB format. It works, but it can be slow. Unfortunately, that has finally caught up with me. I am trying to run it on a 580-page file with literally tens of thousands of elements for it to tag, and it’s taking much longer than I think it should.

I have been running it from a handler so that I could set the undo mode to “fast entire script” (I had it at “auto” with no real difference in performance). I set the time out to 500,000 seconds; after about 21 hours, Applescript did time out.

I tried running it without the handler so I could watch the replies, and I noted that as it worked, it got progressively slower.

I’m pasting the code in the hopes that someone can give me tips on how to make it run at a reasonable speed.

FYI, I’m new to Applescript, but I’ve been working with complex VBA scripts for many years.

tell application "Adobe InDesign CS4"
set typographers quotes of text preferences to false
repeat with myDocumentsCounter from 1 to count documents
set myDocument to document myDocumentsCounter
--Insert "<a>" tag at the beginning of each page
repeat with myPageCounter from (count of pages in myDocument) to 1 by -1
set myPage to page myPageCounter of myDocument
repeat with myTextframeCounter from 1 to (count of text frames in myPage)
set myTextframe to text frame myTextframeCounter in myPage
if (count of characters in myTextframe) is greater than 0 then
if next text frame of myTextframe is not equal to nothing or previous text frame of myTextframe is not equal to nothing then
tell myTextframe
set beginning to "<a id=\"p" & name of myPage & "\"/>"
end tell
end if
end if
end repeat
end repeat
		
--Find text that deviates from its paragraph definition in specific ways and flag that text
set find grep preferences to nothing
set change grep preferences to nothing
repeat with myStoriesCounter from 1 to count stories in myDocument
set myStory to story myStoriesCounter in myDocument
if (count of characters of myStory) > 0 then --ignore empty text columns
if class of parent of parent of item 1 of parent text frames of text column 1 of myStory is spread then --ignore master spreads
if (count of text style ranges in myStory) > 1 then --ignore stories with no changes from defined text style
repeat with myTextStyleRangesCounter from (count of text style ranges in myStory) to 1 by -1
set myTextStyleRange to object reference of text style range myTextStyleRangesCounter in myStory
if font style of myTextStyleRange is not font style of applied paragraph style of myTextStyleRange then --and font style of myTextStyleRange is not font style of applied character style of myTextStyleRange then
set myFontStyle to "<~" & font style of myTextStyleRange & "~>"
tell myTextStyleRange
set {end, beginning} to {"<~~>", "<~" & font style of myTextStyleRange & "~>"}
clear overrides
end tell
end if
end repeat
end if
end if
end if
end repeat
		
--Find/Change with hex codes those characters that will get lost on export but that need to be retained
set find grep preferences to nothing
set change grep preferences to nothing
set myFindList to {"(\\r)(\\r)", "~m", "~>", "~/", "~f", "~s", "~S", "~.", "~4", "~y", "~%", "~<", "~#"}
set myChangeList to {"$1 $2", " ", " ", " ", "  ", " ", " ", " ", " ", "  ", " ", " ", " "}
repeat with myFindListCounter from 1 to count myFindList
set find what of find grep preferences to item myFindListCounter of myFindList
set change to of change grep preferences to item myFindListCounter of myChangeList
tell myDocument
change grep
end tell
end repeat
end repeat
set typographers quotes of text preferences to true
display dialog ("All done!")
end tell

The script seems to be having trouble in the section, “–Find text that deviates from its paragraph definition in specific ways and flag that text”

Thanks for any thoughts.

Browser: Safari 536.26.14
Operating System: Mac OS X (10.8)

If you are making lots of changes to InDesign documents, things start to slow down because all the changes are saved as potential undos. You can often speed things up considerably by doing the equivalent of a “save as” (using save and the providing the path to the existing document) every x changes, where x is something you probably need to experiment with.

Hi. Tens of thousands of changes should be able to be effected in a matter of seconds or minutes. Seeing the liberal usage of repeat loops”several of them nested”my suspicion is that excess work is being performed and that the inflation of some internal list is probably the root cause of the slowdown. The segment you believe is faulty is preceded by find/change value-clearing statements, yet there is no accompanying find/change. A find could dispense with all the if considerations and at least one of the loops in that segment.

–Edited for grammar and link

See this topic for a similar post on eliminating looping with find/change:
http://macscripter.net/viewtopic.php?id=32233

Thanks Shane and Marc for your advice. I have tried doing a save as after every 30 find/change operations, but that doesn’t seem to have appreciably helped. I let the script run for 2 hours this morning, and in that time, it processed fewer than 20 pages of 580.

Rethinking the approach as you suggest, Marc, may be my best bet, so I’ll try that. I am aware that find/change operations would be quicker, but to date, I’ve found no way to do the search that I think I would need. Being new to this, perhaps I just don’t know what to look for in the documentation yet. If you have further thoughts, that would be helpful.

It may not be entirely clear what I’m doing in that sticky section of the script. I’m disappointed with the xhtml files that InDesign generates when I create an EPUB. It discards text elements that I need it to retain (blank lines, for example), “dumbs down” special characters that Adobe Digital Editions can’t display (though iBooks and other e-readers can; em-spaces are an example, but there are many others), and inserts essentially useless span tags around text that is locally formatted (such as an italic word in the middle of a paragraph).

I’ve written this script to tag elements that InDesign handles in a way that I can’t use; I can then run a VBA macro on the xhtml files to convert my planted tags into the xhtml code that I need. The slow-down happens when I loop through the text style ranges looking for those whose font styling deviates from the font styling in the underlying paragraph or character style.

Taking italic for an example, I could write series of find/change routines to apply a sterile character style to all the italic in the document, but there are an infinite number of “italic” font styles”“Italic,” “Itl,” “Slanted,” Oblique," “Obl,” and “Swash” are just a few that I’ve run into, and several of these can be complicated by fonts that add “OSF” or “SC” or “Expert” to the style name”“ItalicOSF” for example. I couldn’t catch them all.

If there’s a way to find every text style range and simply plant codes around them based on on their font style, I’m unaware of it, but I’ll keep looking. If I could do that, then I could eliminate the testing portion of the script and just plant the codes; I could discard the ones I don’t need to worry about (like “Roman” and “Regular” and “Normal”) in my VBA macro.

Thanks again.

m.

If you have to export to epub from InD, perhaps you could export the text as InDesign Tagged Text, plant the codes with your tool of choice, and then re-import the new tagged text file into InD and export to epub.

You could try something like get the index of every text style range, the length of every text style range, and the font style of every text style range. Looping through those lists backwards would tell you what to insert where.

I only have access to CS, which uses different search syntax and is too old to be capable of grep, so this may not be all that useful, however, this example marks up every style range; it could be modified to filter out more things and to work with later versions. Whether or not this proves to be faster will depend on the number of ranges.


tell application "InDesign CS"'s document 1
	repeat with styleRange in (stories's text style ranges's object reference)
		tell styleRange
			set theText to its contents as text
			search for theText replacing with "<~" & its contents's font style & "~>" & theText & "<~~>"
		end tell
	end repeat
end tell

Depending on how many overrides and stories there are, you could actually be dealing with hundreds (or thousands) of thousands of text style ranges, rather than tens; that may cause AS to simply crap out, rather than time out. Does your code work acceptably on a subset of the main document? Try 10-20 pages and note the time it takes. Perhaps the fix is as simple as operating in small chunks.

–edited for clarity

Thanks again to everyone for their suggestions.

I’m dealing with 85,000+ style ranges in this particular book, so find/change operations and repeats are, I think, going to continue to cause problems.

The original code does work on shorter documents. I’ve been informally monitoring the run time, and it looks to be taking about an hour per page, though I think that value increases as the script runs. I started running the script on Friday at 4:30 p.m., and by Monday at 8:30 a.m., it hadn’t completed, so I stopped it. Because I used the undo mode “fast entire script,” I don’t know how far it got.

I’m coming at it from another approach now. I’m trying to eliminate the repeats and ifs. Right now, I’ve got this for that troublesome section:


		tell every text style range in every story in myDocument
			set {end, beginning} to {"<~~>", "<~" & font style & "~>"}
			clear overrides
		end tell

It doesn’t work yet, but I’m exploring/learning/trying to figure out how to fix it. The problem seems to be the “font style” reference in the second line. If I put a literal string into that parameter, the script works, but I haven’t figured out yet how to get the font style property of the text style range.

Well, still no luck. Here’s what I have now for the troublesome middle section:


	set myTextStyleRanges to every text style range in every story in myDocument
	set MyFontStyles to font style of every text style range in every story in myDocument
	repeat with j from (length of myTextStyleRanges) to 1 by -1
		tell item j of myTextStyleRanges
			set {end, beginning} to {"<~~>", "<~" & item j of MyFontStyles & "~>"}
		end tell
	end repeat

I know that I can probably combine the first two lists into a single record, but I’ll leave that alone for now.

What is really confusing me, though, is why the 5th line gives a -10006 error. The exact text of the error is

error "Can’t set end of " morrell 1625" to "<>"." number -10006 from last insertion point of " morrell 1625" to «class <

The text, "&#x2004…1625" is the text that I want to put tags around; the text, “"<~~>"” is the tag I’m trying to insert after the text.

I’m not sure why the insertion point is sending back the error or why there’s an error at all or why the class notation ends the error. I’m probably not “telling” the right object what to do, but I’m at a total loss. Do I need somehow to indicate that “<~~>” is a string, not an object?

As always, I’ll be grateful for any help.

m.

Hi,

have to say that I didn’t get in close touch with you script, but if it slows down cause of to many steps and …

Think about exporting (idml or indesign tagged text) and clean your texts outside the ID-App → finally reimport (tagged Text) → save as Indd (idml)

Is your end/beginning target a reference or text? My older version requires a ref to be explicit. Since your example isn’t within the document’s scope, that may also be the issue.

tell application "InDesign CS"'s document 1
	tell (stories's text style ranges) to set {myTextStyleRanges, MyFontStyles} to {object reference, font style}
   repeat with j from (length of myTextStyleRanges) to 1 by -1
       tell item j of myTextStyleRanges
           set {end, beginning} to {"<~~>", "<~" & item j of MyFontStyles & "~>"}
       end tell
   end repeat
end

Well, I don’t understand how you made it work, but it does. Thanks so much! Now I have to try it on the huge file and see if it completes in less than 3 days.

I’ll let you know.

m.

Hi Hans

Thanks for your thoughts. I had considered the process you describe, but it would end up being more labor intensive than I think what I’m trying to accomplish should be. I also need to be able to retain graphics–often inline graphics–in the files, which I would lose through tagged text. Finally, I need to prevent page breaks from changing until I’ve added my tags, and the only way I could do that using your idea would be to run an Applescript first. It seems to me that if I need to run one script, I might as well have it do everything at one time.

I do appreciate your ideas and your willingness to spend time on my problem, though.

Best wishes

m.

Marc

Thanks once again for your help. :smiley: The script works, and it’s much faster than my original. When I turned in last night, it was still running 6 hours after I’d started it, but when I got up this morning after 14 hours of run time, it was finished. Much faster than the more than 3 days I let the old script run without getting any results. I’ll make a few tweaks to the script to make it a little more efficient, but here’s the full script as it currently exists.


tell application "Adobe InDesign CS4"
	set typographers quotes of text preferences to false
	repeat with myDocumentsCounter from 1 to count documents
		set myDocument to document myDocumentsCounter
		--Insert "<a>" tag at the beginning of each page
		repeat with myPageCounter from (count of pages in myDocument) to 1 by -1
			set mypage to page myPageCounter of myDocument
			repeat with myTextframeCounter from 1 to (count of text frames in mypage)
				set myTextframe to text frame myTextframeCounter in mypage
				if (count of characters in myTextframe) is greater than 0 then
					if next text frame of myTextframe is not equal to nothing or previous text frame of myTextframe is not equal to nothing then
						tell myTextframe
							set beginning to "<a id=\"p" & name of mypage & "\"/>"
						end tell
					end if
				end if
			end repeat
		end repeat
		
		--Find/Change with hex codes those characters that will get lost on export but that need to be retained
		set find grep preferences to nothing
		set change grep preferences to nothing
		set myFindList to {"(\\r)(\\r)", "~m", "~>", "~/", "~f", "~s", "~S", "~.", "~4", "~y", "~%", "~<", "~#"}
		set myChangeList to {"$1 $2", " ", " ", " ", "  ", " ", " ", " ", " ", "  ", " ", " ", " "}
		repeat with myFindListCounter from 1 to count myFindList
			set find what of find grep preferences to item myFindListCounter of myFindList
			set change to of change grep preferences to item myFindListCounter of myChangeList
			tell myDocument
				change grep
			end tell
		end repeat
	end repeat
	
	--Plant tags around all text style ranges; selected tags will be removed and others coerced into HTML using a VBA macro after export
	tell document 1
		tell (stories's text style ranges) to set {myTextStyleRanges, MyFontStyles} to {object reference, font style}
		repeat with j from (length of myTextStyleRanges) to 1 by -1
			tell item j of myTextStyleRanges
				set {end, beginning} to {"<~~>", "<~" & item j of MyFontStyles & "~>"}
				--				display dialog ("Hello")
				clear overrides
			end tell
		end repeat
	end tell
	
	set typographers quotes of text preferences to true
	display dialog ("All done!")
end tell

I doubt it will be much use to anyone, but maybe it can shed some light on other people’s issues.

m.

Hi. I’m glad it finished for you, but will you do this often? 14 hours is still not a good running time. You probably could benefit from using a script object; there are searchable examples on the forum. Alternatively, using my may speed up list access.

"tell [b]my[/b] item j of myTextStyleRanges"

You could also try to iterate through the stories by page, rather than document-wide; the lists being handed to AS would be reduced by roughly 500 times their current size.

Best of luck.

Hi,

these will probably speed up things a bit too:

minimize ID window
turn off preflight
use do script method

and remember Shanes post!!!

Hans-Gerd Claßen

Hi Marc and Hans

I’ll try the things you suggest. I don’t think this will be a common issue for me, though. Most of the books I work on are much shorter and have many thousands fewer text style ranges. Though I agree that 14 hours is unacceptably long for the script, I’m somewhat reassured by the fact that it took InDesign 5 hours to export the book to EPUB. It’s really just an unusual situation, but fortunately, the difficulty pointed me toward you guys, who showed me a faster way to work and helped me learn some more Applescript.

Best wishes, and thanks again.

m.