When the text of an AppleScript text item delimiter is or contains , or ', the text is not delimited there. Escaping it (\, or ') doesn’t work either (whether unicode text or not). How do you specify a text item delimiter that must include single quotes or a back quote? Further, how do you deal with text that has double quotes in it if you want to search it?
Suppose this was your text and your delimiter had to include one of the quote marks in it.
set test to "˜My father has spanked me, and my mother has spanked me; all my aunts and uncles have spanked me for my ˜satiable curtiosity; and still I want to know what the Crocodile has for dinner!'
Then Kolokolo Bird said, with a mournful cry, ˜Go to the banks of the great grey-green, greasy Limpopo River, all set about with fever-trees, and find out.'"
set ASTID to AppleScript's text item delimiters
set AppleScript's text item delimiters to {"˜"}
set test to text items of test
set AppleScript's text item delimiters to ASTID
return test
The original text was copied from a web site, so I’m not sure what the character actually was. Interesting & thanks - your rendition works perfectly as does mine if I insert the marks as you suggest.