Unicode text in Leopard

I am trying to rewrite an InDesign workflow to use CS3 and Database Events instead of useless loops threw a text file but running into issues with unicode text.

My original workflow read a unicode text file using delimiter return and then looped threw each node or line and parsed out values to pass to InDesign. Now I am trying to use the Database Events to convert the text file to sqlLite but having strange results.

–This worked in Tiger
set myData to read myFile as Unicode text using delimiter return

–This should work in Leopard but the data does not delimitate
set myData to read myFile as Unicode text using delimiter {“\n”}

–So I used this as a work around
set myData to read myFile from 3 using delimiter {“\n”}

Using the last read command works and the Event Log shows the data delimitated but when values are parsed out nothing get to the database.

Here is the script
Note I have to give props to Martin Michel for the Database routine, very nice man.

-- author: Martin Michel
-- eMail: martin@joyofscripting
-- created: June 2007
-- requires: Mac OS X 10.4.9
on run
	tell application "Finder"
		set tag_file to (choose file) as alias
		set myFile to open for access tag_file
		set myData to read myFile as Unicode text using delimiter {"\n"}
		--set myData to read myFile from 3 using delimiter {"\n"}
		close access myFile
		set myRecName to name of file tag_file as Unicode text
	end tell
	--Create Database Begin****************************************
	setdbname("webshopDB")
	setdbpath((path to desktop) as Unicode text)
	opendb()
	--Create Database End****************************************
	set rec to createrec(myRecName)
	repeat with aRecord in myData
		try
			set AppleScript's text item delimiters to {"="}
			set myVar to (text item 1 of aRecord as Unicode text)
			createfield(rec, myVar, (text item 2 of aRecord as Unicode text))
		end try
	end repeat
	closedb("saving yes")
	--Save Database Record****************************************
	
	--Open Database Record****************************************
	setdbname("webshopDB")
	setdbpath(path to desktop) as Unicode text
	opendb()
	set rec to getrecbyname(myRecName)
	set mystring to getfieldvalue(rec, "<@useremail@>" as Unicode text)
	--display dialog getfieldvalue(rec, "<@template@>")
	--display dialog getfieldvalue(rec, "<@useremail@>" )
	closedb("saving no")
	quitdbe()
end run

property db : missing value
property dbname : missing value
property dbpath : missing value

-- sets the name of the database to use
on setdbname(newdbname)
	set dbname to newdbname
end setdbname

-- sets the parent folder path of the database to use
-- path must be given in mac style (e.g. "Users:martin:Desktop:")
on setdbpath(newdbpath)
	set dbpath to newdbpath
end setdbpath

-- indicates if the database file already exists
on dbpathexists()
	set fullpath to (dbpath & dbname & ".dbev" as Unicode text)
	try
		set fullalias to fullpath as alias
		return true
	on error
		return false
	end try
end dbpathexists

-- opens the db/creates a new database if the database
-- file does not already exist
on opendb()
	if not dbpathexists() then
		tell application "Database Events"
			set db to make new database with properties {name:dbname, location:dbpath}
		end tell
	else
		tell application "Database Events"
			set db to database (POSIX path of (dbpath & dbname & ".dbev"))
		end tell
	end if
end opendb

-- saves the database
on savedb()
	tell application "Database Events"
		save db
	end tell
end savedb

-- returns the names of all records in the database
on getrecnames()
	tell application "Database Events"
		tell db
			set recnames to name of every record
			return recnames
		end tell
	end tell
end getrecnames

-- returns the ids of all records in the database
on getrecids()
	tell application "Database Events"
		tell db
			set recids to id of every record
			return recids
		end tell
	end tell
end getrecids

-- creates a record in the database
-- record name must be given
on createrec(recname)
	tell application "Database Events"
		tell db
			set newrec to make new record with properties {name:recname}
			return newrec
		end tell
	end tell
end createrec

-- creates a field with a field value in a record
-- record, field name & field value must be given
on createfield(rec, fieldname, fieldvalue)
	tell application "Database Events"
		tell db
			tell rec
				make new field with properties {name:fieldname, value:fieldvalue}
			end tell
		end tell
	end tell
end createfield

-- creates multiple fields with field values in a record
-- record, field names and corresponding field values must be given
on createfields(rec, fieldnames, fieldvalues)
	tell application "Database Events"
		tell db
			tell rec
				repeat with i from 1 to length of fieldnames
					set fieldname to item i of fieldnames
					set fieldvalue to item i of fieldvalues
					make new field with properties {name:fieldname, value:fieldvalue}
				end repeat
			end tell
		end tell
	end tell
end createfields

-- get all records of the database
on getrecs()
	tell application "Database Events"
		tell db
			set allrecs to every record
			return allrecs
		end tell
	end tell
end getrecs

-- get a record by name
on getrecbyname(recname)
	tell application "Database Events"
		tell db
			set rec to first record whose name is recname
			return rec
		end tell
	end tell
end getrecbyname

-- get a record by id
on getrecbyid(recid)
	tell application "Database Events"
		tell db
			set rec to first record whose id is recid
			return rec
		end tell
	end tell
end getrecbyid

-- returns the record when providing the record name
-- attention: returns only one record id even when
-- several record with the same name exist 
on getrecid(recname)
	tell application "Database Events"
		tell db
			set recid to id of first record whose name is recname
			return recid
		end tell
	end tell
end getrecid

-- returns the record name when providing the record id
on getrecname(recid)
	tell application "Database Events"
		tell db
			return name of record recid
		end tell
	end tell
end getrecname

-- returns the field value of a record field
-- record & field name must be given
on getfieldvalue(rec, fieldname)
	tell application "Database Events"
		tell db
			tell rec
				set fieldvalue to value of field fieldname
				return fieldvalue
			end tell
		end tell
	end tell
end getfieldvalue

-- sets the field value of a record field
-- record, field name & field value must be given
on setfieldvalue(rec, fieldname, fieldvalue)
	tell application "Database Events"
		tell db
			tell rec
				set value of field fieldname to fieldvalue
			end tell
		end tell
	end tell
end setfieldvalue

-- indicates if a given record name exists in the database
on hasrecname(recname)
	if recname is in my getrecnames() then
		return true
	else
		return false
	end if
end hasrecname

-- indicates if a record contains a given field name
on hasfieldname(rec, fieldname)
	tell application "Database Events"
		tell db
			tell rec
				set fieldnames to name of every field
				if fieldname is in fieldnames then
					return true
				else
					return false
				end if
			end tell
		end tell
	end tell
end hasfieldname

-- deletes a given field of a given record
on delfieldname(rec, fieldname)
	tell application "Database Events"
		tell db
			tell rec
				delete field fieldname
			end tell
		end tell
	end tell
end delfieldname

-- delete a complete record in the database
on delrec(rec)
	tell application "Database Events"
		tell db
			delete rec
		end tell
	end tell
end delrec

-- deletes all records in the database
on delallrecs()
	tell application "Database Events"
		tell db
			delete every record
		end tell
	end tell
end delallrecs

-- closes the database
-- two modes availabe: close "saving yes" and "saving no"
on closedb(mode)
	tell application "Database Events"
		if mode is "saving yes" then
			close db saving yes
		else if mode is "saving no" then
			close db saving no
		end if
	end tell
	set db to missing value
	set dbame to missing value
	set dbpath to missing value
end closedb

-- quits the Database Events application
on quitdbe()
	tell application "Database Events"
		quit
	end tell
end quitdbe

AppleScript: 2.2
Browser: Safari 525.13
Operating System: Mac OS X (10.5)

Hi,

there are two different delimiter parameters

using delimiter optional unicode text the value that separates items to read.
using delimiters optional list of unicode text .or a list of values that separate items to read

so the proper syntax is

set myData to read myFile as Unicode text using delimiter "\n"

second problem: return is not \n
return is \r or ASCII character 13
linefeed is \n or ASCII character 10

Note: the linefeed constant is new in Leopard

If your are working only with Unicode text, and the code should work in Tiger and Leopard,
I recommend to use this code:

set LF to (ASCII character 10) as Unicode text
set myData to read myFile as Unicode text using delimiter LF

btw: Most of the Unicode text coercions are not necessary, for example the text class of the result of


set myRecName to name of file tag_file

is always Unicode text (Tiger and Leopard)

or similar case, dbpath is defined as Unicode text, so no coercion is needed


set fullpath to (dbpath & dbname & ".dbev")

Thanks for explanation of return and linefeed. It was very confusing to get this work a few years ago but the file comes from a pc and it does not have linefeeds it only has returns but this is what I get if I change to the corrected syntax you gave me.

“<@tagid@>=\n<@fiberbrand@>=ACTIVE\n<@category@>= \n<@width@>=13’ 6"\n<@starrating@>= \n<@stylename1@>=COLOR SUITE II SUPREME\n<@stylename@>=COLOR SUITE II SUPREME\n<@framestyle@>=all\n<@swatchweight@>= \n<@swatchrows@>=10\n<@swatchcols@>=5\n<@swatchsize@>=1,2.750,1.750\n<@colorline1@>=MORNING FROST\n<@colorline2@>=MOON MAN\n<@colorline3@>=COOL POOL\n<@colorline4@>=MIDNIGHT REFLECTIONS\n<@colorline5@>=CARIBBEAN NIGHT\n<@colorline6@>=SHALLOW WATER\n<@colorline7@>=FADED DREAM\n<@colorline8@>=GLACIER SPRINGS\n<@colorline9@>=TRANQUIL\n<@colorline10@>=MOORELAND\n<@colorline11@>=TEMPERED GLASS\n<@colorline12@>=SEA TURTLE\n<@colorline13@>=PARADISE\n<@colorline14@>=CALICO CREEK\n<@colorline15@>=PINEWOODS\n<@colorline16@>=NATURE’S WHISPER\n<@colorline17@>=MERRY-GO-ROUND\n<@colorline18@>=BALLOON\n<@colorline19@>=ALPINE LAGOON\n<@colorline20@>=SPRING VALLEY\n<@colorline21@>=SOLITARY FIELDS\n<@colorline22@>=SOUTHERN PINE\n<@colorline23@>=HILLSIDE RANGE\n<@colorline24@>=BROAD LEAF\n<@colorline25@>=JUNGLE DeEP\n<@colorline26@>=BILLOW\n<@colorline27@>=PRAIRIE GHOST\n<@colorline28@>=SOUTHERN PASTURE\n<@colorline29@>=HEDGEROW\n<@colorline30@>=CHILDHOOD FOREST\n<@colorline31@>=ROLLING HILLS\n<@colorline32@>=FARMERS MARKET\n<@colorline33@>=WINDOW GARDEN\n<@colorline34@>=ENVY\n<@colorline35@>=NORTHERN MOSS\n<@colorline36@>=DAYDREAMER\n<@colorline37@>=EAGLE’S BLUFF\n<@colorline38@>=GROUNDED\n<@colorline39@>=DITHER\n<@colorline40@>=SOD\n<@colorline41@>=FLOWING BREEZE\n<@colorline42@>=FOOTPATH\n<@colorline43@>=SUNWASHED\n<@colorline44@>=LAST CHANCE\n<@colorline45@>=ROADSIDE\n<@colorline46@>=NEW BEGINNINGS\n<@colorline47@>=PAINTED SKY\n<”

If the file comes from a pc it could have both CR and LF.
Try this, it considers CR, LF or CRLF

set myData to paragraphs of (read myFile as Unicode text)

Hi, neweller.

I’m not at my Tiger machine at the moment and don’t have Leopard. But just to clarify, are the “\n” bits in your text literally backslashes followed by n’s? Or have you written them in to represent linefeeds? If they’re literal characters, and they’re where you want to break the text, you’ll have to use AppleScript’s text item delimiters, since only the first character of read’s delimiter is used.

on run
	set tag_file to (choose file)
	set myData to (read tag_file as Unicode text) -- No need to open for access first.
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to "\\n" -- Escaped backslash followed by "n".
	set myData to myData's text items
	set AppleScript's text item delimiters to astid
	set myRecName to name of (info for tag_file)
	--Create Database Begin****************************************
	
	-- etc.
end run

FYI, Script Editor 2.2 (i.e., Leopard) has a preference to “Escape tabs and line breaks in strings” (i.e., show “\t\n\r” instead of a literal tab, linefeed, and carriage return).

Thanks for all the help guys but it turned out I had save the text file as another format besides unicode (Mac OS Roman) and even tho applescript Event Log would show the data something was still hidden the file causing major problems. This morining I threw away all text files on my machine and went back to the server and everything is perfect.

Wheewww, also thanks Bruce Phillips you tip really helped out. I did not know 2.2 actually hands returns, tabs, linebreaks as strings. That really threw me off.