handle large textfile

[Hans-Gerd_Classen]

Hello,

i’ve got a tab-delimited textfile with about 5000 lines and want to handle it as shown below to achieve a InDesign-tagged-file. well, it’s very slowly …
is there a faster way to do it by a shell-script-combination¿ (… or by good AppleScript )

set tid to AppleScript's text item delimiters
set mytext to "butcher	name1	name2	name3	strasse	ort
butcher	name1	name2	name3	strasse	ort
butcher	name1	name2	name3	strasse	ort
butcher	name1	name2	name3	strasse	ort
Gärtner	name1	name2	name3	strasse	ort
Gärtner	name1	name2	name3	strasse	ort
Gärtner	name1	name2	name3	strasse	ort
Gärtner	name1	name2	name3	strasse	ort
maurer	name1	name2	name3	strasse	ort
maurer	name1	name2	name3	strasse	ort
maurer	name1	name2	name3	strasse	ort"

set AppleScript's text item delimiters to "\n"
set mylist to every text item of mytext
set branche to "<pstyle:branche>"
set firm to "<pstyle:rest>"

set newlist to {}
repeat with i from 1 to count of mylist
	set myrow to item i of mylist
	set AppleScript's text item delimiters to "\t" --tabulator
	set rowlist to every text item of myrow
	set AppleScript's text item delimiters to " "
	set myname to firm & (items 2 thru 4 of rowlist) as text
	set myeditedrowlist to {}
	copy branche & item 1 of rowlist to end of myeditedrowlist
	copy myname to end of myeditedrowlist
	copy firm & item 5 of rowlist to end of myeditedrowlist
	copy firm & item 6 of rowlist & return & return to end of myeditedrowlist
	copy myeditedrowlist to end of newlist
end repeat

set finallist to {}
repeat with j from 1 to count of newlist
	set mybranche to item j of newlist
	try
		set myreferrer to item (j - 1) of newlist
		if item 1 of myreferrer contains branche then set branchereferrer to item 1 of myreferrer
		if item 1 of mybranche is branchereferrer then set mybranche to suppress item 1 from mybranche
	end try
	copy mybranche to end of finallist
end repeat
set AppleScript's text item delimiters to return

set thestring to "<ANSI-MAC>" & return & "<vsn:6>" & return & finallist as string
set AppleScript's text item delimiters to tid
(* result
<ANSI-MAC>
<vsn:6>
<pstyle:branche>butcher
<pstyle:rest>name1 name2 name3
<pstyle:rest>strasse
<pstyle:rest>ort


<pstyle:rest>name1 name2 name3
<pstyle:rest>strasse
<pstyle:rest>ort


<pstyle:rest>name1 name2 name3
<pstyle:rest>strasse
<pstyle:rest>ort


<pstyle:rest>name1 name2 name3
<pstyle:rest>strasse
<pstyle:rest>ort


<pstyle:branche>Gärtner
<pstyle:rest>name1 name2 name3
<pstyle:rest>strasse
<pstyle:rest>ort


<pstyle:rest>name1 name2 name3
<pstyle:rest>strasse
<pstyle:rest>ort
 and so on ...
*)

[Craig_Williams]

A 5000 line file parsed in .07 sec.

Call from AppleScript like this:

set converted_text to do shell script "ruby /path/to/ruby_file.rb /path/to/tab_file.txt"

Save this with a .rb extension.

#!/usr/bin/env ruby -w

if ARGV.size != 1
puts “Supply a path to a tab delimited file to parse”
puts “\tExample: #{FILE} /Users/user_name/Desktop/tab_text.txt”
exit
end

file_path = ARGV[0]
if !File.exists?(file_path)
puts “Error: provided file does not exist”
exit
end

mytext = File.read(file_path)
@branche = nil
converted_text = [“”, “vsn:6”]

mytext.each_line do |line|
line_components = line.split(“\s”)
branche = line_components[0]
rest1 = line_components[1…3].join(’ ')
rest2 = line_components[4]
rest3 = line_components[5]
if @branche != branche
converted_text << “pstyle:branche#{branche}”
end
@branche = branche
[rest1, rest2, rest3].each do |rest|
converted_text << “pstyle:rest#{rest}”
end
converted_text << “” # delete this line to remove empty lines in output
end

print converted_text.join(“\n”)

[Bruce_Phillips]

@Craig Williams

Save this with a .rb extension.

I was going to do this with Ruby as well, but you beat me to it.

[Craig_Williams]

@Bruce Phillips

I was going to do this with Ruby as well, but you beat me to it.

It has truly become my favorite language. It is a joy to program with.

[Hans-Gerd_Classen]

Hello Craig,

wow, this is really rubyfast and it’ll give me a pretty good start on my monday morning.

thx a lot

Hans-Gerd Classen

[Craig_Williams]

You’re welcome.