Merge csv files

Hi,

I have several UTF-8 CSV files which I would like to merge.

They are import files that are sent out to be translated and return in the same format.

They are basically the same table with only 2 columns being different, the language code and the value.

I would like to write an AppleScript that asks to select the files and merges them in 1 single table in a file, skipping the first header row of every file apart from the first one.

Anybody could help? :slight_smile:

Something like this?

set theFiles to choose file with multiple selections allowed
set saveFile to POSIX path of (((path to desktop) as text) & "MergedCSVFile.csv")

do shell script "cp " & quoted form of POSIX path of item 1 of theFiles & space & quoted form of saveFile
if (count of theFiles) = 1 then return
repeat with fileToAdd in theFiles
	set x to quoted form of POSIX path of fileToAdd
	do shell script "cat " & x & " | awk 'BEGIN {getline}{print $0}' >> " & quoted form of saveFile
end repeat

Yep, seems fine, MergedCSVFile.csv only contains the first file though?

Then the CSV file has probably a row delimiter return instead of linefeed. You can change that with AWK

Hmmm sorry I’m not sure I understand? :confused: :slight_smile:

Different OS have different line delimiters

linefeed for Unix systems including Mac OS X
linefeed followed by a return on my BBC Basic machine
returned followed by linefeed for Window systems
return for Mac OS 9 and earlier.

CSV saved on a Windows and send to Mac OS have other line delimiters as the AWK command expects. Therefore the line delimiters for AWK has to be changed which you can do in AWK by setting the RS (row separator) property

set theFiles to choose file with multiple selections allowed
set saveFile to POSIX path of (((path to desktop) as text) & "MergedCSVFile.csv")

do shell script "cp " & quoted form of POSIX path of item 1 of theFiles & space & quoted form of saveFile
if (count of theFiles) = 1 then return
repeat with fileToAdd in theFiles
	set x to quoted form of POSIX path of fileToAdd
	do shell script "cat " & x & " | awk 'BEGIN {RS=\"\\r\\n\"; getline}{print $0}' >> " & quoted form of saveFile
end repeat

You still have a Beeb?! :slight_smile:

Oh seems to be working great now, will keep testing.

Sometimes I get xls/xlsx files, is it possible to have an if clause to convert xls/xlsx files to csv UTF-8?

Thanks

Here is a simple code doing the trick without calling awk or cat


--[SCRIPT]
set theFiles to choose file of type {"public.comma-separated-values-text"} with multiple selections allowed
set saveFile to ((path to desktop) as text) & "MergedCSVFile.csv"
my writeTo(saveFile, my recolle(paragraphs of (read item 1 of theFiles), return), text, false) # EDITED
repeat with n from 2 to count of theFiles
	my writeTo(saveFile, return & my recolle(items 2 thru -1 of paragraphs of (read item n of theFiles), return), text, true)
end repeat

--=====

on recolle(l, d)
	local oTIDs, t
	set oTIDs to AppleScript's text item delimiters
	set AppleScript's text item delimiters to d
	set t to "" & l
	set AppleScript's text item delimiters to oTIDs
	return t
end recolle

--=====
(*
Handler borrowed to Regulus6633 - http://macscripter.net/viewtopic.php?id=36861
*)
on writeTo(targetFile, theData, dataType, apendData)
	-- targetFile is the path to the file you want to write
	-- theData is the data you want in the file.
	-- dataType is the data type of theData and it can be text, list, record etc.
	-- apendData is true to append theData to the end of the current contents of the file or false to overwrite it
	try
		set targetFile to targetFile as text
		set openFile to open for access file targetFile with write permission
		if not apendData then set eof of openFile to 0
		write theData to openFile starting at eof as dataType
		close access openFile
		return true
	on error
		try
			close access file targetFile
		end try
		return false
	end try
end writeTo

--=====
--[/SCRIPT]

Yvan KOENIG (VALLAURIS, France) vendredi 7 septembre 2012 18:19:49

Yvan, that works too but loses the first row?

Oops, I read too fast your first message.

I will edit the message with the script.

Yvan KOENIG (VALLAURIS, France) vendredi 7 septembre 2012 19:29:03

Yes I do, I learned a lot of programming on that machine when I was a kid. Unfortunately the external floppy drive is broken and I haven’t started it up again in almost ten years from now but I won’t throw it away. Maybe when I find a compatible floppy drive so I can save my code again, I will give it spin again

Is it possible to insert an if clause to convert xls|xlsx files into UTF-8 csv?

The script at this link seems to work but it breaks the columns when there’s a comma.

http://apple.stackexchange.com/a/15690

So it would be:
“ Select files to merge

  • If they are xls|xlsx files convert them to UTF-8 csv
  • Type output filename and select output folder
  • Merge keeping the header row only of the first file