Bash for loop Script or Service to Copy Folders and Archive in one go?

My suggestion with a few comments:

  • The first line of the script sets the base folder and should be changed to whatever is applicable. The base folder could be set by way of a choose-folder dialog if that’s desired.

  • The script will display a files-not-copied error if a source folder does not exist. The script could easily be modified to avoid this.

  • A good backup of everything should be kept until the script is well-tested.


set baseFolder to "/Users/Robert/Capitol_A02635/"
set backupFolder to quoted form of (baseFolder & "Backup")
set archiveFile to quoted form of (baseFolder & "Backup Archive.zip")
set rsyncFolders to {"Source", "Proof", "Mech", "Print"}
set rmFolders to {"Source", "Proof", "Mech"}

repeat with aFolder in rsyncFolders
	set contents of aFolder to (quoted form of (baseFolder & aFolder)) & space
end repeat

repeat with aFolder in rmFolders
	set contents of aFolder to (quoted form of (baseFolder & aFolder & "/")) & "*" & space
end repeat

try
	do shell script "rsync --archive --update " & (rsyncFolders as text) & space & backupFolder
on error
	display alert "Files not copied. A source folder may not exist." as critical
	error number -128
end try

try
	do shell script "ditto -ck " & backupFolder & space & archiveFile
on error
	display alert "Files copied but archive not created." as critical
	error number -128
end try

display dialog "Delete files?" with icon caution

do shell script "rm -rf " & (rmFolders as text)

I don’t understand.
I started with a folder named “Capitol_A02635” containing :
Mech – a folder with the new contents
Print – a folder with the new contents
Proof – a folder with the new contents
Source – a folder with the new contents
Backup – a folder containing 4 subfolders
Mech – a subfolder with old contents
Print – a subfolder with old contents
Proof – a subfolder with old contents
Source – a subfolder with old contents

After execution I have
Backup – a folder containing 4 subfolders
Mech – a subfolder with the new contents
Print – a subfolder with the new contents
Proof – a subfolder with the new contents
Source – a subfolder with the new contents
Backup_20200506_183025.zip

I assumed that the contained subfolders were named as described in you message.
If the names may be different it’s easy to edit the script accordingly.

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to sourceURL's URLByDeletingLastPathComponent
	set destURL to destURL's URLByAppendingPathComponent:subFolderName
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
tell application "System Events"
	set subfolders to path of every folder of folder mainFolder whose name is not "Backup"
end tell

set subFolderName to "Backup"
repeat with aSub in subfolders
	(my moveItemAt:(POSIX path of aSub ) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
	tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
	set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"
do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

After running this version, the folder contain:

Backup_20200506_183025.zip – the old one
Backup_20200507_124006.zip – the new one
Backup
Prouf – new subfolder
Sourcez – new subfolder
Print – new subfolder replacing the old one
Mechanic – new subfolder
Mech – old subfolder
Proof – old subfolder
Source – old subfolder

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) jeudi 7 mai 2020 12:45:03

I tested Yvan’s script and it completed without error. Yvan’s script deletes the identified subfolders; my script deletes the files and folders inside the subfolders (except the Print subfolder) but not the actual subfolders. The OP is not clear on how he wants this handled but should be able to sort this out on his own.

Thank yo Peavine.

The FreddieMac’s comment was upon my first script which was written assuming that its folder was matching the given description.
In fact it didn’t. Some named subfolders were unavailable (at least the one named : “Source”) and maybe others were available.
My second proposal extract a list of the subfolders available and apply the backup process to those whose name isn’t “Backup”.
It’s always the same problem : to get a correct answer we need to receive a correct question. As I am not a sooth sayer I can’t guess that the asker give only an incomplete description of the real problem.
It’s not the first time I face that and, alas it will not be the last one.
It seems that some askers imagine that we are seeing all what is available on their screen.
It’s not a huge problem, just a bit boring one.

My script has no need to delete the original subfolders from the main folder because it doesn’t apply a Copy task but a Move one which requires less work.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) vendredi 8 mai 2020 16:53:36

Here the OP has already confused you both. If he already has an old Backup.zip file (and not a folder), then you cannot move or duplicate anything into it.

  1. Backup-folder should be created every time new, and the old Backup.zip file will be destroyed (only after the successful creation of a new Backup.zip file).

  2. Only 3 folders should be moved, and the Print folder should be duplicated to Backup-folder (not copied). Maybe copying will work too (I’ve never tested something like that), but I think, just in case, it’s better to work with an independent duplicate of the Print folder.

So, the workaround:

  1. Create new folder named newBackup.
  2. Move 3 folders to it, duplicate folder Print to it.
  3. Make zip of folder newBackup.
  4. Delete Backup.zip file, delete newBackup folder
  5. Rename newBackup.zip file to Backup.zip

You may be right but it’s not what the OP wrote:

What is true is that in my late proposal I forgot to treat the folder Print a different way.
Here is the corrected version.

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to sourceURL's URLByDeletingLastPathComponent
	set destURL to destURL's URLByAppendingPathComponent:subFolderName
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	
	if theName as string is "Print" then
		set {theResult, theError} to fileManager's copyItemAtURL:sourceURL toURL:destURL |error|:(reference)
	else
		set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	end if
	
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
	
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
tell application "System Events"
	set subfolders to path of every folder of folder mainFolder whose name is not "Backup"
end tell

set subFolderName to "Backup"
repeat with aSub in subfolders
	(my moveItemAt:(POSIX path of aSub) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
	tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
	set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"

do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

It moves every subfolder minus the Print one which is copied so the oritinal remains in the main folder.

So, after execution, the folder Capitol_A02635 contain:

Backup
Mech
Mechanical
Print
dossier sans titre
Proof
Prouf
Source
Backup_20200509_105340.zip
Backup_20200509_105956.zip
Backup_20200509_110404.zip
Print
dossier sans titre

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 11:06:40

I still have a vague suspicion that the OP does not need old backups. This is just logical. Most likely, the OP itself should clarify this. After all, old zipped backups perform the opposite task of compressing a folder (saving disk space). So,

Backup(subfolder) may be 1) last Backup.zip file with hidden extension 2) Backup (subfolder), which contains only 1 thing in it - last Backup.zip file

The OP has several working scripts to choose from and should be able to do whatever fine-tuning is required.

I use SuperDuper for regular backups. However, I also use an AppleScript on a frequent basis to backup many hundreds of data files (no zip file created). In writing this script, an important issue became how to backup the changed files without copying files that have not changed. In other words, I don’t want to copy 500 files to a backup when only 10 of those files have changed. That’s why I use rsync with the -update option.

When I wrote the backup script several years ago, I looked at AppleScript and Bash commands that have both an update option and that act recursively but couldn’t find any. I have to be able to maintain this script so ASObjC is out for me. It’s time I reviewed my backup script and will look to see if there’s an option other than rsync.

The cp command included with some linux distributions had an update option but that doesn’t appear to be the case with the macOS cp command. I also couldn’t find an update option with ditto. I’ll do some more research on this and do some timing tests to see what works best.

Copying files and moving is not same thing. Moving files is only renaming them. And, zipping all together is more effective regarding to saved space of the disk.

I want to copy files not move them. The same appears to be the case with the OP and the Print subfolder. For me, saving space is not necessary.

Just to clarify, if I use the Finder duplicate command with the overwrite option, all 500 files in my example are copied even if only 10 files have changed. This seems a waste of time. With rsync and -update, only 10 files are copied. I will run some timing test to see if that’s correct.

Keeping successive archives may be useful.

When we run the script at day 1, the created zip file will contain what was in the Backup folder when it was called. The folder Backup will now contain the subfolders available in Capitol at day 1.

When we run the script at day 2, the created zip file will contain what was in the Backup folder which are the subfolders from day 1. The folder Backup will now contain the files available in Capitol at day 2.

When we run the script at day 3, the created zip file will contain what was in the Backup folder which are the subfolders from day 2. The folder Backup will now contain the files available in Capitol at day 3.

Nothing prove that the datas archived in the 1st zip are available in the 2nd zip or in the 3rd one.
In fact I’m quite sure that except those in subfolder “Print”, they aren’t.

I apologize but I have no reason to imagine that when somebody write about a folder named “Backup” he is in fact writing about a zip file whose extension is hidden.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 16:44:33

When itemA and itemB aren’t on the same device the operation would be different but when itemA and itemB are on the same volume (which is the case here), move itemA to itemB free the original contents of itemB, edit the descriptor of itemB so that it points to the datas of itemA and at last, removes the descriptor of itemA. We have the contents of itemA in a single area of the device.

In every cases, copy itemA to itemB replace the original contents of itemB by the contents of itemA so that it point to an area of the device different than the one pointed by itemA (but may be the area originally used by itemB) and itemA remains where it was. We have the contents of itemA in two areas of the device.

I described the way both commands behave under High Sierra.
If I remember well, under Catalina it’s different. Copy an item just create a new ‘descriptor’ of the item so that we may have 10 copies but have only one physical ‘copy’ of the embedded datas.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 17:15:17

I ran some timing tests with Script Geek. The source folder is on my boot drive and contains 479 files; the destination folder is on a USB 3 external SSD. The first reported time is with the destination folder empty. I then added two small files in the source folder and immediately reran the script. That’s the second reported time. Finally, I repeated these same tests with rsync and Finder but with a thumb drive as the destination drive.

-- Finder
tell application "Finder"
	duplicate folder "Macintosh HD:Users:Robert:Records:" to folder "Save:Test:finder:" with replacing
end tell
-- 838 milliseconds. 991 milliseconds. SSD.
-- 13.498 seconds. 13.754 seconds. Thumb drive.
-- Rsync
do shell script "/usr/bin/rsync --archive --update " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/rsync"
-- 1304 milliseconds. 92 milliseconds. SSD.
-- 13.580 seconds. 1.101 seconds. Thumb drive.
-- Ditto
do shell script "ditto " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/ditto"
-- 508 milliseconds. 532 milliseconds. SSD.
-- Copy (cp)
do shell script "cp -R " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/cp"
-- 483 milliseconds. 486 milliseconds. SSD.

ASObjC script in post 23 below.
1060 milliseconds. 884 milliseconds. SSD.

If most files already exist in the destination folder–which is normally the case for me–rsync with update option seems preferable.

May you try with the ASObjC script ?

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 18:07:19

I’d be happy to do that but you would have to supply the working code. The source folder would be as shown above. The destination folder would be:

“Save:Test:asobjc:”

You may test that:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on copyItemAt:POSIXPath toFolder:POSIXDest
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to current application's class "NSURL"'s fileURLWithPath:POSIXDest
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	if theName as string is "Records" then
		set {theResult, theError} to fileManager's copyItemAtURL:sourceURL toURL:destURL |error|:(reference)
	else
		set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	end if
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
end copyItemAt:toFolder:


-- If you try with a sourceFolder named "Records" it will be copied
-- If it's not named "Records" it will be moved
my copyItemAt:"/Users/Robert/Records" toFolder:"/Volumes/Save/Test/asobjc/"

(*
-- I tested with :
set sourceContainer to path to desktop as string
set destContainer to my remplace(sourceContainer, "SSD 1000", "Macintosh HD")
set folderToCopy to sourceContainer & "Print:" -- or "ableton:"
my copyItemAt:(POSIX path of folderToCopy) toFolder:(POSIX path of destContainer)
*)

#=====
(*
replaces every occurences of d1 by d2 in the text t
*)
on remplace(t, d1, d2)
	local oTIDs, l
	set {oTIDs, AppleScript's text item delimiters} to {AppleScript's text item delimiters, d1}
	set l to text items of t
	set AppleScript's text item delimiters to d2
	set t to l as text
	set AppleScript's text item delimiters to oTIDs
	return t
end remplace

#=====

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 19:08:16

Thank you.

As shell is not my cup of tea, I’m not really concerned by what is available and what isn’t in this area.

At this time I just wish to know which behavior is the one really wanted by the OP.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 23:29:39

That’s not the case with APFS volumes. Copying works very much like moving, with seperate contents only made when one or the other is modified (and even then, only of the modified part).

Yvan,

You could make your script more efficient by ignoring files that have not changed, something like this:

if not (fileManager's contentsEqualAtPath:POSIXPath andPath:(destURL's |path|())) then

But that’s only going to work with files, not folders or packages.

Thank you Shane but I tried to achieve what was described by the OP:

If I read well, it was asked to copy the folders with their subfolders, not to copy only newly added or changed items.
It’s the way I work for my files because I like to keep old versions which may prove to be useful one day. I retrieve them in the date stamped archives.
It resemble to what is done by Time Machine which keep the old versions (as well as the hidden folder .DocumentRevisions-V100 from which I extracted numerous Pages or Numbers documents to help really annoyed askers).
If I understand well the schemes proposed by Peavine replace the old versions by the new ones.
I don’t criticize this behavior, it’s just not what was asked (at least the way I understood the question which may be the wrong one).

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) dimanche 10 mai 2020 10:44:06