Bash for loop Script or Service to Copy Folders and Archive in one go?

Hello,

I often archive my document folders manually and I’d like to automate the process.
I just don’t know if I should use automator, applescript or bash.
my example structure looks like this:

Capitol_A02635 (folder)
Source(subfolder)
Proof(subfolder)
Backup(subfolder)
Mech(subfolder)
Print(subfolder)

Inside the subfolders there are several other folders.
My archive process is to copy All the folders except Backup, into Backup.
Once copied, zip Backup.
Remove all files inside the rest of the folders (except Backup and Print).
Done.

I normally only do this once a week for several jobs but as I grow my library I seem to need automation. Could someone direct me to a source?
I’d like to do it with bash for loop or as a mac automator service, whichever is simpler.

Please note. I’m not asking for a scripting service. I am an applescript student and I don’t mind sweating blood to get the information, which I know is…out there… Thanks.

Here is a script which does some of the tasks.
As is, the zip task must be added.
Doing it with Shell script is easy but I’m searching for a way to achieve it with ASObjC.

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on moveItemAt:POSIXPath toSubFolder:subFolderName
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to sourceURL's URLByDeletingLastPathComponent
	set destURL to destURL's URLByAppendingPathComponent:subFolderName
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
set subFolders to {"Source:", "Proof:", "Mech:", "Print:"}

repeat with aSub in subFolders
	(my moveItemAt:(POSIX path of (mainFolder & aSub)) toSubFolder:"Backup")
end repeat

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mardi 5 mai 2020 18:52:23

Alas, you won’t find it. Well, it can be done – but only as of macOS 10.15.

Thank you Shane.
I found infos about different COMPRESSION scheme but no way to use them.
Now I know that I wasn’t missing something.

Maybe some readers would be interested to know how it may be done in 10.15.

Here is my proposal with the zip part.
I choose to date stamp the archive, you may drop that by editing a dedicated property.

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to sourceURL's URLByDeletingLastPathComponent
	set destURL to destURL's URLByAppendingPathComponent:subFolderName
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
set subFolders to {"Source:", "Proof:", "Mech:", "Print:"}
set subFolderName to "Backup"
repeat with aSub in subFolders
	(my moveItemAt:(POSIX path of (mainFolder & aSub)) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
	tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
	set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"
do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 mai 2020 11:49:08

I finally got around to testing what I thought should work — only to find that it doesn’t, after all.

Thank you Shane

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 mai 2020 15:11:55

Yvan, thank you for the effort.
upon running the script I run into the following error: "Error Number: -2700

“Source” couldn’t be moved to “Backup” because either the former doesn’t exist, or the folder containing the latter doesn’t exist." I have a feeling it’s because I am only backing up the contents of Capitol_A02635 not the folder itself. It also occurred to me that my folder names will change always, so I was looking for a more “generic” form of script. like a bash for loop which will act upon any folder with the already mentioned subfolders.

My suggestion with a few comments:

  • The first line of the script sets the base folder and should be changed to whatever is applicable. The base folder could be set by way of a choose-folder dialog if that’s desired.

  • The script will display a files-not-copied error if a source folder does not exist. The script could easily be modified to avoid this.

  • A good backup of everything should be kept until the script is well-tested.


set baseFolder to "/Users/Robert/Capitol_A02635/"
set backupFolder to quoted form of (baseFolder & "Backup")
set archiveFile to quoted form of (baseFolder & "Backup Archive.zip")
set rsyncFolders to {"Source", "Proof", "Mech", "Print"}
set rmFolders to {"Source", "Proof", "Mech"}

repeat with aFolder in rsyncFolders
	set contents of aFolder to (quoted form of (baseFolder & aFolder)) & space
end repeat

repeat with aFolder in rmFolders
	set contents of aFolder to (quoted form of (baseFolder & aFolder & "/")) & "*" & space
end repeat

try
	do shell script "rsync --archive --update " & (rsyncFolders as text) & space & backupFolder
on error
	display alert "Files not copied. A source folder may not exist." as critical
	error number -128
end try

try
	do shell script "ditto -ck " & backupFolder & space & archiveFile
on error
	display alert "Files copied but archive not created." as critical
	error number -128
end try

display dialog "Delete files?" with icon caution

do shell script "rm -rf " & (rmFolders as text)

I don’t understand.
I started with a folder named “Capitol_A02635” containing :
Mech – a folder with the new contents
Print – a folder with the new contents
Proof – a folder with the new contents
Source – a folder with the new contents
Backup – a folder containing 4 subfolders
Mech – a subfolder with old contents
Print – a subfolder with old contents
Proof – a subfolder with old contents
Source – a subfolder with old contents

After execution I have
Backup – a folder containing 4 subfolders
Mech – a subfolder with the new contents
Print – a subfolder with the new contents
Proof – a subfolder with the new contents
Source – a subfolder with the new contents
Backup_20200506_183025.zip

I assumed that the contained subfolders were named as described in you message.
If the names may be different it’s easy to edit the script accordingly.

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to sourceURL's URLByDeletingLastPathComponent
	set destURL to destURL's URLByAppendingPathComponent:subFolderName
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
tell application "System Events"
	set subfolders to path of every folder of folder mainFolder whose name is not "Backup"
end tell

set subFolderName to "Backup"
repeat with aSub in subfolders
	(my moveItemAt:(POSIX path of aSub ) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
	tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
	set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"
do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

After running this version, the folder contain:

Backup_20200506_183025.zip – the old one
Backup_20200507_124006.zip – the new one
Backup
Prouf – new subfolder
Sourcez – new subfolder
Print – new subfolder replacing the old one
Mechanic – new subfolder
Mech – old subfolder
Proof – old subfolder
Source – old subfolder

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) jeudi 7 mai 2020 12:45:03

I tested Yvan’s script and it completed without error. Yvan’s script deletes the identified subfolders; my script deletes the files and folders inside the subfolders (except the Print subfolder) but not the actual subfolders. The OP is not clear on how he wants this handled but should be able to sort this out on his own.

Thank yo Peavine.

The FreddieMac’s comment was upon my first script which was written assuming that its folder was matching the given description.
In fact it didn’t. Some named subfolders were unavailable (at least the one named : “Source”) and maybe others were available.
My second proposal extract a list of the subfolders available and apply the backup process to those whose name isn’t “Backup”.
It’s always the same problem : to get a correct answer we need to receive a correct question. As I am not a sooth sayer I can’t guess that the asker give only an incomplete description of the real problem.
It’s not the first time I face that and, alas it will not be the last one.
It seems that some askers imagine that we are seeing all what is available on their screen.
It’s not a huge problem, just a bit boring one.

My script has no need to delete the original subfolders from the main folder because it doesn’t apply a Copy task but a Move one which requires less work.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) vendredi 8 mai 2020 16:53:36

Here the OP has already confused you both. If he already has an old Backup.zip file (and not a folder), then you cannot move or duplicate anything into it.

  1. Backup-folder should be created every time new, and the old Backup.zip file will be destroyed (only after the successful creation of a new Backup.zip file).

  2. Only 3 folders should be moved, and the Print folder should be duplicated to Backup-folder (not copied). Maybe copying will work too (I’ve never tested something like that), but I think, just in case, it’s better to work with an independent duplicate of the Print folder.

So, the workaround:

  1. Create new folder named newBackup.
  2. Move 3 folders to it, duplicate folder Print to it.
  3. Make zip of folder newBackup.
  4. Delete Backup.zip file, delete newBackup folder
  5. Rename newBackup.zip file to Backup.zip

You may be right but it’s not what the OP wrote:

What is true is that in my late proposal I forgot to treat the folder Print a different way.
Here is the corrected version.

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
	set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
	set theName to sourceURL's lastPathComponent()
	set destURL to sourceURL's URLByDeletingLastPathComponent
	set destURL to destURL's URLByAppendingPathComponent:subFolderName
	set destURL to destURL's URLByAppendingPathComponent:theName
	set fileManager to current application's NSFileManager's defaultManager()
	if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
		-- the folder already exists, delete it
		fileManager's removeItemAtURL:destURL |error|:(missing value)
	end if
	
	if theName as string is "Print" then
		set {theResult, theError} to fileManager's copyItemAtURL:sourceURL toURL:destURL |error|:(reference)
	else
		set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
	end if
	
	if (theResult as boolean) is false then
		error (theError's localizedDescription() as text)
	end if
	
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
tell application "System Events"
	set subfolders to path of every folder of folder mainFolder whose name is not "Backup"
end tell

set subFolderName to "Backup"
repeat with aSub in subfolders
	(my moveItemAt:(POSIX path of aSub) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
	tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
	set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"

do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

It moves every subfolder minus the Print one which is copied so the oritinal remains in the main folder.

So, after execution, the folder Capitol_A02635 contain:

Backup
Mech
Mechanical
Print
dossier sans titre
Proof
Prouf
Source
Backup_20200509_105340.zip
Backup_20200509_105956.zip
Backup_20200509_110404.zip
Print
dossier sans titre

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 11:06:40

I still have a vague suspicion that the OP does not need old backups. This is just logical. Most likely, the OP itself should clarify this. After all, old zipped backups perform the opposite task of compressing a folder (saving disk space). So,

Backup(subfolder) may be 1) last Backup.zip file with hidden extension 2) Backup (subfolder), which contains only 1 thing in it - last Backup.zip file

The OP has several working scripts to choose from and should be able to do whatever fine-tuning is required.

I use SuperDuper for regular backups. However, I also use an AppleScript on a frequent basis to backup many hundreds of data files (no zip file created). In writing this script, an important issue became how to backup the changed files without copying files that have not changed. In other words, I don’t want to copy 500 files to a backup when only 10 of those files have changed. That’s why I use rsync with the -update option.

When I wrote the backup script several years ago, I looked at AppleScript and Bash commands that have both an update option and that act recursively but couldn’t find any. I have to be able to maintain this script so ASObjC is out for me. It’s time I reviewed my backup script and will look to see if there’s an option other than rsync.

The cp command included with some linux distributions had an update option but that doesn’t appear to be the case with the macOS cp command. I also couldn’t find an update option with ditto. I’ll do some more research on this and do some timing tests to see what works best.

Copying files and moving is not same thing. Moving files is only renaming them. And, zipping all together is more effective regarding to saved space of the disk.

I want to copy files not move them. The same appears to be the case with the OP and the Print subfolder. For me, saving space is not necessary.

Just to clarify, if I use the Finder duplicate command with the overwrite option, all 500 files in my example are copied even if only 10 files have changed. This seems a waste of time. With rsync and -update, only 10 files are copied. I will run some timing test to see if that’s correct.

Keeping successive archives may be useful.

When we run the script at day 1, the created zip file will contain what was in the Backup folder when it was called. The folder Backup will now contain the subfolders available in Capitol at day 1.

When we run the script at day 2, the created zip file will contain what was in the Backup folder which are the subfolders from day 1. The folder Backup will now contain the files available in Capitol at day 2.

When we run the script at day 3, the created zip file will contain what was in the Backup folder which are the subfolders from day 2. The folder Backup will now contain the files available in Capitol at day 3.

Nothing prove that the datas archived in the 1st zip are available in the 2nd zip or in the 3rd one.
In fact I’m quite sure that except those in subfolder “Print”, they aren’t.

I apologize but I have no reason to imagine that when somebody write about a folder named “Backup” he is in fact writing about a zip file whose extension is hidden.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 16:44:33

When itemA and itemB aren’t on the same device the operation would be different but when itemA and itemB are on the same volume (which is the case here), move itemA to itemB free the original contents of itemB, edit the descriptor of itemB so that it points to the datas of itemA and at last, removes the descriptor of itemA. We have the contents of itemA in a single area of the device.

In every cases, copy itemA to itemB replace the original contents of itemB by the contents of itemA so that it point to an area of the device different than the one pointed by itemA (but may be the area originally used by itemB) and itemA remains where it was. We have the contents of itemA in two areas of the device.

I described the way both commands behave under High Sierra.
If I remember well, under Catalina it’s different. Copy an item just create a new ‘descriptor’ of the item so that we may have 10 copies but have only one physical ‘copy’ of the embedded datas.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 17:15:17

I ran some timing tests with Script Geek. The source folder is on my boot drive and contains 479 files; the destination folder is on a USB 3 external SSD. The first reported time is with the destination folder empty. I then added two small files in the source folder and immediately reran the script. That’s the second reported time. Finally, I repeated these same tests with rsync and Finder but with a thumb drive as the destination drive.

-- Finder
tell application "Finder"
	duplicate folder "Macintosh HD:Users:Robert:Records:" to folder "Save:Test:finder:" with replacing
end tell
-- 838 milliseconds. 991 milliseconds. SSD.
-- 13.498 seconds. 13.754 seconds. Thumb drive.
-- Rsync
do shell script "/usr/bin/rsync --archive --update " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/rsync"
-- 1304 milliseconds. 92 milliseconds. SSD.
-- 13.580 seconds. 1.101 seconds. Thumb drive.
-- Ditto
do shell script "ditto " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/ditto"
-- 508 milliseconds. 532 milliseconds. SSD.
-- Copy (cp)
do shell script "cp -R " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/cp"
-- 483 milliseconds. 486 milliseconds. SSD.

ASObjC script in post 23 below.
1060 milliseconds. 884 milliseconds. SSD.

If most files already exist in the destination folder–which is normally the case for me–rsync with update option seems preferable.