Saturday, July 4, 2020

#1 2020-05-05 09:58:45 am

FreddieMac
Member
Registered: 2019-11-14
Posts: 38

Bash for loop Script or Service to Copy Folders and Archive in one go?

Hello,

I often archive my document folders manually and I'd like to automate the process.
I just don't know if I should use automator, applescript or bash.
my example structure looks like this:

Capitol_A02635 (folder)
  Source(subfolder)
  Proof(subfolder)
  Backup(subfolder)
  Mech(subfolder)
  Print(subfolder)

Inside the subfolders there are several other folders.
My archive process is to copy All the folders except Backup, into Backup.
Once copied, zip Backup.
Remove all files inside the rest of the folders (except Backup and Print).
Done.

I normally only do this once a week for several jobs but as I grow my library I seem to need automation. Could someone direct me to a source?
I'd like to do it with bash for loop or as a mac automator service, whichever is simpler.

Please note. I'm not asking for a scripting service. I am an applescript student and I don't mind sweating blood to get the information, which I know is....out there.. Thanks.


Filed under: applescript, Automator, bash, os x

Offline

 

#2 2020-05-05 10:53:07 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Here is a script which does some of the tasks.
As is, the zip task must be added.
Doing it with Shell script is easy but I'm searching for a way to achieve it with ASObjC.

Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on moveItemAt:POSIXPath toSubFolder:subFolderName
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to sourceURL's URLByDeletingLastPathComponent
   set destURL to destURL's URLByAppendingPathComponent:subFolderName
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
set subFolders to {"Source:", "Proof:", "Mech:", "Print:"}

repeat with aSub in subFolders
   (my moveItemAt:(POSIX path of (mainFolder & aSub)) toSubFolder:"Backup")
end repeat

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mardi 5 mai 2020 18:52:23

Last edited by Yvan Koenig (2020-05-05 11:01:46 am)

Offline

 

#3 2020-05-05 05:42:33 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 6395

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Yvan Koenig wrote:

Doing it with Shell script is easy but I'm searching for a way to achieve it with ASObjC.



Alas, you won't find it. Well, it can be done -- but only as of macOS 10.15.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/
latenightsw.com

Offline

 

#4 2020-05-06 03:49:15 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Thank you Shane.
I found infos about different COMPRESSION scheme but no way to use them.
Now I know that I wasn't missing something.

Maybe some readers would be interested to know how it may be done in 10.15.

Here is my proposal with the zip part.
I choose to date stamp the archive, you may drop that by editing a dedicated property.

Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to sourceURL's URLByDeletingLastPathComponent
   set destURL to destURL's URLByAppendingPathComponent:subFolderName
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
set subFolders to {"Source:", "Proof:", "Mech:", "Print:"}
set subFolderName to "Backup"
repeat with aSub in subFolders
   (my moveItemAt:(POSIX path of (mainFolder & aSub)) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
   tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
   set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"
do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 mai 2020 11:49:08

Last edited by Yvan Koenig (2020-05-06 04:27:33 am)

Offline

 

#5 2020-05-06 05:57:11 am

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 6395

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Yvan Koenig wrote:

Maybe some readers would be interested to know how it may be done in 10.15.



I finally got around to testing what I thought should work — only to find that it doesn't, after all.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/
latenightsw.com

Offline

 

#6 2020-05-06 07:12:00 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Thank you Shane

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 mai 2020 15:11:55

Offline

 

#7 2020-05-06 02:58:25 pm

FreddieMac
Member
Registered: 2019-11-14
Posts: 38

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Yvan Koenig wrote:

Here is a script which does some of the tasks.
As is, the zip task must be added.
Doing it with Shell script is easy but I'm searching for a way to achieve it with ASObjC.

Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on moveItemAt:POSIXPath toSubFolder:subFolderName
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to sourceURL's URLByDeletingLastPathComponent
   set destURL to destURL's URLByAppendingPathComponent:subFolderName
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
set subFolders to {"Source:", "Proof:", "Mech:", "Print:"}

repeat with aSub in subFolders
   (my moveItemAt:(POSIX path of (mainFolder & aSub)) toSubFolder:"Backup")
end repeat

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mardi 5 mai 2020 18:52:23


Yvan, thank you for the effort.
upon running the script I run into the following error: "Error Number: -2700

“Source” couldn’t be moved to “Backup” because either the former doesn't exist, or the folder containing the latter doesn't exist." I have a feeling it's because I am only backing up the contents of Capitol_A02635 not the folder itself. It also occurred to me that my folder names will change always, so I was looking for a more "generic" form of script. like a bash for loop which will act upon any folder with the already mentioned subfolders.

Offline

 

#8 2020-05-06 08:35:02 pm

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 530

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

My suggestion with a few comments:

* The first line of the script sets the base folder and should be changed to whatever is applicable. The base folder could be set by way of a choose-folder dialog if that's desired.

* The script will display a files-not-copied error if a source folder does not exist. The script could easily be modified to avoid this.

* A good backup of everything should be kept until the script is well-tested.

Applescript:


set baseFolder to "/Users/Robert/Capitol_A02635/"
set backupFolder to quoted form of (baseFolder & "Backup")
set archiveFile to quoted form of (baseFolder & "Backup Archive.zip")
set rsyncFolders to {"Source", "Proof", "Mech", "Print"}
set rmFolders to {"Source", "Proof", "Mech"}

repeat with aFolder in rsyncFolders
   set contents of aFolder to (quoted form of (baseFolder & aFolder)) & space
end repeat

repeat with aFolder in rmFolders
   set contents of aFolder to (quoted form of (baseFolder & aFolder & "/")) & "*" & space
end repeat

try
   do shell script "rsync --archive --update " & (rsyncFolders as text) & space & backupFolder
on error
   display alert "Files not copied. A source folder may not exist." as critical
   error number -128
end try

try
   do shell script "ditto -ck " & backupFolder & space & archiveFile
on error
   display alert "Files copied but archive not created." as critical
   error number -128
end try

display dialog "Delete files?" with icon caution

do shell script "rm -rf " & (rmFolders as text)

Last edited by peavine (2020-05-10 09:21:57 am)


2018 Mac mini - macOS Catalina

Offline

 

#9 2020-05-07 04:47:24 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

FreddieMac wrote:
Yvan Koenig wrote:

Here is a script which does some of the tasks.
As is, the zip task must be added.
Doing it with Shell script is easy but I'm searching for a way to achieve it with ASObjC.

Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on moveItemAt:POSIXPath toSubFolder:subFolderName
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to sourceURL's URLByDeletingLastPathComponent
   set destURL to destURL's URLByAppendingPathComponent:subFolderName
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
set subFolders to {"Source:", "Proof:", "Mech:", "Print:"}

repeat with aSub in subFolders
   (my moveItemAt:(POSIX path of (mainFolder & aSub)) toSubFolder:"Backup")
end repeat

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mardi 5 mai 2020 18:52:23


Yvan, thank you for the effort.
upon running the script I run into the following error: "Error Number: -2700

“Source” couldn’t be moved to “Backup” because either the former doesn't exist, or the folder containing the latter doesn't exist." I have a feeling it's because I am only backing up the contents of Capitol_A02635 not the folder itself. It also occurred to me that my folder names will change always, so I was looking for a more "generic" form of script. like a bash for loop which will act upon any folder with the already mentioned subfolders.




I don't understand.
I started with a folder named "Capitol_A02635" containing :
Mech -- a folder with the new contents
Print -- a folder with the new contents
Proof -- a folder with the new contents
Source -- a folder with the new contents
Backup -- a folder containing 4 subfolders
    Mech -- a subfolder with old contents
    Print -- a subfolder with old contents
    Proof -- a subfolder with old contents
    Source -- a subfolder with old contents

After execution I have
Backup -- a folder containing 4 subfolders
    Mech -- a subfolder with the new contents
    Print -- a subfolder with the new contents
    Proof -- a subfolder with the new contents
    Source -- a subfolder with the new contents
Backup_20200506_183025.zip

I assumed that the contained subfolders  were named as described in you message.
If the names may be different it's easy to edit the script accordingly.


Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to sourceURL's URLByDeletingLastPathComponent
   set destURL to destURL's URLByAppendingPathComponent:subFolderName
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
tell application "System Events"
   set subfolders to path of every folder of folder mainFolder whose name is not "Backup"
end tell

set subFolderName to "Backup"
repeat with aSub in subfolders
   (my moveItemAt:(POSIX path of aSub ) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
   tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
   set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"
do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

After running this version, the folder contain:

Backup_20200506_183025.zip -- the old one
Backup_20200507_124006.zip -- the new one
Backup
    Prouf -- new subfolder
    Sourcez -- new subfolder
    Print -- new subfolder replacing the old one
    Mechanic -- new subfolder
    Mech -- old subfolder
    Proof -- old subfolder
    Source -- old subfolder


Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) jeudi 7 mai 2020 12:45:03

Last edited by Yvan Koenig (2020-05-08 08:44:09 am)

Offline

 

#10 2020-05-08 08:06:05 am

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 530

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

FreddieMac wrote:

Yvan, thank you for the effort.
upon running the script I run into the following error: "Error Number: -2700

“Source” couldn’t be moved to “Backup” because either the former doesn't exist, or the folder containing the latter doesn't exist." I have a feeling it's because I am only backing up the contents of Capitol_A02635 not the folder itself. It also occurred to me that my folder names will change always, so I was looking for a more "generic" form of script. like a bash for loop which will act upon any folder with the already mentioned subfolders.



I tested Yvan's script and it completed without error. Yvan's script deletes the identified subfolders; my script deletes the files and folders inside the subfolders (except the Print subfolder) but not the actual subfolders. The OP is not clear on how he wants this handled but should be able to sort this out on his own.

FreddieMac wrote:

Inside the subfolders there are several other folders.
My archive process is to copy All the folders except Backup, into Backup.
Once copied, zip Backup.
Remove all files inside the rest of the folders (except Backup and Print).
Done.


2018 Mac mini - macOS Catalina

Offline

 

#11 2020-05-08 08:55:57 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

peavine wrote:


I tested Yvan's script and it completed without error. Yvan's script deletes the identified subfolders; my script deletes the files and folders inside the subfolders (except the Print subfolder) but not the actual subfolders. The OP is not clear on how he wants this handled but should be able to sort this out on his own.

FreddieMac wrote:

Inside the subfolders there are several other folders.
My archive process is to copy All the folders except Backup, into Backup.
Once copied, zip Backup.
Remove all files inside the rest of the folders (except Backup and Print).
Done.



Thank yo Peavine.

The FreddieMac's comment was upon my first script which was written assuming that its folder was matching the given description.
In fact it didn't. Some named subfolders were unavailable (at least the one named : "Source") and maybe others were available.
My second proposal extract a list of the subfolders available and apply the backup process to those whose name isn't "Backup".
It's always the same problem : to get a correct answer we need to receive a correct question. As I am not a sooth sayer I can't guess that the asker give only an incomplete description of the real problem.
It's not the first time I face that and, alas it will not be the last one.
It seems that some askers imagine that we are seeing all what is available on their screen.
It's not a huge problem, just a bit boring one.

My script has no need to delete the original subfolders from the main folder because it doesn't apply a Copy task but a Move one which requires less work.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) vendredi 8 mai 2020 16:53:36

Last edited by Yvan Koenig (2020-05-08 08:59:34 am)

Offline

 

#12 2020-05-09 12:21:45 am

KniazidisR
Member
Registered: 2019-03-03
Posts: 1259

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Here the OP has already confused you both. If he already has an old Backup.zip file (and not a folder), then you cannot move or duplicate anything into it.

1) Backup-folder should be created every time new, and the old Backup.zip file will be destroyed (only after the successful creation of a new Backup.zip file).

2) Only 3 folders should be moved, and the Print folder should be duplicated to Backup-folder (not copied). Maybe copying will work too (I've never tested something like that), but I think, just in case, it's better to work with an independent duplicate of the Print folder.

So, the workaround:

1) Create new folder named newBackup.
2) Move 3 folders to it, duplicate folder Print to it.
3) Make zip of folder newBackup.
4) Delete Backup.zip file, delete newBackup folder
5) Rename newBackup.zip file to Backup.zip

Last edited by KniazidisR (2020-05-09 12:51:14 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#13 2020-05-09 03:06:48 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

KniazidisR wrote:

Here the OP has already confused you both. If he already has an old Backup.zip file (and not a folder), then you cannot move or duplicate anything into it.

1) Backup-folder should be created every time new, and the old Backup.zip file will be destroyed (only after the successful creation of a new Backup.zip file).

2) Only 3 folders should be moved, and the Print folder should be duplicated to Backup-folder (not copied). Maybe copying will work too (I've never tested something like that), but I think, just in case, it's better to work with an independent duplicate of the Print folder.

So, the workaround:

1) Create new folder named newBackup.
2) Move 3 folders to it, duplicate folder Print to it.
3) Make zip of folder newBackup.
4) Delete Backup.zip file, delete newBackup folder
5) Rename newBackup.zip file to Backup.zip



You may be right but it's not what the OP wrote:

Inside the subfolders there are several other folders.
My archive process is to copy All the folders except Backup, into Backup.
Once copied, zip Backup.
Remove all files inside the rest of the folders (except Backup and Print).
Done.



What is true is that in my late proposal I forgot to treat the folder Print a different way.
Here is the corrected version.

Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property dateTimeStamp : true
-- true --> insert a date time stamp in the name of the zip file
-- false --> doesn't insert such date time stamp in the name of the zip file

on moveItemAt:POSIXPath toSubFolder:subFolderName
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to sourceURL's URLByDeletingLastPathComponent
   set destURL to destURL's URLByAppendingPathComponent:subFolderName
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   
   if theName as string is "Print" then
       set {theResult, theError} to fileManager's copyItemAtURL:sourceURL toURL:destURL |error|:(reference)
   else
       set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   end if
   
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
   
end moveItemAt:toSubFolder:

set mainFolder to (path to desktop as string) & "Capitol_A02635:"
tell application "System Events"
   set subfolders to path of every folder of folder mainFolder whose name is not "Backup"
end tell

set subFolderName to "Backup"
repeat with aSub in subfolders
   (my moveItemAt:(POSIX path of aSub) toSubFolder:subFolderName)
end repeat
set path2Backup to mainFolder & subFolderName
if dateTimeStamp then
   tell (current date) to set stamp to "_" & (((its year) * 10000 + (its month) * 100 + (its day)) as text) & "_" & text 2 thru -1 of ((1000000 + (its hours) * 10000 + (its minutes) * 100 + (its seconds)) as text)
else
   set stamp to ""
end if

set path2ZippedOne to path2Backup & stamp & ".zip"

do shell script "ditto -ck " & quoted form of POSIX path of path2Backup & " " & quoted form of POSIX path of path2ZippedOne

It moves every subfolder minus the Print one which is copied so the oritinal remains in the main folder.

So, after execution, the folder Capitol_A02635 contain:

Backup
    Mech
    Mechanical
    Print
        dossier sans titre
    Proof
    Prouf
    Source
Backup_20200509_105340.zip
Backup_20200509_105956.zip
Backup_20200509_110404.zip
Print
    dossier sans titre

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 11:06:40

Last edited by Yvan Koenig (2020-05-09 03:07:54 am)

Offline

 

#14 2020-05-09 03:24:47 am

KniazidisR
Member
Registered: 2019-03-03
Posts: 1259

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

I still have a vague suspicion that the OP does not need old backups. This is just logical. Most likely, the OP itself should clarify this. After all, old zipped backups perform the opposite task of compressing a folder (saving disk space). So,

Backup(subfolder) may be 1) last Backup.zip file with hidden extension 2) Backup (subfolder), which contains only 1 thing in it - last Backup.zip file

Last edited by KniazidisR (2020-05-09 05:51:16 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#15 2020-05-09 08:13:53 am

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 530

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

The OP has several working scripts to choose from and should be able to do whatever fine-tuning is required.

I use SuperDuper for regular backups. However, I also use an AppleScript on a frequent basis to backup many hundreds of data files (no zip file created). In writing this script, an important issue became how to backup the changed files without copying files that have not changed. In other words, I don't want to copy 500 files to a backup when only 10 of those files have changed. That's why I use rsync with the -update option.

When I wrote the backup script several years ago, I looked at AppleScript and Bash commands that have both an update option and that act recursively but couldn't find any. I have to be able to maintain this script so ASObjC is out for me. It's time I reviewed my backup script and will look to see if there's an option other than rsync.

The cp command included with some linux distributions had an update option but that doesn't appear to be the case with the macOS cp command. I also couldn't find an update option with ditto. I'll do some more research on this and do some timing tests to see what works best.

Last edited by peavine (2020-05-09 08:19:34 am)


2018 Mac mini - macOS Catalina

Offline

 

#16 2020-05-09 08:19:13 am

KniazidisR
Member
Registered: 2019-03-03
Posts: 1259

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

peavine wrote:

I don't want to copy 500 files to a backup when only 10 of those files have changed.


Copying files and moving is not same thing. Moving files is only renaming them. And, zipping all together is more effective regarding to saved space of the disk.


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#17 2020-05-09 08:22:26 am

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 530

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

KniazidisR wrote:
peavine wrote:

I don't want to copy 500 files to a backup when only 10 of those files have changed.


Copying files and moving is not same thing. Moving files is only renaming them. And, zipping all together is more effective regarding to saved space of the disk.



I want to copy files not move them. The same appears to be the case with the OP and the Print subfolder. For me, saving space is not necessary.

Just to clarify, if I use the Finder duplicate command with the overwrite option, all 500 files in my example are copied even if only 10 files have changed. This seems a waste of time. With rsync and -update, only 10 files are copied. I will run some timing test to see if that's correct.

Last edited by peavine (2020-05-09 08:51:12 am)


2018 Mac mini - macOS Catalina

Offline

 

#18 2020-05-09 08:45:11 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

KniazidisR wrote:

I still have a vague suspicion that the OP does not need old backups. This is just logical. Most likely, the OP itself should clarify this. After all, old zipped backups perform the opposite task of compressing a folder (saving disk space). So,

Backup(subfolder) may be 1) last Backup.zip file with hidden extension 2) Backup (subfolder), which contains only 1 thing in it - last Backup.zip file




Keeping successive archives may be useful.

When we run the script at day 1, the created zip file will contain what was in the Backup folder when it was called. The folder Backup will now contain the subfolders available in Capitol at day 1.

When we run the script at day 2, the created zip file will contain what was in the Backup folder which are the subfolders from day 1. The folder Backup will now contain the files available in Capitol at day 2.

When we run the script at day 3, the created zip file will contain what was in the Backup folder which are the subfolders from day 2. The folder Backup will now contain the files available in Capitol at day 3.

Nothing prove that the datas archived in the 1st zip are available in the 2nd zip or in the 3rd one.
In fact I'm quite sure that except those in subfolder "Print", they aren't.

I apologize but I have no reason to imagine that when somebody write about a folder named "Backup" he is in fact writing about a zip file whose extension is hidden.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 16:44:33

Offline

 

#19 2020-05-09 09:15:25 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

peavine wrote:
KniazidisR wrote:
peavine wrote:

I don't want to copy 500 files to a backup when only 10 of those files have changed.


Copying files and moving is not same thing. Moving files is only renaming them. And, zipping all together is more effective regarding to saved space of the disk.



I want to copy files not move them. Also, just for me, saving space is not necessary.



When itemA and itemB aren't on the same device the operation would be different but when itemA and itemB are on the same volume (which is the case here), move itemA to itemB free the original contents of itemB, edit the descriptor of itemB so that it points to the datas of itemA and at last, removes the descriptor of itemA. We have the contents of itemA in a single area of the device.

In every cases, copy itemA to itemB replace the original contents of itemB by the contents of itemA so that it point to an area of the device different than the one pointed by itemA (but may be the area originally used by itemB) and itemA remains where it was. We have the contents of itemA in two areas of the device.

I described the way both commands behave under High Sierra.
If I remember well, under Catalina it's different. Copy an item just create a new 'descriptor' of the item so that we may have 10 copies but have only one physical 'copy' of the embedded datas.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 17:15:17

Offline

 

#20 2020-05-09 09:31:14 am

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 530

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

I ran some timing tests with Script Geek. The source folder is on my boot drive and contains 479 files; the destination folder is on a USB 3 external SSD. The first reported time is with the destination folder empty. I then added two small files in the source folder and immediately reran the script. That's the second reported time. Finally, I repeated these same tests with rsync and Finder but with a thumb drive as the destination drive.

Applescript:

-- Finder
tell application "Finder"
   duplicate folder "Macintosh HD:Users:Robert:Records:" to folder "Save:Test:finder:" with replacing
end tell
-- 838 milliseconds. 991 milliseconds. SSD.
-- 13.498 seconds. 13.754 seconds. Thumb drive.

Applescript:

-- Rsync
do shell script "/usr/bin/rsync --archive --update " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/rsync"
-- 1304 milliseconds. 92 milliseconds. SSD.
-- 13.580 seconds. 1.101 seconds. Thumb drive.

Applescript:

-- Ditto
do shell script "ditto " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/ditto"
-- 508 milliseconds. 532 milliseconds. SSD.

Applescript:

-- Copy (cp)
do shell script "cp -R " & quoted form of "/Users/Robert/Records" & space & quoted form of "/Volumes/Save/Test/cp"
-- 483 milliseconds. 486 milliseconds. SSD.

ASObjC script in post 23 below.
1060 milliseconds. 884 milliseconds. SSD.

If most files already exist in the destination folder--which is normally the case for me--rsync with update option seems preferable.

Last edited by peavine (2020-05-11 07:24:27 am)


2018 Mac mini - macOS Catalina

Offline

 

#21 2020-05-09 10:07:24 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

May you try with the ASObjC script ?

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 18:07:19

Offline

 

#22 2020-05-09 10:10:16 am

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 530

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

I'd be happy to do that but you would have to supply the working code. The source folder would be as shown above. The destination folder would be:

"Save:Test:asobjc:"


2018 Mac mini - macOS Catalina

Offline

 

#23 2020-05-09 11:10:36 am

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

You may test that:

Applescript:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on copyItemAt:POSIXPath toFolder:POSIXDest
   set sourceURL to current application's class "NSURL"'s fileURLWithPath:POSIXPath
   set theName to sourceURL's lastPathComponent()
   set destURL to current application's class "NSURL"'s fileURLWithPath:POSIXDest
   set destURL to destURL's URLByAppendingPathComponent:theName
   set fileManager to current application's NSFileManager's defaultManager()
   if (destURL's checkResourceIsReachableAndReturnError:(missing value)) as boolean then
       -- the folder already exists, delete it
       fileManager's removeItemAtURL:destURL |error|:(missing value)
   end if
   if theName as string is "Records" then
       set {theResult, theError} to fileManager's copyItemAtURL:sourceURL toURL:destURL |error|:(reference)
   else
       set {theResult, theError} to fileManager's moveItemAtURL:sourceURL toURL:destURL |error|:(reference)
   end if
   if (theResult as boolean) is false then
       error (theError's localizedDescription() as text)
   end if
end copyItemAt:toFolder:


-- If you try with a sourceFolder named "Records" it will be copied
-- If it's not named "Records" it will be moved
my copyItemAt:"/Users/Robert/Records" toFolder:"/Volumes/Save/Test/asobjc/"

(*
-- I tested with :
set sourceContainer to path to desktop as string
set destContainer to my remplace(sourceContainer, "SSD 1000", "Macintosh HD")
set folderToCopy to sourceContainer & "Print:" -- or "ableton:"
my copyItemAt:(POSIX path of folderToCopy) toFolder:(POSIX path of destContainer)
*)


#=====
(*
replaces every occurences of d1 by d2 in the text t
*)

on remplace(t, d1, d2)
   local oTIDs, l
   set {oTIDs, AppleScript's text item delimiters} to {AppleScript's text item delimiters, d1}
   set l to text items of t
   set AppleScript's text item delimiters to d2
   set t to l as text
   set AppleScript's text item delimiters to oTIDs
   return t
end remplace

#=====

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 19:08:16

Offline

 

#24 2020-05-09 03:31:28 pm

Yvan Koenig
Member
Registered: 2006-09-14
Posts: 4533

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Thank you.

As shell is not my cup of tea, I'm not really concerned by what is available and what isn't in this area.

At this time I just wish to know which behavior is the one really wanted by the OP.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) samedi 9 mai 2020 23:29:39

Offline

 

#25 2020-05-09 06:03:38 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 6395

Re: Bash for loop Script or Service to Copy Folders and Archive in one go?

Yvan Koenig wrote:

In every cases, copy itemA to itemB replace the original contents of itemB by the contents of itemA



That's not the case with APFS volumes. Copying works very much like moving, with seperate contents only made when one or the other is modified (and even then, only of the modified part).


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/
latenightsw.com

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)