Tuesday, May 11, 2021

#1 2021-03-31 07:35:17 am

CWilcke
Member
Registered: 2012-06-14
Posts: 12

Sort files in folders

Hi,

I it possible to sort n move files in folders by applescript?

I do have one Folder with 30k+ files and would like to split it and move about 300 files into a new folder.

Each folder should not have more than these 300 files.

Best,


Filed under: applescript, automation

Offline

 

#2 2021-04-01 04:14:23 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1797

Re: Sort files in folders

Hi,

One simple solution is this (Caution: test it on the copy of folder firstly):

Applescript:


set MAX to 300
set sourceFolder to (choose folder) as text

tell application "Finder"
   set theFiles to sort files of folder sourceFolder by name
   set subFolderIndex to 1
   repeat while (theFiles is not {})
       set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
       set filesMoved to 0
       repeat while ((filesMoved < MAX) and (theFiles is not {}))
           move contents of item 1 of theFiles to subFolder
           set filesMoved to filesMoved + 1
           set theFiles to rest of theFiles
       end repeat
       set subFolderIndex to subFolderIndex + 1
   end repeat
end tell

Last edited by KniazidisR (2021-04-01 04:15:01 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 14.1
Ram: 4 GB

Offline

 

#3 2021-04-07 12:13:19 am

CWilcke
Member
Registered: 2012-06-14
Posts: 12

Re: Sort files in folders

perfect! Thank You!

Offline

 

#4 2021-04-07 07:51:50 pm

wch1zpink
Member
Registered: 2011-08-20
Posts: 18

Re: Sort files in folders

KniazidisR wrote:

Hi,

One simple solution is this (Caution: test it on the copy of folder firstly):

Applescript:


set MAX to 300
set sourceFolder to (choose folder) as text

tell application "Finder"
   set theFiles to sort files of folder sourceFolder by name
   set subFolderIndex to 1
   repeat while (theFiles is not {})
       set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
       set filesMoved to 0
       repeat while ((filesMoved < MAX) and (theFiles is not {}))
           move contents of item 1 of theFiles to subFolder
           set filesMoved to filesMoved + 1
           set theFiles to rest of theFiles
       end repeat
       set subFolderIndex to subFolderIndex + 1
   end repeat
end tell


I tested the above code in Script Editor on 5000 identical text files,  named from 0001 to 5000.  The results of timing the code was 490 seconds

While using Finder.app to move, copy, duplicate, etc., on only a few files or folders at a time,  is generally OK, it becomes extremely inefficient when trying to process dozens, hundreds, or even thousands of items at a time.

It is also worth noting that creating a list of files or folders with Finder.app becomes much quicker by coercing it to an alias list.

Applescript:

set sourceFolder to (choose folder) as text
set MAX to 300
tell application "Finder" to set theFiles to files of folder sourceFolder as alias list

Once that alias list has been created, there is no need to use the sort command because the created alias list will automatically be sorted. (I do believe)

A more efficient solution would be to use Finder to generate the alias list first, then System Events.app to move the files instead of Finder.

Applescript:

set sourceFolder to (choose folder) as text
set MAX to 300

tell application "Finder" to set theFiles to files of folder sourceFolder as alias list

tell application "System Events"
   set subFolderIndex to 1
   repeat while (theFiles is not {})
       set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
       set filesMoved to 0
       repeat while ((filesMoved < MAX) and (theFiles is not {}))
           move contents of item 1 of theFiles to subFolder
           set filesMoved to filesMoved + 1
           set theFiles to rest of theFiles
       end repeat
       set subFolderIndex to subFolderIndex + 1
   end repeat
end tell

The results of timing this code was 47 seconds

Offline

 

#5 2021-04-08 03:13:38 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1797

Re: Sort files in folders

Hello, wch1zpink.

I ran detailed tests here with a folder containing 1690 files. Here are the test results:

My original script (227 seconds) works correctly, but it is slow. Your script runs 5-6 times faster (45 seconds), but it is wrong because it breaks sorting by name. In order for it to work correctly, you will have to abandon the alias list + coerce the Finder references into the System Events references inside the repeat loop. And this increases the execution time (158 seconds):

Applescript:

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set theFiles to (sort files of folder sourceFolder by name)

tell application "System Events"
   set subFolderIndex to 1
   repeat while (theFiles is not {})
       set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
       set filesMoved to 0
       repeat while ((filesMoved < MAX) and (theFiles is not {}))
           move file ((contents of item 1 of theFiles) as text) to subFolder
           set filesMoved to filesMoved + 1
           set theFiles to rest of theFiles
       end repeat
       set subFolderIndex to subFolderIndex + 1
   end repeat
end tell
-- 158 seconds

And here, I wrote other Finder version, which has same speed (165 seconds) with correct System Events version above:

Applescript:

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set theFiles to sort files of folder sourceFolder by name
set theCount to count theFiles

set N to theCount div MAX
if N > 0 then
   tell application "Finder"
       repeat with i from 1 to N
           set subFolder to make new folder at folder sourceFolder with properties {name:i}
           repeat with j from (i - 1) * MAX + 1 to i * MAX
               move contents of item j of theFiles to subFolder
           end repeat
       end repeat
   end tell
end if

set M to theCount mod MAX
if M > 0 then
   tell application "Finder"
       set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
       repeat with j from theCount - M + 1 to theCount
           move contents of item j of theFiles to subFolder
       end repeat
   end tell
end if
-- 165 seconds

Note: I think, adapting this last script for System Events instead of Finder will produce one fastest script:

Applescript:

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set theFiles to sort files of folder sourceFolder by name
set theCount to count theFiles

set N to theCount div MAX
if N > 0 then
   tell application "System Events"
       repeat with i from 1 to N
           set subFolder to make new folder at folder sourceFolder with properties {name:i}
           repeat with j from (i - 1) * MAX + 1 to i * MAX
               move file ((contents of item j of theFiles) as text) to subFolder
           end repeat
       end repeat
   end tell
end if

set M to theCount mod MAX
if M > 0 then
   tell application "System Events"
       set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
       repeat with j from theCount - M + 1 to theCount
           move file ((contents of item j of theFiles) as text) to subFolder
       end repeat
   end tell
end if
-- 130 seconds

Last edited by KniazidisR (2021-04-08 04:58:18 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 14.1
Ram: 4 GB

Offline

 

#6 2021-04-08 05:51:11 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1797

Re: Sort files in folders

Here I wrote fastest script (31 seconds for 1690 files). It uses sorting names instead of sorting files, and speed of System Events:

Applescript:


use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set fileNames to name of files of folder sourceFolder
set sortedNames to sortListOfStrings(fileNames) -- you can use other sorting method as well
set theCount to count sortedNames

set N to theCount div MAX
if N > 0 then
   tell application "System Events"
       repeat with i from 1 to N
           set subFolder to make new folder at folder sourceFolder with properties {name:i}
           repeat with j from (i - 1) * MAX + 1 to i * MAX
               move file (sourceFolder & item j of sortedNames) to subFolder
           end repeat
       end repeat
   end tell
end if

set M to theCount mod MAX
if M > 0 then
   tell application "System Events"
       set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
       repeat with j from theCount - M + 1 to theCount
           move file (sourceFolder & item j of sortedNames) to subFolder
       end repeat
   end tell
end if


on sortListOfStrings(theList)
   -- convert list to Cocoa array
   set theArray to ¬
       current application's NSArray's arrayWithArray:theList
   -- sort the array using a specific function
   set theArray to ¬
       theArray's sortedArrayUsingSelector:"localizedStandardCompare:"
   -- return the sorted array as an AppleScript list
   return theArray as list
end sortListOfStrings

Last edited by KniazidisR (2021-04-08 05:55:46 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 14.1
Ram: 4 GB

Offline

 

#7 2021-04-08 12:12:07 pm

wch1zpink
Member
Registered: 2011-08-20
Posts: 18

Re: Sort files in folders

Actually, this following approach bypasses Finder completely (which proves to be the major bottleneck)

I tested this code in Script DeBugger on 5000 identical text files,  named from 0001 to 5000.  The results of timing the code was 46 seconds.

Applescript:

global startTime, timeTaken

set sourceFolder to (choose folder) as text
set MAX to 300
startTimer()

set theFiles to paragraphs of (do shell script "find " & quoted form of POSIX path of sourceFolder & " -type f -maxdepth 1 | sort -f")

tell application "System Events"
   set subFolderIndex to 1
   repeat while (theFiles is not {})
       set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
       set filesMoved to 0
       repeat while ((filesMoved < MAX) and (theFiles is not {}))
           move file ((contents of item 1 of theFiles) as text) to subFolder
           set filesMoved to filesMoved + 1
           set theFiles to rest of theFiles
       end repeat
       set subFolderIndex to subFolderIndex + 1
   end repeat
end tell
log endTimer()

to startTimer()
   set startTime to current date
   return startTime
end startTimer

to endTimer()
   set timeTaken to ((current date) - startTime as string) & " Seconds"
   activate
   ignoring application responses
       display alert "Script Timer" message "Running This Code Took " & ¬
           timeTaken buttons {"OK"} giving up after 7
   end ignoring
   return timeTaken
end endTimer

RESULT 46 SECONDS



I tested this code in Script DeBugger on 5000 identical text files,  named from 0001 to 5000.  The results of timing the code was 73 seconds.

Applescript:

use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

global startTime, timeTaken

set sourceFolder to (choose folder) as text
set MAX to 300
startTimer()

tell application "Finder" to set fileNames to name of files of folder sourceFolder
set sortedNames to sortListOfStrings(fileNames) -- you can use other sorting method as well
set theCount to count sortedNames

set N to theCount div MAX
if N > 0 then
   tell application "System Events"
       repeat with i from 1 to N
           set subFolder to make new folder at folder sourceFolder with properties {name:i}
           repeat with j from (i - 1) * MAX + 1 to i * MAX
               move file (sourceFolder & item j of sortedNames) to subFolder
           end repeat
       end repeat
   end tell
end if

set M to theCount mod MAX
if M > 0 then
   tell application "System Events"
       set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
       repeat with j from theCount - M + 1 to theCount
           move file (sourceFolder & item j of sortedNames) to subFolder
       end repeat
   end tell
end if

log endTimer()

to startTimer()
   set startTime to current date
   return startTime
end startTimer

to endTimer()
   set timeTaken to ((current date) - startTime as string) & " Seconds"
   activate
   ignoring application responses
       display alert "Script Timer" message "Running This Code Took " & ¬
           timeTaken buttons {"OK"} giving up after 7
   end ignoring
   return timeTaken
end endTimer

on sortListOfStrings(theList)
   -- convert list to Cocoa array
   set theArray to ¬
       current application's NSArray's arrayWithArray:theList
   -- sort the array using a specific function
   set theArray to ¬
       theArray's sortedArrayUsingSelector:"localizedStandardCompare:"
   -- return the sorted array as an AppleScript list
   return theArray as list
end sortListOfStrings

RESULT 73 SECONDS

Last edited by wch1zpink (2021-04-08 12:35:55 pm)

Offline

 

#8 2021-04-10 08:17:42 pm

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 872

Re: Sort files in folders

I decided to see if ASObjC might improve the timing results and wrote the script below. It took 18 seconds to move 5,000 files located on an external SSD, although better results might be achieved by rewriting the move handler. To set a base, I timed KniazidisR's script from post 2 and it took 132 seconds.

A few comments:

* The script assumes that the sourceFolder contains the files that will be moved and nothing else.
* The script errors if a source file already exists in the target folder.
* The files are sorted in lexicographic rather than numeric order.
* The script needs some error notification.
* This script should be used with test files until found to be reliable.

Applescript:

use framework "AppKit"
use framework "Foundation"
use scripting additions

on main()
   set sourceFolder to "/Volumes/Store/Source Folder/"
   set targetFolder to "/Volumes/Store/Target Folder/"
   set theFiles to getFiles(sourceFolder)
   set folderCounter to 0
   
   repeat with i from 1 to (count theFiles)
       if (i - 1) mod 300 = 0 then
           set folderCounter to folderCounter + 1
           set newFolder to targetFolder & "Folder " & folderCounter
           makeFolder(newFolder)
       end if
       set theFile to POSIX path of ((item i of theFiles) as text)
       moveFile(theFile, newFolder)
   end repeat
end main

on getFiles(theFolder)
   set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
   set theFiles to current application's NSFileManager's defaultManager()'s contentsOfDirectoryAtURL:theFolder includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
   set sortDescriptors to current application's NSSortDescriptor's sortDescriptorWithKey:"lastPathComponent" ascending:true
   set sortDescriptors to current application's NSArray's arrayWithObject:sortDescriptors
   set sortedFiles to (theFiles's sortedArrayUsingDescriptors:sortDescriptors)
   return sortedFiles
end getFiles

on makeFolder(theFolder)
   set fileManager to current application's NSFileManager's defaultManager()
   fileManager's createDirectoryAtPath:theFolder withIntermediateDirectories:false attributes:(missing value) |error|:(missing value)
end makeFolder

on moveFile(theFile, theFolder)
   set sourceFile to current application's NSString's stringWithString:theFile
   set sourceName to sourceFile's lastPathComponent()
   set targetFolder to current application's NSString's stringWithString:theFolder
   set targetFile to targetFolder's stringByAppendingPathComponent:sourceName
   set fileManager to current application's NSFileManager's defaultManager()
   set {theResult, theError} to fileManager's moveItemAtPath:sourceFile toPath:targetFile |error|:(reference)
end moveFile

main()

Last edited by peavine (2021-04-10 09:21:43 pm)


2018 Mac mini - macOS Catalina

Offline

 

#9 2021-04-10 09:26:20 pm

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1797

Re: Sort files in folders

peavine wrote:


* The files are sorted in lexicographic rather than numeric order.


Cool script, peavine. As I see, it skips hidden files like my scripts. I have question only with sorting. Your script sorts not exactly like Finder sorting by name:

"myFile1" -- For me, it is preferable this Finder sorting by name order.
"myFile2"
"myFile10"

Your script sorts like this:

"myFile1"
"myFile10"
"myFile2"

You can sort exactly the Finder way using localizedStandardCompare selector:

Applescript:


set sortDescriptors to current application's NSSortDescriptor's sortDescriptorWithKey:"lastPathComponent" ascending:true selector:"localizedStandardCompare:"

TEST:

Applescript:


use framework "AppKit"
use framework "Foundation"
use scripting additions

on sortByNameFinderWay(theFolder)
   set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
   set theFiles to current application's NSFileManager's defaultManager()'s contentsOfDirectoryAtURL:theFolder includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
   set sortDescriptors to current application's NSSortDescriptor's sortDescriptorWithKey:"lastPathComponent" ascending:true selector:"localizedStandardCompare:"
   set sortDescriptors to current application's NSArray's arrayWithObject:sortDescriptors
   set sortedFiles to (theFiles's sortedArrayUsingDescriptors:sortDescriptors)
   return sortedFiles
end sortByNameFinderWay

my sortByNameFinderWay(POSIX path of (choose folder)) as list

Tip:  you can get fileManager only once, out of repeat loop. Symply make fileManager global variable.

Last edited by KniazidisR (2021-04-10 11:41:18 pm)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 14.1
Ram: 4 GB

Offline

 

#10 2021-04-10 11:45:55 pm

peavine
Member
From:: Prescott, Arizona
Registered: 2018-09-04
Posts: 872

Re: Sort files in folders

Thanks KniazidisR for looking at my script and for the suggestions. I revised my script to create file manager only once and I optimized the getFiles handler. These changes reduced the timing result with 5,000 files to 14.5 seconds. I also changed the sort selector, which works as you indicate. Using an AppleScript to move 30,000 files may be a risky thing to do, and so I used copy rather than move in the script.

Applescript:

use framework "Foundation"
use scripting additions

on main()
   set sourceFolder to "/Volumes/Store/Source Folder/"
   set targetFolder to "/Volumes/Store/Target Folder/"
   set filesPerFolder to 300
   set fileManager to current application's NSFileManager's defaultManager()
   set theFiles to getFiles(sourceFolder, fileManager)
   set folderCounter to 0
   
   repeat with i from 1 to (count theFiles)
       if (i - 1) mod filesPerFolder = 0 then
           set folderCounter to folderCounter + 1
           set newFolder to targetFolder & "Folder " & folderCounter
           makeFolder(newFolder, fileManager)
       end if
       copyFile((item i of theFiles), newFolder, fileManager)
   end repeat
end main

on getFiles(theFolder, fileManager)
   set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
   set enumOptions to (current application's NSDirectoryEnumerationSkipsPackageDescendants as integer) + (current application's NSDirectoryEnumerationSkipsHiddenFiles as integer)
   set theFiles to (((fileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{} options:enumOptions errorHandler:(missing value))'s allObjects())'s |path|)
   return (theFiles's sortedArrayUsingSelector:"localizedStandardCompare:") as list
end getFiles

on makeFolder(theFolder, fileManager)
   fileManager's createDirectoryAtPath:theFolder withIntermediateDirectories:false attributes:(missing value) |error|:(missing value)
end makeFolder

on copyFile(theFile, theFolder, fileManager)
   set sourceFile to current application's NSString's stringWithString:theFile
   set sourceName to sourceFile's lastPathComponent()
   set targetFolder to current application's NSString's stringWithString:theFolder
   set targetFile to targetFolder's stringByAppendingPathComponent:sourceName
   set theResult to fileManager's copyItemAtPath:sourceFile toPath:targetFile |error|:(missing value)
   if theResult is false then display dialog "An error was encountered while copyng the files" buttons "OK" cancel button 1 default button 1 with icon stop
end copyFile

main()

Last edited by peavine (2021-04-19 06:41:22 pm)


2018 Mac mini - macOS Catalina

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)