Monday, December 11, 2017

#1 2015-09-10 03:49:52 pm

robdut
Member
Registered: 2009-09-02
Posts: 301

Fast and accurate folder size rehashed

Hi All,

Reliving an old thread.

http://macscripter.net/viewtopic.php?id=36220

I have tried several different methods of getting size of folder (as it appears in finder) using FSGetCatalogInfoBulk, NSFileManager, NSURLFileSizeKey etc.  and they all come up considerably short. Just something faster than "get physical size" and Finder. Any new thoughts and why are these methods not showing the finder size, or is the finder size wrong?

Cheers,  Rob

Last edited by robdut (2015-09-10 05:15:57 pm)

Offline

 

#2 2015-09-10 06:53:07 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 5197

Re: Fast and accurate folder size rehashed

Sometimes the Finder is indeed wrong. However, this seems to do a pretty good job:

Applescript:

use scripting additions
use framework "Foundation"

set theFolder to "/Users/shane/Desktop/Explorer stuff"
set totalSize to its sizeOfFolder:theFolder
display dialog (its formatAsBytes:totalSize)

on sizeOfFolder:folderPosixPath
   set anNSURL to current application's |NSURL|'s fileURLWithPath:folderPosixPath
   set totalSize to 0
   set theEnumerator to current application's NSFileManager's defaultManager()'s enumeratorAtURL:anNSURL includingPropertiesForKeys:{current application's NSURLTotalFileSizeKey, current application's NSURLIsDirectoryKey} options:0 errorHandler:(missing value)
   repeat
       set nextURL to theEnumerator's nextObject()
       if nextURL is missing value then exit repeat
       set theDict to nextURL's resourceValuesForKeys:{current application's NSURLTotalFileSizeKey, current application's NSURLIsDirectoryKey} |error|:(missing value)
       if (theDict's objectForKey:(current application's NSURLIsDirectoryKey)) as boolean is false then
           set thisSize to theDict's objectForKey:(current application's NSURLTotalFileSizeKey)
           set totalSize to totalSize + (thisSize as real)
       end if
   end repeat
   return totalSize
end sizeOfFolder:

on formatAsBytes:theValue
   set theNSByteCountFormatter to current application's NSByteCountFormatter's alloc()'s init()
   theNSByteCountFormatter's setIncludesActualByteCount:true
   return (theNSByteCountFormatter's stringFromByteCount:theValue) as text
end formatAsBytes:

NSURLTotalFileSizeKey requires 10.7, and NSByteCountFormatter requires 10.8.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/

Offline

 

#3 2015-09-10 10:43:42 pm

robdut
Member
Registered: 2009-09-02
Posts: 301

Re: Fast and accurate folder size rehashed

Thanks Shane,

I like it and better than supposedly deprecated FSGetCatalogInfoBulk...

It does get the exact ( at least what Finder agrees on) bytes. FSGetCatalogInfoBulk also returns the correct bytes.

So a folder (lots of files nested) on my desktop shows  6,113,994 bytes and via  formatAsBytes:theValue

Shane human readable = 6.1 MB (6,113,994 bytes)

I guess I am happy to get the actual bytes regardless of the translation. Below for 6,113,994 bytes:

Yours = 6.1 MB

Finder says 7 MB

The two methods below = 5.83 MB

Cheers,  Rob

Applescript:

+(id)transformedValue:(id)value
   {
       double convertedValue = [value doubleValue];
       int multiplyFactor = 0;

       NSArray *tokens = [NSArray arrayWithObjects:@"bytes",@"KB",@"MB",@"GB",@"TB",nil];

       while (convertedValue > 1024) {
           convertedValue /= 1024;
           multiplyFactor++;
       }
       return [NSString stringWithFormat:@"%4.2f %@",convertedValue, [tokens objectAtIndex:multiplyFactor]];
   }

Applescript:

on getCalculatedSize_(newsize)
       set newsize to newsize as integer
       if newsize as integer < 1000 then
           return (my roundThis_(2, newsize) as string) & " KB"
           else if newsize as integer < (1000 * 1000) then
           return (my roundThis_(2, ((newsize / 1000 * 10)) / 10) as string) & " MB"
           else if newsize as integer < (1000 * 1000 * 1000) then
           return (my roundThis_(2, (newsize / 1000 / 1000 * 10) / 10) as string) & " GB"
           else
           return (my roundThis_(2, (newsize / 1000 / 1000 / 1000 * 10) / 10) as string) & " TB"
       end if
   end getCalculatedSize_

on roundThis_(num, n)
   set num to num as number
   set n to n as integer
   set x to 10 ^ num
   (((n * x) + 0.5) div 1) / x
end roundThis_

Offline

 

#4 2015-09-10 11:02:11 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 5197

Re: Fast and accurate folder size rehashed

robdut wrote:

Below for 6,113,994 bytes:

Yours = 6.1 MB

Finder says 7 MB


I presume you're quoting the value the Finder gives for "on disk", which is calculated differently. To get that, you use NSURLTotalFileAllocatedSizeKey instead of NSURLTotalFileSizeKey.

The two methods below = 5.83 MB


Disk space is generally calculated using 1000 rather than 1024; that's what you need to use if you want to match the Finder. But you can also change the handler if you'd prefer:

Applescript:

on formatAsBytes:theValue
   set theNSByteCountFormatter to current application's NSByteCountFormatter's alloc()'s init()
   theNSByteCountFormatter's setIncludesActualByteCount:true
   theNSByteCountFormatter's setCountStyle:(current application's NSByteCountFormatterCountStyleFile)
   return (theNSByteCountFormatter's stringFromByteCount:theValue) as text
end formatAsBytes:

If you need to support versions before 10.8, by all means roll your own. Otherwise, it's pointless work. NSByteCounter gives you more control, as well as localized units.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/

Offline

 

#5 2015-09-11 08:20:35 am

robdut
Member
Registered: 2009-09-02
Posts: 301

Re: Fast and accurate folder size rehashed

Hi Shane,

Your formatAsBytes is great. I did some speed tests on several large folders (iPhoto, Library etc..) using your method, fastFolderSizeAtFSRef, and finder:

Here is iPhoto test:

09:01:12 > fastFolderSizeAtFSRef bytes = 1.11622613768E+11
09:01:12 >  fastFolderSizeAtFSRef (using Shane formatAsBytes ) = 111.62 GB (111,622,613,768 bytes)
09:01:12 > Time diff fastFolderSizeAtFSRef = 4
09:02:06 >  Shane size = 1.11622613768E+11
09:02:06 > Shane human size = 111.62 GB (111,622,613,768 bytes)
09:02:06 > Time diff Shane= 54

The finder took about 4 seconds via "size of (info for the/file)" so is equal to fastFolderSizeAtFSRef. As you can see the fastFolderSizeAtFSRef is fast but of course deprecated.

I imagine Finder uses something similar to the deprecated fastFolderSizeAtFSRef.The time diff was similar on all folders I tried.

Cheers, Rob


fastFolderSizeAtFSRef (from Daniel Jalkut) I of course reference from an OBJc class file.

Applescript:

+ (unsigned long long) fastFolderSizeAtFSRef:(FSRef*)theFileRef{
   FSIterator thisDirEnum = NULL;
   unsigned long long totalSize = 0;
   // Iterate the directory contents, recursing as necessary
   if (FSOpenIterator(theFileRef, kFSIterateFlat, &thisDirEnum) ==
           noErr)
   {
       const ItemCount kMaxEntriesPerFetch = 256;
       ItemCount actualFetched;
       FSRef fetchedRefs[kMaxEntriesPerFetch];
       FSCatalogInfo fetchedInfos[kMaxEntriesPerFetch];

       OSErr fsErr = FSGetCatalogInfoBulk(thisDirEnum,kMaxEntriesPerFetch, &actualFetched, NULL, kFSCatInfoDataSizes | kFSCatInfoNodeFlags | kFSCatInfoRsrcSizes,fetchedInfos,fetchedRefs, NULL, NULL);

       while ((fsErr == noErr) || (fsErr == errFSNoMoreItems))
       {
           ItemCount thisIndex;
           for (thisIndex = 0; thisIndex < actualFetched; thisIndex++)
           {
               // Recurse if it's a folder
               if (fetchedInfos[thisIndex].nodeFlags &
                       kFSNodeIsDirectoryMask)
               {
           totalSize += [self fastFolderSizeAtFSRef:(&fetchedRefs[thisIndex])];
               }
               else
               {
                // add the size for this item
                   totalSize += fetchedInfos
                   [thisIndex].dataLogicalSize;
                   totalSize += fetchedInfos[thisIndex].rsrcLogicalSize;
               }
           }
           if (fsErr == errFSNoMoreItems)
           {
               break;
           }
       }
       FSCloseIterator(thisDirEnum);
   }
   return totalSize;
}

Offline

 

#6 2015-09-11 07:05:11 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 5197

Re: Fast and accurate folder size rehashed

Interesting. I originally used FSGetCatalogInfoBulk() in ASObjC Runner, and changed to the NSURL method later, but I didn't see such a big performance hit. I was probably testing smaller folders, and timing the full turn-around.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)