Tree.sb to list folder contents

[GeorgeMcGinn]

POST UPDATED: I removed the entire directory from my IDE and ran it again, and I get the same error on the end of the the next directory in the list.

Also, it seems that your program is dropping the first letter of the file names.

Here is a directory list of the file that your program stops at, and as you can see here, the E in Exercise is there, but in the post below, it is missing from the variables in the debug screen.

I checked the contents file and it is missing there as well.

[Dutchman]

Instead of building a tree of your root-folder you could also build trees from sub-folders, thus reducing the possible size-problem.
Please try by changing the variable Dir$ (now: Dir$="") to a subfolder of moderate size.
AND, to be sure, I have reloaded the code-section in my first post.
The screenshot shows the contents of my browser-folder, no letters missing:

screenshot.PNG (147.87 KiB) Viewed 3879 times

[GeorgeMcGinn]

EDITED: That works great.

I will be adding the display for file size as I am trying to find some of my files that are large and need to store elsewhere, except that will not be an option with the new release of SmartBASIC as I will have to store large files. But the directory listing should give you the file size as well.

Thanks for looking into this.

I'll give that a try and let you know how it works, but my guess is it is a memory issue as when I deleted large files I saw more in the listing.

[Dutchman]

I added size info to each file in the tree so that large files can be detected.
The format is "#.##" in Megabytes.
The code in the first post has been updated.

screenshot.PNG (193.18 KiB) Viewed 3862 times

[rbytes]

I did a rescan of my files, looking for large ones. It wasn’t easy to spot them when scrolling through a large number, so in the code section ‘write contents’, I made a small change.

'--- write contents

....

if size>1E6 then
FILE File$ PRINT Leader$&F$&size$&" ***************************************************************"
ELSE
FILE File$ PRINT Leader$&F$&size$
ENDIF

Now the larger files catch my eye even with a fast scroll. You can change the 1E6 (1 meg) to whatever you want to define as large.

[Dutchman]

Smart

[GeorgeMcGinn]

Great change.

However, many files do not even register a size and many show as 0.00Mb. So I added the following code that prints in either Bytes or Mb, depending on file size.

I put the following above Rbytes code:

if size>1E6 then
size$=" ("&STR$(size/1E6,"#.##")&"Mb)"
else
size$=" ("&STR$(size,"#")&" Bytes)"
endif

Then the file looks like below. However, doing it the way above means that you will get long byte sizes for sizes above .25Mb or .5Mb

You can always change the the IF statement "if size>.25E6 then ... or what ever fraction you want. Now the report looks like this:

#/App Development/JSON-URL File Management/Output/
LOGFile.txt (994 Bytes)
#/App Development/JSON-URL File Management/Output/Test Results/
Conditions.txt (0.26Mb) (***NOTE: File size exceeded .25E6, and was displayed properly)
Forecast.txt (46947 Bytes)
LOGFile.txt (952 Bytes)
NWS LOGFile.txt (2014 Bytes)
OUT-Conditions.txt (3864 Bytes)
OUT-Conditions.txt (3384 Bytes)
RPT-Conditions.txt (3618 Bytes)
RPTFile.txt (2.73Mb) ***************************************************************

If you would like, I have a very simple formatting routine that will put the right commas in the number of bytes, or even Mb >999Mb, if you wish, I can add it for you.

George.

[rbytes]

I would like to see it. Thanks.

[GeorgeMcGinn]

Instead of keeping it a function (I was having problems with the decimal numbers of the megabytes) I embedded it into the code. My additions are marked in RED code. (I put the function in the Library subfolder in Basic Programs).

Two tables were created, one for bytes and the other for megabytes, with the megabytes only having 2 decimal places. You can change that to as many as you like if you want more precision.

For bytes, all I did was determine the length of the number and used that as the index into the format array FMI$. If the number of bytes were 1234, then the length of 4, used as the index returned "#,###" format.

For decimals, it was a little bit tricky. All I needed to know was the size of the integer. So I first had to separate it from the decimal, and make sure that the value was atleast 1, as the OPTION BASE 1 was used, and a 0 value returns a length of 0. Then I passed the entire number using the format of the integer for the format mask. For example, if size/1E6=0.1234, then 0 was extracted, and since its length is zero, length was made a 1, and the format returned "#.##" and the number displayed was 0.12.

I hope that explains how I integrated my Function into this program. The simple function is the second program listing.

I have also included parts of my contents.tree file to show you how the numbers now look.

The next change could add all the bytes together and in the report printed at thee end of the run, it can display the total size of the directory you are listing.

Code: Select all

'Tree.sb, by Dutchman, September 2017
'Modified by Rbytes and George McGinn, October 2017
'Makes File$ with contents of folder Dir$
'Folders are printed with leading "#"
'
'r'
OPTION BASE 1

'*** Formatting Setup, Integers and two-decimals
DIM FMI$(10)
FMI$(1)  = "#"
FMI$(2)  = "##"
FMI$(3)  = "###"
FMI$(4)  = "#,###"
FMI$(5)  = "##,###"
FMI$(6)  = "###,###"
FMI$(7)  = "#,###,###"
FMI$(8)  = "##,###,###"
FMI$(9)  = "###,###,###"
FMI$(10) = "#,###,###,###"

DIM FMD$(10)
FMD$(1)  = "#.##"
FMD$(2)  = "##.##"
FMD$(3)  = "###.##"
FMD$(4)  = "#,###.##"
FMD$(5)  = "##,###.##"
FMD$(6)  = "###,###.##"
FMD$(7)  = "#,####,###.##"
FMD$(8)  = "##,###,###.##"
FMD$(9)  = "###,###,###.##"
FMD$(10) = "#,###,###,###.##"
''

Dir$="/App Development"   '*** Change Directory name as desired
File$="contents.tree"
Indent$="   "             '*** adds to each folder-level, e.g. "#" or "   "

Content.numbers=0
IF LEFT$(REVERSE$(Dir$),1)<>"/" THEN Dir$&="/"
IF NOT FILE_EXISTS(Dir$) THEN 
  PRINT "Folder """&Dir$&""" not found"
  STOP
ENDIF
IF FILE_EXISTS(File$) THEN FILE File$ DELETE
CALL Content(File$,Dir$,Indent$)
TEXT CLEAR
PRINT "Scanned folder """&Dir$&"""."
PRINT "Found "&Files&" files in "&Dirs&" folders."
PRINT "Content written to """&File$&"""."
END

DEF Content(File$,Dir$,Indent$) 
' Write contents of folder Dir$ to File$
' Folder-names are preceded by "#"
' Indent$ is added at each level
' If Content.number<>0 then names 
'	will be preceded by level-number&"|"
' Dirs and Files-count are written to 
'	.Dirs and .Files
'---------- by Dutchman, 2017 ----------
IF numbers THEN Level$=Level&"|" ELSE Level$=""
FILE File$ PRINT Tab$&Level$;"#";Dir$
.Dirs+=1
Tab$&=Indent$
level+=1
IF numbers THEN Level$=Level&"|" ELSE Level$=""
Leader$=Tab$&level$
'--- list files and dirs
Content$="" ' clear for this folder
DIR Dir$ LIST FILES F1$,n1
DIR Dir$ LIST DIRS F2$,n2
FOR i=1 TO n1
  Content$&=F1$(i)&","
NEXT i
FOR i=1 TO n2
  Content$&="#"&F2$(i)&","
NEXT i
'--- write contents
WHILE LEN(Content$)>1
  p=INSTR(Content$,",")
  F$=LEFT$(Content$,p-1)
  Content$=RIGHT$(Content$,LEN(Content$)-p)
  IF INSTR(F$,"#")<1 THEN ' is file
    size=FILE_SIZE(Dir$&F$)
'r'
'**    if size>.25E6 then
'**       size$=" ("&STR$(size/1E6,"#.##")&"Mb)"
'**    else
'**         size$=" ("&STR$(size,"#")&" Bytes)"
'**    endif
    LB=LEN(STR$(size))
    LM=LEN(STR$(size/1E6))
    LM=LEN(STR$(INTEG(LM)))!IF LM=0 THEN LM=1
    if size>.25E6 then
       size$=" ("&STR$(size/1E6,.FMD$(LM))&"Mb)"
    else
         size$=" ("&STR$(size,.FMI$(LB))&" Bytes)"
    endif
''
    if size>1E6 then
       FILE File$ PRINT Leader$&F$&size$&" ***************************************************************"
    ELSE
       FILE File$ PRINT Leader$&F$&size$
    ENDIF
REM    FILE File$ PRINT Leader$&F$&size$
    .Files+=1
  ELSE ' is folder
    PRINT CHR$(9632);
    f$=Dir$&RIGHT$(f$,LEN(F$)-1)&"/"
    CALL Content(File$,f$,Indent$)
  ENDIF
END WHILE
RETURN
END DEF

Here is a snippet of the content file:

#/App Development/File Compare/Data copy/
ELIZA copy.sb (9,137 Bytes)
ELIZA.sb (9,737 Bytes)
testempty.txt (2 Bytes)
#/App Development/File Compare/Images/
CompareBG.jpg (156,821 Bytes)
scope.jpg (0.28Mb)
#/App Development/File Compare/Lib/
drawinfobar.txt (1,962 Bytes)
#/App Development/File Compare/Support Code/
File Compare v3.0.testsb (12,984 Bytes)
File Comparison v1.5.sb (2,798 Bytes)
File Comparison v2.0.sb (9,031 Bytes)
File Comparison v2.5 copy.sb (11,593 Bytes)
File Comparison.sb (2,061 Bytes)
GetScreenInput.sb (1,960 Bytes)
HelpButtonCycle.txt (1,664 Bytes)
Stepping Stone To A Version Control System.txt (1,493 Bytes)
#/App Development/JSON-URL File Management/Data/PoemGenerator/
data.verb (2.75Mb) ***************************************************************
ogden_2000.txt (25,149 Bytes)
verb.txt (0.55Mb)
#/App Development/JSON-URL File Management/JSON & XML Files/
2DTowerSchema.xml.txt (3,669 Bytes)
JSON Example.txt (1,269 Bytes)
SBForumEmail.csv.txt (26,535 Bytes)
XLMSchema.dtd.txt (20,700 Bytes)

I would like to see it. Thanks.

[GeorgeMcGinn]

EDIT: If you copied the code before 7:10am EST, it has been replaced as there was an error when psize=0. It flagged all items with '************" regardless of size. Corrected code has been replaced here. - George

In this version I corrected the size of a computer byte so the file sizes are the actual size taking up space, added a variable so you can display files of a specific size or larger, or all of them, and in the final display added a line that shows, regardless of what you choose to display, the total bytes and megabytes in the entire folder.

First, 1024 is the size of a byte of memory, not 1000. So I corrected it all the way up to megabytes. Now you get the real space that a file is taking up.

I also added a variable that allows you to control the size that gets flagged and gives you both bytes and megabytes.

Variable Content.psize controls what files by size gets displayed. psize=0 displays all. psize=1024000 will only display files that are 1 megabyte or larger.

psize input is in bytes. So for files that are 2 megabytes or larger, psize=2048000. You can also use inbetween sizes, such as .5Mb, psize=512000.

Another change was to add, regardless of what is displayed, the total number of bytes and megabytes in the display at the end of the program. Now you can tell how much space the entire folder and sub-folders hold.

Next update will provide a GUI to control the run of the program and not only create the file contents.tree, it will display all the files in the GUI.

Code: Select all

'Tree.sb, by Dutchman, September 2017
'Modified by Rbytes and George McGinn, October 2017
'Modified by George McGinn, October 2017 (to use the correct values for bytes/megabytes & chose minimum file size to display)
'Makes File$ with contents of folder Dir$
'Folders are printed with leading "#"
'

OPTION BASE 1

'*** Formatting Setup, Integers and two-decimals
DIM Content.FMI$(12)
Content.FMI$(1)  = "#"
Content.FMI$(2)  = "##"
Content.FMI$(3)  = "###"
Content.FMI$(4)  = "#,###"
Content.FMI$(5)  = "##,###"
Content.FMI$(6)  = "###,###"
Content.FMI$(7)  = "#,###,###"
Content.FMI$(8)  = "##,###,###"
Content.FMI$(9)  = "###,###,###"
Content.FMI$(10) = "#,###,###,###"
Content.FMI$(11) = "##,###,###,###"
Content.FMI$(12) = "###,###,###,###"

DIM Content.FMD$(11)
Content.FMD$(1)  = "#.##"
Content.FMD$(2)  = "##.##"
Content.FMD$(3)  = "###.##"
Content.FMD$(4)  = "#,###.##"
Content.FMD$(5)  = "##,###.##"
Content.FMD$(6)  = "###,###.##"
Content.FMD$(7)  = "#,####,###.##"
Content.FMD$(8)  = "##,###,###.##"
Content.FMD$(9)  = "###,###,###.##"
Content.FMD$(10) = "#,###,###,###.##"
Content.FMD$(11) = "##,###,###,###.##"

Content.psize=512000              '*** Print files only this size or larger. 0=print all files
Content.byte=1024                 '*** Sixe of a byte in computer definition
Content.megabyte=1024000          '*** Size of a megabyte based on computer definition

Content.numbers=0
tbytes=0
tmegabytes=0

Dir$="/App Development"    '*** Change Directory name as desired
File$="contents.tree"
Indent$="   "              '*** adds to each folder-level, e.g. "#" or "   "

IF LEFT$(REVERSE$(Dir$),1)<>"/" THEN Dir$&="/"
IF NOT FILE_EXISTS(Dir$) THEN 
   PRINT "Folder """&Dir$&""" not found"
   STOP
ENDIF

IF FILE_EXISTS(File$) THEN FILE File$ DELETE
CALL Content(File$,Dir$,Indent$)
TEXT CLEAR


tsize$=STR$(tbytes)
fnd=INSTR(tsize$,"E")
IF fnd<>-1 THEN
   LB=VAL(RIGHT$(tsize$,2))+1
ELSE
   LB=LEN(STR$(tbytes))
ENDIF

LM=LEN(STR$(tmegabytes))
LM=LEN(STR$(INTEG(LM)))
IF LM=0 THEN LM=1

tsizeline$="Total Bytes (Megabytes) in folder: "&STR$(tbytes,Content.FMI$(LB))&" ("&STR$(tmegabytes,Content.FMD$(LM))&")"

PRINT "Scanned folder """&Dir$&"""."
PRINT "Found "&Files&" files in "&Dirs&" folders."
PRINT "Content written to """&File$&"""."
PRINT tsizeline$

'** PRINT "Total Bytes (Megabytes) in folder = "&tbytes&" ("&tmegabytes&")"

END


DEF Content(File$,Dir$,Indent$) 
' Write contents of folder Dir$ to File$
' Folder-names are preceded by "#"
' Indent$ is added at each level
' If Content.number<>0 then names 
'	will be preceded by level-number&"|"
' Dirs and Files-count are written to 
'	.Dirs and .Files
'---------- by Dutchman, 2017 ----------

IF numbers THEN Level$=Level&"|" ELSE Level$=""
FILE File$ PRINT Tab$&Level$;"#";Dir$
.Dirs+=1
Tab$&=Indent$
level+=1
IF numbers THEN Level$=Level&"|" ELSE Level$=""
Leader$=Tab$&level$

'--- list files and dirs
Content$="" ' clear for this folder
DIR Dir$ LIST FILES F1$,n1
DIR Dir$ LIST DIRS F2$,n2

FOR i=1 TO n1
    Content$&=F1$(i)&","
NEXT i

FOR i=1 TO n2
    Content$&="#"&F2$(i)&","
NEXT i

'--- write contents
WHILE LEN(Content$)>1
      p=INSTR(Content$,",")
      F$=LEFT$(Content$,p-1)
      Content$=RIGHT$(Content$,LEN(Content$)-p)

      IF INSTR(F$,"#")<1 THEN                        '** is file
         size=FILE_SIZE(Dir$&F$)!bsize$=STR$(size)
         fnd=INSTR(bsize$,"E")
         IF fnd<>-1 THEN
            LB=VAL(RIGHT$(bsize$,2))+1
         ELSE
            LB=LEN(STR$(size))
         ENDIF
         LM=LEN(STR$(size/megabyte))
         LM=LEN(STR$(INTEG(LM)))
         IF LM=0 THEN LM=1

         .tbytes=.tbytes+size
         .tmegabytes=.tmegabytes+(size/megabyte)

         IF psize=0 THEN
            IF size>=megabyte THEN
               size$=" ("&STR$(size/megabyte,FMD$(LM))&"Mb or "&STR$(size,FMI$(LB))&" Bytes)"
               FILE File$ PRINT Leader$&F$&size$&" **********"
            ELSE
               size$=" ("&STR$(size,FMI$(LB))&" Bytes)"
               FILE File$ PRINT Leader$&F$&size$
            ENDIF
         ELSE
            IF size>=megabyte THEN
               size$=" ("&STR$(size/megabyte,FMD$(LM))&"Mb or "&STR$(size,FMI$(LB))&" Bytes)"
               FILE File$ PRINT Leader$&F$&size$&" **********"
            ENDIF
         ENDIF
         .Files+=1
      ELSE ' is folder 
         PRINT CHR$(9632);
         f$=Dir$&RIGHT$(f$,LEN(F$)-1)&"/"
         CALL Content(File$,f$,Indent$)
      ENDIF

END WHILE

RETURN 

ENDDEF

The first image shows the last line that now displays the total bytes (total megabytes in parentheses) regardless of the value in Content.psize.

The second image shows part of the run of the program listed here with psize=512000 or .5Mb.