Page 1 of 1

Recursively Listing Files and Folders

Posted: Sat Feb 27, 2021 6:05 am
by matt7
Below is a standalone library for creating a list of either all files or all folders recursively from a starting directory. The two main functions are:
  • DIRS_LIST_RECURSIVE(dir$)
  • FILES_LIST_RECURSIVE(dir$)
Here is a usage example:

Code: Select all

{{/lib/files/list_recursive}}    ' whatever you name it and wherever you place it
SET OUTPUT FONT SIZE 14

' List all directories recursively starting from Smart Basic's root directory
dir$ = "/"
DIRS_LIST_RECURSIVE(dir$)
PRINT "All dirs starting at """ & dir$ & """"
PRINT "(" & DIRS_LIST_RECURSIVE.nDirs & " Found)"
FOR i = 0 TO DIRS_LIST_RECURSIVE.nDirs-1
  PRINT DIRS_LIST_RECURSIVE.dirs$(i)
NEXT i
PRINT ""

 ' List all files recursively starting one level above the current directory
dir$ = ".."
FILES_LIST_RECURSIVE(dir$)
PRINT "All files starting in """ & dir$ & """"
PRINT "(" & FILES_LIST_RECURSIVE.nFiles & " Found)"
FOR i = 0 TO FILES_LIST_RECURSIVE.nFiles-1
  PRINT FILES_LIST_RECURSIVE.files$(i)
NEXT i

' Example of how to copy the resulting array into a local scope
nFiles = FILES_LIST_RECURSIVE.nFiles
DIM files$(nFiles)
COPY1$(FILES_LIST_RECURSIVE.files$, files$)
After you call one of the functions you can DIM a local array and copy the contents from the function's scope into the new array. The library also includes a COPY1$ function which I find convenient.

You can also write to an array named OPT.dirs_exclude$() and populate it with directory paths that you want to exclude from the searches. For example, I have a /Backups folder that is large and has archived code that I'm not interested in seeing all of its subfolders. This exclude functionality is more useful for performing find and replaces recursively, but that is a separate library file. If anyone is interested I can also put that here on the forum.

Here is the library:

Code: Select all

'g'
/*=============================================================

Functions:

Functions for recursively listing all files or folders in all subdirectories:

    DIRS_LIST_RECURSIVE(dir$)
    FILES_LIST_RECURSIVE(dir$)

Functions for managing the list of directories to exclude from list recursive operations:

    CLEAR_EXCLUDE_DIRS()
    DEFAULT_EXCLUDE_DIRS()

Parameters:

    OPT.dirs_exclude$()

===============================================================

For both DIRS_LIST_RECURSIVE and FILES_LIST_RECURSIVE, the directories listed in OPT.dirs_exclude$() will be ignored, unless it is the directory provided to the function (input dir$).

For each functions' output arrays, DIRS_LIST_RECURSIVE.dirs$() and FILES_LIST_RECURSIVE.files$(), each item is a folder or file path. This path is relative to the input directory. For example, an input of ".." (the directory up one level) will result in an array will all elements beginning with "../". Note however that the default OPT.dirs_exclude$ list are absolute paths that begin at the root directory, "/". Overwrite this list of directories to exclude if needed.

===============================================================

DIRS_LIST_RECURSIVE creates a one-dimensional array that lists all folders within the specified folder and recursively down all of its subfolders. This list includes the input directory itself.

  Inputs:

    dir$    Directory to recursively traverse

  Outputs:

    DIRS_LIST_RECURSIVE.dirs$()    Array of dir paths
    DIRS_LIST_RECURSIVE.nDirs      Size of dirs$()

===============================================================

FILES_LIST_RECURSIVE creates a one-dimensional array that lists all files within the specified folder and recursively down all of its subfolders.

  Inputs:

    dir$    Directory to recursively traverse

  Outputs:

    FILES_LIST_RECURSIVE.files$()    Array of file paths
    FILES_LIST_RECURSIVE.nFiles      Size of files$()

=============================================================*/

''
'b'
'==============================================================
'  Variable initialization
'==============================================================

''
DEFAULT_EXCLUDE_DIRS()

.ERR = -1
.OK = 0

DIM BF.s1$(1000)
BF_RESIZE_S1.resizeFactor = 5

'b'
'==============================================================
'  Functions
'==============================================================

''
DEF CLEAR_EXCLUDE_DIRS ()

  DIM OPT.dirs_exclude$(1)

END DEF

'==============================================================

DEF DEFAULT_EXCLUDE_DIRS ()

'c' Only these DATA statements need to be changed (a minimum of one DATA statement is required)

  DATA ""
  'DATA "/Backups"
  'DATA "/Data"
  '... etc.

''
  n = 0
  WHILE DATA_EXIST()
    READ dir$
    n += 1
  END WHILE
  RESTORE

  DIM OPT.dirs_exclude$(n)
  FOR i = 0 TO n-1
    READ dir$
    IF NOT ENDS_WITH(dir$,"/") THEN dir$ &= "/"
    OPT.dirs_exclude$(i) = dir$
  NEXT i

END DEF

'==============================================================

DEF DIRS_LIST_RECURSIVE (dir$)

  nDirs = 0       ' Init in case of error
  DIM dirs$(1)

  IF NOT IS_DIR(dir$) THEN RETURN .ERR
  IF NOT ENDS_WITH(dir$,"/") THEN dir$ &= "/"

  GET DIM BF.s1$ XSIZE bufSz
  BF.s1$(0) = dir$
  iRead = 0
  iWrite = 1

  DO
    DIR BF.s1$(iRead) LIST DIRS dirs$, n
    IF iWrite+n > bufSz THEN bufSz = BF_RESIZE_S1(iWrite+n)
    FOR i = 0 TO n-1
      dir$ = BF.s1$(iRead) & dirs$(i) & "/"
      IF MATCH(OPT.dirs_exclude$, dir$) > -1 THEN CONTINUE i
      BF.s1$(iWrite) = dir$
      iWrite += 1
    NEXT i
    iRead += 1
  UNTIL iRead >= iWrite

  nDirs = iWrite
  DIM dirs$(nDirs)
  COPY1$(BF.s1$, dirs$)
  SORT dirs$

  RETURN .OK

END DEF

'==============================================================

DEF FILES_LIST_RECURSIVE (dir$)

  nFiles = 0       ' Init in case of error
  DIM files$(1)

  IF NOT IS_DIR(dir$) THEN RETURN .ERR

  DIRS_LIST_RECURSIVE(dir$)

  GET DIM BF.s1$ XSIZE bufSz
  iWrite = 0
  FOR iRead = 0 TO DIRS_LIST_RECURSIVE.nDirs-1
    dir$ = DIRS_LIST_RECURSIVE.dirs$(iRead)
    DIR dir$ LIST FILES files$, n
    IF iWrite+n > bufSz THEN bufSz = BF_RESIZE_S1(iWrite+n)
    FOR i = 0 TO n-1
      BF.s1$(iWrite) = dir$ & files$(i)
      iWrite += 1
    NEXT i
  NEXT iRead

  nFiles = iWrite
  IF nFiles > 0 THEN
    DIM files$(nFiles)
    COPY1$(BF.s1$, files$)
    SORT files$
  ELSE
    DIM files$(1)
  END IF

  RETURN .OK

END DEF

'b'
'==============================================================
'  Other Functions
'==============================================================

''
DEF IS_FILE (path$)

  IF NOT FILE_EXISTS(path$) THEN RETURN .FALSE

  IF INSTR(path$, "/") = -1 THEN
    dir$ = "."
    file$ = path$
  ELSE
    idxFilename = LEN(path$) - INSTR(REVERSE$(path$), "/")
    dir$ = LEFT$(path$, idxFilename)
    file$ = RIGHT$(path$, LEN(path$)-idxFilename)
  END IF

  DIR dir$ LIST FILES files$, nFiles
  IF MATCH(files$, file$) = -1 THEN RETURN .FALSE

  RETURN .TRUE

END DEF

'==============================================================

DEF IS_DIR (path$)

  IF path$ = "" THEN RETURN .TRUE       ' Equivalent to "."
  IF IS_FILE(path$) THEN RETURN .FALSE
  IF NOT FILE_EXISTS(path$) THEN RETURN .FALSE

  RETURN .TRUE

END DEF

'==============================================================

DEF BF_RESIZE_S1 (minSize)

  GET DIM BF.s1$ XSIZE bufSize
  DIM TEMP.a$(bufSize)
  COPY1$(BF.s1$, TEMP.a$)

  DO bufSize *= resizeFactor
  UNTIL bufSize >= minSize

  DIM BF.s1$(bufSize)
  COPY1$(TEMP.a$, BF.s1$)

  RETURN bufSize

END DEF

'==============================================================

DEF COPY1$ (from$(), to$())

  GET DIM from$ XSIZE n
  GET DIM to$ XSIZE m

  n = MIN(n,m)

  FOR i = 0 TO n-1
    to$(i) = from$(i)
  NEXT i

END DEF

'==============================================================

DEF MATCH (array$(), string$)

  GET DIM array$ XSIZE n

  FOR i = 0 TO n-1
    IF array$(i) = string$ THEN RETURN i
  NEXT i

  RETURN -1    ' string$ not found (this is not an error)

END DEF

'==============================================================

DEF ENDS_WITH (str$, end$)

  IF RIGHT$(str$,LEN(end$)) = end$ THEN RETURN 1 ELSE RETURN 0

END DEF

'==============================================================

Re: Recursively Listing Files and Folders

Posted: Wed Mar 03, 2021 3:57 pm
by matt7
Just updated the library code with some bug fixes.

Re: Recursively Listing Files and Folders

Posted: Wed Mar 17, 2021 9:51 am
by Dutchman
That function did already exist: "Tree.sb to list folder contents", viewtopic.php?f=20&t=2008
Several users have added additional features in their replies and it has been used for an archiving and installation program: "MakeInstall.sb": viewtopic.php?f=20&t=2030#p13345