Page 1 of 1

Faster file read & write data routines (INPUT/OUTPUT)

Posted: Sun Apr 10, 2016 3:21 am
by Dav
These routines kind of mimic Qbasics file INPUT £ OUTPUT functions. They read and write many bytes at a time. It uses the file readdim and writedim to read and write data instead of the slower read & write on byte at a time method.

The demo code makes a 5000 byte file and reads & writes data to it using both methods to show the speed difference. Helpful when loading and saving large file data.

- Dav

Code: Select all

'InputOutput.txt
'Faster File data read & write routines.
'Works like Qbasic file input/output ones.
'Reads and writes chunks of data at a time.
'Faster than the one byte read/write method.
'Reads/writes using readdim/writedim instead.
'Code by Dav, April/2016

option base 1

'make a test file to use first

f$="testfile.dat"
print "INPUT & OUTPUT functions test."
print "Making testfile.dat of 5000 bytes to use..."
print
for t = 1 to 5000
   file f$ write 33
next t

'read file using file read byte method

print "Reading 'file read' one byte method..."
t1=time()
a$=""
file f$ setpos 0
for t = 1 to 5000
   file f$ read c
   a$=a$&chr$(c)  'build data...
next t
print "Read 5000 bytes in ";
print time()-t1;" secs"
print

'read file using new file input method

print "Reading new INPUT$ method..."
t1=time()
a$=input$(f$,1,-1) 'load whole file (-1)
print "Read 5000 bytes in ";
print time()-t1; " secs"
print

print "Writing 'file write' one byte method..."
t1=time()
file f$ setpos 0
for t = 1 to 5000
   file f$ write asc(mid$(a$,t,1))
next t
print "Wrote 5000 bytes in ";
print time()-t1;" secs"
print

print "Writing new OUTPUT method..."
t1=time()
output(f$,1,a$)
print "Wrote 5000 bytes in ";
print time()-t1; " secs"
print

end



'=====================================

def input$(f$,pos,bytes)

'Reads data from file - returns it as input$
'
'f$    = file to read
'pos   = position in file to start reading
'bytes = how many bytes to read
'
'note: if bytes=-1 then reads entire file.

  'make sure values are in bounds
  if pos < 1 then pos=1
  if bytes>file_size(f$) or bytes=-1 then 
     bytes=file_size(f$)
  end if

  'read entire file into m array
  file f$ setpos 0
  file f$ readdim m
  
  'read location, build data
  t$=""
  for r=pos to (pos+bytes-1)
    t$=t$&chr$(m(r))
  next r
  
  'return file data as input$
  input$=t$

end def

'==================================

def output(f$,pos,out$)

'writes data out to given file.
'
'f$   = file to write to
'pos  = position in file to start saving to
'out$ = the data to write to file

  'load file into m array
  file f$ setpos 0
  file f$ readdim m

  'change m array data
  s=1
  for r=pos to (pos+(len(out$))-1)
    m(r)= asc(mid$(out$,s,1))
    s=s+1
  next r

  'save file with m changes
  file f$ setpos 0
  file f$ writedim m

end def
   
'===================================

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Sun Apr 10, 2016 8:54 am
by SBRuss
Nice. My first foray into sB will be doing quite a bit of data manipulation, so this could be useful.

One thing I didn't realise though, is there no limit on the size of a string in sB?

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Sun Apr 10, 2016 9:22 am
by Mr. Kibernetik
SBRuss wrote:One thing I didn't realise though, is there no limit on the size of a string in sB?
No, there is no pre-defined limit.

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Sun Apr 10, 2016 11:25 am
by Dav
SBRuss wrote:Nice. My first foray into sB will be doing quite a bit of data manipulation, so this could be useful.
Thanks, SBRuss. Welcome to the forum. I read your post in the other area about storing large numbers and file information. Here's an adaptation of these file routines that may interest you. I use these for a database kind of file that stores 100 blocks of bytes in a file for information. You can adjust it to suit your needs.

- Dav

Code: Select all

'Database file manipulation example.
'Makes and uses a file containing 100 byte blocks of info.

option base 1

'Let's make a 900 byte file for testing
'It contains 9 unique 100-byte blocks.
'Each block has special data.
'block 1 is full of 100 1's,
'block 2 is full of 100 2's, etc...

f$="900.dat"

for b = 1 to 9
   c$=b
   for t = 1 to 100
     c$=b
     file f$ write asc(c$)
   next t
next b

'Now let's read blocks and print its data.
'It should contain 100 numbers each.
'block will contain 5's, etc...

print "block 5 data: "; ReadBlock$(f$,5,100)
print "block 3 data: "; ReadBlock$(f$,3,100)

print
print "Changing block 5 data..."
print

a$="This data is now in block 5..."
SaveBlock(f$, 5, a$,100)

print "block 5 data: "; ReadBlock$(f$,5,100)

def ReadBlock$(f$,block,bytes)
  p=block*bytes-(bytes-1)
  file f$ setpos 0
  file f$ readdim m
  t$=""
  for r=p to (p+bytes-1)
    t$=t$&chr$(m(r))
  next r
  ReadBlock$=t$
end def


def SaveBlock(f$,block,dat$,bytes)

  'if data length is < bytes,
  'add spaces to make it equal bytes.
  if len(dat$) < bytes then
    do
      dat$=dat$&" "
      if len(dat$)=bytes then break
    until 0
  end if

  'load file into m array
  file f$ setpos 0
  file f$ readdim m

  p=block*bytes-(bytes-1)

  s=1
  for r=p to (p+bytes-1)
    m(r)= asc(mid$(dat$,s,1))
    s=s+1
  next r

  'save file with m changes
  file f$ setpos 0
  file f$ writedim m
  
end def

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Mon Apr 11, 2016 3:15 am
by rbytes
When I run this code to the end and then check the contents of 900.dat, it contains only the text "This data is now in block 5..."

If I debug pause the code right after 900.dat is first saved, the contents are correct:

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222223333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444455555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555556666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777788888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888889999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Mon Apr 11, 2016 3:31 am
by Dav
Whoops. I bet I posted an early buggy version of the block saving program. I'll have to wait till tomorrow to check it out though (got to get up early in the morning for a job interview) Thanks for reporting this.

- Dav

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Mon Apr 11, 2016 3:43 am
by rbytes
Thanks, Dav. It should be very useful.

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Mon Apr 11, 2016 10:06 am
by SBRuss
I'm back to work today after holidays, so my experimentation is on hold for now, but I'll certainly have a look at whatever you put up.

Thanks.

Re: Faster file read & write data routines (INPUT/OUTPUT)

Posted: Mon Apr 11, 2016 4:57 pm
by Dav
I fixed the block code above. it was a just a typo when calling the save Routine, it was saving 900 bytes instead 100. I also fixed some other typos. The code is updated.

- Dav