A direct access database system

User avatar
rbytes
Posts: 1338
Joined: Sun May 31, 2015 12:11 am
My devices: iPhone 11 Pro Max
iPad Pro 11
MacBook
Dell Inspiron laptop
CHUWI Plus 10 convertible Windows/Android tablet
Location: Calgary, Canada
Flag: Canada
Contact:

Re: A direct access database system

Post by rbytes »

It could take me a while. Lots of holiday activities over here, along with -20 C temperatures :o One of the coldest starts to winter in years.

Thanks for the new info.
Got the add_rec working now. Probably I had a typo somewhere.
Will edit_rec be developed further?

I was experimenting today using fields on a graphics screen to do the data_dump and other functions. I noticed that name and surname are reversed. If I have the meaning correct, surname should be the last name.
The only thing that gets me down is gravity...

User avatar
rbytes
Posts: 1338
Joined: Sun May 31, 2015 12:11 am
My devices: iPhone 11 Pro Max
iPad Pro 11
MacBook
Dell Inspiron laptop
CHUWI Plus 10 convertible Windows/Android tablet
Location: Calgary, Canada
Flag: Canada
Contact:

Re: A direct access database system

Post by rbytes »

I spent way too much time on this project today, but it is taking shape. Below are some screen grabs.

I made a couple of changes. I still use last names as keys, but have reworked the logic slightly so that they are properly labeled as surnames.

Maybe you can clear up a mystery for me. I noticed that the spacing was quite irregular in the data statements that initialize the records in the database . I thought I would clean that up and just have single spaces dividing the fields, and no leading or trailing spaces in each data statement. But this has created issues where some surnames now have letters chopped off on the left in the data dump. If I then go back to the data statements and add some leading spaces to the problem surnames, this fixes that particular record when viewed in the data dump, but I don't think it is a viable solution. It causes that surname not to be found when I do a search!

I also have tried adding new records using the Add button, and once again the surnames are sometimes chopped off on the left when I look at the data dump. You can see an example in a screenshot of the data dump, below. Lydia Carter's surname is chopped.

So I will wait to hear how this can be fixed. I am also wondering if you have plans for a function to sort the records alphabetically by surname.

I will hold off posting the code until I get Datamine more functional.
Attachments
IMG_8691.PNG
IMG_8691.PNG (4.81 MiB) Viewed 6014 times
IMG_8692.PNG
IMG_8692.PNG (4.8 MiB) Viewed 6014 times
IMG_8685.PNG
IMG_8685.PNG (3.62 MiB) Viewed 6016 times
IMG_8690.PNG
IMG_8690.PNG (232.89 KiB) Viewed 6016 times
The only thing that gets me down is gravity...

User avatar
sarossell
Posts: 195
Joined: Sat Nov 05, 2016 6:31 pm
My devices: iPad Mini 2, iPhone 5, MacBook Air, MacBook Pro
Flag: United States of America
Contact:

Re: A direct access database system

Post by sarossell »

This is all starting to look really amazing. I'm excited to see where it all leads. So much has already been accomplished. Imagine the possibilities. :)
smart BASIC Rocks!

- Scott : San Diego, California

Henko
Posts: 814
Joined: Tue Apr 09, 2013 12:23 pm
My devices: iPhone,iPad
Windows
Location: Groningen, Netherlands
Flag: Netherlands

Re: A direct access database system

Post by Henko »

sarossell wrote:This is all starting to look really amazing. I'm excited to see where it all leads. So much has already been accomplished. Imagine the possibilities. :)
I'm curious too. It is a "team" development now, but quite different from the IT projects i used to do as a free-lance projectleader :lol:

User avatar
sarossell
Posts: 195
Joined: Sat Nov 05, 2016 6:31 pm
My devices: iPad Mini 2, iPhone 5, MacBook Air, MacBook Pro
Flag: United States of America
Contact:

Re: A direct access database system

Post by sarossell »

Agreed, we've sort of stumbled onto a collaborative, international, open source project using the forum as a kind of old school BBS instead of using Github. I'm ecstatic to see what comes of it all - what CAN come of it all using BASIC and a "BBS" in our modern iPad world. :D
smart BASIC Rocks!

- Scott : San Diego, California

User avatar
rbytes
Posts: 1338
Joined: Sun May 31, 2015 12:11 am
My devices: iPhone 11 Pro Max
iPad Pro 11
MacBook
Dell Inspiron laptop
CHUWI Plus 10 convertible Windows/Android tablet
Location: Calgary, Canada
Flag: Canada
Contact:

Re: A direct access database system

Post by rbytes »

It does feel good to be on the cutting edge of international collaboration. I didn't mention this in my last post, but there are some very useful code snippets that can be adapted from various Basic Programs posts. The Datamine toolbar is based on code originated by Dav, an American Forum member. I like the way it expands the usable screen area and gives more information.

In some future version I intend to add a file requester so that different databases can be loaded. Most likely I will use the file requester originally posted by Henko, which I have adapted and used in many of my programs.
The only thing that gets me down is gravity...

User avatar
sarossell
Posts: 195
Joined: Sat Nov 05, 2016 6:31 pm
My devices: iPad Mini 2, iPhone 5, MacBook Air, MacBook Pro
Flag: United States of America
Contact:

Re: A direct access database system

Post by sarossell »

Looks great. I look forward to seeing the code.
smart BASIC Rocks!

- Scott : San Diego, California

Henko
Posts: 814
Joined: Tue Apr 09, 2013 12:23 pm
My devices: iPhone,iPad
Windows
Location: Groningen, Netherlands
Flag: Netherlands

Re: A direct access database system

Post by Henko »

I lost quite some text today by typing text directly in a posting, without having logged in. So, here we go again, this time using a wireless keyboard and using Notepad as an intermediate medium that also keeps typed text intact.

First of all, i fixed the truncation error. It happened tor the record wich was written into a new slot at the end of the file.The filepointer cannot be larger than the length of the file. So, in order to extend the database, a dummy record is now first appended to the file. After that the correct filepointer for the new record can be set.

There was a language problem (those bloo&^# foreigners! -:)) with me about names. I interchanged the meaning of "name" and "surname". In the testcase, The surname is the key, that is field (1), the second field in the record.

The irregular spacing between the record fields was done intentionally for testing purposes. Actually, the SPLIT command is used and it works fine. I tested with minimal spacing as you did, and the results were ok.
The big advantage of "packing" the fields into one string is that the "low level" functions are independent of the record format, they need not be adapted for a specific record format.

The DATAMINE panel looks promizing. There are some remarks:
The buttons NEXT and PREV have no meaning. There is no logical key sequence in this data organization. The records are scattered over the database by the obscure hashing algorithm. Even the linked list mechanism does not reflect any logical key sequence, it connects totally different keys, which happen to be assigned an identical slot adres by the hashing function.
But i already coded a list, displaying all keys in a sorted order. There you may select a key to have the corresponding record displayed in a separate window, and you have the previous and the next record "at hand". The list and the record window are page-based, so the the background remains intact upon closing the list and the window. It is shown in the accompanying code.
Another point is the LOAD button. If it means loading the database for the first time, or appending a load of records from a "disc" file, i would call it an IMPORT function. It can easily be coded, if the input is presented in a file with the Key$,Record$ format.
Final remark: do you have an idea how to cope with a record layout with numerous fields on the DATAMINE panel?

When the number of records in the database grows far beyond the initial guess, then it is desirable to entend the primary area, to keep the number of accesses in the overflow area minimal. But it is never a necessity, the database will keep functioning.
The extreme case is a primary area of only 1 slot. The first record will take that slot, all other records will form one long linked list in the overflow area. This means that getting a record with given key will cause reading half of the database on the average. This is theory, but could be tested easily with the little testDB.

Deletion of records: this function merely marks the slot as "empty", by writing "","","0" to it. This means that a large part of the "deleted" record remains visible, but it does no harm. If it is a record in the overflow area, the links around it are properly modified to "skip" that slot.
By the way, i use the "hex-viewer" to inspect the database in detail.

An important issue is the uniqueness of the key$. The ADD_REC() function will not add a record with an alraeady existing key in the DB. Some "low level" functions and perhaps some "user" functions would become far more complex if this uniqueness would not be required.

In the accompanying code, i made a dissection into "user" functions and "low level" functions, the latter beiing record layout independent. I've put those in an include file, as i did with other apps. Some of those functions are rather tricky; if they are modified by other persons, we loose a common basis for working on this little project. In the "user" functions, the app dependent code pieces are in yellow.

Code: Select all

graphics ! graphics clear .8,.8,.8
fill color .8,.8,.8 ! draw color 0,0,0
first_time=1
f$="testdb"
if first_time then init_db(f$)

db(f$)     ' open database f$
edit_rec("Brady")
' disp_rec()
' del_rec("Carter")
add_rec("Flintstone", "Fred Flintstone delivery 42 4400") 
key_list()
page "keys" show
rec_window(400,50,240,400,.8,.8,.8,1)
page "dbrec" show
do ! slowdown
  if bp("keys_close") then page "keys" hide
  if bp("dbrec_close") then page "dbrec" hide 
until forever
end

def rec_window(xs,ys,ww,hh,R,G,B,alpha)
name$="dbrec"
page name$ set 
page name$ frame xs,ys,0,0
page name$ color R,G,B,alpha
button name$&"_close" title "❎" at ww-30,5 size 22,22
set buttons custom ! draw color 0,0,0
button name$&"bottom" title "" at -6,hh-3 size ww+12,3
button name$&"left" title "" at 0,-6 size 3,hh+12
button name$&"right" title "" at ww-3,-6 size 3,hh+12
button name$&"upper1" title "" at -6,0 size ww+12,3
button name$&"upper2" title "" at -6,30 size ww+12,3
tx$="window for viewing and editing records"
field "temp" text tx$ at 10,140 size ww-20,60 ML RO
page name$ hide
page name$ frame xs,ys,ww,hh
end def


def db(fn$)       ' open database f$ (read meta data in slot 0)
'y'
n_fields=5        ' # of fields in record
''
dim f_name$(n_fields)   ' test DB has staff records with 5 fields
file fn$ setpos 0
file fn$ readline db_name$,size$,prim$,free$
S_size=size$ ! N_prim=prim$ ! S_free=free$
empty_slot$="" ! for i=1 to S_size ! empty_slot$ &= " " ! next i
restore to fields
'y'
for i=0 to n_fields-1 ! read f_name$(i) ! next i
fields:
data "Name","Surname","Dept.  ","Age    ","Salary "
'' 
end def

def user_edit$(rec$)
' this user function is called by edit_rec(key$)
'y'
' user code to modify the contents of rec$
''
return rec$
end def

def disp_rec(key$)    ' temporary function for test purpose
dim field$(db.n_fields)
do
  input "Surname? ": key$
  if key$="stop" or key$="Stop" or key$="" then break
  record$=get_record$(key$)
  if record$="not found" then
    print ! print record$ ! continue
    end if
  split record$ to field$,n with " "
  print
  for i=0 to n-1 ! print db.f_name$(i) & " - " & field$(i) ! next i
  until forever
end def

def init_db(f$)
'y'
  create_db(f$,"Test_database",50,10)
  restore to staff
  for i=0 to 8
    read rec$
    split rec$ to field$,n with " "  ' to extract the key
    key$=field$(1)
    add_rec(key$,rec$)
    next i
staff:
  data "John Friedman    sales      32  4500  "
  data " Betty Longa     ledger     28  3700"
  data " Ann Strady    sales      43  5300  "
  data "John-John Brady  production 25  3250  "
  data "   Lydia Carter  sales      36  3300 "
  data "  Peter Pan    R&D    48   4100"
  data " Reginald Nobody  production  23 3100  "
  data " Mary Poppins  sales  29  3650 "
  data " Elly Sunshine   production  32 3560 "
''
end def



{db_util}
' def create_db(fn$,db_name$,S_size,N_rec)
' def add_rec(key$,rec$)
' def edit_rec(key$)
' def del_rec(key$)
' def get_record$(key$)
' def get_adr(key$)
' def hash(tt$)
' def key_list()
' def get_1st_free()
' def put_1st_free(adr)
' def db_dump()
' def list_window(name$,titel$,cont$(),size,xs,ys,ww,hh,R,G,B,alpha)
' def pos(adr)
' def f_grey()
' def f_yellow()
' def d_black()
' def d_blue()
' def bp(a$)
And here is the include file, presently to reside in the same directory as the main program

Code: Select all

' Create a direct access database with fixed length slots
' fn$      - filename on "disc"
' db_name$ - database name for readability
' S_size   - slotsize for key + record + link fields
' N_rec    - estimated max. # of records
'
def create_db(fn$,db_name$,S_size,N_rec)
N_prim=N_rec          ' # of slots in primary area
S_free=N_rec+1        ' initial first free slot
'
dim slot(S_size)
if file_exists(fn$) then file fn$ delete
'
   ' create space for the database
for i=0 to N_prim ! file fn$ writedim slot ! next i
'
   ' write slot number 0 with meta data
file fn$ setpos 0
file fn$ writeline db_name$,str$(S_size),str$(N_prim),str$(S_free)
'
   ' init the slots in the primary area
for i=1 to N_rec
  file fn$ setpos S_size*i
  file fn$ writeline "","","0"
  next i
db(fn$)
end def

def add_rec(key$,rec$)
adr=get_adr(key$)
if adr>0 then return -1    ' record with same key already present
if adr<0 then              ' free slot in primary area
  adr=-adr
  pos(adr)  ! file db.fn$ writeline key$,rec$,"0"
  return adr
  end if
new=hash(key$)
pos(new)    ! file db.fn$ readline aux$,trec$,link$
rec_to=get_1st_free()
pos(rec_to) ! file db.fn$ writeline db.empty_slot$
pos(rec_to) ! file db.fn$ writeline key$,rec$,link$
pos(new)    ! file db.fn$ writeline aux$,trec$,str$(rec_to)
put_1st_free(rec_to+1)
end def

def edit_rec(key$)
adr=get_adr(key$)
if adr>0 then
  pos(adr) ! file db.fn$ readline key$,rec$,link$
  rec$=user_edit$(rec$)   ' user function to edit fields in rec$
  pos(adr) ! file db.fn$ writeline key$,rec$,link$
  end if
end def

def del_rec(key$)
prd_adr=hash(key$)
pos(prd_adr) ! file db.fn$ readline prd_key$,prd_rec$,prd_link$
if prd_key$=key$ then
  suc_adr=val(prd_link$)
  if suc_adr=0 then
    pos(prd_adr) ! file db.fn$ writeline "","","0"
    return 0
    end if
  pos(suc_adr) ! file db.fn$ readline suc_key$,suc_rec$,suc_link$
  pos(prd_adr) ! file db.fn$ writeline suc_key$,suc_rec$,suc_link$
  pos(suc_adr) ! file db.fn$ writeline "","","0"
  return 1
  end if
do
  suc_adr=val(prd_link$)
  pos(suc_adr) ! file db.fn$ readline suc_key$,suc_rec$,suc_link$
  if suc_key$=key$ then
    prd_link$=suc_link$
    pos(prd_adr) ! file db.fn$ writeline prd_key$,prd_rec$,prd_link$
    pos(suc_adr) ! file db.fn$ writeline "","","0"
    return 1
    end if
  prd_adr=suc_adr
  pos(prd_adr) ! file db.fn$ readline prd_key$,prd_rec$,prd_link$
  until suc_link$="0"
return 0
end def

' retrieve record with key <key$> from the DB
' returns <not found> if not found
'
def get_record$(key$)
slot=get_adr(key$)
if slot>0 then 
  pos(slot) ! file db.fn$ readline t$,rec$,t$
  return rec$
  else
  return "not found"
  end if
end def

' find adress of record with given key
' key$ - the key on which to search
' function returns one of 3 values:
' adr > 0 : record found in slot <adr>    (update or delete record)
' adr = 0 : no record found with that key  (error key or add new record)
' adr < 0 : not found, but slot <abs(adr)> is free (add new record)
'
def get_adr(key$)
adr=hash(key$)
while adr>0
  pos(adr) ! file db.fn$ readline aux$,rec$,link$
  if aux$=key$ then return adr
  if aux$="" then return -adr
  adr=val(link$)
  end while
return 0
end def

' transform key tt$ into a DB adress (slotnumber)
'
def hash(tt$)
nt=len(tt$) ! adr=1
for i=0 to nt-1 ! adr+=asc(mid$(tt$,i,1))-32 ! adr*=7 ! next i
return 1+adr%db.N_prim
end def

def key_list()
f$=db.fn$ ! slot=db.S_size ! nkeys=-1
file f$ setpos 999999 ! nslot=floor(file_pos(f$)/slot)
dim keys$(nslot)
for i=1 to nslot
pos(i) ! file f$ readline key$,t$,t$
if key$<>"" then ! nkeys+=1 ! keys$(nkeys)=key$ ! end if
next i
list_window("keys","Record keys",keys$,nkeys,50,50,180,500,.8,.8,.8,1)
end def

' returns the adress of the first free slot
'
def get_1st_free()
pos(0) ! file db.fn$ readline t$,t$,t$,t$
return val(t$)
end def

' edits the adress of the first free slot
'
def put_1st_free(adr)
pos(0) ! file db.fn$ readline  t1$,t2$,t3$,t4$
pos(0) ! file db.fn$ writeline t1$,t2$,t3$,str$(adr)
return
end def

def db_dump()
print
f$=db.fn$ ! slot=db.S_size
file f$ setpos 999999 ! nslot=floor(file_pos(f$)/slot)
for i=1 to nslot
pos(i) ! file f$ readline key$,rec$,link$
if key$<>"" then print i;"  ";key$;"  ";rec$;"  ";link$
next i
end def

def list_window(name$,titel$,cont$(),size,xs,ys,ww,hh,R,G,B,alpha)
dim temp$(size+1)
for i=0 to size ! temp$(i)=cont$(i) ! next i
sort temp$
page name$ set 
page name$ frame xs,ys,0,0
page name$ color R,G,B,alpha
set buttons custom ! d_black() ! f_grey()
button name$&"_close" title "❎" at ww-30,5 size 22,22
button name$&"bottom" title "" at -6,hh-3 size ww+12,3
button name$&"left" title "" at 0,-6 size 3,hh+12
button name$&"right" title "" at ww-3,-6 size 3,hh+12
button name$&"upper1" title "" at -6,0 size ww+12,3
button name$&"upper2" title "" at -6,30 size ww+12,3
button name$&"title" title titel$ at 20,3 size ww-60,27
set lists custom
list name$ text temp$ at 2,32 size ww-4,hh-34
page name$ hide
page name$ frame xs,ys,ww,hh
end def

def pos(adr)
file db.fn$ setpos adr*db.S_size
end def

def f_grey() ! fill color .8,.8,.8 ! end def
def f_yellow() ! fill color 1,1,.7 ! end def
def d_black() ! draw color 0,0,0 ! end def
def d_blue() ! draw color 0,0,1 ! end def
def bp(a$) = button_pressed(a$)
IMG_1320.PNG
IMG_1320.PNG (330.91 KiB) Viewed 5953 times

User avatar
sarossell
Posts: 195
Joined: Sat Nov 05, 2016 6:31 pm
My devices: iPad Mini 2, iPhone 5, MacBook Air, MacBook Pro
Flag: United States of America
Contact:

Re: A direct access database system

Post by sarossell »

Well, now that there's a growing library of functions, what should we name this new database management system of yours? smartBASE? sBASE? HenkoBase? HSK (Henko Slot Key)? Here's a list of other names that exist: http://tinyurl.com/gtvjr67
smart BASIC Rocks!

- Scott : San Diego, California

User avatar
rbytes
Posts: 1338
Joined: Sun May 31, 2015 12:11 am
My devices: iPhone 11 Pro Max
iPad Pro 11
MacBook
Dell Inspiron laptop
CHUWI Plus 10 convertible Windows/Android tablet
Location: Calgary, Canada
Flag: Canada
Contact:

Re: A direct access database system

Post by rbytes »

Thanks, Henko. I will take a look at the new code tonight. I am booked for Christmas shopping at IKEA this afternoon. :lol:

Some quick responses. I agree that it is best if you handle the low-level functions. I changed the data statements to put the surname first, and then made field 0 the key, since it was now the surname. Those changes corrected the labeling. Perhaps you have already done this correction some other way.

Regarding more or fewer fields. This one is a bit tougher. If there are only a small number of fields contained in the database, I don't think it would be good esthetically to have a bunch of blank slots on screen. Also some slots, such as those containing notes, will need to be quite a bit larger, or at least taller, such as the image window. Whether all of the tools need to be on screen for manipulating the layout is a good question. There could be pop-up menu that allows for setting the field layout and linking them to data. The other alternative would be some statements near the front of the code that could be changed to allow for more or fewer on-screen fields. If this program is to have maximum flexibility in being able to maintain multiple databases, then there will need to be a data header added to each database that describes its layout - how many fields, where positioned, what data links to them, etc. but I see that as an evolution that will take some time to reach

Re Next and Previous. These buttons make sense if the records can be called up alphabetically by surname, which your sorted list does well too. A possible combination could be that Next and Previous buttons step through the names in the list sequentially by alphabet, and the pop-up list gives random access to any name without needing to type it into the search field.

The Load button I envisioned to load distinctly different databases. I didn't include a save button since saves are performed continuously. But there may need to be at least a Confirm button for layout changes.

Regarding a name for the program, I did pick a name that I thought might work well, and I used it in the screen shots I posted last night. I like something short with a bit of flair, which ideally includes a pun or dual meaning. Naming it after one or both of us as the main collaborators doesn't appeal to me as much :?:
The only thing that gets me down is gravity...

Post Reply