Programming challenge

[rbytes]

Smart Basic creates a root directory when it is installed. I have never anything in it deleted or overwritten by upgrades.

I did manage to do it myself though, while I was creating my FIle Manager program on my iPad. Somehow I issued the command DELETE "/", probably through a coding error. "/" is an alias for the root directory. Every smart Basic project I had worked on disappeared. The root folder was empty, emptier than it would be with a fresh install, because even the supplied Examples folder was gone. Glad that I had backups on my phone and in Dropbox, but it took me weeks to restore everything.

MR. K. says that he has since made changes to prevent this from occurring. But who is going to test them!!?

[sarossell]

I understand that's the case for the interpreter, but my concern is when an app is compiled and submitted for the Apple App Store and then later updated. The app ipa file is replaced, but where do you save data files for the ipa to access on the iDevice?

[Mr. Kibernetik]

If you create an app using smart BASIC SDK, publish it in App Store and later upgrade it, then all files created internally in your app will be kept.
This is because each app in iOS has its own Documents folder which is preserved during update, and sB writes its files in this Documents folder.

[sarossell]

So, it sounds like it's all automatic. That's cool.

[Henko]

I I'm still trying to wrap my head around how to create a database app using arrays. I wrote an address book program back in the early 80s, so I'm fairly confident I can figure it out.

Can you tell more about that?

If the following conditions exist:
- Large database (sequential read of the entire DB takes considerable seconds)
- Records have about the same length
- reading and writing of individual records using a key occurs often and on regular basis,

then i might have something for you in SB.

[rbytes]

I am also interested in a fast way to read database info, Henko, so I hope you will post it.

[sarossell]

The "database" program I created in the past (way back in 1982) in BASIC was a group of single dimension arrays of the same size; name$(100), address$(100), city$(100), etc. The version of BASIC I was using had specific commands for saving arrays to a file. So it was relatively simple to refer to the array value as a key for each entry. All of the array string values for a specific array number were related (i.e., the name for name$(1) was associated with the address for address$(1), etc). I was thinking about creating a simple comma delimited text files: key1, name, address...key2, name, address, etc. This would allow for a kind of random access for when an entry was deleted. The file could simply save the key entries in any order and the data could be of variable length as long as it didn't include a comma. The entire contents of all arrays would have to be loaded at run though, which could take a little bit of time as the "database" grows. After they're loaded in memory though, it should be instant access from the arrays. The user should only notice a slow run from a cold start after using the app for quite a while to make several entries. By then, they'll have become accustomed to the initial wait for the array load - assuming they don't have the program running on background all the time in the first place.

[sarossell]

I emailed the developer of the online icon generator app called Icon Slayer at http://www.gieson.com/Library/projects/ ... on_slayer/# to request the addition of an automatically generated json file to make icon development a real snap. He happily included the function. You just upload your image to the app on line, make whatever changes you like and then Export the icon images and json data in a zip file. It's got some nice graphical modification options as well. It's my free, go-to solution until perhaps one of you clever folks write something similar in sB. ;@)

[Henko]

I have no idea about the database knowledge of the programming enthousiasts in this forum, so i will handle this subject as if talking to newbees. For those who know more and better than me, apologies.

Databases are usually kept on an external storage device, which is slow compared with internal memory and has a considerable "search and access time". In such case, the application helds only one record in memory for processing (apart from buffering techniques).
But on a device like an iPad, the "harddisk" is as fast as the "internal memory". The DB (database) is then kept in one ore more files. If the DB is not too large, it is loaded entirely in arrays and processed "in memory". This method has a disadvantage: if the progam crashes, all editing is lost. Solution: save the entire DB upon each edit. This is bearable (not/hardly noticable), if the saving times are minimal.

Suppose, that the DB is large enough to require annoying saving times or even an annoying loading time.
In such case, the DB is kept as a file on "disk", and processing the DB is done on an individual record basis. This requires a direct access method, with wich a record can be retrieved from the DB without the need for a sequentially search.

There are some methods to do that. The principle is that a record with a given key (one field or a combination of fields from a record) is to be added, deleted, or retrieved for editing from the DB. The different methods do nothing else than transforming the key into an adress (filepointer) in the DB where to get or put the record.
Let's look to 3 such methods

1 Indexed sequential / fixed length
The records of the DB are stored in "fixed length" slots. The "slot size" has to accomodate the largest record in the DB.
All keys of the DB records are duplicated with their appropriate slot number in a separate file. The filepointer for a record = slot number x slot size.
This index file is sorted on key and kept sorted thereafter when adding new records.
The index file is also kept on "disk", is loaded entirely when the app starts, and saved upon each editing of the indexfile itself. New records are simple appended to the DB file, getting slot number Nrec+1, and a new entry is inserted in the index file, keeping it sorted on the keys. Deletion of records is simply done by setting the slot number in the index file to -1. As the slots are fixed length, such entries in the index file and the corresponding slots in the DB may be used for new records (mind the sorted poperty of the index file).
The big advantage of the sorted index file is that a binary search can be used to find a given key, which is practically timeless, even for very large amounts of records.
The advantage of this method and method 2 over method 3 is that the DB may be accessed with more than one key(combi), each key(combi) having its own index file.
This method allows that the DB may be processed sequentially without use of the index file (the slotsize beiing known). This is an advantage if the DB is kept on a physical "discdrive", but not in case of RAM like memory (like on the iPad).

2 Indexed sequential / variable length
Same as method 1, but the DB area is not divided in fixed length slots, it is one lineair adress (filepointer) space. Each entry in the indexfile does not contain a slot number, but right away the filepointer where to access the record. Adding new records is only by appending; sequential processing of the DB not possible without the index file.

3 Random access using hashing (fixed length slots only)
The DB is defined as a number of fixed length (accommodating the largest record length, plus a "linked list" pointer) slots. One part of the slots is named the "primary area", the rest as "secondary area".
An algorithm is designed which must map the key(combi) evenly spreaded into the primary area. The key(combi) may facilitate billions of records, depending on the length of the key, but the amount of slots in the primary area relates somehow to the actual expected number of records. Even if the amount of "primary" slots is equal or greater than the amount of records, there is always a possibility that two or even more than two records with different keys get the same slot number assigned by the hashing algorithm. Such records are called "synonimes", and the "secondary area" is used to store them in a "linked list" for each group of synonimes. The first entry of each linked list resides of course in the primary area. If the algorithm spreads the keys reasonably over the primary area, the size of that area can be selected such that a given percentage of all records have no synonimes, i.e. do require only one access to the DB. Accessing a record for editing with this method is rather staightforward, adding and deleting records requires editing a linked list mechanism.

In the case of an iPad, an implemetation of these methods can be realized by combining all fields of a record in one string with some separator between the fields, hence one record is one string. The field of a record may be extracted by the SPLIT statement. The random access possibility within the DB can then be realized using the FILE F$ READLINE and FILE F$ WRITELINE statements in combination with the
FILE F$ SETPOS X (X = slot number x slotsize) statement.

To be continued... ( with some coding for case 3)

[sarossell]

Thanks Henko!

That was very informative and will definitely be useful for my projects. I didn't understand a word of the third option though, so I'm looking forward to hearing more about that. In the meantime, I'm going to have to go back and read it again. I was playing around with the idea of having comma delimited variable length fields separated by key number and then search by key sequentially, but that would be slow since it would have to read through the entire file. After reading your message, I realized one huge oversight for me was not putting each record on a single line. The SETPOS/READ/WRITELINE combo is an ideal solution to that end. Thanks for that. I also had not anticipated the need for an index, so again, thanks. I'm intrigued to learn how to efficiently manage editing and deleting records as well as sorting and searching for specific data. Back in the 80s, when I was using a 3.5 Mhz Z80-based computer, slow was the operative norm. Reading files sequentially from a tape drive was just your only option. I was spoiled in a way with having to load the array data into memory from a single file. Now...Sakes!

As technology and my career progressed since 1982, I naturally stepped on board the SQL bandwagon and never looked back at BASIC for the specific idea of database design...until now. It's refreshing in an odd, nostalgic way. It requires more thought, and I like that. This is going to be a wild ride. :@)