Page 1 of 3

HEXtoBIN for pure HEX-files

Posted: Wed Sep 06, 2017 2:56 pm
by Dutchman
The hexdump-files generated for the "HEXdump2BIN"-program from viewtopic.php?f=20&t=1937 has about five times the size of the original file. A pure hex-file however should have two times the original size.
In OSX such a file can be generated with the command: hexdump -ve '1/1 "%.2x"' inputfile > outputfile
For Windows you can use the DOS-program "sfk.exe" which can be downloaded from
https://www.windows10download.com/swiss-file-knife/
See info in the attached file "BIN to HEX procedure"
BIN to HEX procedure.txt
(1.52 KiB) Downloaded 411 times
The following code translates the hex-file to the original binary file. The file should contain pure HEX-code without carriage-return or other ASCII characters.
Choose Select=1 or 2 or 3 for testing. The selection section can simply be extended with user-files.

Code: Select all

'HEXtoBIN by Dutchman, September 2017
'Transform hex-file to binary file
'In OSX-terminal the hex-file can be generated with
'	hexdump -ve '1/1 "%.2x"' inputfile > outputfile
'In Windows in the DOS-command line window
' 	with the program sfk.exe
'See file "BIN to HEX procedure" for details
'
'
'==== SELECT OPTIONS ====
Select=1 'Test1, Test2, Test3, User1, User2
ON Select GOTO Test1, Test2, Test3, User1, User2
'
'==== User filenames ====
' File names to select 
User1: ' set filenames InFile$ and BinFile$
  InFile$=""
  BinFile$=""
GOTO Preset
User2: ' set filenames InFile$ and BinFile$
  InFile$=""
  BinFile$=""
GOTO Preset
'
'==== Test files ====
Test1: ' small file
  Infile$="alarm.wavHEX"
  BinFile$="alarm.wav"
GOTO Preset
Test2: ' large file
  Infile$="hardy.mp3HEX"
  BinFile$="Hardy.mp3"
GOTO Preset
Test3: ' wrong file-format
  Infile$="alarm.wavHEXdump"
  BinFile$="alarm.wav"
GOTO Preset
'
Preset:'==== PRESETS ====
OPTION BASE 1
Steps=40 ' number of progress steps
MinSize=5E5 ' minimum filesize to show progress
'
'==== MAIN ====
'--- test Infile
IF Infile$="" THEN
  PRINT "Infile$ is not set" ! STOP
ELSE ! IF NOT FILE_EXISTS(InFile$) THEN
  PRINT "InFile$ """&Infile$&""" NOT found"
  STOP ! ENDIF
ENDIF
'
'--- Read file and determine size
filesize=FILE_SIZE(InFile$)
PRINT "HEXtoBIN operating on file """&InFile$&""""
PRINT "Filesize=";"#":filesize
DIM BIN(Filesize/2)
TIME RESET
PRINT "Reading file"
FILE InFile$ READLINE line$
Tr=TIME()
linesize=LEN(Line$)
IF Linesize<>Filesize THEN
  PRINT "*** ERROR *** Invalid file-format"
  STOP
ENDIF
nd=linesize/2
'
'--- set stepsize np
IF nd>=MinSize THEN 
  np=FLOOR(nd/steps) ' set progress steps
ELSE ! np=nd+1 ! ENDIF
'
'--- Transfer stringdata into numeric array
PRINT "Decoding to bytes"
DIM Bin(nd)
TIME RESET
FOR i=1 TO nd
  Bin(i)=DEC(MID$(Line$,2*i-1,2))
  IF i%np=0 THEN PRINT CHR$(9632);'show progress
NEXT i
Td=TIME() 'decode time
IF nd>np THEN PRINT 'terminate progress
'
'--- Write to binary file
IF BinFile$="" THEN 
  PRINT "*** ERROR *** BinFile$ not set"
  STOP
ENDIF
IF FILE_EXISTS(BinFile$) THEN FILE BinFile$ DELETE
FILE BinFile$ WRITEDIM Bin
Tw=TIME()-Td ' write time
PRINT
PRINT "#":nd;"bytes written to """&BinFile$&"""."
'
'---- write run-times
PRINT "Run-time:"
F$="#.##"
PRINT "  Read: ";f$:Tr;"seconds"
PRINT "  Decode: ";f$:Td;"seconds"
PRINT "  Write: ";f$:Tw;"seconds"
'
'--- Test binary file
IF Select<4 THEN
  TIME RESET
  PRINT "Test on binary file """&BinFile$&"""."
  MUSIC 1 LOAD BinFile$
  MUSIC 1 PLAY
  DO ! UNTIL NOT MUSIC_PLAYING("1")
  FILE BinFile$ DELETE
  PRINT "Time played: ";f$:TIME();"seconds."
ENDIF
PRINT "Done"
END
The requested file "alarm.wavHEX" for Select=1 is attached as alarm.wavHEX.txt
Remove the extension ".txt" before start of the program.
alarm.wavHEX.txt
(33.26 KiB) Downloaded 388 times
For the longer test with Select=2, the file "hardy.mp3HEX" is required and is attached. Remove the ".txt"-extension before use.
hardy.mp3HEX.txt
(5.87 MiB) Downloaded 423 times
In case the file contains carriage -return characters the program will abort.
That is tested with Select=3 with the file "alarm.wavHEXdump" which is generated with hexdump without format parameters in OSX
The HEX-file is attached, remove ".txt" before use.
alarm.wavHEXdump.txt
(82.16 KiB) Downloaded 406 times
The complete HEX2BIN-folder can be downloaded from https://www.dropbox.com/sh/z9j037m0trn1 ... AfEFa?dl=0
The DOS-program "sfk.exe" is also added to the Dropbox-folder and attached here:
sfk.exe.txt
(1.78 MiB) Downloaded 370 times
Remove the extension ".txt" before use.

Re: HEX2BIN for pure HEX-files

Posted: Thu Sep 07, 2017 1:53 pm
by Dutchman
I have added a selection facility for test-files and user-files at the start of the program.
I also added a test on a large file, a song of Françoise Hardy as mp3HEX-file of about 6 megabytes.
That file is decoded and played if Select=2.
The HEX-file is attached in the initial post of this topic. Remove the ".txt" extension before use.

Furthermore the screen-info is extended with the expected decode time and it displays run-times for the different phases.
During the decode of large files the progress is indicated by printing dots.
The Dropbox-folder has been updated.
The following screenshot is of decoding the mp3HEX-file
hardy's screenshot.PNG
hardy's screenshot.PNG (127.68 KiB) Viewed 4892 times

Re: HEX2BIN for pure HEX-files

Posted: Thu Sep 07, 2017 7:23 pm
by Dutchman
The code is adapted for usage with hex-files containing carriage-return characters.
Also info is added about BIN to HEX generation of files under windows.
A third test has been added for files with carriage-return characters.
The procedure description file has been extended and renamed to "BIN to HEX procedure" because now it relates to OSX and DOS.
It remembers me to the (non-good) old times of DOS 3.8 :evil:

Re: HEX2BIN for pure HEX-files

Posted: Thu Sep 07, 2017 7:44 pm
by rbytes
Working perfectly for me now. Thanks again. I suspected I might be able to go back to your original code and use the line-by-line extraction, but I'm glad I didn't have to. ;) You are very good at this file manipulation business, and I am not in your league. :lol:

There are a couple of small typos in the User Options section. Binfile$ is spelled Binfiel$. I fixed in my copy.

Next I will encoding and extracting my 2.2 MB .mp3.

Re: HEX2BIN for pure HEX-files

Posted: Thu Sep 07, 2017 8:20 pm
by Dutchman
Thanks for the correction.
There was also an error in the timing due to the introduction of the "expected time" section. :(
The code is corrected now.

Re: HEX2BIN for pure HEX-files

Posted: Thu Sep 07, 2017 11:50 pm
by Mr. Kibernetik
Following your discussion, I made BIN/TXT and TXT/BIN converters for Windows using my new SPL language: viewtopic.php?f=75&t=1943#p11783
These programs can be used on any Windows 10 computer to create BIN and TXT files, and they encode/decode files according to Dutchman's HEX standard. Hope it will be useful.

Re: HEX2BIN for pure HEX-files

Posted: Thu Sep 07, 2017 11:58 pm
by rbytes
Yes, that will be very useful. Working in a DOS window is less fun than it used to be. :)

Re: HEX2BIN for pure HEX-files

Posted: Fri Sep 08, 2017 1:06 am
by rbytes
Will you be removing Dropbox then as Apple requires?

Re: HEX2BIN for pure HEX-files

Posted: Fri Sep 08, 2017 1:08 am
by Mr. Kibernetik
rbytes wrote:
Fri Sep 08, 2017 1:06 am
Will you be removing Dropbox then as Apple requires?
Removing Dropbox will make sB practically useless. So I will not do it until sB update is absolutely necessary.

Re: HEX2BIN for pure HEX-files

Posted: Fri Sep 08, 2017 1:25 am
by rbytes
Ton, I need a refresher on how to copy my 4.5 megs of hex text. I thought I could do it in Dropbox, since it will display very large text files. But then it crashes when I select all of the text and try to copy it. That happens even with no other apps open, so is likely a memory issue.

I'm wondering how and where I can copy the text of the hex file so I can create a new file in Smart Basic and paste it in.

Up until now I have just been copying the hex file from my PC tablet to Dropbox and then to Smart Basic on my iPad, but that won't work if Dropbox is removed from Smart Basic.

I will try pasting the text into an email.