It is a mystery that the data survives JPEG compression.
I have made some tests and have to conclude that SB is to blame for stripping data from JPEG-files.
See viewtopic.php?f=28&p=13331
DATA_viaJPG for multiple files
- Dutchman
- Posts: 851
- Joined: Mon May 06, 2013 9:21 am
- My devices: iMac, iPad Air, iPhone
- Location: Netherlands
- Flag:
Re: DATA_viaJPG for multiple files
Last edited by Dutchman on Thu Jan 19, 2023 3:16 pm, edited 2 times in total.
- GeorgeMcGinn
- Posts: 435
- Joined: Sat Sep 10, 2016 6:37 am
- My devices: IPad Pro 10.5in
IMac
Linux i386
Windows 7 & 10 - Location: Venice, FL
- Flag:
- Contact:
Re: DATA_viaJPG for multiple files
Actually, to us photographers, it not a mystery at all.
JPG Files have two components: the image and a text area referred to us as EXIF Data.
The two areas are separated. The EXIF stores just about all the technical data such as exposure, camera used and serial number, GPS Of where the image was taken, lens used including manufacture, and other fields where you can add a copyright, caption, image description, and transmission fields used to send images to AP, Getty Images, Reuters, etc. You can also provide various tags such as subject, location, and sub-tags. The amount of data, which is stored after the actual image is impressive.
There are, if my recollection is correct, 3 user-defined fields that you can put as much data, or anything you want in it.
Without looking at SB's source code (and I need to relook at Rbytes code as it has been awhile) when you write anything to the image, it is after the image EOF indicator (unless you are writing an image editor), then there are special code to access the image directly, you are writing your data in the text part of the JPG file, and when compression is applied and reapplied, it stops at the images EOF tag.
This is why any text data and your encoded data survives.
Now there is a caveat when editing images that is separate from image compression. I've noticed this in Photoshop and quite frankly it is a pain in the ass, because in certain circumstances when you save images resized, edited, and recompressed, sometimes all that exposure info, captions, copyright, image title, GPS, etc, disappears. This occurs when you save the file to a format, like .GIF, that does not support this text data. It is simply dropped.
That's a non-technical explanation as to why your data remains, even if you recompress the image.
JPG Files have two components: the image and a text area referred to us as EXIF Data.
The two areas are separated. The EXIF stores just about all the technical data such as exposure, camera used and serial number, GPS Of where the image was taken, lens used including manufacture, and other fields where you can add a copyright, caption, image description, and transmission fields used to send images to AP, Getty Images, Reuters, etc. You can also provide various tags such as subject, location, and sub-tags. The amount of data, which is stored after the actual image is impressive.
There are, if my recollection is correct, 3 user-defined fields that you can put as much data, or anything you want in it.
Without looking at SB's source code (and I need to relook at Rbytes code as it has been awhile) when you write anything to the image, it is after the image EOF indicator (unless you are writing an image editor), then there are special code to access the image directly, you are writing your data in the text part of the JPG file, and when compression is applied and reapplied, it stops at the images EOF tag.
This is why any text data and your encoded data survives.
Now there is a caveat when editing images that is separate from image compression. I've noticed this in Photoshop and quite frankly it is a pain in the ass, because in certain circumstances when you save images resized, edited, and recompressed, sometimes all that exposure info, captions, copyright, image title, GPS, etc, disappears. This occurs when you save the file to a format, like .GIF, that does not support this text data. It is simply dropped.
That's a non-technical explanation as to why your data remains, even if you recompress the image.
George McGinn
Computer Scientist/Cosmologist/Writer/Photographer
Member: IEEE, IEEE Computer Society
IEEE Sensors Council & IoT Technical Community
American Association for the Advancement of Science (AAAS)
Computer Scientist/Cosmologist/Writer/Photographer
Member: IEEE, IEEE Computer Society
IEEE Sensors Council & IoT Technical Community
American Association for the Advancement of Science (AAAS)
- GeorgeMcGinn
- Posts: 435
- Joined: Sat Sep 10, 2016 6:37 am
- My devices: IPad Pro 10.5in
IMac
Linux i386
Windows 7 & 10 - Location: Venice, FL
- Flag:
- Contact:
Re: DATA_viaJPG for multiple files
I missed this post. Using a non-compressed or what we in photography call RAW formats (because the images do not lose data no matter how many times they are saved) like PNG or TIFF or PSD, are better choices to preserve your encoded data.
Mr. Kibernetik wrote: ↑Mon Jul 23, 2018 4:24 pmMost probably ALBUM IMPORT command recompresses JPG file instead of making a plain copy of the file from the gallery (it uses system import function). So, all extra markers will be wiped out.
I would suggest using non-compressed PNG file format (if it is possible to import PNGs) and encode information inside pixels. Each pixel can keep 4 bytes of information and even recompression will not destroy data.
George McGinn
Computer Scientist/Cosmologist/Writer/Photographer
Member: IEEE, IEEE Computer Society
IEEE Sensors Council & IoT Technical Community
American Association for the Advancement of Science (AAAS)
Computer Scientist/Cosmologist/Writer/Photographer
Member: IEEE, IEEE Computer Society
IEEE Sensors Council & IoT Technical Community
American Association for the Advancement of Science (AAAS)
- GeorgeMcGinn
- Posts: 435
- Joined: Sat Sep 10, 2016 6:37 am
- My devices: IPad Pro 10.5in
IMac
Linux i386
Windows 7 & 10 - Location: Venice, FL
- Flag:
- Contact:
Re: DATA_viaJPG for multiple files
I haven't done much coding using SB images, but I have noticed that images imported by SB or saved by SB are changed to 72DPI.
Mr. K, is this done on purpose, possibly to make the image size smaller and fit into the buffers?
For those not familiar with working with images, 144DPI contains twice as much photo data, and its file size is larger. That is why web images set at a DPI higher than 72 either takes too much time to load or doesn't load at all (webservers have a timer on image downloads).
On the dedicated servers I had on the Internet, I set the max download times for 12 or 24 seconds per image. Also, you can set the max time to load webpages and even how long CGI script can run.
If I had a regular user, any script running more than 30 to 45 seconds got terminated. For my university clients running scripts on large data, I would set their max to 999.
Mr. K, is this done on purpose, possibly to make the image size smaller and fit into the buffers?
For those not familiar with working with images, 144DPI contains twice as much photo data, and its file size is larger. That is why web images set at a DPI higher than 72 either takes too much time to load or doesn't load at all (webservers have a timer on image downloads).
On the dedicated servers I had on the Internet, I set the max download times for 12 or 24 seconds per image. Also, you can set the max time to load webpages and even how long CGI script can run.
If I had a regular user, any script running more than 30 to 45 seconds got terminated. For my university clients running scripts on large data, I would set their max to 999.
Dutchman wrote: ↑Mon Jul 23, 2018 2:04 pmI have created a variant in which the data is stored within the description of the image in COM (comment) segment(s). So the data is stored before the EOI marker. See the extension under 20180723 of my first post in this thread.
I assumed that this would solve the problem. But unfortunately it turned out differently. This data is also stripped. It even appears that the Exif data is also being changed. For example, the resolution is changed from 144dpi to 72dpi.
Initially I thought I could solve it by changing an adjustment in the Photo app: "TRANSFER TO MAC OR PC" changed from "Automatic" to "Keep Originals".
setting photo-transfer.PNG
But that proved to be no solution.
Eventually I found information on the internet that the app that delivers the photo to the library is responsible for whether or not the photo is stripped. Therefore, I can only conclude that the problem AND the solution must be found in SB.
If that is the case, then in a new version of SB that is adapted to it, data transfer should be possible in this way.
Then I wonder if Mr. K. wants and can investigate this.
George McGinn
Computer Scientist/Cosmologist/Writer/Photographer
Member: IEEE, IEEE Computer Society
IEEE Sensors Council & IoT Technical Community
American Association for the Advancement of Science (AAAS)
Computer Scientist/Cosmologist/Writer/Photographer
Member: IEEE, IEEE Computer Society
IEEE Sensors Council & IoT Technical Community
American Association for the Advancement of Science (AAAS)
-
- Posts: 2
- Joined: Wed May 22, 2024 1:51 pm
- My devices: The BASIC product provides essential features that are perfect for beginners. Make sure to review all the features included in this package.
- Location: India
- Flag:
Re: DATA_viaJPG for multiple files
To avoid data loss, use non-compressed formats like PNG or try using online jpeg compressor
tool to ensure minimal impact on your data. If data is missing, check if your software preserves metadata.
tool to ensure minimal impact on your data. If data is missing, check if your software preserves metadata.