Web scraping and email

[MarkP]

I'd like to use sB to load a web page end extract certain information displayed in HTML on the web page, a process known as "web scraping".

I can't use the sB BROWSER function to do this, because that just displays the page - I need to extract info from it.

Then, I need to programmatically send an email, and using the information I "scraped", include an m4a audio clip (already in my local sB directory).

Any suggestions on how to do this using sB? I could do it with C++, but I really want to use my favorite language, Smart Basic!

[rbytes]

Most of what you want to do can be accomplished in SB.

1. Web scraping: The SB manual section named NET is where you can find HTTP commands for downloading data from web pages. I wrote a couple of programs that could send queries to various weather data services, receive weather information in JSON format over the web, then parse it to extract the weather data or forecasts for my choice of cities.

This link is to WeatherScope. https://kibernetik.pro/forum/viewtopic. ... ope#p11483

2. Sending HTML text to a mail message. Some time back, I posted a program called SB Notes. Here is the link: https://kibernetik.pro/forum/viewtopic. ... tes#p13108

By pressing the MAIL button, you can send the contents of the browser to Mail. I now use JavaScript, so the device's default mail browser opens immediately with all of the content you choose.

3. Attaching a file. You would think this would be easy given the above SB capabilities - but unfortunately the SB BROWSER command can not send attachments to The Mail app. That is more likely a limitation of the Mail app. SB does send links, but then you would need to keep your sound files on a server so that the links would download the sound files.

[Mr. Kibernetik]

You can use BROWSER_TEXT$ function from Interface section to extract information from a web page.

[Dutchman]

With the D&Dbrowser viewtopic.php?f=20&t=2029 you can simply download the complete page