Domo Jupyter Notebook Web scraper

ajkircher01

Hello

I am interested in building a python web scrapper using Selenium using DOMOs online Jupyter notebook. The Domo I don't want to use the Python SDK. The reason is because I need this to scrape daily and if I do this through Python SDK I will need my computer to be on to do this.

I only article I could find on this topic was the one below which is on connectors.

https://community-forums.domo.com/main/discussion/41232/website-scraping

The reason I need Selenium is because I need it to log in and go to a certain page to get a table. The challenge with Selenium is it requires a web driver to be installed for it to work.

Is any help would be greatly appreciated.

GrantSmith

You'd need to reach out to Domo Support directly as this would require them to do a custom environment installation for you if they even allow it.

Have you looked into other packages which wouldn't require a driver like beautiful soup?

ajkircher01

I need to log into a website to get where I need to go. I was under the impression that beautiful soup struggled with this. I found this while searching https://stackoverflow.com/questions/23102833/how-to-scrape-a-website-which-requires-login-using-python-and-beautifulsoup
I will report back if any of this is useful.