Website Scraping
Hi Everyone
Has anyone ever used Domo and its connectors to web scrape a website? Jut wondering if any of the connectors have the ability
Thanks
LB
Comments
-
I haven't experimented to see if a connector exists that does that but it sounds like something that's better suited for scripting out in a programming language and sending the results to Domo via the Domo SDK.
Example: in Python use Selenium or Beautiful Soup to scrape the contents of a site and then format it as a CSV to upload to Domo.
Maybe other people have different use cases that might answer it better but I'm not sure any connector will do that. Do you have an example of a site you would be scraping?
**Make sure to like any users posts that helped you and accept the ones who solved your issue.**0 -
Hi
Thanks for the response. Agreed Python is the way or Sceaming Frog, but a connector would be easier
THx
0 -
I too am currently working on a project involving creating a card with info from scraped websites. I do not believe Domo has the capability to do this since it requires some scripting, but I do have some good resources/code I've been working on for basic scraping if you run into any issues!
0 -
I'm looking into this, did anyone come up with a solution?
0 -
I'd recommend using a scripting language like Python (with the beautiful soup package). You could possibly utilize an ETL with some regular expressions but there's much more flexibility and power with a scripting language than you would have with Domo.
**Was this post helpful? Click Agree or Like below**
**Did this solve your problem? Accept it as a solution!**0
Categories
- All Categories
- 1.7K Product Ideas
- 1.7K Ideas Exchange
- 1.5K Connect
- 1.2K Connectors
- 295 Workbench
- 6 Cloud Amplifier
- 8 Federated
- 2.8K Transform
- 98 SQL DataFlows
- 608 Datasets
- 2.1K Magic ETL
- 3.8K Visualize
- 2.5K Charting
- 712 Beast Mode
- 50 App Studio
- 39 Variables
- 668 Automate
- 170 Apps
- 446 APIs & Domo Developer
- 45 Workflows
- 7 DomoAI
- 34 Predict
- 14 Jupyter Workspaces
- 20 R & Python Tiles
- 391 Distribute
- 111 Domo Everywhere
- 274 Scheduled Reports
- 6 Software Integrations
- 116 Manage
- 113 Governance & Security
- Domo Community Gallery
- 31 Product Releases
- 9 Domo University
- 5.3K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 104 Community Announcements
- 4.8K Archive