Best scraping solutions on Reddit
16 reviews from r/learnpython, r/dataengineering
16 reviews from
and
By Brand
/By Product
#1
Scrapy
4.3
(4)
"I use Scrapy for my scraping needs."
"Scrapy is a powerful and flexible framework for large-scale web scraping projects."
"Scrapy is FAST. It has its quirks and unnecessary complexities tho."
"Scrapy is also a decent framework"
#2
Bright Data
4.3
(3)
"Bright Data’s Scraping Browser simplifies scraping even the most challenging sites."
"Take a look at Bright Data. They have a service called web unlocker."
"Beautiful soup is a staple but if you’re looking for stacked up solutions then probably something like Bright data."
#3
Playwright
4.0
(3)
"Playwright supports multiple browsers and is excellent for automated testing and scraping."
"Have a look into playwright if you need to access pages that have to load javascript."
"Playwright seems to work very well as a possible alternative."
#4
Proxycurl
5.0
(1)
"It’s a great tool for pulling detailed LinkedIn profiles, including work history, roles, and more, without the hassle of captchas."
#5
Parsel
4.0
(1)
"Parsel and either aiohttp or playwright"
#6
lxml
4.0
(1)
"I start with lxml with xpath queries; works most of the time and usually is the easiest and fastest solution."
#7
Splinter
4.0
(1)
"It is better with Splinter (Selenium wrapper) and Stere (Page object model wrapper for Splinter)."
#8
Selenium
4.0
(1)
"Selenium is essential for scraping sites with dynamic content or heavy JavaScript."
#9
Puppeteer
4.0
(1)
"Puppeteer works well with complex websites and is particularly useful for JavaScript-heavy sites."
Discover your audience
GummySearch is an audience research toolkit for 130,000 unique communities on Reddit.
If you are looking for startup problems to solve, want to validate your idea or find your first customers online, GummySearch is for you.
Sign up for free, get community insights in minutes.
Tell me more
Get started
