Building Web Crawlers for Data Acquisition with Python Scrapy Video
- Be able to explain the functionality and use cases of Scrapy. Scrapy is an open source web crawling framework written in Python for extracting the data you need from websites. It is built on top of Twisted, an asynchronous networking framework.
- Learn about the UrlLib2 and Requests modules for reading and opening web pages. Beautiful Soup is used for extracting data points and Selenium is a tool for writing automated tests for web applications.
- Install Scrapy and build your first scrapy spider.
- Combine Xpath with Scrapy. Xpath is a handy tool for extracting html tags.
- Build a more advanced Scrapy spider. Be able to explain the Scrapy Architecture.
- Deploy and schedule a spider through ScrapingHub.
- Log in to websites using Scrapy.
Instructor: Advait Jayant
Length: 2 hours
Access period: For one year starting from purchase date