Dynasty League Football Web Scraper

This collection of web scrapers collects fantasy football rankings and player stats from dynastyleaguefootball.com. The scrapers can be found here: https://github.com/mdane117/dynasty_league_football/tree/main

First, the ranking data gets collected in order to get a list of all relevant players on the site. Note that this requires a login to gain access. Otherwise, the scraper will only collect the first 10 records. It uses Selenium to open the site and enter the login details. Once logged in, the web scraper will collect the Quarterback, Running Back, Wide Receiver, and Tight End rankings. I then use the file_aggregator.py to aggregate the rankings.

The aggregated rankings gives a complete list of all the relevant players, including the player IDs and their unique URLs. There are two types of scrapers, one that scrapes the season stats and another scrapes the individual game stats. Within those two types, there is a scraper for the QB and another for the other three positions. This is because the QB will have different stats compared to the other positions (e.g. passing attempts vs receptions, passing yards vs receiving yards).

Note that the DLF website only needs the login details for the rankings, but once the rankings and the unique player URLs are collected, the scraper does not need a log in to access players’ individual stat pages.