Build a scraping system with dashboard.

Develop a scraping system which will scrape all listings in a websites directory. This project will include creating 3 custom site scrapers and a dashboard to monitor and manage them. After complete we will create additional project to add more sites. - The initial build will include 3 sites to be scraped (we will add more in future project) - The dashboard must provide list of all sites we are scraping - The ability to start/pause scraping runs - Real time live status of each scrape (how many pages scraped total/per run - Time offset (how much time to wait between runs - Default wait 10 sec) - Listing checker (to check a url before site goes to it to make sure the script never scrapes or goes to the same page twice). - Should be able to run (and see live stats) on more then one site at the same time - Ability to add a proxy via dashboard - Ability to set scrapers user agent info - Ability to check to see if the listing was already scraped from one of the other scraper modules and update missing info: - Feature to scrape all web urls we found in the scrapes, open the site, Save a jpg of the homepage, navigate to contact us page and scrape and email addresses found, go to about us and scrape page text