Build nokogiri scraper to populate MySQL DB

Stagecoach Software Pty Ltd (trading as RateMyAgent) has fully functioning ruby Nokogiri scrapers populating a MySQL database from certain websites. The scrapers are based on a VPS and run on daily cron jobs. The scrapers generate thousands of rows of data per day in two key tables plus further rows in related tables. The database is designed to function as a backend to a forthcoming website.

This project requires a freelancer to adapt the existing scraper code to operate on a seventh website operating in the same industry and displaying the same type of data.

The existing code is available on github. A sample development database is available on Heroku. These may be inspected by approved bidders before they accept the project but development is to take place on another development database on the VPS. All code must be posted to and deployed through github. The new scarper must follow and use existing code as much as practicable.

Sundry other minor changes to the code and database may be required as the project progresses to help achieve the corporate objective of providing a useful and functioning website. The freelancer must ensure that the current production database and existing scrapers continue to operate without interference throughout the project. Fixing bugs identified within a week after production are to be included in the price.

Further written instructions including identification of the site to be scraped will be provided to approved bidders. Bidders are free to revise their bids upon studying the instructions, the code or the existing database.

This project follows freelancer Project ID’s 5477803, 5129765, 5066285, 4975478, 5799583, 6103569 and 6195613. Further similar scraping projects for other sites will be offered to the successful bidder if this project is completed successfully.

Bidders without demonstrable experience in Ruby, MySQL and Web Scraping will be rejected. Bids placed within minutes of the project being posted will also be rejected as indicative of no thought being applied to the project requirements.

Ruby

Project ID: #6246290

About the project