Automated Web Scraping in PHP or Python saved in MySQL Database
$8-15 USD / hour
I need code that gets information scraped from many websites and updates a MySQL database as well as storing images.
I will supply a MySQL DB Diagram with design and fields needed to be scraped. It can be modified if needed.
It will be hosted on Amazon EC2 so I will have flexibility in code/plugin support.
I need the most efficient form of scraping to the MySQL DB, saving images, and compressing the images for the web. I do not want a full GUI webkit running. Any library for scraping/image compression that is not processor intensive.
Images will be hosted on Amazon S3. If that's a quick addon to save images there after compressing, that would be great!
Websites scraped are simple html to javascript/aspx. This will be an ongoing thing as more websites will needed to be scraped.
I need someone I can work to guide me in DB design and making sure it's efficient. Also if there are better options than PHP/Python/Amazon, suggestions would be great! Shouldn't take more than 5 minutes of chit chat. I will supply one website to scrape and my DB design and work from there!
It will be ran daily or multiple times a day. It will rescrape the site and skip/update existing entries and add new ones.
Project ID: #4630632
About the project
13 freelancers are bidding on average $14/hour for this job
Seasoned web scraper. I worked on many similar projects, I have big experience in data mining projects. I can finish this task in short time, with the best quality.
Hi, I have advanced web scrapping solutions. Please provide more details and lets start it!
Thanks for inviting us to bid on this project. We have a team of highly qualified and creative professionals. Give us a chance to show our talents and we assure you quality.
What type of information are you looking to scrap and how many websites I just would like an overview to see it in my head more clearly... Thanks