Control panel for run spiders with Scrapy - Python

Cancelled Posted Jun 27, 2015 Paid on delivery
Cancelled Paid on delivery

I'm need a control panel to manage the spiders: schedule, pause, stop, run, verify the scraped itens and more.

-Clean and simple interface;

-Run under a localhost machine (Windows);

-Developed with a simple language like own python or php.

This service is the best example, once is from the Scrapy framework developers: [login to view URL]

============

Basically the scraper is working this way:

1. The spider scrape the data and export to a json file inside a S3 bucket. Also the images are exported to the same bucket

2. An [login to view URL] script read this json's files and export to a MySQL database

3. A [login to view URL] script read all the products in the database table and group the similar products based on some rules

Also I'll need a resource to insert, schedule and run scripts like [login to view URL] and [login to view URL]

============

On the attached file, please discard the "Login" resource. Will not be necessary.

MySQL Python Web Scraping

Project ID: #7950338

About the project

4 proposals Remote project Active Jun 28, 2015

4 freelancers are bidding on average $520 for this job

mantislin

Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi

$250 USD in 5 days
(377 Reviews)
7.9
lufte

Hi. I specialize in web scraping with Scrapy. I can write an app on top of scrapyd (the Scrapy daemon) that fulfills your requirements. I would write the app on python but there are two options, let me know your prefer More

$400 USD in 15 days
(19 Reviews)
4.8
vxflores

i've been developing many web scrapers for freelancer mostly in pytthon and php and many other web related projects, so I'm aware of the project needs and the python platform in special. Given the requirements I can de More

$400 USD in 15 days
(12 Reviews)
3.8