Find Jobs
Hire Freelancers

Need planning and advise for Huge dynamic Scraping Project

€8-30 EUR

Cancelled
Posted over 4 years ago

€8-30 EUR

Paid on delivery
This is yet just a pre-step to the main job. For now i need advises, planning and cost estimation of the project. 1. 600,000 pages need to be scraped, preferred on a daily basis from a free webpage (will be mentioned in further chat) 2. "banning" must be prevented, technical infrastructure must be stable and secured 3. i need a realization plan if this is technically possible and a REAL time and cost plan, including storage formats and servers too (mysql, excel etc) - need to know all technical specs of the project 4. i will only chat further on with persons that read this terms and put PASSWORD:SKYSCRAPER in head of bidding, information text in freelancer 5. this does not necessarily be the person that will take over the future main job but preferred, if wished i can share a "sample file" of wished file, will also tell page to be scraped, who wants to be a candidate for the main project can give overall budget plus dynamic costs (servers) etc but has to give also sample work later on 6. the main project will need not only BACK END skills but also FRONT END / UI Skills, preferred both
Project ID: 23098934

About the project

14 proposals
Remote project
Active 4 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hi, I will build a Java application to scrape data from target website. Based on your input, to scrape 600,000 pages we need: 1) Linux/UNIX VPS (15 EUR - 30 EUR per month approx for 30GB - 60GB RAM) 2) Proxy servers for rotating IP's (100 EUR per month approx) 3) Application design, development, testing, deployment, server setup, database setup etc costs 300 EUR approximately(price depends on the complexity) 4) 10 - 15 days to finish entire project. (num of days depends on the complexity of the website) Please contact me to discuss more about your project. Looking forward to hearing from you soon. Thank you
€30 EUR in 1 day
5.0 (12 reviews)
5.9
5.9
14 freelancers are bidding on average €58 EUR for this job
User Avatar
Hi there, I am scraping expert, I have did more than 500+ scraping project, please check my feedback then you will know. Can we discuss more details about this project? then I will provide example data/script for you. Thanks, Lin
€60 EUR in 2 days
5.0 (324 reviews)
7.7
7.7
User Avatar
PASSWORD:SKYSCRAPER Hi. Sure we can talk about it. I have 6+ years in web scrapping and I can advice you what are the best tool and practices to use. Send me for now websites you want to crawl and which fields data you want to get. Thx, waiting for these details and hope to collaborate.
€200 EUR in 2 days
5.0 (130 reviews)
7.6
7.6
User Avatar
"Hello Dear, Yes, I can work for you as per your project description with amazing exploration and experimental research abilities of mine organization is perfect for that you may require. My work fulfills as well as keep you returning consistently. It is because of the importance behind this field that I am confident & serious when taking these types of projects. Let me introduce with professionalism and skill that exists to supply the best. Regards, DataExpress."
€30 EUR in 2 days
5.0 (3 reviews)
3.6
3.6
User Avatar
PASSWORD:SKYSCRAPER Hello. I already work with high load scrappers and have wide glance what possible to do and what fails if you want gather big data ammounts. I ready ansver to your questions.
€34 EUR in 4 days
5.0 (5 reviews)
2.9
2.9
User Avatar
PASSWORD:SKYSCRAPER Dear Client, I am scraping expert, using python and its libraries to extract data. According to your job description if there is low security or bot detection on target site, then I can create a scraper which can extract 30000 query in 24 hours, and it will be deploy on VPS (which cost average $10), and also I can create GUI in python just for scraper and output into desired format. Hope, we can more discuss on chat. Thanks Shafqat Ali
€19 EUR in 7 days
5.0 (4 reviews)
2.2
2.2
User Avatar
PASSWORD:SKYSCRAPER ⭐ ⭐ ⭐ ⭐ ⭐ PROPER BUDGET && SHORT TIME && HIGH QUALITY ⭐ ⭐ ⭐ ⭐ ⭐ How are you. i've review your post carefully and interestly. I have a specialty in Web Scraping. You will be fascinated by my skills. i'd like to help and finish your project. So, I think that I can complete your project perfectly. Also, I can satisfy you with time and quality. And I will always be with your time. You won't regret to hire me. Please let's discuss more detail through via chat. Best regards
€100 EUR in 3 days
0.0 (0 reviews)
0.0
0.0
User Avatar
PASSWORD:SKYSCRAPER I've already faced all the possible issues and I'm confident to provide a robust architecture for the crawling operation. The largest crawling system I've built gathered 7 million companies from the largest web markets (ebay, alibaba, ...) I managed to tackle performance, cost and security issues and all this knowledge can be served for this project. This is my first freelancer.com bid, till now I was mostly working with companies but I recently switched to freelancing. Looking forward to your message.
€200 EUR in 1 day
0.0 (0 reviews)
0.0
0.0
User Avatar
i m doing python machine learning so i am glat if i get some work to do related to my profile Relevant Skills and Experience i have knowledge of ms office , python and many libraries in python i hava a good hand on excel and good communication skills
€23 EUR in 1 day
0.0 (0 reviews)
0.0
0.0
User Avatar
PASSWORD:SKYSCRAPER ways of avoiding banning : throttling requests, using proxies real time : 4 -5 max storage format : database (sqlite3 preferred), filestorage (possible, not preferred) cost plan : depends on the whether we use our own system for scraping or an instance from cloud service provider such as AWS. Text me to chat further.
€30 EUR in 7 days
0.0 (1 review)
0.0
0.0

About the client

Flag of GERMANY
frankfurt, Germany
5.0
6
Payment method verified
Member since Dec 31, 2014

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.