Find Jobs
Hire Freelancers

Serach Engine URL Scraper

$30-100 USD

Cancelled
Posted over 18 years ago

$30-100 USD

Paid on delivery
I need a tool that will scrape the urls from google search results pages, and write these urls to a text file. I am envisaging a php, server based solution, however I am open to other options if you belive they are more feasable. The starting page should have space to enter the search term and to specify how many results (or results pages) to collect. Any search terms can be entered into this box and submitted. These search terms determine which urls the script will collect from google. For example, if the following is entered into the text box: "Send an Email to the Forum Administrator Contact Us" The script should collect the url, complete with http:// from the page below, as well all subsequent pages until X number of urls have been collected. [login to view URL] When this process is completed, there should be a list of around 59,000 urls. Note the num=100 in the url gets google to display 100 results per page. Note how the url changes as subsequent pages of results are viewed. Google will place a temp block on a ip if too many requests are made in a certain time period. So there must be support for proxy servers built into the script. The urls should be written to a txt file with one per line. If you feel this may be a performace issue when the list gets big, I welcome suggestions e.g. 1000 per file, or even using mysql. Other Points ============= 1. Code should be commented. 2. Installation instructions should be given. Any questions let me know. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform Any. I prefer a php server based solution. But will consider other scripting languages.
Project ID: 3164359

About the project

9 proposals
Remote project
Active 18 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
9 freelancers are bidding on average $61 USD for this job
User Avatar
See private message.
$85 USD in 7 days
4.9 (105 reviews)
7.3
7.3
User Avatar
See private message.
$85 USD in 7 days
5.0 (226 reviews)
7.2
7.2
User Avatar
See private message.
$51 USD in 7 days
5.0 (69 reviews)
5.5
5.5
User Avatar
See private message.
$42.50 USD in 7 days
5.0 (32 reviews)
4.5
4.5
User Avatar
See private message.
$85 USD in 7 days
4.2 (19 reviews)
4.5
4.5
User Avatar
See private message.
$76.50 USD in 7 days
5.0 (4 reviews)
3.9
3.9
User Avatar
See private message.
$22.10 USD in 7 days
4.8 (9 reviews)
3.0
3.0
User Avatar
See private message.
$85 USD in 7 days
0.0 (1 review)
0.0
0.0
User Avatar
See private message.
$12.75 USD in 7 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of
5.0
2
Member since Dec 7, 2006

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.