Find Jobs
Hire Freelancers

Data collection program

$30-250 USD

In Progress
Posted over 15 years ago

$30-250 USD

Paid on delivery
I have a need to harvest data from a web site on a weekly basis and need a program to do the work. I currently do it with a program I wrote but I need something more robust in order to run this more often. The project is simple: download some html files and then extract the data in them and put them in a standard delimited ASCII file. The program will simply query a web site to get the html pages that are available. There are about 100 per calendar date and the need is to be able to download the files starting with the current date and going to an end date. This means downloading between 35,000 and 100,000 html files each time the program runs. The program should begin by getting the html files and storing them in a temporary folder. The second step of the process is to parse the html files and extract the data contained in them. The HTML files keep the same format and are easy to extract as they are simple lists. ALL fields must be extracted. The list have header information, so the header information must be repeated on each record of data created to ensure that the information stays together. For example, it will give the name and then a list of all the clients for that name. The extracted data must be saved in a standard ASCII file format where each field is delimited by a character to be configurable in the program (example a tab). I will then take this data and import it into a database system for processing. I do not need a program with a fancy user interface. It needs to be simple and functional. It must work on Windows XP Pro. Attached is a ZIP file containing samples of the html files as well as the web site information. The provider must submit a final program in executable format with all necessary files, and also the source code. He must have tested the program and must submit the results for one run between two dates, of a year time period (example: the provider can run the program on february 1st 2009 and put the end date february 1st 2010. The data collected must be submited to show the program works.). If the provider does a good job on this, there are several other similar projects available for him. In the future when the format of these html files changes, I will ask the provider to modify the program. This is a simple project but please only bid if you have done this type of work before and are sure you can deliver the work. I do not want to waste your time or mine. If you have questions please message me before you bid. Thank you
Project ID: 364777

About the project

40 proposals
Remote project
Active 15 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hi! Please check PMB for demo.
$30 USD in 1 day
5.0 (3 reviews)
2.5
2.5
40 freelancers are bidding on average $119 USD for this job
User Avatar
We can help in your project, please check PMB to see our related experience.
$250 USD in 4 days
4.8 (281 reviews)
8.2
8.2
User Avatar
I can do this job for you. See PM for details.
$80 USD in 2 days
5.0 (600 reviews)
7.8
7.8
User Avatar
Hi, More info is in the PM. Best Regards, Yousef
$245 USD in 3 days
5.0 (70 reviews)
7.2
7.2
User Avatar
i like this kind of job, will do it easily.
$100 USD in 1 day
5.0 (58 reviews)
7.0
7.0
User Avatar
$250 USD in 0 day
5.0 (85 reviews)
6.5
6.5
User Avatar
Hello, please refer your PMB. Thank you.
$200 USD in 5 days
4.6 (86 reviews)
7.0
7.0
User Avatar
I worked on many similar scraping projects before. I'm a professional scrapper working in C#, C++, php. I can finish and deliver the program in a fastest possible time.
$100 USD in 2 days
5.0 (34 reviews)
6.6
6.6
User Avatar
We are ready to do this project
$40 USD in 6 days
5.0 (65 reviews)
5.8
5.8
User Avatar
I've done a lot similar projects. I have special modules in Python. PyCurl+MultiThreads (with errors reprocessing).
$50 USD in 1 day
5.0 (14 reviews)
5.5
5.5
User Avatar
Hi, I am currently working on a scrapper project which is quite similar to this one. I will do this using C#. I am an expert with data processing and text extraction, that is my field of work. I would like to have a long term working relation with you and I'm sure I will deliver up to your needs. I'm open for discussion so hope to hear back from you soon. Regards, Ancosys
$100 USD in 8 days
4.9 (74 reviews)
5.7
5.7
User Avatar
Hello, Please Check PMB
$70 USD in 2 days
5.0 (20 reviews)
5.1
5.1
User Avatar
Very interested in your data collection project. Please check your PMB. Thanks.
$150 USD in 3 days
5.0 (20 reviews)
4.9
4.9
User Avatar
Please see PMB for details
$200 USD in 3 days
5.0 (19 reviews)
4.7
4.7
User Avatar
Hi there, I am a expert data extractor, I have been doing it for over 11 years. I have completed many tasks both on GAF and other sites including extracting info from websites and other places. Please see my reviews for previous references and let me know if you require anything further. Kind regards, Nash
$220 USD in 2 days
4.9 (14 reviews)
4.6
4.6
User Avatar
Please refer to PMB
$200 USD in 3 days
5.0 (4 reviews)
3.9
3.9
User Avatar
Please See PMB.
$100 USD in 7 days
5.0 (3 reviews)
2.6
2.6
User Avatar
I have a ready to go software for you which saves data in csv, text(ascii) and few other formats as well. It will help you to extensive extract data directly from the website without first extracting the html files. But in the case if it's compulsory for you to extract html files and then extract data indirectly from the html files, then this software has that capacity also. I have complete solution for you whether you want to extract directly or indirectly via html files. I am a security expert with world records in my field and lots of global achievements. I kindly request you to have a look at my GAF profile for further details. I provide exclusive and unique services which require excellent talent and expertise and which no one other then me can perform. I hope for a long term relationship with you for such extraction services. Kalpesh Sharma
$150 USD in 2 days
4.6 (2 reviews)
2.8
2.8
User Avatar
I can complete it for u... I am expert in AJAX, WSDL, Web Services & Clients, JSF, J2ME, J2SE, J2EE/EJB3, JavaScript, NetBeans IDE v6.0, JBOSS, TOMCAT, HTML, XSLT, XML, ORACLE, MS-ACCESS, MySQL, SQL Server, etc... I am also readily available for chat (10 hrs/day) in skype / gtalk / MSN / YAHOO Meesenger Available Days : Mon - Fri Available Time : 8:30 AM to 6:30 PM Singapore Time [GMT + 8] Winners never Quits Quitters never Wins
$126 USD in 2 days
4.7 (4 reviews)
1.9
1.9
User Avatar
we are ready to start.
$50 USD in 2 days
3.8 (2 reviews)
1.4
1.4
User Avatar
Please see PM
$55 USD in 2 days
0.0 (1 review)
0.0
0.0

About the client

Flag of CANADA
Montreal, Canada
5.0
72
Payment method verified
Member since Oct 23, 2008

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.