Find Jobs
Hire Freelancers

Convert MySQL and epoll based web crawler to use asynchronous MySQL calls to speed up web crawler. -- 2

£20-250 GBP

Closed
Posted almost 4 years ago

£20-250 GBP

Paid on delivery
See my question on stackoverflow for more details about what this requires: [login to view URL] I have a working epoll based web crawling architecture which consumes over 3 Gbps of bandwidth on my dedicated server. When I add MySQL code to keep the state of the frontier and to save crawled information the web crawler slows down. Feedback to my question suggests I need to use asynchronous calls to MySQL so that the web crawler can continue to create new connections while waiting for query results. I need someone with experience of asynchronous MySQL programming in C to fix this for me.
Project ID: 25829787

About the project

3 proposals
Remote project
Active 4 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
3 freelancers are bidding on average £217 GBP for this job
User Avatar
I am very proficient in c and c++. I have 18 years c++ developing experience now, and have worked for more than 10 years. My work is online game developing, and mainly focus on server side, using c++ under Linux environment. I made many great projects using c++, for example, I made the tools which could convert java codes into c++ scripts, of course garbage collection included, this was very similar to a compiler, and was very complex. I also made our own mobile game using c++, I can show you the demo of client, if you like. I am very proficient in java also. I have a very good review on Freelancer.com, I never miss a project once I accept the job, you can check my review. Trust me, please let expert help you.
£150 GBP in 2 days
4.9 (151 reviews)
7.1
7.1
User Avatar
This problem is not easy. My solution 1: MySQL uses connection pooling to improve efficiency. (The number of connections is still limited, if you wait for processing, then it will affect the speed of the crawler). 2: When mysql is busy, the state is cached to disk. 3: mysql is idle to synchronize data 3: Of course, the above operation requires multiple threads.
£300 GBP in 7 days
5.0 (3 reviews)
4.0
4.0
User Avatar
I CAN COMPLETE YOUR PROJECT EASILY Hello, I am programmer with enough experiences on C/C++ for 13 years. C/C++ is nice programming language and I am majoring on it for long years so I have confident to complete your project by C/C++. I can work by full time for your project and trying to satisfy the requirements. I am designing the project from scratch and structure it well. I am following the project's design process exactly, and if any error is found, f ix it within 24 hours and provide maintenance free of charge for a month. Client's idea + Programmer's good coding is the perfect result, this is my idea. I want to keep long-term relationship for customer so I want to give the nice result. If you want the perfect result, please contact with me.
£200 GBP in 7 days
3.1 (5 reviews)
4.5
4.5

About the client

Flag of UNITED KINGDOM
London, United Kingdom
5.0
1
Payment method verified
Member since Aug 13, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.