Find Jobs
Hire Freelancers

I need a web crawler

$750-1500 USD

Closed
Posted over 4 years ago

$750-1500 USD

Paid on delivery
I need a webcrawler that can find websites that are using or have Flash elements on their website. Furthermore it needs to be able to differentiate the language of the website. I am ONLY interested in finding German and English language websites! The project literally requires that I find ONLY websites in German and English language. That could be a whole number of different domain endings such as .de .at .ch for German language and .com .[login to view URL] .[login to view URL] [login to view URL] and so forth for English. But I definitely don't want websites that are in any other language (french, spanish, italian, polish, russian, japanese, chinese etc). The crawler needs to generate a list of domains and their e-mail addresses for THAT language. The crawler needs to have a GUI where the above parameters most be adjustable. It also must have the possibility to check against a blacklist, so if he hits a known domain or email, it gets skipped from the beginning. There is a second stage where the crawler is supposed to find all websites that do not have an SSL certificate. And a third stage to find those websites that use cookies, but do not have a cookie banner. If the crawler can do all 3 of the above things, that much the better. But the first stage is to find websites with Flash elements. I need a minimum of 2 million addresses per language I am looking forward to your offers!
Project ID: 20352419

About the project

42 proposals
Remote project
Active 5 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
42 freelancers are bidding on average $1,294 USD for this job
User Avatar
Hi, I can design and develop a web crawler for you. I am familiar with related libraries and frameworks such as Tensorflow, Keras, Theano, Torch Scipy, Scikit-learn. and I am skilled in Python, R, Lua, Go etc. So this kind project is very familiar to me and I am feeling a strong confidence in this area. Looking forward to your response. Regards, FA
$3,000 USD in 35 days
4.7 (209 reviews)
9.5
9.5
User Avatar
I have checked your requirements where you mentioned that you need a webcrawler that can find websites that are using or have Flash elements on their website. Please invite me in chat so we can discuss in detail.
$1,500 USD in 20 days
4.9 (188 reviews)
8.9
8.9
User Avatar
Hello, Dear How are you? I have checked your project description and am ready for discussing with you about project as Web scraping developer. I have experienced in PHP , Python, Web Scraping. I will work very hard and best for you. Best Regards
$1,125 USD in 20 days
5.0 (107 reviews)
8.2
8.2
User Avatar
Hello, I read your description and got what to do. I am looking forward to discussing more for your project. I am ready now, please message me to start your project. I have 7+ years of experience in this field I am proud of 100% 5 stars and 100% completion rate. https://www.freelancer.com/u/mingah429.html From YueXi
$1,125 USD in 7 days
5.0 (21 reviews)
8.0
8.0
User Avatar
With Freelancer Preferred badge bound to give 100% Quality (Let's Chat) Hello, A great team is here for your service!! I read the project description properly, and I agreed to fulfill 100% of all your requirements. I can start your important project right away. I am 24x7 available on chat, so i can serve you whenever you want. Please check the PMB. Let's Lets start! To achieve goal with 100% quality we have team of 27 Professional Software Engg. Reply us so that we can move ahead to get this software successfully run. Regards Dsv Infosolutions Pvt. Ltd.
$1,125 USD in 7 days
4.6 (210 reviews)
8.3
8.3
User Avatar
Hello, I love to create Web crawler that can find out German and English language websites! using flash on their websites. I see web crawler should generate a list of domains and their e-mail addresses for THAT language. I do have few questions to discuss, here are few related projects : 1. Do you focus on website related to specific domain? 2. Can you share UI of screen where parameter is set for crawler >>>> Portfolio We are mainly focusing on creating Web applications, REST services, client-server apps, data-oriented apps with Python, Django framework. Here are few projects done by us : 1. Aadhaar Number Masking ( Mask the Last 8 digits of Aadhaar number ) 2. Incredible India Chatbot 3. Document Processing (Provided a Document we have to extract certain fields as required.) 4. Invoice Processing (Provide an invoice we have to extract Total,Company Name,Date,etc .) 5. Card Detection (On a document we have to detect the number of cards(Aadhaar Card,Voter card,etc)) >>> About Us We are a group of companies (Techelogy LLC)registered at Charlotte UNITED STATES with ODC(Offshore Delivery Center) in India. We have professionals with strong experience familiar with AGILE,SCRUM as well. Kindly open chatbox to further discuss project in detail Thanks Neeraj
$1,000 USD in 7 days
4.9 (428 reviews)
7.6
7.6
User Avatar
Greetings! Seeing is believing - please check my recent reviews and portfolios first. I am proficient in Web scrapping and Data mining, and I have already finished several smiliar projects before. I am confident in my develop skills and big or middle project are no problem for me. I just read your project description carefully, but still no enough for full understanding. Please ping me soon, then we can discuss your project in detail and jump to start without time wasting. I am always here to help you. Regards
$1,125 USD in 7 days
4.9 (38 reviews)
7.4
7.4
User Avatar
Hi. I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert and smart technic with web scraping (1000 over projects for scraping), automation and bot(500 over projects for automation and bot), selenium, mobile (android, ios native and others). Especially, I am an expert in betting project and gambling games and sites(oddsportal,bet365, bwin, pinnacle, betfair...) ,,, shopping auto buyer(adidas, footlocker, eastbay, supreme and etc) So, I can solve the captcha automatically. Please contact me, then we can discuss about the details and the price. And I am free in recent days, so I can do my best for your project. Thanks.
$1,125 USD in 7 days
4.9 (116 reviews)
7.1
7.1
User Avatar
Hello I am a talented Web developer. To detect Flash elements, I can scrape Web pages from each website using selenium. In this way, we can also detect languages. The second and third stage is a bit difficult but I can do it. Hope to chat with you in details. King Regards.
$1,000 USD in 7 days
4.9 (31 reviews)
7.1
7.1
User Avatar
Hi, I'm experienced web scraping developer with 5 years of hands-on experience. I have created several scraping projects and even developed automatic scraping bots. I'm good at python programming language and used scrapy, selenium, chrome driver, Beautiful Soap, lxml, greasemonkey and so on. Let's have an interview. Regards, ChunYan.
$1,500 USD in 20 days
4.8 (39 reviews)
7.0
7.0
User Avatar
Hello, Greeting for the day! I recently came across your job post and feel that I can assist you perfectly. I am a senior web developer with an experience of 8+ years, working proficiently on various domains which includes PHP, PHP based MVC frameworks ( CI, Laravel, CakePHP)and its major CMS(WordPress, Joomla, Magento). In last 8 years I have successfully executed 350+ remarkable sites and delivered top notch results with 5-star ratings. I have a strong hold on frontend technologies and scripting like HTML5, CSS3, Javascript, jQuery, AngularJS, NodeJS, Ajax, Bootstrap, responsive designing, Photoshop, Graphic designing, parallax designing. I always follow W3C standards, SEO and responsiveness as a general practice. Anticipate your response and hope that I will get a chance to discuss the project further. Thanks and Looking Forward Saurabh Mehta
$900 USD in 7 days
4.8 (48 reviews)
6.7
6.7
User Avatar
hello,dear. I have read all your requirements for 'I need a web crawler' and I fully understood it. I am confident and I am sure that I am able to finish this project. Please come in contact with me, so that we can discuss any details via chat:) Skills: PHP, Python, Web Scraping
$1,250 USD in 7 days
4.9 (18 reviews)
6.4
6.4
User Avatar
Hi there, I’ve checked the project details and I can do this project perfectly. I have huge experience in working with these projects. If you need sample work just ping me. Waiting for your quick reply. Look forward to hearing from you soon. Best Regards
$1,500 USD in 30 days
4.9 (107 reviews)
6.6
6.6
User Avatar
Highly experienced web scraper developer. My web scrapers scrape more than 10 Million pages each day without getting blocked. Please check my reviews here to know more about the quality you can expect : https://www.freelancer.com/u/sayno2bugs Lets start off ASAP and I assure we both have a smile at the end of the work. Nitin
$1,500 USD in 10 days
5.0 (31 reviews)
6.4
6.4
User Avatar
Hi there! My name is Orestes and I am a Software Engineer with specialty in Java/Spring. I represent a team of engineers (so our 'language' stack is actually quite big: Java, PHP, C#, Javascript, Python among others) that have undertaken too many projects in the past (including Web / Desktop Applications, Web Services, RESTful APIs, Game Development etc) so I can guarantee that you (and your project) are in safe hands with us! :) I would like to state that even though your description has given us a basic idea for the project, I would like to discuss with in order to clarify the details. That is why the price of my bid is yet to be discussed and we can decide on a fair price for you and for us! Feel free to get in touch! Orestes
$1,854 USD in 20 days
4.8 (11 reviews)
6.6
6.6
User Avatar
Hi, I am very interested to work on your project. I have 17+ years experience in Web Development using a large variety of programming languages, frameworks, database architecture, APIs, CRMs, ERPs, mobile development, web design, development tools. There are more then 1000 projects(small/medium/high complexity) successfully implemented and few of them related with yours can be seen on my freelancer.com portfolio. * * Note that my bid is relative and will be adjusted after detailed estimation of implementation effort/coast ** Let's start our collaboration by sending me all details in a written document which will be used as reference during project implementation Thank you! AdiC
$1,500 USD in 27 days
5.0 (60 reviews)
5.8
5.8
User Avatar
Hello! I am glad to place a bid on your project. I am good at Python and its scraping modules including selenium, scrappy, and so on. I have read your description and I can do your project perfectly. I can do your project in time and I can provide you full service. Looking forward to your reply. Thanks. Regards.
$1,125 USD in 7 days
5.0 (36 reviews)
5.4
5.4
User Avatar
I can help you to make solid big steps in your project. I'm +15 years experience in consulting and digital services with MBA and Six Sigma Black Belt. I've made hundreds of professional presentations, logos and brand identities. I can boost your sales by just choosing the right elements to be included in your identity. I have the experience. I've made numerous marketing strategies and data statistical analysis projects as well. I'm also full stack web developer who can build you a complete web application, from idea to web without any issue. please contact me to check sample of my previous work and you will be delighted.
$750 USD in 7 days
5.0 (14 reviews)
5.5
5.5
User Avatar
Hi,dear. I am a senior software developer. I am familiar with web scraping. I have just checked your project description, I am able to complete this project. I am looking forward to your response. Thanks.
$1,250 USD in 15 days
5.0 (4 reviews)
4.8
4.8
User Avatar
We cannot grab algorithm of other app's. It is not technically right. We can scrape data of others apps It is possible through python webservice or selenium web driver. We cannot grab the logic or backend of any app but we can crawl the data as per your requirements and will give you in GUI format as per mention in your job requirements. Recently I have scraped Amazon product by using amazon scrapping tool where 10K products are scrapped in .CSV file within 5 minutes. I have proof of it.
$1,125 USD in 5 days
4.9 (15 reviews)
5.0
5.0

About the client

Flag of SOUTH AFRICA
Kensington, South Africa
4.0
1
Member since Sep 12, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.