Find Jobs
Hire Freelancers

spark DataFrame Operations - Spark Session - 27/04/2018 01:01 EDT

$10-30 USD

Closed
Posted almost 6 years ago

$10-30 USD

Paid on delivery
Data Frame1: EmpNo EmpName Salary E123 Tom 2000 E124 RAM 2000 E125 TAM 2000 E126 SAM 2000 E124 RAM 4000 E126 SAM 6000 E125 TAM 9000 E123 Tom1 4000 Transform this DataFrame to DataFrame2: EmpNo EmpName Salary rownum E123 Tom 2000 1 E124 RAM 2000 1 E125 TAM 2000 1 E126 SAM 2000 1 E126 SAM1 5000 2 E124 RAM 4000 2 E126 SAM 6000 3 E125 TAM 9000 2 E123 Tom1 4000 2 Here is the summary: -- Duplicate EMpnos should be indexed(as shown in the rownum column) -- The order of index should be based on salary. -- Need All of the below approaches -- Should be optimized and should be runnable on a cluster. 1. Using RDD(a. using SparkContext, [login to view URL]) 2. Using DataFrame(a. using SparkContext, [login to view URL]) 3. Using DataSet(a. using SparkContext, [login to view URL]) DataFrame Operations should contain both . notation and sql notation. Action Items: 1) Development 2) Testing 3) Demo 4) Any corrections/small enhancements(if required)
Project ID: 16796241

About the project

12 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
12 freelancers are bidding on average $25 USD for this job
User Avatar
hello, senior data engineer, I have worked from more than 6 years on software development, I know well spark, scala etc, I believe I can resolve your problem.
$25 USD in 1 day
5.0 (2 reviews)
2.9
2.9
User Avatar
Hi I am interested in this job. I believe that the skills and experiences I have gained at this position make me an ideal candidate for this position. I have extensive experience in implementing Spark-core, Spark-sql and Spark-Streaming. As Developer, I have developed strong programming and debugging skills. I have completed dozens of PoC for my customers and received praise for the before time completion and that in Scala and Java. Beside Apache Spark, I have also gained extensive experience in Jenkins and Automated / Unit Testing. My experience has afforded me the opportunity to become familiar with the some of the best used architectures in the programming. In particular, through my work I have become heavily involved in writing the good applications and debugging the high severity issues. It has also helped me in designing the best application model. I am always eager to learn about new emerging technologies in the world of programming and Devops Integration. I would love to bring the passion for programming and sharing the good practices with your application. I am confident that my experience in the Big Data and my programming skills in Scala / Java / Python will qualify me for consideration. If you would like, I can discuss my experience with you. I look forward to meeting with you and discussing my qualifications in more detail. PS: I am certified Spark, Scala and Java Developer. Beside that I held many certificates in different domains from BigData University.
$25 USD in 1 day
4.8 (1 review)
1.5
1.5
User Avatar
Hi Dear, I'm available now for urgent working for you 40/hr for data entry work with 100% correction I will do Fast and accurate Data entry and Data processing form according to the requirement. The First priority is to provide the professional quality work for the project for your complete guaranteed satisfaction on the lowest budget. I Have expertise from 8 years in Data Entry, Data Processing, Data Analysis, and Survey, Excel Files filling, Internet Searching, Web Scraping, Google Searching, Email Hunting, Email Searching, Contact detail, Name, Email, Adress, etc. Expertise in Web Development, Social Media Marketing, Digital Marketing, Article & Content Writing. Thanks for you're time & Consideration. Please looking forward me. You Can take interview for further discussion & Price Negotiation. Please feel free to contact me anytime. Please must contact for your accurate Data processing work. Thanks & Best Regards, Zohaib Hussain
$10 USD in 1 day
0.0 (0 reviews)
0.0
0.0
User Avatar
Scala and spark expert. Have written numerous algorithms in spark/scala. Most recently wrote cure algorithm in scala/spark. The problem presented is interesting. Can complete this project with specs2 mocking. I can tune to run efficiently on yarn mode as well.
$35 USD in 2 days
0.0 (0 reviews)
0.0
0.0
User Avatar
A proposal has not yet been provided
$30 USD in 2 days
0.0 (0 reviews)
0.0
0.0
User Avatar
excel expert Relevant Skills and Experience BS in maths
$25 USD in 1 day
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
orlando, United States
5.0
1
Payment method verified
Member since Mar 7, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.