Spark SQL Databricks Expert Needed
₹600-1500 INR
Paid on delivery
I'm currently seeking a skilled freelancer proficient in Databricks Spark SQL to assist with a specific project focused on data transformation and manipulation within my organization. This project will require a detailed understanding and hands-on experience with Spark SQL, particularly in the context of Databricks, to achieve the goals set out for improving and optimizing our data processing and analysis capabilities.
**Core Tasks:**
- Implement data transformation techniques to refine and restructure datasets.
- Perform data manipulation tasks to update multiple records in Delta tables based on sophisticated business logic, ensuring data integrity and relevance.
**Specific Requirements:**
- **Update Strategy:** The key task involves setting different values for varied subsets of records in a Delta table. This task is crucial for tailoring our dataset in alignment with dynamic business requirements and insights derived from data analysis.
- **Technical Proficiencies:** Proficiency in Databricks, Spark SQL, and an understanding of Delta tables are essential. Familiarity with data transformation and manipulation techniques including but not limited to data filtering, aggregation, and record updates based on specific conditions will be indispensable.
- **Data Insight:** Ability to interpret requirements and translate them into efficient Spark SQL queries for precise data updates.
**Ideal Skills and Experience:**
- Demonstrated experience with Databricks and Spark SQL.
- Strong background in data engineering or similar field, with specific experience in data transformation and manipulation.
- Familiarity with handling and updating Delta tables.
- Excellent problem-solving skills and the ability to work independently to implement data transformations and updates based on complex business logic.
This project offers an opportunity to engage deeply with Spark SQL within Databricks, making significant contributions towards the optimization of our data handling capabilities. If you have the skills and experience outlined above, I look forward to your proposal and potentially working together to achieve our data optimization goals.
Project ID: #37773808
About the project
7 freelancers are bidding on average ₹1114 for this job
Hello, I am writing to express my strong interest in Databricks Spark SQL developer position. The key responsibilities align perfectly with my expertise and experience in Azure data technologies and data engineering. More
I have been working as Azure Data Engineer and Databricks Engineer from 4.6 years. I have taught Databricks to more than 20000 students via YouTube and Online classes
Greetings I am a professional Data Engineer, with significant experience and expertise in the domain, including DataBricks and PySpark. I have worked with a few of the world's leading organizations as a Data Engineer More
Hi, I have reviewed the project details and understood your requirement of a Databricks Pyspark Developer who can to assist you with the project focused on data transformation and manipulation. I am capable of completi More
Having proficiency in sql tuning , sql optimization, manipulation and data transformation. Ability to perform work well before the deadline
I have very good experience in SQL server database and development for web application. Please connect for more details and discuss
