Hadoop Jobs

I am looking for a freelancer with experience in Spark Structured Streaming to help with my real-time data processing project. Purpose: The purpose of this project is to perform real-time data processing using Spark Structured Streaming. Data Source: The data for this project will be sourced from Kafka. Data Transformations and Output Operations: I have specific requirements for data transformations and output operations. Ideal Skills and Experience: - Strong knowledge and experience in Spark Structured Streaming - Proficiency in working with Kafka - Ability to perform complex data transformations - Experience in implementing various output operations - Strong problem-solving skills - Attention to detail and ability to meet deadlines If you have the necessary skills and ex...

I am seeking assistance with a research project focused on data warehouse implementation, specifically in the area of cloud-based data warehouses. Skills and experience required for this project include: - Strong knowledge of data warehousing concepts and principles - Experience with cloud-based data warehousing platforms, such as Amazon Redshift or Google BigQuery - Proficiency in data modeling and designing data warehouse schemas - Understanding of ETL (Extract, Transform, Load) processes and tools - Ability to analyze and integrate data from multiple sources - Familiarity with SQL and other programming languages for data manipulation and analysis The deliverable for this project is a comprehensive report that summarizes the research findings and provides recommendations for implemen...

I am in need of a freelancer who can help me with setting up AWS EMR Spark sessions. The project requirements are as follows: Spark Version: - The session setup should be done using Spark 3.0. AWS Region: - The EMR clusters are already set up and ready, so there is no specific AWS region requirement. Ideal Skills and Experience: - Strong experience in AWS EMR and Spark. - Knowledge of Spark 3.0 and its features. Please note that the EMR clusters are already set up, so the focus of this project is on the session setup using Spark 3.0.

HDFS Setup Configuration: 1 NameNode 3 DataNodes 1 SecondaryNameNode Requirements: Assuming your team has three students: Tom (999900012), Jerry (999900034), Mike (999900056) Configure the hostname of each DataNode: Hostname of DataNode 1: Tom Hostname of DataNode 2: Jerry Hostname of DataNode 3: Mike Set the last two digits of the IP address of each DataNode: IP address of DataNode 1: IP address of DataNode 2: IP address of DataNode 3: Submission Requirements: Submit the following screenshots: Use commands to create three directories on HDFS, named after the first name of each team member. Use commands to upload the Hadoop package to HDFS. Use commands to show the IP addresses of all DataNodes. Provide detailed information (ls -l) of the blocks on each DataNode. Provi...

Looking for a developer who is good in onboarding Enterprise Redis for the client . Should be available to work on remote machine . Should be very good with REDIS onboarding and sizing .

"We are seeking a Senior Data Engineer who possesses extensive experience and proficiency in a range of key technologies and tools. The ideal candidate should have a strong background in Python, demonstrating skillful use of this programming language in data engineering contexts. Proficiency in Apache Spark is essential, as we rely heavily on this powerful analytics engine for big data processing. Experience with PySpark, the Python API for Spark, is also crucial. In addition to these core skills, we require expertise in AWS cloud services, particularly AWS Glue and Amazon Kinesis. Experience with AWS Glue will be vital for ETL operations and data integration tasks, while familiarity with Amazon Kinesis is important for real-time data processing applications. Furthermore, the candida...

Job Title: Informatica Cloud MDM Architect/Senior Developer Location: HYD/Remote Duration: Full Time Required Skills: • At least 12+Years of experience in designing, developing, and implementing Informatica MDM solutions with at least one end to end project experience using Informatica Cloud. • Experience in architecting Informatica Master data management in a large enterprise integrating diverse ERP systems (such as Salesforce, SAP) and implementing an effective, efficient, and easy to maintain batch/real time/near real time integrations. • Strong experience in Informatica SaaS Multidomain MDM components and their interaction for solutioning – Cloud Data Quality (CDQ), Cloud Data Integration (CDI), Cloud Application Integration (CAI),...