Home
Jobs

Posted:3 days ago| Platform: Linkedin logo

Apply

Skills Required

data developer kafka pyspark python scala spark sql hadoop design processing hive extraction analysis scheduling automate integrity scalability gcp learning governance compliance

Work Mode

On-site

Job Type

Full Time

Job Description

We are looking for an immediate joiner and experienced Big Data Developer with a strong background in Kafka, PySpark, Python/Scala, Spark, SQL, and the Hadoop ecosystem. The ideal candidate should have over 5 years of experience and be ready to join immediately. This role requires hands-on expertise in big data technologies and the ability to design and implement robust data processing solutions. Responsibilities Design, develop, and maintain scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark. Work extensively with the Kafka and Hadoop ecosystem, including HDFS, Hive, and other related technologies. Write efficient SQL queries for data extraction, transformation, and analysis. Implement and manage Kafka streams for real-time data processing. Utilize scheduling tools to automate data workflows and processes. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Ensure data quality and integrity by implementing robust data validation processes. Optimize existing data processes for performance and scalability. Requirements Experience with GCP. Knowledge of data warehousing concepts and best practices. Familiarity with machine learning and data analysis tools. Understanding of data governance and compliance standards. This job was posted by Arun Kumar K from krtrimaIQ Cognitive Solutions. Show more Show less

More Jobs at krtrimaIQ Cognitive Solutions

Data Engineer

Bengaluru, Karnataka, India

5.0 - 5.0 yrs
|

Salary: Not disclosed

Data Engineer

India

4.0 - 4.0 yrs
|

INR 10 - 17 Lacs

Data Engineer

Bengaluru, Karnataka, India

5.0 - 5.0 yrs
|

Salary: Not disclosed

Data Engineer

Basavanagudi, Bengaluru, Karnataka

Experience: Not specified

|

Salary: Not disclosed

Data Engineer

Bengaluru, Karnataka, India

5.0 - 5.0 yrs
|

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Data Interview Now

My Connections krtrimaIQ Cognitive Solutions

Download Chrome Extension (See your connection in the krtrimaIQ Cognitive Solutions )

chrome image
Download Now
krtrimaIQ Cognitive Solutions

4 Jobs

RecommendedJobs for You

Vestas logo

Data Engineer

Vestas

Chennai

Mount Talent Consulting logo

Data Engineer

Mount Talent Consulting

Hyderabad, Pune, Bengaluru

Revature logo

Data Engineer

Revature

Chennai

HCLTech logo

Data Engineer

HCLTech

Hyderabad, Chennai, Bengaluru

SP Software logo

Data Engineer

SP Software

Bengaluru

Centific logo

Data Engineer

Centific

Hyderabad

Avigna.AI logo

Data Engineer

Avigna.AI

Pune

Mindsprint logo

Data Engineer

Mindsprint

Chennai, Bengaluru

Zentek Infosoft logo

Data Engineer

Zentek Infosoft

Hyderabad, Chennai, Bengaluru

Ideagen logo

Data Engineer

Ideagen

Hyderābād