Data Engineer

Job Role : Data Engineer

Experience : 12 - 16 years

Notice Period : Immediate - 30 days

Job Location : Bengaluru or PanIndia

Primary Skills : Bigdata, Spark, Scala and AWS

Secondary Skills : Kubernetes, Kafka

5+ years of software development experience with deep understanding of algorithms, data structures, design patterns, data pipelines and data warehousing.

Algorithms, data structures, design patterns required forà

Data processing (Sorting, filtering and aggregating large datasets)

Optimization (reducing time and space complexity in ETL)

Search and stream data (implementing fast lookups, sliding window, sampling, etc.)

• Strong hands-on experience in one or more programming languages such as Java, scala, Python or similar languages

Programming languages hands-on experience is required to handle complex data transformations

• Strong hands-on experience in distributed processing using Spark and Hive.

In Big data environment, experience in large scale distributed infrastructures for

data processing is required (Hadoop, Spark, Hive, Kafka, Redis)

• Hands on experience with AWS Cloud Computing platforms and understanding of scalability and reliability issues.

In Modern Data solutions cloud experience is required and mandatory

• Good understanding of Kubernetes including its core components and architecture

For CI/CD support, Cloud Compatibility, Auto scaling, schedule and manage jobs related to ETL, streaming, etc..