Data Engineer ( Scala Spark) - Coimbatore - Hybrid

Position Name: Data Engineer

Location: Coimbatore (Hybrid 3 days per week)

Work Shift Timing: 1.30 pm to 10.30 pm (IST)

Mandatory Skills: SCALA, Spark, Python, Data bricks

Good to have: Java & Hadoop

The Role:

• Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
• Constructing infrastructure for efficient ETL processes from various sources and storage systems.
• Leading the implementation of algorithms and prototypes to transform raw data into useful information.
• Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations.
• Creating innovative data validation methods and data analysis tools.
• Ensuring compliance with data governance and security policies.
• Interpreting data trends and patterns to establish operational alerts.
• Developing analytical tools, programs, and reporting mechanisms.
• Conducting complex data analysis and presenting results effectively.
• Preparing data for prescriptive and predictive modeling.
• Continuously exploring opportunities to enhance data quality and reliability.
• Applying strong programming and problem-solving skills to develop scalable solutions.

Requirements:

• Experience in the

  Big Data technologies (Hadoop, Spark, Nifi, Impala).

• Hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
• High proficiency in

  Scala/Java and Spark

  for applied large-scale data processing.
• Expertise with big data technologies, including

  Spark, Data Lake, and Hive.

• Solid understanding of batch and streaming data processing techniques.
• Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion.
• Expert-level ability to write complex, optimized SQL queries across extensive data volumes.
• Experience on

  HDFS, Nifi, Kafka.

• Experience on

  Apache Ozone, Delta Tables, Databricks, Axon(Kafka), Spring Batch, Oracle DB

• Familiarity with

  Agile

  methodologies.
• Obsession for service observability, instrumentation, monitoring, and alerting.
• Knowledge or experience in architectural best practices for building data lakes.

Interested candidates share your resume at Neesha1@damcogroup.com along with the below mentioned details :

Total Exp :

Relevant Exp in Scala & Spark :

Current CTC:

Expected CTC:

Notice period :

Current Location: