AWS Data Engineer

Job Title: AWS Data Engineer

Location: Pune, Jaipur, Bengaluru, Hyderabad, Noida

Duration: Fulltime

Positions: Multiple

Responsibilities:

• Defines, designs, develops and test software components/applications using AWS - (Databricks on AWS,

AWS Glue, Amazon S3, AWS Lambda, Amazon Redshift, AWS Secrets Manager)

• Strong SQL skills with experience.

• Experience handling Structured and unstructured datasets.

• Experience in Data Modeling and Advanced SQL techniques.

• Experience implementing AWS Glue, Airflow, or any other data orchestration tool using latest technologies

and techniques.

• Good exposure in Application Development.

• The candidate should work independently with minimal supervision.

Must Have:

• Hands-on experience with distributed computing framework like Databricks, Spark-Ecosystem (Spark Core,

PySpark, Spark Streaming, SparkSQL)

• Willing to work with product teams to best optimize product features/functions.

• Experience on Batch workloads and real-time streaming with high volume data frequency

• Performance optimization on Spark workloads

• Environment setup, user management, Authentication and cluster management on Databricks

• Professional curiosity and the ability to enable yourself in new technologies and tasks.

• Good understanding of SQL and a good grasp of relational and analytical database management theory and

practice.

Good To Have:

• Hands-on experience with distributed computing framework like Databricks.

• Experience with Databricks migration from On-premise to Cloud OR Cloud to Cloud

• Migration of ETL workloads from Apache Spark implementations to Databricks

• Experience on Databricks ML will be a plus

• Migration from Spark 2.0 to Spark 3.5

Key Skills:

• Python, SQL and PySpark

• Big Data Ecosystem (Hadoop, Hive, Sqoop, HDFS, HBase)

• Spark Ecosystem (Spark Core, Spark Streaming, Spark SQL) / Databricks

• AWS (AWS Glue, Databricks on AWS, Lambda, Amazon Redshift, Amazon S3, AWS Secrets Manager)

• Data Modelling, ETL Methodology