AWS Databricks (data engineer)

Job Title

Location: Pune/ Mumbai/Bangalore/Chennai/Hyderabad

Experience: 5 to 12 years

Requirements

Candidate must be experienced working in projects involving:

• Primarily looking for a Data Engineer with expertise in processing data pipelines using Databricks, Spark, and SQL on Hadoop distributions like AWS EMR, Databricks, Cloudera, etc.
• Should be very proficient in performing large-scale data operations using Databricks and overall very comfortable using Python.
• Familiarity with AWS compute, storage, and IAM concepts.
• Experience working with S3 Data Lake as the storage tier.
• Any ETL background (Talend, AWS Glue, etc.) is a plus but not required.
• Cloud data warehouse experience (Snowflake, etc.) is a huge plus.
• Carefully evaluates alternative risks and solutions before taking action.
• Optimizes the use of all available resources.
• Develops solutions to meet business needs that reflect a clear understanding of the objectives, practices, and procedures of the corporation, department, and business unit.

Skills

• Hands-on experience with Databricks, Spark, SQL, AWS Cloud platform — especially S3, EMR, Databricks, Cloudera, etc.
• Experience with Shell scripting.
• Exceptionally strong analytical and problem-solving skills.
• Relevant experience with ETL methods and retrieving data from dimensional data models and data warehouses.
• Strong experience with relational databases and data access methods, especially SQL.
• Excellent collaboration and cross-functional leadership skills.
• Excellent communication skills, both written and verbal.
• Ability to manage multiple initiatives and priorities in a fast-paced, collaborative environment.
• Ability to leverage data assets to respond to complex questions that require timely answers.
• Working knowledge of migrating relational and dimensional databases on the AWS Cloud platform.