Lead Data Engineer

Job Description:

9 years of industry experience including cloud technologies.

Very strong hands-on experience in Databricks with AWS cloud service in Data engineering/Data processing hands-on.

Must have experience in IBM StreamSets development.

Hands on experience in AWS Cloud-based development and integration

Proficiency in programming languages Scala and PLSQL for data processing and application development

Practical experience with Data Engineering and the accompanying DevOps & DataOps workflows

Strong knowledge on Pyspark and Python, Databricks SQL - Data engineering pipeline

Experience in offshore/onshore model and Ability and agile methodology.

Gathering requirements, understand the business need and regular discussion with tech on design, development activities.

Should have good experience working with client architect/design team to understand the architecture, requirement and work on the development.

Experience working in a Financial Industry.

Certification on Databricks and AWS will be added advantage.

Skills:

Rest APIs, SQL, Unit Testing, Snowflake, Amazon Web Services, Communication

Skills, Analytical Skills, Advanced SQL, Airflow, Apache Kafka, GBI Framework,

Data Engineering, Data Visualization, ETL Framework, GSF, Kubernetes, Python