Aws Data Engineer

Experience Level

4-8 years

Role Overview

Should have Clinical trial /Pharma/Health care domain experience

Required Skills

Data Engineering & ETL

• Python Development:

Strong proficiency in Python for data engineering

Experience with data processing libraries (Pandas, NumPy)

Knowledge of PySpark for large-scale data processing

• Modern ETL Tools & Frameworks:

dbt (data build tool) for data transformation

Apache Airflow for workflow orchestration

Great Expectations for data validation

Prefect or Dagster for modern dataflow

Experience with stream processing using Apache Kafka or AWS Kinesis

• Data Warehousing & Lakes:

Experience building and maintaining data lakes

Data modeling and schema design

Cloud & Infrastructure

• AWS Expertise:

AWS Glue for ETL processes

Amazon S3 for data storage

RDS Aurora for relational database operations

Redshift for data warehousing

Lambda for serverless applications

Step Functions for workflow orchestration

Route 53 and VPC for networking

EMR for big data processing

• Database Management:

Strong knowledge of MySQL

Experience with complex queries and optimization

Data pipeline performance tuning

Data quality monitoring and testing

• Frontend Development:

Basic proficiency in React, Angular, or jQuery

Experience creating data visualization interfaces

Understanding of HTML, CSS, and JavaScript

General Technical Skills

• Experience with CI/CD pipelines
• Proficiency with Git and version control
• Data governance and security best practices
• Documentation and technical writing

Nice to Have

• Advanced Data Tools:

Apache Beam

Apache Flink

Kubernetes for data infrastructure

Ray for distributed computing

• Additional Databases:

MongoDB

Snowflake

Cassandra

Elasticsearch

Redis

• DevOps Experience:

Infrastructure as Code (Terraform, CloudFormation)

Docker and container orchestration

Monitoring tools (Prometheus, Grafana)

Log management (ELK Stack)

• Data Science Collaboration:

Experience working with ML pipelines

Feature store implementation

Understanding of ML Operations (MLOps)

What We're Looking For

The ideal candidate will be a data engineering specialist who can:

• Design and implement scalable data architectures
• Build efficient and maintainable data pipelines
• Ensure data quality and reliability
• Balance technical excellence with practical solutions
• Collaborate effectively with data scientists and analysts