Job
Description
As a Specialist Data Engineer at our company, you will play a crucial role in designing, developing, and maintaining data pipelines to extract data from various sources and populate data lakes and data warehouses. Your primary responsibilities will include: - Collaborating with Data Analysts, Data Scientists, and Machine Learning Engineers to identify and transform data for ingestion, exploration, and modeling - Working with the data governance team to implement data quality checks and maintain data catalogs - Using orchestration, logging, and monitoring tools to build resilient pipelines - Developing pipelines to ingest data into cloud data warehouses - Analyzing data using SQL - Utilizing serverless AWS services like Glue, Lambda, and StepFunctions - Employing Terraform Code to deploy on AWS - Containerizing Python code using Docker - Using Git for version control and understanding various branching strategies - Building pipelines to work with large datasets using PySpark - Developing proof of concepts using Jupyter Notebooks - Working as part of an agile team - Creating technical documentation as needed Qualifications required for this role include: - Bachelors Degree or equivalent experience in a relevant field such as Engineering or Computer Science - 4-8 years of relevant experience - Good experience with AWS services like S3, ECS, Fargate, Glue, StepFunctions, CloudWatch, Lambda, and EMR - Proficiency in Python, PySpark, SQL, Git, Docker, and Terraform - Ability to work in cross-functional teams Preferred experience and skills include any AWS developer or architect certification and knowledge of agile development methodology. The Human Health Division of our organization prioritizes a patient-first, profits-later ideology. Our team of professionals is dedicated to bringing medicines to customers worldwide. As a Specialist Data Engineer at our company, you will play a crucial role in designing, developing, and maintaining data pipelines to extract data from various sources and populate data lakes and data warehouses. Your primary responsibilities will include: - Collaborating with Data Analysts, Data Scientists, and Machine Learning Engineers to identify and transform data for ingestion, exploration, and modeling - Working with the data governance team to implement data quality checks and maintain data catalogs - Using orchestration, logging, and monitoring tools to build resilient pipelines - Developing pipelines to ingest data into cloud data warehouses - Analyzing data using SQL - Utilizing serverless AWS services like Glue, Lambda, and StepFunctions - Employing Terraform Code to deploy on AWS - Containerizing Python code using Docker - Using Git for version control and understanding various branching strategies - Building pipelines to work with large datasets using PySpark - Developing proof of concepts using Jupyter Notebooks - Working as part of an agile team - Creating technical documentation as needed Qualifications required for this role include: - Bachelors Degree or equivalent experience in a relevant field such as Engineering or Computer Science - 4-8 years of relevant experience - Good experience with AWS services like S3, ECS, Fargate, Glue, StepFunctions, CloudWatch, Lambda, and EMR - Proficiency in Python, PySpark, SQL, Git, Docker, and Terraform - Ability to work in cross-functional teams Preferred experience and skills include any AWS developer or architect certification and knowledge of agile development methodology. The Human Health Division of our organization prioritizes a patient-first, profits-later ideology. Our team of professionals is dedicated to bringing medicines to customers worldwide.
