Senior Data Engineer

Role: Senior Data Engineer

Description:

The Senior Data Engineer plays a pivotal role within our Data Engineering team, leading the design,

development, and maintenance of robust data pipelines and infrastructure. This position involves

managing multiple projects, collaborating with cross-functional teams, and delivering high-quality data

solutions that align with business objectives. The ideal candidate brings extensive experience in data

engineering, with expertise in technologies such as PySpark, AWS Glue, and S3, and a proven ability

to lead complex data initiatives.

To excel in this role, you must be a proactive problem-solver with excellent communication skills,

capable of translating complex technical concepts into actionable insights for business stakeholders.

You should be adept at leading teams, managing project timelines, and ensuring that all deliverables

meet the highest standards of quality and compliance.

Responsibilities:

• Lead functional teams or projects, managing multiple data engineering initiatives

simultaneously to ensure timely delivery.

• Design, implement, and maintain scalable ETL pipelines using Databricks, PySpark, AWS

Glue, and S3 to process and transform large datasets.

• Query and analyze complex datasets using SQL and Python to support business intelligence

and analytics initiatives.

• Develop and document data models, schemas, and standards to ensure data consistency,

usability, and governance.

• Monitor, troubleshoot, and optimize data pipeline performance and reliability, minimizing

downtime and maximizing efficiency.

• Expert in designing and optimizing Databricks-based data architectures, developing ETL/ELT

pipelines, and ensuring data governance and compliance for large-scale, pharmaceutical

cloud data systems.

• Implement robust data validation and testing techniques to ensure high data quality and

integrity.

• Communicate potential risks, mitigations, and business impacts clearly and promptly to

stakeholders.

• Define technical requirements and design solutions leveraging existing technologies and

industry best practices.

• Solve complex data-related problems by analyzing multiple sources of information and

applying innovative approaches.

• Ensure all data, processes, and pipelines adhere to security protocols and compliance

standards.

About You (Desired Profile):

• 6+ years of experience in data engineering, with a focus on big data technologies and cloud

platforms.

• Proficiency in PySpark, AWS Glue, S3, SQL, and Python.

• Demonstrated experience leading data engineering teams or projects.

• Strong analytical and problem-solving skills, with the ability to handle complex data

challenges.

• Excellent communication and interpersonal skills, enabling collaboration with technical and

non-technical stakeholders.

• Deep understanding of data modeling, schema design, and database management.

• Knowledge of data quality assurance techniques and compliance requirements.

• Bachelors degree in computer science, Information Systems, or a related field; advanced

degree preferred.

• Pharmaceutical data knowledge is Plus

• Certifications in relevant technologies (e.g., AWS Certified Data Analytics, Databricks

Certified Data Engineer) is REQUIRED.

Good to Have

• Integration with Veeva Vault CRM, Veeva Network, Veeva Align