Data Engineer

Job Description

Role Description: We are looking for highly motivated expert Data Engineer who can own the design & development of complex data pipelines, solutions and frameworks, with deep domain knowledge of End-to-End (E2E) Regulatory Integrated Product Teams (IPTs) within the biotech or pharmaceutical industry in biotech or life sciences or pharma. The ideal candidate will be responsible to design, develop, and optimize data pipelines, data integration frameworks, and metadata-driven architectures that enable seamless data access and analytics The ideal candidate brings a strong foundation in data engineering, combined with an understanding of regulatory processes, scientific data requirements, and structured submission formats such as eCTD, IDMP, xEVMPD, and SPL. Roles & Responsibilities: Design, develop, and manage ETL/ELT pipelines to ingest, transform, and deliver regulatory data from diverse source systems. Collaborate with the Regulatory IPT to understand data requirements for global health authority submissions and ensure data readiness. Build and maintain regulatory data models and structured datasets aligned with submission standards (e.g., eCTD structure, IDMP specifications). Enable seamless data integration across regulatory, clinical, and quality platforms, ensuring consistency and traceability. Implement and maintain data validation and quality checks to ensure accuracy, integrity, and compliance of submission data. Support data lineage, metadata tagging, and audit trail capabilities to meet GxP and regulatory requirements. Proactively identify and implement opportunities to automate tasks and develop reusable frameworks Work in an Agile and Scaled Agile (SAFe) environment, collaborating with cross-functional teams, product owners, and Scrum Masters to deliver incremental value Use JIRA, Confluence, and Agile DevOps tools to manage sprints, backlogs, and user stories. Support continuous improvement, test automation, and DevOps practices in the data engineering lifecycle Collaborate and communicate effectively with the product teams, with cross-functional teams to understand business requirements and translate them into technical solutions Must-Have Skills: Deep domain knowledge of Manufacturing and/or Process Development and/or Supply Chain in biotech or life sciences or pharma. Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies. Proficiency in workflow orchestration, performance tuning on big data processing. Strong understanding of AWS services Ability to quickly learn, adapt and apply new technologies Familiarity with tools such as Veeva Vault RIM, Argus, ArisGlobal, or other regulatory information systems. Experience with metadata management, data lineage tools, and data quality frameworks. Strong problem-solving and analytical skills Excellent communication and teamwork skills Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices. Good-to-Have Skills: Data Engineering experience in Biotechnology or pharma industry Experience in writing APIs to make the data available to the consumers Experienced with SQL/NOSQL database, vector database for large language models Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops Education and Professional Certifications Bachelors degree and 4 to 8 years of Computer Science, IT or related field experience OR Masters degree and 1 to 4 years of Computer Science, IT or related field experience AWS Certified Data Engineer preferred Databricks Certificate preferred Scaled Agile SAFe certification preferred Soft Skills: Excellent analytical and troubleshooting skills. Strong verbal and written communication skills Ability to work effectively with global, virtual teams High degree of initiative and self-motivation. Ability to manage multiple priorities successfully. Team-oriented, with a focus on achieving team goals. Ability to learn quickly, be organized and detail oriented. Strong presentation and public speaking skills.