Job
Description
Expert Data Engineer Role Description Let's do this. Let's change the world. We are looking for a highly motivated Expert Data Engineer who can own the design and development of complex data pipelines, solutions, and frameworks. The ideal candidate will be responsible for designing, developing, and optimizing data pipelines, data integration frameworks, and metadata-driven architectures that enable seamless data access and analytics. This role requires deep expertise in big data processing, distributed computing, data modeling, and governance frameworks to support self-service analytics, AI-driven insights, and enterprise-wide data management. Roles & Responsibilities Design, develop, and maintain complex ETL/ELT data pipelines in Databricks using PySpark, Scala, and SQL to process large-scale datasets. Understand biotech/pharma or related domains & build highly efficient data pipelines for complex data migration and deployment. Implement solutions for unified data access, governance, and interoperability across hybrid cloud environments. Ingest and transform structured and unstructured data from databases (PostgreSQL, MySQL, SQL Server, MongoDB), APIs, logs, event streams, images, PDFs, and third-party platforms. Ensure data integrity, accuracy, and consistency through rigorous quality checks and monitoring. Expertise in data quality, data validation, and verification frameworks. Innovate, explore, and implement new tools and technologies to enhance efficient data processing. Identify automation opportunities and develop reusable frameworks. Work in an Agile and Scaled Agile (SAFe) environment, collaborating with cross-functional teams, product owners, and Scrum Masters. Use JIRA, Confluence, and Agile DevOps tools to manage sprints, backlogs, and user stories. Support continuous improvement, test automation, and DevOps practices in the data engineering lifecycle. Collaborate effectively with product teams to translate business requirements into technical solutions. Must-Have Skills Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL, Apache Spark, AWS, Python, SQL, and Scaled Agile (SAFe) methodologies. Proficiency in workflow orchestration and big data performance tuning. Strong understanding of AWS services with hands-on experience. Quick adaptability to new technologies. Strong problem-solving and analytical skills. Excellent communication and teamwork skills. Experience with Scaled Agile Framework (SAFe), Agile delivery, and DevOps practices. Good-to-Have Skills Data Engineering experience in Biotechnology or Pharma industry. Experience in writing APIs to make data available to consumers. Expertise in SQL/NoSQL databases and vector databases for large language models. Strong experience in data modeling and performance tuning for OLAP and OLTP databases. Software engineering best practices, including version control (Git, Subversion), CI/CD (Jenkins, Maven), automated unit testing, and DevOps. Education and Professional Certifications Any degree with 6-8 years of experience. Preferred certifications: AWS Certified Data Engineer Databricks Certificate Scaled Agile SAFe certification Soft Skills Excellent analytical and troubleshooting skills. Strong verbal and written communication skills. Ability to work effectively with global, virtual teams. High degree of initiative and self-motivation. Ability to manage multiple priorities successfully. Team-oriented mindset focused on achieving team goals. Ability to learn quickly, be organized, and detail-oriented. Strong presentation and public speaking skills.
