Associate Data Engineer

Job Description

We are looking for an Associate Data Engineer with deep expertise in writing data pipelines to build scalable, high-performance data solutions. The ideal candidate will be responsible for developing, optimizing, and maintaining complex data pipelines, integration frameworks, and metadata-driven architectures that enable seamless access and analytics. This role requires a strong understanding of big data processing, distributed computing, data modeling, and governance frameworks to support self-service analytics, AI-driven insights, and enterprise-wide data management. Roles & Responsibilities Develop complex ETL/ELT data pipelines to process large-scale datasets. Contribute to design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions. Ensure data integrity, accuracy, and consistency through rigorous quality checks and monitoring. Explore and implement new tools and technologies to enhance the ETL platform and pipeline performance. Identify automation opportunities and develop reusable frameworks. Understand biotech/pharma domains and build highly efficient data pipelines for complex data migration and deployment. Work in an Agile and Scaled Agile (SAFe) environment, collaborating with cross-functional teams, product owners, and Scrum Masters. Use JIRA, Confluence, and Agile DevOps tools to manage sprints, backlogs, and user stories. Support continuous improvement, test automation, and DevOps practices in the data engineering lifecycle. Collaborate effectively with product teams to translate business requirements into technical solutions. Must-Have Skills Experience in Data Engineering, focusing on Databricks, AWS, Python, SQL, and Scaled Agile methodologies. Proficiency in big data processing and transformation using Databricks, Apache Spark, Delta Lake, and distributed computing concepts. Strong understanding of AWS services with hands-on experience. Quick adaptability to new technologies. Strong problem-solving and analytical skills. Excellent communication and teamwork skills. Experience with Scaled Agile Framework (SAFe), Agile delivery, and DevOps practices. Good-to-Have Skills Data Engineering experience in Biotechnology or Pharma industry. Exposure to APIs, full-stack development. Experience with SQL/NoSQL databases and vector databases for large language models. Expertise in data modeling and performance tuning for OLAP and OLTP databases. Software engineering best practices, including version control (Git, Subversion), CI/CD (Jenkins, Maven), automated unit testing, and DevOps. Education and Professional Certifications Any degree with 2-5 years of experience. Preferred certifications: AWS Certified Data Engineer Databricks Certificate Scaled Agile SAFe certification Soft Skills Excellent analytical and troubleshooting skills. Strong verbal and written communication skills. Ability to work effectively with global, virtual teams. High degree of initiative and self-motivation. Ability to manage multiple priorities successfully. Team-oriented mindset focused on achieving team goals. Ability to learn quickly, be organized, and detail-oriented. Strong presentation and public speaking skills.