DataOps Engineer

You will be a critical part of the data engineering team, working at the intersection of software engineering, DevOps, and data analytics. Your responsibilities will include: - Designing, developing, and managing robust, scalable, and efficient ETL/ELT pipelines using Python and modern DataOps methodologies. - Implementing data quality checks, pipeline monitoring, and error handling mechanisms. - Building data solutions using cloud-native services on AWS (e.g., S3, ECS, Lambda, CloudWatch). - Containerizing applications using Docker and orchestrating using Kubernetes for scalable deployments. - Working with infrastructure-as-code tools and CI/CD pipelines to automate deployments. - Designing and optimizing data models using PostgreSQL, Redis, and PGVector for high-performance storage and retrieval. - Supporting feature stores and vector-based storage for AI/ML applications. - Driving Agile ceremonies (daily stand-ups, sprint planning, retrospectives) and ensuring successful sprint delivery. - Reviewing pull requests (PRs), conducting code reviews, and enforcing security and performance standards. - Collaborating closely with product owners, analysts, and architects to refine user stories and technical skills. Qualifications required for this role: - 10+ years of experience in Data Engineering, DevOps, or Software Engineering roles with a focus on data products. - Strong hands-on experience with Python, Docker, Kubernetes, and AWS (especially S3 and ECS). - Proficiency with relational and NoSQL databases such as PostgreSQL, Redis, and experience with PGVector is a strong plus. - Deep understanding of CI/CD pipelines, GitHub workflows, and modern source control practices. - Experience working in Agile/Scrum environments with strong collaboration and communication skills. - Passion for building clean, well-documented, and scalable code in a collaborative environment. - Familiarity with DataOps principles, including automation, testing, monitoring, and deployment of data pipelines.,