Data Ops

As the Lead Data Infrastructure Engineer at CBG, you will be responsible for developing and managing a robust data infrastructure and pipelines across the network. Your role is crucial in enabling the delivery of data as a product, ensuring high-quality, accessible, and timely data for advanced analytics, reporting, and decision-making across all operational and business functions. Your key responsibilities will include designing, implementing, and managing scalable data architecture for collecting, storing, and processing large volumes of data from CBG plant systems. You will own the cloud/on-prem data lake, data warehouse, and structured databases supporting real-time and batch processing. Additionally, you will develop and maintain robust, automated data pipelines using modern ETL/ELT tools, ensuring reliability, efficiency, and monitoring of all data flows from source to destination systems. Moreover, you will implement processes to ensure data accuracy, consistency, completeness, and freshness, working closely with Data Governance and Compliance teams to define standards, validation rules, and audit trails. Collaborating with data scientists, business analysts, application teams, and plant operations will be essential to understand and prioritize data requirements and enable self-service data access through APIs, secure dashboards, and curated datasets. You will also be responsible for maintaining a data catalogue and lineage tracking system to improve data discoverability and reusability across the organization. Providing documentation and training on data schema, usage, and access policies will be part of your role. Ensuring data is stored and accessed securely, following best practices in encryption, role-based access, and regulatory compliance is a key aspect of your responsibilities. The ideal candidate for this role should have a B.E. / B.Tech degree with expertise in SQL, Python, DevOps, and distributed data technologies such as Spark and Kafka. Experience with cloud platforms like AWS, Azure, or GCP, and associated data services is necessary. A strong understanding of CI/CD for data pipelines and MLOps integration is required. Familiarity with industrial data sources like OPC-UA, MQTT, and SCADA systems will be highly desirable. Excellent leadership, documentation, and stakeholder communication skills are essential for success in this role.,