Data engineer with Gen AI- Balewadi, pune- hybrid

Job Description

Role & responsibilities We are seeking a skilled Data Engineer with advanced expertise in Python, PySpark, Databricks, and Machine Learning, along with a working knowledge of Generative and Agentic AI. This role is critical in ensuring data integrity and driving innovation across enterprise systems. You will design and implement ML-driven solutions to enhance Data Governance & Data Privacy initiatives through automation, self-service capabilities, and scalable, AI-enabled innovation. Key Responsibilities: Implement ML and Generative/Agentic AI solutions to optimize Data Governance processes. Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks. Develop automation frameworks to support data quality, lineage, classification, and access control. Develop and deploy machine learning models to uncover data patterns, detect anomalies, and enhance data governance and privacy compliance Collaborate with data stewards, analysts, and governance teams to build self-service data capabilities. Work with Databricks, Azure Data Lake, AWS, and other cloud-based data platforms for data engineering. Build, configure, and integrate APIs for seamless system interoperability. Ensure data integrity, consistency, and compliance across systems and workflows. Integrate AI models to support data discovery, metadata enrichment, and intelligent recommendations. Optimize data architecture to support analytics, reporting, and governance use cases. Monitor and improve the performance of ML/AI components in production environments. Stay updated with emerging AI and data engineering technologies to drive continuous innovation. Technical Skills: Strong programming skills in Python, PySpark, SQL for data processing and automation. Experience with Databricks and Snowflake (preferred) for building and maintaining data pipelines. Experience with Machine Learning model development and Generative/Agentic AI frameworks (e.g. LLMs, Transformers, LangChain) especially in the Data Management space Experience working with REST APIs & JSON for service integration Experience working with cloud-based platforms such as Azure, AWS, or GCP Power BI dashboard development experience is a plus. Soft Skills: Strong problem-solving skills and attention to detail. Excellent communication and collaboration abilities, with experience working across technical and business teams