AI Data Engineer

Role Overview: You will be a Data Engineer at Operation Center, India (INOPC-PG), contributing to the development of a global value chain to optimize value for Hitachi Energy customers. Your role in the Transformers BU will involve providing high-quality engineering and technology solutions as part of Hitachi Energy's Global Footprint strategy. Key Responsibilities: - Demonstrate technical expertise in data analytics across a team of diverse technical competencies. - Develop and maintain accurate and scalable data pipeline and infrastructure, including SQL Warehouse, Data Lakes, on Cloud platforms like MS Azure, Databricks. - Collaborate with business stakeholders to understand data lineage, definitions, and extraction methods. - Write production-grade SQL and PySpark code for creating data architecture. - Consolidate SQL databases from multiple sources, perform data cleaning, manipulation, and prepare for analytics and machine learning. - Utilize data visualization tools such as Power BI to generate professional-quality dashboards and reports. - Create high-quality documentation for data processing in various projects to ensure reproducibility. - Ensure compliance with external and internal regulations, procedures, and guidelines, in alignment with Hitachi Energy's core values of safety and integrity. Qualifications Required: - BE / B.Tech in Computer Science, Data Science, or related discipline with a minimum of 5 years of relevant working experience. - 5 years of data engineering experience, with knowledge of lake house architecture, data integration framework, ETL/ELT pipeline, orchestration/monitoring, and star schema data modeling. - Proficiency in Python/PySpark and SQL (especially in PySpark, Python, and Spark SQL). - Hands-on experience of 2-3 years in data engineering using Databricks as the main tool, focusing more than 60% of the time on Databricks. - Familiarity with different Databricks components (DLT, workflow, Unity catalog, SQL warehouse, CI/CD) in addition to using notebooks. - Experience in Microsoft Power BI. - Proficiency in spoken & written English language is mandatory for this role.,