Senior Data Engineer

As a Senior Data Engineer at Persistent Ltd., you will be responsible for designing and developing scalable data pipelines using Azure Databricks, PySpark, and Delta Lake. Your key responsibilities will include: - Implementing and optimizing data warehousing solutions using Snowflake. - Integrating AI capabilities into data workflows using tools like ChatGPT, Copilot, Databricks AI Functions, or Snowflake Copilot. - Collaborating with data scientists and analysts to enable AI-powered insights, including summarization, classification, and sentiment analysis. - Building and maintaining ETL/ELT workflows to ensure data quality, performance, and governance. - Working with stakeholders to understand business requirements and translating them into technical solutions. - Ensuring security, compliance, and performance optimization across cloud data platforms. - Documenting architecture, workflows, and best practices for AI-enhanced data engineering. Qualifications required for this role include: - 6-12 years of experience in data engineering, with expertise in Azure Databricks, PySpark, and Snowflake. - Deep understanding of Lakehouse architecture, Delta Lake, and data warehousing principles. - Proficiency in Python, SQL, and Spark for building scalable data pipelines and transformation workflows. - Experience integrating AI tools such as ChatGPT, Copilot, or Databricks AI Functions for intelligent automation and enhanced analytics. - Knowledge of Snowflake Copilot, Snowpark, and AI-powered query generation. - Strong grasp of data governance, security, and metadata management. - Experience with CI/CD pipelines, DevOps practices, and infrastructure-as-code tools. - Ability to collaborate effectively with data scientists, analysts, and business stakeholders. - Excellent problem-solving, communication, and documentation skills. Additionally, Persistent Ltd. offers a competitive salary and benefits package, a culture focused on talent development, opportunities to work with cutting-edge technologies, employee engagement initiatives, annual health check-ups, and insurance coverage for self, spouse, children, and parents. Persistent Ltd. is committed to fostering diversity and inclusion in the workplace, supporting hybrid work and flexible hours, and providing an accessibility-friendly office environment. If you have specific requirements due to disabilities, you are encouraged to inform during the application process or at any time during your employment. Join Persistent Ltd. and unleash your full potential at persistent.com/careers.,