8 - 12 years

0 Lacs

Posted:2 days ago| Platform: Shine logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

You have a strong background in data engineering or a related field with over 8 years of experience. Your expertise includes hands-on experience with Azure Databricks, Spark, Python/Scala, CICD, and scripting for data processing. You have worked with multiple file formats such as Parquet, Delta, and Iceberg, and have knowledge of streaming technologies like Kafka for real-time data ingestion. Additionally, you are experienced in data governance and data security in Azure, and have a deep understanding of Azure Data Services. Key Responsibilities: - Utilize Azure Databricks, Spark, Python, and Scala for scalable and distributed data processing. - Work with various file formats like Parquet, Delta, and Iceberg for data processing. - Implement data governance and data security measures in Azure. - Collaborate with cross-functional teams to develop and optimize CI/CD pipelines. - Use version control tools like Git, Jenkins, and agile methodologies for efficient project management. - Ensure data quality and reliability by working with data lakes, data warehouses, and modern data architectures. Qualifications Required: - 8+ years of experience in data engineering or a related field. - Strong proficiency in Spark, Spark Streaming, Python, Scala, and SQL. - Knowledge of data formats like Iceberg, Parquet, ORC, and Delta Lake. - Familiarity with Azure Data Services and cloud infrastructure principles. - Experience with CI/CD pipelines, version control, and agile methodologies.,

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You