Senior Data Engineer - Azure Databricks

As a Senior Data Engineer at our company, you will play a crucial role in leading the design, development, and optimization of scalable, secure, and high-performance data solutions in Databricks. Your responsibilities will include: - Leading Data Architecture & Development by designing and optimizing data solutions in Databricks - Building and maintaining ETL/ELT pipelines, integrating Databricks with Azure Data Factory or AWS Glue - Implementing machine learning models and AI-powered solutions for business innovation - Collaborating with data scientists, analysts, and business teams to translate requirements into technical designs - Enforcing data validation, governance, and security best practices for trust and compliance Additionally, you will be responsible for mentoring junior engineers, conducting code reviews, and promoting continuous learning. Staying updated with the latest trends in Databricks, Azure Data Factory, cloud platforms, and AI/ML to suggest and implement improvements is also a key part of your role. Qualifications required for this position include: - Bachelor's or master's degree in Computer Science, Information Technology, or a related field - 5+ years of experience in data engineering with a minimum of 3 years of hands-on experience in Databricks - Expertise in data modeling, Data Lakehouse architectures, ELT/ETL processes, SQL, Python or Scala, and integration with Azure Data Factory or AWS Glue - Strong knowledge of cloud platforms (Azure preferred) and containerization (Docker) - Excellent analytical, problem-solving, and communication skills - Demonstrated experience in mentoring or leading technical teams Preferred skills for this role include experience with Generative AI technologies, familiarity with other cloud platforms like AWS or GCP, and knowledge of data governance frameworks and tools. Please note that the job posting was referenced from hirist.tech.,