DataBricks Manager

As an experienced Data Engineer, you will be responsible for designing and implementing high-performance data ingestion pipelines using technologies like Databricks. Your key responsibilities will include: - Developing scalable and reusable frameworks for ingesting, processing, and enriching datasets using Databricks. - Integrating end-to-end data pipelines to ensure efficient transportation of data from various source systems to target data stores. - Implementing data ingestion and ETL processes on Databricks, leveraging technologies such as Spark, Delta Lake, and MLflow. - Utilizing expertise in Spark, SQL, Python, and Scala for data processing and analytics on Databricks. - Collaborating with engineering teams to define best practices in data engineering, including agile methodologies, coding standards, and CI/CD processes. Qualifications required for this role include: - Bachelor's or higher degree in Computer Science, Data Engineering, or a related discipline, or equivalent (minimum 10+ years overall work experience). - At least 7+ years of experience in developing data ingestion, data processing, and analytical pipelines using big data technologies, relational databases, and data warehouse solutions. - Strong knowledge of cloud migration strategies and tools, including Databricks-specific solutions. - Familiarity with Big Data file formats and compression techniques. - Proficiency in cloud architectures and understanding the pros and cons of different migration approaches. Additionally, you should have proven ability to work independently under pressure while managing multiple priorities and possess excellent communication skills, both written and verbal, to collaborate effectively in a team environment.,