Databricks Engineer

Role Overview: OneMagnify is a global performance marketing organization that focuses on brand marketing, technology, and analytics to accelerate business growth and provide valuable customer insights. As a Databricks Engineer in the Data Engineering team, you will architect, build, and maintain data infrastructure on the Databricks Lakehouse Platform. Your role will involve collaborating with data scientists, analysts, and engineers to deliver data solutions that drive the business forward. Key Responsibilities: - Architect, develop, and deploy scalable and reliable data infrastructure and pipelines using Databricks and Spark. - Design and implement data models and data warehousing solutions with a focus on performance and scalability. - Optimize data processing frameworks and infrastructure for maximum efficiency and cost-effectiveness. - Collaborate with data scientists and analysts to understand their data needs and engineer solutions. - Implement robust data quality frameworks, monitoring systems, and alerting mechanisms. - Design, build, and maintain efficient ETL/ELT processes. - Integrate Databricks with various data sources, systems, and APIs. - Contribute to the definition and implementation of data governance, security, and compliance policies. - Stay current with the latest advancements in Databricks, cloud data engineering standard methodologies, and related technologies. Qualifications Required: - Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience). - 5+ years of experience in data engineering or a similar role with a strong emphasis on building and maintaining data infrastructure. - Deep understanding and practical experience with the Databricks Lakehouse Platform and its core engineering aspects. - Expert-level proficiency in working with big data processing frameworks, particularly Apache Spark. - Strong hands-on experience with programming languages such as Python (PySpark) and/or Scala. - Solid experience with SQL and data warehousing principles, including schema design and performance tuning. - Proven experience with cloud platforms such as AWS, Azure, or GCP. - Comprehensive understanding of data modeling, ETL/ELT architecture, and data quality engineering principles. - Excellent problem-solving, analytical, and debugging skills. - Strong communication and collaboration skills, with the ability to explain technical concepts to both technical and non-technical audiences. Additional Company Details (if present): OneMagnify is committed to employee growth and development, providing a supportive environment for over 700 colleagues to thrive and achieve their personal best. The company has been recognized as a Top Workplace, Best Workplace, and Cool Workplace in the United States for 10 consecutive years. They were also recently recognized as a Top Workplace in India. OneMagnify believes in providing equal opportunities for all employees and actively seeks like-minded individuals to join their team.,