Job
Description
We are seeking a highly skilled Senior Data Engineer with strong experience in cloud-based data engineering and AI solutions. The ideal candidate will bring a combination of software engineering, big data processing, and cloud infrastructure expertise. This role requires hands-on experience in building data pipelines on AWS or Azure , with a strong emphasis on Databricks , Spark , and Python . You will work within Agile teams , contributing to the design, development, and deployment of scalable, high-performance data solutions. A passion for quality, testing, and continuous learning is key to success in this role. Required Qualifications Bachelor's degree or equivalent in Computer Science , Computer Engineering , or a related field. 7+ years of experience in data or software engineering roles. 3+ years of experience building cloud-based data pipelines and AI/ML data solutions on AWS or Azure . 3+ years of hands-on experience with Python and Apache Spark . Deep experience with Databricks , including: Spark-based processing Building and managing scalable data pipelines Use of Databricks notebooks and platform tools for analytics and machine learning Key Responsibilities Design, build, and maintain cloud-native data pipelines that support AI/ML models and large-scale analytics workloads. Leverage Databricks and Spark to process structured and unstructured data efficiently. Collaborate with cross-functional teams, including data scientists, product owners, and DevOps engineers, in an Agile environment. Apply testing methodologies and best practices to ensure high-quality, resilient data solutions. Contribute to code reviews , data modeling , and the development of data quality frameworks . Stay current on evolving tools, technologies, and industry trends to continuously improve processes and architecture. Required Skills Strong Python programming and data manipulation skills. Expertise in Apache Spark and large-scale distributed data processing. In-depth experience with Databricks as a development and orchestration environment. Solid understanding of cloud architecture and data services on AWS or Azure . Experience working in Agile teams with modern development and deployment practices. Strong written and verbal communication skills with the ability to convey technical topics clearly. Desirable Skills Databricks certification (Developer or Data Engineer). Experience working within a DevOps delivery model , including CI/CD for data pipelines. Understanding of quality and compliance frameworks , and applying standards in production environments. Prior experience in industries with large, complex data environments , or consulting/vendor roles.
