Scala spark developer

Job Description

Job Description Position Overview: We are seeking an experienced Scala Spark Developer with 3-4 years of hands-on experience to join our dynamic data engineering team. The ideal candidate will possess a deep understanding of big data processing frameworks and demonstrate proficiency in developing and optimizing scalable data solutions using Apache Spark and Scala. Responsibilities: Design, develop, and maintain robust data processing applications utilizing Apache Spark and Scala. Collaborate with cross-functional teams, including data scientists and analysts, to architect efficient data pipelines and workflows. Optimize and fine-tune Spark applications for enhanced performance and scalability. Conduct thorough code reviews, testing, and debugging to ensure the delivery of high-quality software solutions. Integrate diverse data sources into the data infrastructure, ensuring data integrity and consistency. Stay abreast of emerging trends and advancements in big data technologies and distributed computing. Qualifications : Bachelors degree in Computer Science, Engineering, or a related discipline. professional experience in developing applications with Apache Spark and Scala. Strong understanding of big data ecosystems, including Hadoop and Hive. Proficiency in data integration and ETL processes. Advanced skills in SQL and experience with both relational and NoSQL databases. Demonstrated problem-solving abilities and meticulous attention to detail. Excellent communication skills and a proven track record of effective teamwork. Preferred Skills: Experience with additional programming languages such as Python or Java. Familiarity with cloud platforms like AWS, Azure, or Google Cloud. Understanding of data pipeline and workflow management tools. Knowledge of data serialization formats such as Parquet, Avro, or ORC. Experience with real-time data processing and streaming technologies.