Job
Description
Role & responsibilities Experience: 5 - 8 years Employment Type: Full-Time Job Summary: We are looking for a highly skilled Scala and Spark Developer to join our data engineering team. The ideal candidate will have strong experience in building scalable data processing solutions using Apache Spark and writing robust, high-performance applications in Scala. You will work closely with data scientists, data analysts, and product teams to design, develop, and optimize large-scale data pipelines and ETL workflows. Key Responsibilities: Develop and maintain scalable data processing pipelines using Apache Spark and Scala. Work on batch and real-time data processing using Spark (RDD/DataFrame/Dataset). Write efficient and maintainable code following best practices and coding standards. Collaborate with cross-functional teams to understand data requirements and implement solutions. Optimize performance of Spark jobs and troubleshoot data-related issues. Integrate data from multiple sources and ensure data quality and consistency. Participate in design reviews, code reviews, and provide technical leadership when needed. Contribute to data modeling, schema design, and architecture discussions. Required Skills: Strong programming skills in Scala . Expertise in Apache Spark (Core, SQL, Streaming). Hands-on experience with distributed computing and large-scale data processing. Experience with data formats like Parquet, Avro, ORC, and JSON. Good understanding of functional programming concepts. Familiarity with data ingestion tools (Kafka, Flume, Sqoop, etc.). Experience working with Hadoop ecosystem (HDFS, Hive, YARN, etc.) is a plus. Strong SQL skills and experience working with relational and NoSQL databases. Experience with version control tools like Git. Preferred Qualifications: Bachelor's or Masters degree in Computer Science, Engineering, or related field. Experience with cloud platforms like AWS, Azure, or GCP (especially EMR, Databricks, etc.). Knowledge of containerization (Docker, Kubernetes) is a plus. Familiarity with CI/CD tools and DevOps practices. ndidate profile
