Job Description

Position Overview: We are seeking a highly skilled and experienced Sr. Java Spark Developer to join our dynamic team. The ideal candidate will design, develop, and maintain scalable and efficient data processing systems using Java and Apache Spark. You will work closely with data scientists, analysts, and other engineering teams to implement data-driven solutions and optimize performance. This role requires a strong background in backend services and distributed computing, along with hands-on experience in big data technologies and cloud platforms. Key Responsibilities: Design, develop, and maintain scalable and efficient data processing systems using Java and Apache Spark. Collaborate with data scientists, analysts, and engineering teams to implement data-driven solutions and optimize performance. Develop, test, and deploy Spark applications to process large volumes of data efficiently. Design ETL pipelines to process batch and streaming data, ensuring data quality, integrity, and scalability. Troubleshoot, debug, and optimize existing Spark applications and Java code for enhanced performance. Deploy and manage big data applications on cloud platforms such as AWS or Azure. Participate in code reviews, mentor junior developers, and promote best practices within the team. Stay updated with emerging trends and technologies related to big data, Spark, and Java. Software Requirements: Java Apache Spark Hadoop ecosystem (HDFS, YARN, Hive, etc.) is a plus ETL tools and frameworks Big Data technologies (HBase, Kafka, etc.) Data processing frameworks (Apache Flink, FaaS, Apache Beam, etc.) Relational and NoSQL databases Cloud platforms (AWS, Azure, GCP) Kubernetes and Docker (preferred) Scala (preferred) Technical Skills: Core Skills: Strong proficiency in Java development focused on backend services and distributed computing. Hands-on experience with Apache Spark for developing batch and streaming applications. Proficient in designing and implementing ETL pipelines. Big Data Technologies: Experience with Hadoop ecosystem components. Familiarity with HBase, Kafka, and other big data technologies. Knowledge of data processing frameworks like Apache Flink, FaaS, or Apache Beam. Databases: Proficiency in working with both relational and NoSQL databases. Cloud Platforms: Good understanding and experience with cloud technologies (AWS, Azure, GCP). Additional Skills: Experience with Kubernetes and Docker for containerized Spark jobs. Knowledge of Graph Databases, Machine Learning, or AI technologies. Experience with Scala is a plus. Experience: 4-7 years of strong experience in Java development, with a focus on backend services and distributed computing. Hands-on experience with Apache Spark, including the development of batch and streaming applications. Experience with big data technologies, ETL pipelines, and cloud platforms. Day-to-Day Activities: Develop and maintain data processing systems using Java and Spark. Collaborate with various teams to implement data-driven solutions. Design, test, and deploy Spark applications. Optimize and debug existing applications and code. Manage deployments on cloud platforms. Participate in code reviews and mentor junior developers. Stay updated with the latest trends and technologies in big data. Qualifications: Bachelor's degree in Computer Science, Engineering, or related field. Strong problem-solving skills and ability to troubleshoot complex systems. Familiarity with Agile methodologies (Scrum, Kanban). Excellent communication and teamwork skills. Soft Skills: Strong analytical and problem-solving abilities. Team player with excellent communication skills. Ability to mentor and guide junior developers. Adaptability to fast-paced and dynamic environments. Continuous learning mindset to stay updated with emerging technologies.