Spark Scala Applications Development Technology Lead Analyst - Vice President

Role Overview: As a Lead Spark Scala Engineer under ICG TTS Operations Technology (OpsTech) Group, you will be responsible for developing and optimizing production-grade Spark applications in Scala. Your role will involve providing technical leadership, conducting code reviews, enforcing best practices, and optimizing performance. Your expertise in Spark, Scala, RDBMS, CI/CD tools, and other relevant technologies will be crucial in contributing to applications, systems analysis, and programming activities. Key Responsibilities: - Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization. - Provide technical leadership and guidance to a team of data engineers, fostering a culture of technical excellence, collaboration, and continuous improvement. - Conduct rigorous code reviews, establish coding standards, and enforce best practices for Spark Scala development, data governance, and data quality. - Identify and resolve performance bottlenecks in Spark applications through advanced tuning techniques. - Demonstrate profound understanding of Spark's architecture, execution model, and optimization techniques. - Design and build complex data pipelines (ETL/ELT) using Spark SQL, Spark Streaming, and Spark Core. - Exhibit expert-level proficiency in Scala programming, including functional programming paradigms and object-oriented design. - Utilize strong hands-on experience with the broader Hadoop ecosystem and related big data technologies. - Apply solid understanding of relational databases and NoSQL databases for effective data processing. - Demonstrate excellent communication, interpersonal, and leadership skills to articulate complex technical concepts to diverse audiences. - Showcase exceptional analytical and problem-solving abilities with meticulous attention to detail. Qualifications Required: - Hands-on knowledge of SPARK, Py-Spark, Scala, Java, and RDBMS like MS-SQL/Oracle. - Familiarity with CI/CD tools like LightSpeed, uDeploy. - Proficiency in Scala programming, Spark SQL, Spark Streaming, and Spark Core. - Strong hands-on experience with the broader Hadoop ecosystem and related big data technologies. - Solid understanding of relational databases and NoSQL databases. - Bachelors degree/University degree or equivalent experience. (Note: Additional details of the company were not provided in the job description.),