Big Data / Scala / Python Engineering Lead - VP

Role Overview: As an Applications Development Technology Lead Analyst at our company, you will be a senior member responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your main objective in this role will be to lead applications systems analysis and programming activities. You should have at least two years of experience in building and leading highly complex, technical data engineering teams, with an overall of 10+ years of hands-on Data Engineering experience. Key Responsibilities: - Partner with multiple management teams to ensure appropriate integration of functions and identify necessary system enhancements - Manage a data-focused product and ML platform - Design, develop, and optimize scalable distributed data processing pipelines using Apache Spark and Scala - Resolve high impact problems/projects through evaluation of complex business processes - Ensure application design adheres to the overall architecture blueprint - Develop standards for coding, testing, debugging, and implementation - Manage, hire, and coach software engineering teams - Experience with large-scale distributed web services and processes around testing, monitoring, and SLAs - Provide in-depth analysis to define issues and develop innovative solutions - Serve as an advisor or coach to mid-level developers and analysts Qualifications Required: - 7 to 10+ years of hands-on experience in big data development, focusing on Apache Spark, Scala, and distributed systems - High proficiency in Scala-based functional programming for developing robust and efficient data processing pipelines - Strong experience with Apache Spark, Hadoop ecosystem tools such as Hive, HDFS, and YARN, AIRFLOW, DataOps, Data Management - Advanced knowledge of Scala and a good understanding of Python for data engineering tasks - Solid understanding of data modeling principles and ETL processes in big data environments - Strong ability to analyze and solve performance issues in Spark jobs and distributed systems - Familiarity with Git, Jenkins, and other CI/CD tools for automating the deployment of big data applications Additional Details: The company is focused on technology and applications development, with a strong emphasis on big data processing and data engineering. The environment encourages collaboration and innovation, with opportunities to work on robust data pipelines and architectures. Experience in real-time data streaming, financial services context, and leadership in data engineering are considered desirable skills for this position.,