Systems Integration Advisor

Technical Lead with 10+ yrs of experience as well as expertise in data engineering and analytics in

• In-depth knowledge of Apache Spark, Pyspark and Databricks architecture.
• Strong programming skills in languages such as Python or Scala.
• Experience with data engineering concepts, ETL processes, and data modeling.
• Proven experience as a Technical Lead with a focus on Databricks and big data technologies on Azure Cloud Platform.
• Design and implement scalable and efficient data processing architectures using Databricks.
• Handson experience in writing python code, pyspark notebooks and Azure Cloud, Configure Databricks clusters and ensure adherence to best practices and coding standards
• Make strategic decisions regarding data organization, storage, and processing.
• Lead and mentor a team of data engineers and data scientists.
• Provide technical guidance and support to ensure the team's success.
• Oversee the implementation of data processing workflows, analytics, and machine learning models using Databricks.
• Optimize the performance of big data processing tasks and analytics jobs.
• Collaborate with cross-functional teams, including data science, business intelligence, and infrastructure teams.
• Ensure seamless integration of data pipelines with various systems.
• Ensure consistency, quality, and maintainability of the codebase.
• Optimize queries and resolve technical challenges during development and production.
• Manage the technical aspects of projects, including timelines and resource allocation.
• Coordinate with stakeholders to ensure project goals are met.
• Excellent leadership and communication skills.