Lead Data Engineer - SQL and AWS

Job Description

About the Role: Gartner is looking for passionate and motivated Lead Data Engineers who are excited to foray into new technologies and help build / maintain data driven components for realizing business needs. This role is in Gartner Product Delivery Organization (PDO). PDO Data Engineering teams are high velocity agile teams responsible for developing and maintaining components crucial to customer-facing channels, data science teams, and reporting & analysis. These components include but are not limited to Spark jobs, REST APIs, AI/ML model training & inference, MLOps / devops pipelines, data transformation & quality pipelines, data lake & data catalogs, data streams etc. What you will do : Ability to lead and execute a mix of small/medium sized projects simultaneously Owns success, takes responsibility for successful delivery of solutions from development to production. Mentor and guide team members Explore and create POC of new technologies / frameworks Should have significant experience working directly with Business users in problem solving Excellent Communication and Prioritization skills. Should be able to interact and coordinate well with other developers / teams to resolve operational issues. Should be self-motivated and a fast learner to ramp up quickly with a fair amount of help from team members. Must be able to estimate development tasks with high accuracy and deliver on time with high quality while following coding guidelines and best practices. Identify systemic operational issues and resolve them. What you will need 6+ years of post-college experience in data engineering, API development or related fields Must have Demonstrated experience in data engineering, data science, or machine learning. Experience working with data platforms - building and maintaining ETL flows and data stores for ML and reporting applications. Skills to transform data, prepare it for analysis, and analyze it - including structured and unstructured data Ability to transform business needs into technical solutions Demonstrated experience of cloud platforms (AWS, Azure, GCP, etc.) Experience with languages such as Python, Java, SQL Experience with tools such as Apache Spark, Databricks, AWS EMR Experience with Kanban or Agile Scrum development Experience with REST API development Experience collaboration tools such as Git, Jenkins, Jira, Confluence Experience with Data modeling and Database schema / table design #LI-SP7