Senior Data Engineer

Role & responsibilities

  4+ years of experience in data engineering, data architecture, data platforms & analytics 

  At least 3+ years of experience with Databricks, PySpark, Python, and SQL. 

  Consulting / customer-facing experience, working with external clients across a variety of industry markets 

  Comfortable writing code in both Python and SQL 

  Proficiency in SQL and experience with data warehousing solutions 

  Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one 

  Strong understanding of data modeling, ETL processes, and data architecture principles. 

  Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals 

  Familiarity with CI/CD for production deployments - GitHub, Azure DevOps, Azure Pipelines 

  Working knowledge of MLOps methodologies 

  Design and deployment of performant end-to-end data architectures 

  Experience with technical project delivery - managing scope and timelines Documentation and white-boarding skills 

  Experience working with clients and managing conflicts 

  Build skills in technical areas that support the deployment and integration of Databricks-based solutions to complete customer projects 

  Good to have Databricks Certifications