Data Engineer

Key Skills and Experience

• Expert Proficiency in SQL:

  Mastery in writing complex SQL queries, optimizing performance, and working with large-scale datasets.

• Expert Proficiency in Python:

  Extensive experience with Python for data workflows, pipeline automation, and data manipulation tasks.

• Pipeline Development at Scale:

  Demonstrated ability in building and maintaining data pipelines that process large, complex datasets efficiently and reliably, using tools such as DataSwarm or Airflow.

• Data Quality Monitoring & Alerting:

  Experience in setting up, validating, and maintaining data quality rules to ensure high data reliability across pipelines and tables.

• Stakeholder Collaboration:

  Proven ability to work directly and collaboratively with diverse technical and non-technical stakeholders to capture and address their data engineering needs.

• Independent Task Delivery:

  Able to own and deliver assigned data engineering tasks end-to-end, effectively managing priorities and deadlines with minimal supervision.

• Documentation Experience:

  Creates and maintains thorough technical documentation for own and related work to ensure transparency, maintainability, and team knowledge sharing

Preferred Qualifications

• Education

  : Bachelors or Masters degree in Computer Science, Engineering, Information Systems, or related field.

• Experience:

  3+ years as a Data Engineer or in roles working on large-scale data systems.

• Tech Stack Familiarity:

  Experience with Meta Stack and tools such as Hive, Presto, and DataSwarm.

• AI and Automation:

  Prior experience applying AI and automation to optimize and enhance data engineering processes.