Data Engineer

Key Responsibilities:

• Design, implement, and operate scalable data pipelines for business intelligence and machine learning.
• Develop and maintain core data frameworks and key infrastructures.
• Create and support ETL processes to ingest data from various sources into our data warehouse.
• Design and model data warehouses for efficient and cost-effective reporting.
• Collaborate with data analysts and scientists to optimize data warehouse structures for reporting and insights.
• Improve software development processes and team productivity continuously.
• Define and enforce data governance processes, including data discovery, lineage, access control, and quality assurance.
• Perform code reviews and ensure data quality across all pipelines and processes.

Qualifications:

• Experience:

  35 years overall, with at least 2+ years in data engineering/infrastructure.

• Technology Stack:

• Big Data Technologies: Hive, Spark, PySpark (Batch & Streaming), Airflow, Redshift, Delta Lake
• Cloud: AWS (Strong architectural experience preferred)
• Data Tools: Databricks (strong plus), BI tools (Metabase, Power BI – a plus)
• DevOps knowledge is preferable but not mandatory
• Strong understanding of data warehousing and data ecosystem concepts
• Proven experience building and maintaining scalable data pipelines
• Experience with APIs for business application data ingestion
• Familiarity with data governance and quality assurance best practices
• Experience in product-based companies or startups is highly preferred