Data Engineer (Data Pipeline Development) 7+ Years - Kuwait (On - Site)

Job Title – Data Engineer (Data Pipeline Development)

Location: Kuwait

Experience: 7+ Years

Job Summary:

Data Engineer

Key Responsibilities:

Data Pipeline Development:

• Extract data from upstream systems (e.g., Seabed, Sahala, and others).
• Design and implement data ingestion mechanisms (batch and streaming) into centralised environments (Data Lake / Data Warehouse).
• Integrate multiple data sources ensuring schema consistency and alignment.

Data Quality (QC) Implementation:

• Define and implement data quality checks (Completeness, Accuracy, Consistency, Timeliness, Duplicate handling).
• Build automated logging, alerts, and reports for QC failures.
• Leverage frameworks such as Great Expectations, Deequ, dbt tests for QC validation.

Data Transformation:

• Apply transformation rules to convert raw data into dashboard-ready formats.
• Implement business rules, filtering, aggregation, and enrichment.
• Prepare datasets for analytical dashboards and operational workflows.

System & Dashboard Integration:

• Ensure seamless integration of processed data into BI dashboards (Power BI, Tableau, QlikSense).
• Expose data to APIs, automation scripts, and system workflows.
• Optimize performance for dashboards and consuming applications.

Documentation & Handover:

• Document pipeline architecture, data flow, transformation logic, and QC rules.
• Prepare user guides and technical documentation for maintenance and enhancements.

Testing & Validation:

• Perform source-to-target data validation.
• Test transformation logic, QC rules, and integration with dashboards/systems.
• Conduct end-to-end testing for reliability and accuracy.

Required Skills & Qualifications:

• Technical Expertise:

  SQL, Python/Scala, Spark, ETL tools (ADF, Airflow, Talend, Glue, SSIS).

• Data Platforms:

  Hands-on experience with Data Lakes (e.g., S3, ADLS) and Data Warehouses (Snowflake, Redshift, Synapse, BigQuery).

• BI/Visualization:

  Integration experience with Power BI, Tableau, or QlikSense.

• Cloud Experience:

  Azure / AWS / GCP services for data pipelines.

• Version Control & CI/CD:

  Git, DevOps pipelines, automated deployments.

• Other Skills:

  Data Quality frameworks (Great Expectations, Deequ), strong problem-solving, documentation, and testing practices.

Preferred Qualifications:

• Experience in large-scale data engineering projects in the Middle East / Kuwait market.
• Familiarity with upstream systems like Seabed and Sahala.
• Exposure to real-time/streaming architectures (Kafka, Kinesis, Event Hub).