Senior Data Engineer

Role:

Job Title:

Work Experience:

Education:

Department:

Work location:

About the Role:

We are seeking a highly skilled Data Engineer with 4 - 8 years of hands-on experience in designing and developing scalable, reliable, and efficient data solutions. The ideal candidate will have a strong background in cloud platforms (AWS or Azure), experience in building both batch and streaming data pipelines, and familiarity with modern data architectures including event-driven and medallion architectures.

Key Responsibilities:

• Design, build, and maintain scalable data pipelines (batch and streaming) to process structured and unstructured data from various sources.
• Develop and implement solutions based on event-driven architectures using technologies like Kafka, Event Hubs, or Kinesis.
• Architect and manage data workflows based on the Medallion architecture (Bronze, Silver, Gold layers).
• Work with cloud platforms (AWS or Azure) to manage data infrastructure and storage, compute, and orchestration services.
• Leverage cloud-native or open-source tools for data transformation, orchestration, monitoring, and quality checks.
• Collaborate with data scientists, analysts, and product manager to deliver high-quality data solutions.
• Ensure best practices in data governance, security, lineage, and observability.

Required Skills & Qualifications:

• Strong experience in cloud platforms: AWS (e.g., S3, Glue, Lambda, Redshift) or Azure (e.g., Data Lake, Synapse, Data Factory, Functions).
• Proven expertise in building batch and streaming pipelines using tools like Spark, Flink, Kafka, Kinesis, or similar.
• Practical knowledge of event-driven architectures and experience with message/event brokers.
• Hands-on experience implementing Medallion architecture or similar layered data architectures.
• Familiarity with data orchestration tools (e.g., Airflow, Azure Data Factory, AWS Step Functions).
• Proficiency in SQL, Python, or Scala for data processing and pipeline development.
• Exposure to open-source tools in the modern data stack (e.g., dbt, Delta Lake, Apache Hudi, Great Expectations).

Preferred Qualifications:

Experience with containerization and CI/CD for data workflows (Docker, GitHub Actions, etc.).

Knowledge of data quality frameworks and observability tooling.

Experience with Delta Lake or Lakehouse implementations.

Strong problem-solving skills and ability to work in fast-paced environments.