Data Engineer || Carnation Infotech || Remote

Job Summary

As a Data Engineer , you will be responsible for designing, building, and maintaining the infrastructure and tools that enable the organization to collect, store, and analyze large volumes of data. You will work closely with data scientists, analysts, and other engineers to ensure that our data systems are reliable, efficient, and scalable.

Key Responsibilities:

Data Pipeline Development:

Data Modeling:

ETL Processes:

Data Quality:

Data Integration:

Performance Tuning:

Documentation:

Collaboration:

Compliance and Security:

Qualifications

Experience

3+ years of experience in data engineering domain.

Technical Skills:

• Experience with ETL process (data extraction, data cleansing, data transformation, data monitoring, data modelling)
• Experience with Microsoft Azure (

  Azure Data Factory

  ,

  Azure Databricks

  , Azure Storage, Azure SQL Database, Azure Synapse)
• Familiarity with Big Data systems (batch & stream processing, Spark, Hive)
• Experience with two or more coding languages (including

  Python

  with

  pyspark

  )
• Experience with SQL (

  Spark SQL

  , T-SQL),
• Familiarity with code versioning (GIT, Azure DevOps)
• Experience with various data formats (e.g. CSV, JSON, Parquet, Delta Lake)
• Knowledge of distributed systems concepts and principles (consistency, availability, safety, durability, reliability, fault-tolerance)

Soft Skills:

• Strong problem-solving and analytical skills
• Excellent communication and collaboration skills
• Ability to work in a fast-paced, dynamic environment

Eager to learn new things and passionate about technology