Data Engineer

Senior Data Engineer – Azure/AWS Databricks

📍 Location: [Add Location] | 💼 Experience: 7+ Years | 🏢 Employment Type: [Full-time/Contract]

Senior Data Engineer

Responsibilities

• Design, develop, and maintain scalable data pipelines and architectures in

  Azure/AWS Databricks

  .
• Optimize complex queries and ensure high-performance execution on the Spark platform.
• Work with structured and unstructured data to design and implement effective data models and storage solutions.
• Develop and deploy data applications and solutions on cloud environments (Azure, AWS).
• Perform performance tuning and code optimization within Databricks environments.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that drive business insights.
• Troubleshoot data-related issues and ensure data quality, reliability, and governance.

Qualifications

• Minimum

  7+ years of experience

  in the data analytics and engineering field.
• Proven expertise with

  Azure/AWS Databricks

  for big data solutions.
• Strong proficiency in

  Python, Scala, Spark, PySpark

  , and advanced

  SQL

  .
• Solid understanding of

  data warehousing, data lake architectures

  , and big data management tools.
• Hands-on experience with

  HIVE, HBase, Impala, Parquet

  , and data ingestion tools (

  Sqoop, Kafka, Flume

  ).
• Experience in

  Airflow

  for workflow orchestration and automation.
• Strong analytical, problem-solving, and performance tuning skills.
• Excellent communication skills with the ability to engage directly with customers and stakeholders.

Nice to Have:

• Experience with

  Cloudera

  ecosystem and other big data platforms.
• Knowledge of data governance, security, and compliance best practices.