Data Engineer

About The Role

Key Responsibilities

• Design, build, and maintain ETL/ELT pipelines for efficient data ingestion, transformation, and storage.
• Collaborate with data analysts, data scientists, and stakeholders to understand data requirements and deliver actionable solutions.
• Implement and maintain data models and transformation workflows using Databricks.
• Optimize performance of large-scale distributed data processing jobs using PySpark.
• Ensure data quality, consistency, and integrity through validation and monitoring.
• Work with Azure cloud data platforms (e.g., Azure Data Lake, Azure Data Factory, Azure Databricks, Azure Synapse) to manage data storage and pipelines.
• Troubleshoot and resolve data-related technical issues.
• Participate in code reviews, documentation, and best practice enforcement for data engineering workflows.
  

Required Skills And Qualifications

• Minimum 2 years of experience in Data Engineering or a related role.
• Experience writing code in Python language.
• Proficiency in PySpark for distributed data processing.
• Expertise in Databricks for data engineering, transformation, and workflow development.
• Strong knowledge of SQL and relational databases.
• Familiarity with data warehousing solutions such as Snowflake, BigQuery, or Redshift.
• Experience with cloud platforms, with strong preference for Azure.
• Understanding of data pipelines, ETL/ELT processes, and data architecture principles.
• Knowledge of version control systems such as Git.
• Strong problem-solving skills and attention to detail.
  

Preferred Skills

• Working knowledge of DBT for data modeling and transformation.
• Experience with orchestration tools like Airflow or Prefect.
• Knowledge of streaming data pipelines such as Kafka or Kinesis.
• Familiarity with containerization using Docker or Kubernetes.
• Understanding of data governance, security, and compliance practices.
  

Perks And Benefits Of Working At Algoscale

• Opportunity to collaborate with leading companies across the globe.
• Opportunity to work with the latest and trending technologies.
• Competitive salary and performance-based bonuses.
• Comprehensive group health insurance.
• Flexible working hours and remote work options (for some positions only).
• Generous vacation and paid time off.
• Professional learning and development programs and certifications.