Senior Data Engineer

About the project:

Responsibilities:

• Design complex ETL processes of various data sources in the data Lakehouse
• Build new and maintain existing data pipelines using Python to improve efficiency and latency
• Improve data quality through anomaly detection by building and working with internal tools to measure data and automatically detect changes
• Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
• Perform data modeling and improve our existing data models for analytics
• Collaborate with SMEs, architects, analysts, data scientists, and others to build solutions that integrate data from many of our enterprise data sources
• Partner with stakeholders, including data, design, product, and executive teams, and assist them with data-related technical issues
  

Requirements:

• 3-5 years of commercial experience in building and maintaining complex data pipelines
• Proficiency in Python and SQL
• Well-versed in the optimization of ETL processes
• Solid skills in Bash scripting
• Solid knowledge of relational SQL and NoSQL databases: PostgreSQL/Redshift/MongoDB, Snowflake
• Knowledge of job scheduling and orchestration tools (Airflow etc.)
• Good knowledge of cloud services (Azure)
• Solid expertise in data processing tools for Azure: Elasticsearch, Azure Data Factory, Databricks, Azure Synapse
• Decent knowledge of CI/CD (Docker, Jenkins, Terraform, Git)
• Good understanding of algorithms and data structures
• Experience in the schema and dimensional data design
• Excellent communication skills, both written and verbal
• English level: upper intermediate+
  

Nice to have:

• Databricks certification
• Microsoft Azure certification
  

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits
• not applicable for freelancers