Data Engineer

Job Title: Data Engineer

Location:

Position: Full-time

Department: Annalect Engineering

Position Overview

Annalect is currently seeking a data engineer to join our technology team. In this role you will build Annalect products which sit atop cloud-based data infrastructure. We are looking for people who have a shared passion for technology, design & development, data, and fusing these disciplines together to build cool things. In this role, you will work on one or more software and data products in the Annalect Engineering Team. You will participate in technical architecture, design, and development of software products as well as research and evaluation of new technical solutions.

Key Responsibilities:

• Designing, building, testing, and deploying data transfers across various cloud environments (Azure, AWS, Snowflake, etc).
• Developing data pipelines, monitoring, maintaining, and tuning. Write at-scale data transformations in SQL and Python.
• Perform code reviews and provide leadership and guidance to junior developers.
• Ability to learn and teach new technologies.
• Provide guidance on leading edge data governance processes, architecture, and technologies.
• Setting up and maintaining data infrastructures to support various use cases.

Required Skills

• Curiosity in learning the business requirements that are driving the engineering requirements.
• Interest in new technologies and eagerness to bring those technologies and out of the box ideas to the team.
• 5+ years of SQL experience.
• 5+ years of professional Python experience.
• Preferred familiarity with Snowflake, AWS, GCP, Azure cloud environments.
• Intellectual curiosity and drive; self-starters will thrive in this position.
• Passion for Technology: Excitement for new technology, bleeding edge applications, and a positive attitude towards solving real world challenges.

Additional Skills

• BS, MS or PhD in Computer Science, Engineering, or equivalent real-world experience.
• Experience with big data and/or infrastructure. Bonus for having experience in setting up Petabytes of data so they can be easily accessed. Understanding of data organization, ie partitioning, clustering, file sizes, file formats.
• Experience working with classical relational databases (Postgres, Mysql, MSSQL).
• Experience with Snowflake, Hive, Spark, Redshift, or other data processing tools
• Experience building scalable data pipelines (Airflow experience a plus)
• Proven ability to independently execute projects from concept to implementation to launch and to maintain a live product
• Interest or experience in ML technologies (TensorFlow, PyTorch or SageMaker, BigQueryML).
• Proven ability to independently execute projects from concept to implementation to launch and to maintain a live product.