Lead Data Engineer

Responsibilities:

• Build new data acquisition and transformation pipelines using advanced data processing and cloud technologies.
• Collaborate with the broader technology team, including information architecture and data integration teams, to align pipelines with strategic initiatives.
  

What Were Looking For:

• Bachelors degree in computer science or a related field, with at least 8+ years of professional software development experience.
  
• Must have:

  Programming languages commonly used for data processing,

  Data orchestration and workflow management systems,

  Distributed data processing framework, relational database management systems,

  Big data processing frameworks

Minimum 8+ years of strong backend development experience

Advance SQL programming skills

Understanding of database performance tuning in large datasets

Excellent logical, analytical and communication skills are essential, with strong verbal and writing proficiencies

Experience in conducting application design and code reviews

Proficiency with one or more of the following technologies:

Object-oriented programming

Programing Languages (Python)

Scripting (Bash, Scala, Powershell)

Good knowledge on Cloud Computing

Database systems (SQL, MS SQL)

Exposure to Data Engineering & Big Data technologies like Hadoop/Databricks, Spark/Scala, Nifi & ETL is a plus

Knowledge of Fundamentals, or financial industry highly preferred.

Experience on Docker is a plus

Experience working in cloud computing environments such as AWS, Azure or GCP

• Experience with large-scale data processing platforms.
• Deep understanding of RESTful services, good API design, and object-oriented programming principles.
• Proficiency in object-oriented or functional scripting languages.
• Good working knowledge of relational and NoSQL databases.
• Experience in maintaining and developing software in production environments utilizing cloud-based tools.
• Strong collaboration and teamwork skills, along with excellent written and verbal communication abilities.
• Self-starter and motivated individual with the ability to thrive in a fast-paced software development environment.
• Agile experience is highly desirable.
• Experience with data warehousing and analytics platforms will be a significant advantage.