Data Engineer OpenData Commercial

We are seeking a skilled Data Engineer to design and build reusable and configurable data pipelines using a mix of standard cloud-based and proprietary data platforms. This role involves collaborating with data scientists to develop AI/ML proofs of concept into full implementations, creating documentation, and providing informal training to data analysts. You will also work closely with other engineering and product teams and provide guidance to more junior data engineers.

What You'll Do:

• Design and build reusable and configurable data pipelines

  using a mix of standard cloud-based and proprietary data platforms.
• Work with a data scientist to

  develop AI/ML proofs of concept into full implementations

  .

• Create documentation and provide informal training

  for data analysts on the configuration and use of tools and pipelines.
• Work with other engineering and product teams to

  understand proprietary platforms and provide input and feedback

  .

• Provide guidance to more junior data engineers

  on best practices.
• Work with security teams to ensure that all servers, platforms, and other resources

  meet security requirements

  .

Requirements:

• BS degree in Computer Science, Engineering, or a related subject.

• 4+ years of experience in Data Engineering roles.

• Experience developing

  sophisticated data pipelines in cloud-based environments (e.g., AWS)

  using scalable data processing tools (e.g., Apache Spark).

• Data modeling experience.

• Demonstrated ability to work with others

  , particularly providing guidance to other data engineers.
• Ability to

  communicate around complex ideas and topics in English

  with both technical and non-technical individuals.

Nice to Have:

• Familiarity with

  Agile methodologies

  .

• DevOps skills

  , especially CI/CD experience.
• Configuring and maintaining

  cloud-based cluster computing resources and orchestration systems

  (e.g., EC2 instances, Kubernetes clusters, Elastic Beanstalk).