Data Engineer II

Responsibilities

• Authoring and design of Big Data ETL platforms and pipelines in SCOPE, Scala, SQL, Python, or C#.
• Data extraction across a wide variety of data sources.
• Data cleaning, preprocessing, and transformation for further analysis by data analysts.
• Data Validation framework from source to endpoints ensuring data quality and integrity.
• Enable the Data Scientists and Analysts to do more with data across all aspects of the development lifecycle.
• Contribute to a data-driven culture as well as a culture of experimentation across the organization.
• Provide new and improve upon existing data platform offerings with a fundamental understanding of the end-to-end scenarios.
• Enable Data Scientists and other partners to do more with data across all aspects of the development lifecycle by helping democratize data.

Qualifications

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience.
• Proven coding and debugging skills in C#, C++, Java, or SQL.
  
• Ability to work and communicate effectively across disciplines and teams.
  
• Familiarity with distributed data processing and analysis, such as Map-Reduce or Apache Spark.
• 2+ years of experience in data engineering.
• Understanding and experience with data cloud computing technologies such as – Azure Synapse, Azure Data Factory, SQL, Azure Data Explorer, Power BI, PowerApps, Hadoop, YARN, Apache Spark.
• Excellent analytical skills with systematic and structured approach to software design.