Data Engineer III

Work Schedule

Environmental Conditions

Job Description

Job Summary:

We need a diligent Data Engineer III to join our dedicated data team, contributing to impactful projects.

Key Responsibilities:

• Design, develop, and maintain scalable and robust data pipelines for both batch and real-time processing.
• Extract, transform, and load (ETL) data from a wide variety of structured and unstructured data sources including:

RESTful and SOAP APIs

1. Databases (SQL, NoSQL)

Cloud storage (e.g., S3, Google Cloud Storage)

1. File formats (e.g., JSON, CSV, XML, Parquet)

Web scraping tools where appropriate

• Build reusable data connectors and integration solutions to automate data ingestion.
• Collaborate with internal collaborators to understand data needs and guarantee data is accessible and user-friendly.
• Monitor and optimize pipeline performance and troubleshoot data flow issues.
• Ensure data governance, security, and quality standards are strictly applied across all pipelines.
• Maintain and scale data warehouse or data lake environments (e.g., Snowflake, Redshift, BigQuery).
• Apply data manipulation and analysis libraries like Pandas, Polars, or Dask for efficient handling of large datasets.
• Build and craft data flow and architecture diagrams to visually represent data pipelines, system integrations, and data models, ensuring clarity and alignment among technical and non-technical collaborators.

Requirements:

Technical Skills:

• Proficiency in SQL and at least one programming language (Python, Pyspark, Java, Scala).
• Experience with data pipeline and workflow tools (e.g., Apache Airflow, AWS Data Pipeline).
• Knowledge of relational and non-relational databases (e.g., Oracle, SqlServer, MongoDB).
• Strong data modeling and data warehousing skills.

Education & Experience:

• Bachelor’s degree or equivalent experience in Computer Science, Engineering, Information Systems, or related field (Master’s a plus).
• 5+ years of proven experience in a data engineering or similar role.

Soft Skills:

• Strong analytical and problem-solving abilities.
• Excellent communication and collaboration skills.
• Diligent and proactive approach.