Data Engineer

Job Summary:

As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines and ETL processes. You will also provide production support for our data infrastructure, ensuring data reliability and performance. The ideal candidate will have 2-4 years of experience in data engineering, SQL, PL/SQL, Snowflake, strong Python programming skills, and a solid understanding of production environments.

Key Responsibilities:

• Data Pipeline Development:

• Design, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and store large volumes of data.

• Ensure data quality, consistency, and reliability across various data sources.
• Design and build software for large-scale data extraction and transformation from streaming, structured and unstructured data sources.

• Data Modeling and Architecture:

• Develop and optimize data models to support business intelligence and analytics.

• Collaborate with data scientists and analysts to understand data requirements and deliver appropriate data solutions.

• Production Support:

• Monitor and maintain the health of data pipelines and ETL processes in production environments.

• Troubleshoot and resolve data-related issues promptly to ensure minimal disruption to business operations.
• Implement and maintain data monitoring and alerting systems to proactively identify and address potential issues.

• Collaboration and Communication:

• Work closely with cross-functional teams including data scientists, analysts, and software engineers to deliver high-quality data solutions.

• Communicate technical concepts and solutions effectively to both technical and non-technical stakeholders.

Qualifications:

• Experience:

• 2-4 years of experience in data engineering or a related field.

• Undergraduate or higher degree in Computer Science or related disciplines
• Proven experience with Python programming, including experience with libraries such as Pandas, NumPy, and SQLAlchemy.

• Technical Skills:

• Strong understanding of data warehousing concepts and technologies.

• Strong SQL, PL/SQL, Snowflake and Python skillset
• Experience with ETL tools and frameworks (e.g., Apache Airflow, Talend, or similar).
• Strong proficiency in SQL for data manipulation and querying.
• Familiarity with cloud platforms such as AWS, GCP, or Azure.

• Production Support:

• Experience in monitoring and maintaining data pipelines in production environments.

• Strong problem-solving skills and the ability to troubleshoot complex data issues.
• Familiarity with data monitoring and alerting tools (e.g., Datadog, New Relic).

• Soft Skills:

• Excellent communication and collaboration skills.

• Ability to work in a fast-paced, dynamic environment.
• Strong analytical and problem-solving abilities.

Preferred Qualifications:

• Experience with big data technologies such as Hadoop, Spark, or Kafka.
• Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).
• Experience with version control systems (e.g., Git).

What We Offer:

• Competitive salary and benefits package.
• Opportunities for professional growth and development.
• A collaborative and innovative work environment.
• The chance to work on exciting and challenging projects.