Data Engineer

Role & responsibilities

1. Data Architecture Design: Design and implement scalable, reliable, and maintainable data architectures.

2. Data Integration: Develop ETL processes to integrate data from various sources into a centralized data warehouse. Ensure data quality and integrity throughout the integration process.

3. Database Management: Administer and optimize databases for high performance and availability. Implement security measures to safeguard data against unauthorized access.

4. Data Modelling: Create and maintain data models for efficient storage and retrieval of information. Collaborate with data scientists and analysts to translate data needs into effective structures.

5. Coding and Scripting: Utilize programming languages (e.g., Python, SQL) for developing and maintaining data pipelines.

6. Performance Monitoring: Monitor data processing systems to identify and resolve performance bottlenecks.

Preferred candidate profile

1. Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

2. Experience 8-10 years.

3. Proven experience as a Data Engineer with expertise in migration strategies.

4. Proficiency in Python and SQL; expert in Views/Stored procedures.

5. Experience with ETL tools and processes.

6. Familiarity with big data technologies (e.g., Hadoop, Spark) is a plus.

7. Knowledge of defining and running ETL pipelines.

Additional Necessary Skills:

1. Capability to create data pipelines for KPI Dashboards.

2. Optimization of databases (Cloud) for efficient resource utilization.

3. Expertise in various database technologies with the ability to apply technology to use cases.

4. Experience with database caching, decoupling the database from reports.

5. Proficient in using microservices for data ingestion.

Shift Time:

1:30 PM to 10:30 PM