Data Engineer

In this role at Markovate, you will be responsible for designing, developing, and maintaining scalable data pipelines for ingestion, processing, and storage. Your tasks will involve optimizing data architectures and models, developing and maintaining ETL processes, ensuring data quality and security, collaborating with cross-functional teams, monitoring and troubleshooting data workflows, and implementing data integration solutions using tools like Apache Spark, Kafka, and Airflow. Additionally, you will be documenting data processes and flow diagrams for internal transparency. Key Responsibilities: - Design, develop, and maintain scalable data pipelines for ingestion, processing, and storage. - Optimize data architectures and models for efficient storage and retrieval. - Develop and maintain robust ETL processes to transform and load data into data warehouses and lakes. - Ensure data quality, integrity, and security across all systems. - Collaborate with data scientists, analysts, and business teams to meet evolving data needs. - Monitor and troubleshoot data workflows to ensure high performance and availability. - Implement data integration solutions using tools like Apache Spark, Kafka, and Airflow. - Document data processes, architectures, and flow diagrams for internal transparency. Qualifications Required: - Bachelor's degree in Computer Science, Information Technology, or a related field. - Minimum of 3 years of experience in data engineering and architecture. - Strong interpersonal skills with the ability to mentor and collaborate effectively. - Ability to work under tight deadlines in fast-paced environments. - A self-starter who can work independently and as part of a team.,