Data Engineer

As a Data Engineer at our company, your role will involve building, maintaining, and monitoring sustainable infrastructure with a focus on developing and maintaining high-volume, low-latency data platforms using the latest open-source frameworks, highly available RESTful services, and back-end Cloud based/open source systems. Your responsibilities will include: - Executing day-to-day operation of the data platform to ensure top-notch performance of data platform systems for customer-facing applications. - Working on features to make the data platform more robust and resilient. - Driving automation of application deployment for production environments. - Identifying and supporting areas for process and efficiency improvement within platform operations. - Taking ownership of your projects and working in small cross-functional teams to drive projects to closure. - Being highly collaborative with a strong belief in continuous improvement. - Demonstrating the ability to stay focused and calm in a fast-paced environment. To be successful in this role, you should have: - A BE/B-Tech/BS/Meng/MSc in computer science or a related technical field from top-tier colleges or equivalent practical experience. - 2.5-5 years of experience working on platform infrastructure based on Cloud and open-source systems and automation. - Proficiency in programming languages, particularly Python. - Experience with Django/Flask for building APIs. - Knowledge of databases including SQL querying & database understanding (RDBMS: MySQL, PostgreSQL; NoSQL: MongoDB, Elasticsearch, DynamoDB). - Familiarity with big data tools & distributed systems such as Spark, Redshift, and Kafka. - Experience with Cloud Computing, preferably AWS. - Familiarity with orchestration tools like Airflow and Docker. It would be good to have experience with: - Monitoring/Logging frameworks like ELK and Grafana. - Scalable Compute solutions like Kafka and Snowflake. - Workflow management libraries and tools. - Container orchestration systems like Kubernetes. - Code-as-Infrastructure/Pipeline-as-Code tools like Terraform and Chef. Additionally, understanding and functional experience in Big Data Engineering practices such as Data Warehousing, ingestion pipelines, ELT, data migration, and QC would be beneficial for this role.,