Data Engineer - Python Programming

Job Description

Key Responsibilities Develop, maintain, and optimize robust data ingestion and ETL/ELT pipelines using Python on the Databricks platform. Work collaboratively with the architecture team to align implementation with long-term data strategies. Leverage Delta Lake for data storage optimization and performance tuning. Utilize Prometheus for monitoring the health and performance of data pipelines and services. Ensure quality, reliability, and scalability of the data platform across multiple domains and use cases. Drive automation and reusability in data processing pipelines to reduce manual overhead. Participate in code reviews, peer programming, and continuous improvement efforts. Ensure adherence to best practices in coding, security, and documentation. Engage in team standups and collaborative sessions in the office 3 days a Skills : 5+ years of professional experience in Python development, primarily in data engineering use cases. Strong experience working with Databricks and Delta Lake. Proficient in SQL, data manipulation, and working with large-scale structured and semi-structured datasets. Hands-on experience with monitoring tools like Prometheus. Solid understanding of modern data architecture, especially around data lakes and distributed processing. Deep understanding of scalable system design, performance optimization, and best coding practices. Familiarity with CI/CD workflows and Git-based version Skills : Experience with Spark (especially PySpark). Exposure to other monitoring tools like Grafana, ELK stack, etc. Knowledge of cloud platforms such as AWS, Azure, or GCP. Understanding of data governance and security practices in enterprise Skills : Strong analytical and problem-solving skills. Excellent verbal and written communication. Ability to collaborate effectively across globally distributed teams. Proactive, self-motivated, and innovative mindset (ref:hirist.tech)