Data Engineer

Data Engineer who can work on data lake system, which has been implemented on Databricks.
Required Skills:Must be proficient in the major languages such as Scala, Spark, and Kafka (any variant). Write elegant and maintainable code, and be comfortable with picking up new technologies.Proficient in working with distributed systems and have experience with different distributed processing frameworks that can handle data in batch and near real-time pipelines.Experience in working with AWS cloud services and Databricks (preferred) to build end-to-end data lakehouse solutions that bring different systems togetherCapability to understand and implement metadata-based data processing framework and methodologiesExperience in working with CICD process - Jenkins, & GIT ActionsExperience in working with monitoring and control stack - Prometheus/GrafanaKey job responsibilities:Drive the backbone of our organizational data platform by building robust pipelines that turn complex data into actionable insights using AWS and the Databricks platformDesign scalable, maintainable data systems with strong observabilityMonitor, log, alert, and ensure system robustnessBuild efficient pipelines for structured and unstructured dataInterface with Product engineering, regulatory, and legal teams as necessary to ensure Lakehouse’s compliance adherenceWrite clean, efficient code that handles massive amounts of structured and unstructured data