Databricks Engineer

You have 5+ years of experience in designing and building data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. You possess hands-on expertise with streaming and messaging systems like Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub. Your experience includes creating producers, consumers, and topics, and integrating them into downstream processing. You have a deep understanding of relational databases and Change Data Capture (CDC). You are proficient in SQL Server, Oracle, or other RDBMSs, and have experience capturing change events using tools like Debezium or native CDC tools, transforming them for downstream consumption. Your proficiency extends to programming languages such as Python, Scala, or Java, along with solid knowledge of SQL for data manipulation and transformation. You also have cloud platform expertise, including experience with Azure or AWS services for data storage, compute, and orchestration (e.g., ADLS, S3, Azure Data Factory, AWS Glue, Airflow, Databricks, DLT). Furthermore, you have knowledge of data modeling and warehousing, including familiarity with data Lakehouse architectures, Delta Lake, partitioning strategies, and performance optimization. You are also well-versed in version control and DevOps practices, with experience in Git, CI/CD pipelines, and the ability to automate deployment and manage infrastructure as code.,