Senior Consultant

5 to 8 years of experience designing and building data pipelines using
Apache Spark, Databricks or equivalent bigdata frameworks.Hands on expertise with streaming and messaging systems such asApache Kafka (publish subscribe architecture), Confluent Cloud,RabbitMQ or Azure Event Hub. Experience creating producers,consumers and topics and integrating them into downstreamprocessing.Deep understanding of relational databases and CDC. Proficiencyin SQL Server, Oracle or other RDBMSs; experience capturingchange events using Debezium or native CDC tools andtransforming them for downstream consumption.Proficiency in programming languages such as Python, Scala orJava and solid knowledge of SQL for data manipulation andtransformation.Cloud platform expertise. Experience with Azure or AWS services fordata storage, compute and orchestration (e.g., ADLS, S3, AzureData Factory, AWS Glue, Airflow, DBX, DLT).Data modelling and warehousing. Knowledge of data Lakehousearchitectures, Delta Lake, partitioning strategies and performanceoptimisation.Version control and DevOps. Familiarity with Git and CI/CDpipelines; ability to automate deployment and manageinfrastructure as code.Strong problem solving and communication skills. Ability to workwith cross functional teams and articulate complex technicalconcepts to nontechnical stakeholders.