Senior Data Engineer

Essential for this role:
Education and Qualifications: • Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent. • Minimum seven plus years of experience in data analytics field • Experience with Azure/AWS Databricks • Experience in building and optimizing data pipelines, architectures and data sets • Excellent experience in Scala or Python, PySpark and SQL • Ability to troubleshoot and optimize complex queries on the Spark platform • Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques • Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS • Hands on experience in performance tuning and optimizing code running in Databricks environment • Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment Technical / Professional Skills : • Excellent communication skills / should interact directly with the customer • Azure/AWS Databricks • Python / Scala / Spark / PySpark • HIVE / HBase / Impala / Parquet • Sqoop, Kafka, Flume • Very Strong SQL and RDBMS • Airflow • Jenkins / Bamboo • Github / Bitbucket • Nexus • Knowledge on Delta Live Tables (DLT) will be an added advantage Good to have: • Relevant accredited certifications for Azure, AWS, Cloud Engineering, and/or Databricks • Data Engineering expertise in Big Data Management using Cloudera, Python, Hive, Scala, Data Warehouse, Data Lake, AWS, Azure