SQL Data Engineer

Job Description

Key Skills: Python, Data Engineer, SQL Roles & Responsibilities: Design, develop, test, deploy, maintain and improve software. Develop flowcharts, layouts and documentation to identify requirements & solutions. Write well designed & high-quality testable code. Produce specifications and determine operational feasibility. Integrate software components into fully functional platform. Apply pro-actively & perform hands-on design and implementation of best practice CI/CD. Coaching & mentoring of other Service Team members. Develop/contribute to software verification plans and quality assurance procedures. Document and maintain software functionality. Troubleshoot, debug and upgrade existing systems, including participating in DR tests. Deploy programs and evaluate customer feedback. Contribute to team estimation for delivery and expectation management for scope. Comply with industry standards and regulatory requirements. Ensuring throughout project life cycle, the data component of the solution i.e. data management (data generation, standardisation, Integration, transformation, migration, delivery, etc) is aligned with the Data Architecture pattern/ principles/policies and working closely with the solution designers. Identification and escalation of any potential gaps in the target data architecture, including likely impact of gaps. Responsible for ensuring the Data design will fit for business purpose and deliver expected benefits with minimal risk/data impact to the other business unit across the enterprise, who are users of the specific data. In addition, the data engineer to ensure the design caters for potential future usage of data by customers across the enterprise. Experience Requirments: 3-8 years technical experience (within financial services industry preferred). Technical Domain experience (Subject Matter Expertise in Technology or Tools). Solid experience, knowledge and skills in Data Engineering, BI/software development such as ELT/ETL, data extraction and manipulation in Data Lake/Data Warehouse/Lake House environment. Hands on programming experience in writing Python, SQL, Unix Shell scripts, in a complex enterprise environment. Experience in configuration management using Ansible/Jenkins/GIT. Hands on experience working on relational Databases - Oracle or MS SQL or DB2 (SQL/PLSQL, Database design, Normalisation, Execution plan analysis, Index creation and maintenance, Stored Procedures). Knowledge of Data Modelling techniques and practices used for a Data Warehouse/Data Mart application. Quality engineering development experience (CI/CD - Jenkins). Experience with Source Control Tools - Github or BitBucket. Skilled in querying data from a range of data sources that store structured and unstructured data. Desired Experience: Hands on expertise in SQL and PL/SQL in Oracle pref. 19c. Hands on exposure on Bash/Shell scripting & Control M for job scheduling Experience with Source code control - GitHub, VSTS etc. Experience with UNIX command-line tools for complex file operations Understanding and ability to translate/physicalise Data Models (Star Schema, Data Vault 2.0 etc). Data integration experience in complex multi asset/multi-platform/ distributed technology environments involving data migration strategy, decommission plans, etc. Understanding of data management concepts, Data Architecture Principles, Data Controls, Data Policies and Standards and possible causes of Data risks/impacts in the organisation. Good to Have Exposure to Big Data (Hive-QL/DataBricks/Redshift SQL). Exposure on python scripting. Exposure to Cloud/AWS service including S3, Glue, Athena, Data Pipeline, etc. Knowledge or understanding of Power BI. Essential Capabilities Enthusiasm for technology, keeping up with latest trends. Ability to articulate complex technical issues and desired outcomes of system enhancements. Proven analytical skills and evidence-based decision making. Excellent problem solving, troubleshooting & documentation skills. Strong written and verbal communication skills. Excellent collaboration and interpersonal skills. Strong delivery focus with an active approach to quality and auditability. Ability to work under pressure and excel within a fast-paced environment Ability to self-manage tasks. Agile software development practices. Qualification: UG / PG in Computer Science.