Senior Data Engineering Consultant

Primary Responsibilities:

• Prepare detail design for the Solution by discussing with PO and business. Consult with Business, Subject Master Expert, users to evaluate business requirements, evaluate existing or proposed systems, and prepare detailed technical design document from which programs will be written
• Walk-thru the technical design document with Business/SME/QA group and support team
• Prepare test planning & setup test data. Execute test planning based on project requirements
• Ensuring timely deliveries for all assigned tasks
• Participation in Design reviews, Test Case reviews, and Production Support readiness reviews for new releases
• Support QA, UAT and Release phases of the project via defect analysis, change maintenance, etc.
• Adhere to all quality requirements
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• Undergraduate degree or equivalent experience
• 10+ years of experience in Azure,Databricks, datafactory, airfolw Spark/Scala, SQOOP and Kafka. Proficient in coding, testing, implementing, debugging, and documenting the complex Spark Jobs
• Good hands-on experience on setting up Kafka streams and consuming from Kafka streams
• Experience in RDBMS like Oracle, SQL Server, Netezza, DB2 etc.
• Good knowledge in Unix and shell scripting
• Good understanding of databricks, datafactory, Spark Architecture
• Good understanding of Data Architecture and Azure Cloud
• Understanding of QA and testing automation process
• Understanding of the business needs and designs programs and systems that match the complex business requirements and records all the specifications that are involved in the development and coding process
• Proven ability to resolve moderate problems associated with the designed programs and provides technical guidance on complex programming
• Proven ability to be involved in creating proper technical documentation in the work assignments
• Proven ability to ensure that all the standard requirements have been met and is involved in performing the technical analysis
• Proven ability in assisting the project manager by compiling information from the current systems, analyzing the program requirements, and ensuring that it meets the specified time requirements
• Proven ability to work independently
• Proven solid problem solving, analytical kills, Good communication and presentation skills, Good attitude and self-motivated