Senior Data Engineering Lead

Caring. Connecting. Growing together.

Functions may include database architecture, engineering, design, optimization, security, and administration as well as data modeling, big data development, Extract, Transform, and Load (ETL) development, storage engineering, data warehousing, data provisioning and other similar roles. Responsibilities may include Platform-as-a-Service and Cloud solution with a focus on data stores and associated eco systems. Duties may include management of design services, providing sizing and configuration assistance, ensuring strict data quality, and performing needs assessments. Analyzes current business practices, processes and procedures as well as identifying future business opportunities for leveraging data storage and retrieval system capabilities. Manages relationships with software and hardware vendors to understand the potential architectural impact of different vendor strategies and data acquisition. May design schemas, write SQL or other data markup scripting and helps to support development of Analytics and Applications that build on top of data. Selects, develops and evaluates personnel to ensure the efficient operation of the function. Generally, work is self-directed and not prescribed.

Primary Responsibilities:

• Come up with architecture and design on various aspects like extensibility, scalability, security, design patterns etc., against a predefined checklist and ensure that all relevant best practices are followed
• Execute POCs to make sure that suggested design/technologies meet the requirements
• Possess/acquire strong troubleshooting skills and be interested in performing. Troubleshooting of issues in different desperate technologies and environments.
• Architecting with modern technology stack and Designing Public Cloud Application leveraging in Azure
• Migrating existing pipelines in BDPass to Azure Cloud
• Works with less structured, more complex issues
• Serves as a resource to others
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• Graduate degree or equivalent experience
• Experience with big data technologies, such as Apache Hadoop and Apache Spark
• Experience with cloud computing platforms, such as Amazon Web Services (AWS) and Microsoft Azure
• Experience with data security and privacy concerns
• Solid programming skills, particularly in languages commonly used for data engineering, such as Python, Java, and SQL
• Knowledge of data warehousing principles and best practices
• Familiarity with data modeling and data management techniques
• Familiarity with software development principles and agile methodologies
• Proven solid problem-solving and critical thinking abilities
• Proven excellent communication and collaboration skills