Job Overview
We are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multi-disciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition, transformation, loading and processing of data from a multitude of disparate data sources, including structured and unstructured data for advanced analytics and machine learning in a big data environment.
Responsibilities:
- Engineer a modern data pipeline to collect, organize, and process data from disparate sources.
- Performs data management tasks, such as conduct data profiling, assess data quality, and write SQL queries to extract and integrate data
- Develop efficient data collection systems and sound strategies for getting quality data from different sources
- Consume and analyze data from the data pool to support inference, prediction and recommendation of actionable insights to support business growth.
- Design and develop ETL processes using tools and scripting. Troubleshoot and debug ETL processes. Performance tuning and opitimization of the ETL processes.
- Provide support to new of existing applications while recommending best practices and leading projects to implement new functionality.
- Collaborate in design reviews and code reviews to ensure standards are met. Recommend new standards for visualizations.
- Learn and develop new ETL techniques as required to keep up with the contemporary technologies.
- Reviews the solution requirements and architecture to ensure selection of appropriate technology, efficient use of resources and integration of multiple systems and technology.
- Support presentations to Customers and Partners
- Advising on new technology trends and possible adoption to maintain competitive advantage
Experience Needed:
- 8+ years of related experience is required.
- A BS or Masters degree in Computer Science or related technical discipline is required
- ETL experience with data integration to support data marts, extracts and reporting
- Experience connecting to varied data sources
- Excellent SQL coding experience with performance optimization for data queries.
- Understands different data models like normalized, de-normalied, stars, and snowflake models. Worked with transactional, temporarl, time series, and structured and unstructured data.
- Experience on Azure Data Factory and Azure Synapse Analytics
- Worked in big data environments, cloud data stores, different RDBMS and OLAP solutions.
- Experience in cloud-based ETL development processes.
- Experience in deployment and maintenance of ETL Jobs.
- Is familiar with the principles and practices involved in development and maintenance of software solutions and architectures and in service delivery.
- Has strong technical background and remains evergreen with technology and industry developments.
- At least 3 years of demonstrated success in software engineering, release engineering, and/or configuration management.
- Highly skilled in scripting languages like PowerShell.
- Substantial experience in the implementation and exectuion fo CI/CD processes.
Additional Requirements
- Demonstrated ability to have successfully completed multiple, complex technical projects
- Prior experience with application delivery using an Onshore/Offshore model
- Experience with business processes across multiple Master data domains in a services based company
- Demonstrates a rational and organized approach to the tasks undertaken and an awareness of the need to achieve quality.
- Demonstrates high standards of professional behavior in dealings with clients, colleagues and staff.
- Is able to make sound and far reaching decisions alone on major issues and to take full responsibility for them on a technical basis.
- Strong written communication skills. Is effective and persuasive in both written and oral communication.
- Experience with gathering end user requirements and writing technical documentation
- Time management and multitasking skills to effectively meet deadlines under time-to-market pressure
