Senior Data Engineer

Job Description

The Senior Data Engineer is responsible to implement / execute the Data strategy, by building scalable & optimized ETL/Data Pipelines, deliver high quality data products by implementing Data Quality, transformations across storage layers, collaborating with business teams to translate product requirements into an efficient data model, providing secured data access to BI / Reporting teams. Responsibilities: Ability to build Scalable Data / ETL pipelines in Batch & Realtime patterns for BI / Reporting Platforms Should be able to create automation workflows / Orchestrations for the Data Pipelines Define and enforce best practices for data engineering, including coding standards, source control, and CI/CD processes to minimize technical debt and ensure high-quality code deployments. Should be able to understand different cloud Platforms, preferably on AWS Platform Services for Data & Analytics Should be able to create Design Documents/Mapping documents Should have hands on experience in developing & optimizing Medium to Large Data Warehouses (on any of Cloud Data Warehouses), preferably Snowflake, SQL Server, MongoDB, real time DB's Should be aware of Data Architecture/ Design Patterns in the areas of Data Ingestion / Curation / Data Consumption / Reporting Semantic Models Aware of Data Ingestion Tools, Data Quality Management, Metadata Management, Data Lineage, Data Security Should be able to monitor Audit the logging to troubleshoot issues and provide a fix to reduce business impacts Should work collaboratively with Data Architect in Designing & Developing Reusable utilities, understand & align to Architecture Design principles Should be able to communicate & Collaborate with all stakeholders (Director level, Business Units, other Architects, Product Managers, Scrum Masters Qualifications: Should hold a Bachelors degree of engineering/MCA with good Academic CGPA Should have at least 6-10 years of total IT experience in Software Development, with 2 years exclusively in ETL Design in Data Warehousing projects Snowflake (Stored procedures, Tasks, Streams, Snow pipes, Dynamic Tables) Open Table formats (Iceberg tables) on Snowflake/AWS SQL Server, MongoDB, Data Analysis, SQL Query proficiency Realtime Databases (like Cassandra, DynamoDB) Replication tools (DMS, Qlik Replica) Nice To Have dBT tool Python/Spark Data Governance Processes SSIS Integrations Knowledge on Tableau/Power BI tools Solid understanding of Agile development process and software release processes. Must be a self-starter who is highly organized, hands-on, and a team player. Should be able to independently do Unit testing Should be able to do peer-peer Code reviews, Code walk-through sessions Detail-oriented, with strong written and verbal communication.