Data Engineer

Project Role : Data Engineer
Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills : Data Engineering
Good to have skills : NA
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary: In this role, you will work to increase the domain data coverage and adoption of the Data Platform by promoting a connected user experience through data. You will increase data literacy and trust by leading our Data Governance and Master Data Management initiatives. You will contribute to the vision and roadmap of self-serve capabilities through the Data Platform. The senior data engineer develops data pipelines extracting and transforming data as governed assets into the data platform, improves system quality by identifying issues and common patterns and developing standard operating procedures; and enhances applications by identifying opportunities for improvement, making recommendations, and designing and implementing systems. Roles and responsibilities: • (MUST HAVE) Extensive experience with cloud data warehouse like Snowflake, AWS Athena, and SQL databases like PostgreSQL, MS SQL Server. Experience with NoSQL databases like AWS DynamoDB and Azure Cosmos is a plus. • (MUST HAVE) Solid experience and clear understanding of DBT. • (MUST HAVE) Experience working with AWS and/or Azure CI/CD DevOps technologies, and extensive debugging experience. • Good understanding of data modeling, ETL, data curation, and big data performance tuning. • Experience with data ingestion tools like Fivetran is a big plus. • Experience with Data Quality and Observability tools like Monte Carlo is a big plus. • Experience working and integrating with Event Bus like Pulsar is a big plus. • Experience integrating with a Data Catalog like Atlan is a big plus. • Experience with Business Intelligence tools like PowerBI is a plus. • An understanding of unit testing, test driven development, functional testing, and performance • Knowledge of at least one shell scripting language. • Ability to network with key contacts outside own area of expertise. • Must possess strong interpersonal, organizational, presentation and facilitation skills. • Must be results oriented and customer focused. • Must possess good organizational skills. Technical experience & Professional attributes: • Prepare technical design specifications based on functional requirements and analysis documents. • Implement, test, maintain and support software, based on technical design specifications. • Improve system quality by identifying issues and common patterns and developing standard operating procedures. • Enhance applications by identifying opportunities for improvement, making recommendations, and designing and implementing systems. • Maintain and improve existing codebases and peer review code changes. • Liaise with colleagues to implement technical designs. • Investigating and using new technologies where relevant • Provide written knowledge transfer material. • Review functional requirements, analysis, and design documents and provide feedback. • Assist customer support with technical problems and questions. • Ability to work independently with wide latitude for independent decision making. • Experience in leading the work of others and mentor less experienced developers in the context of a project is a plus. • Ability to listen and understand information and communicate the same. • Participate in architecture and code reviews. • Lead or participate in other projects or duties as need arises. Education qualifications: • Bachelor’s degree in computer science, Information Systems, or related field; or equivalent combination of education/experience. Master’s degree is a plus. • 5 years or more of extensive experience developing mission critical and low latency solutions. • At least 3 years of experience with developing and debugging distributed systems and data pipelines in the cloud. Additional Information: The Winning Way behaviors that all employees need in order to meet the expectations of each other, our customers, and our partners. • Communicate with Clarity - Be clear, concise and actionable. Be relentlessly constructive. Seek and provide meaningful feedback. • Act with Urgency - Adopt an agile mentality - frequent iterations, improved speed, resilience. 80/20 rule – better is the enemy of done. Don’t spend hours when minutes are enough. • Work with Purpose - Exhibit a “We Can” mindset. Results outweigh effort. Everyone understands how their role contributes. Set aside personal objectives for team results. • Drive to Decision - Cut the swirl with defined deadlines and decision points. Be clear on individual accountability and decision authority. Guided by a commitment to and accountability for customer outcomes. • Own the Outcome - Defined milestones, commitments and intended results. Assess your work in context, if you’re unsure, ask. Demonstrate unwavering support for decisions. COMMENTS: The above statements are intended to describe the general nature and level of work being performed by individuals in this position. Other functions may be assigned, and management retains the right to add or change the duties at any time.