Cloud Engineer

Job Description

Location – Greater Noida Mandatory Skills – AWS, Snowflake, Python/Pyspark, SQL, ETL, Datawarehousing Desired Skills – Data Modelling, Dimensional Modelling Detailed JD – Must be extremely proficient in Data Warehouse ETL Design/Architecture , dimensional/relational data modelling. Experience in atleast one ETL development project, writing/analyzing complex stored procedures. Should have entry level/intermediate experience in Python/PySpark – working knowledge on spark/pandas dataframe, spark multi-threading, exception handling, familiarity with different boto3 libraries, data transformation and ingestion methods, ability to write UDF. Snowflake – Familiarity with stages and external tables, commands in snowflake like copy, unload data to/from S3, working knowledge of variant data type, flattening nested structure thru SQL, familiarity with marketplace integrations, role-based masking, pipes, data cloning, logs, user and role management is nice to have . Familiarity with Coalesce/dbt is an added advantage for this job. Collibra integration experience for Data Quality and Governance in ETL pipeline development is nice to have. AWS – Should have hands-on experience with S3, Glue (jobs, triggers, workflow, catalog, connectors, crawlers), CloudWatch, RDS and secrets manager. AWS - VPC, IAM, Lambda, SNS, SQS, MWAA is nice to have . Should have hands-on experience with version controlling tools like github , working knowledge on configuring, setting up CI/CD pipelines using yaml, pip files. Streaming Services – Familiarity with Confluent Kafka or spark streaming, or Kinesis (or equivalent) is nice to have . Data Vault 2.0 (hubs satellite links) experience will be a PLUS. Interpersonal Highly proficient in Publisher, PowerPoint, SharePoint, Visio, Confluence and Azure DevOps Working knowledge of best practices in value-driven development (requirements management, prototyping, hypothesis-driven development, usability testing) Good communicator with problem solving mindset and focus on process improvement Consistently demonstrates clear and concise written and verbal communication skills Good interpersonal skills, ability to interact with Senior Management Highly self-motivated with a strong sense of initiative Excellent multitasking skills and task management strategies Ability to work well in a team environment, meet deadlines, demonstrate good time management, and multi-task in a fast-paced project environment. Responsibilities Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT counterparts (onshore and offshore) Highly proficient in the architecture and development of an event driven data warehouse; streaming, batch, data modeling, and storage Advanced database knowledge; creating/optimizing SQL queries, stored procedures, functions, partitioning data, indexing, and reading execution plans Skilled experience in writing and troubleshooting Python/PySpark scripts to generate extracts, cleanse, conform and deliver data for consumption Expert level of understanding and implementing ETL architecture; data profiling, process flow, metric logging, and error handling Support continuous improvement by investigating and presenting alternatives to processes and technologies to an architectural review board Develop and ensure adherence to published system architectural decisions and development standards Multi-task across several ongoing projects and daily duties of varying priorities as required Interact with global technical teams to communicate business requirements and collaboratively build data solutions Show more Show less