Principal Consultant - Sr. Developer - DBT+Snowflake

Sr.Data

In this role, the Sr.Data Engineer is responsible for providing technical direction and lead a group of one or more developer to address a goal.

• Develop, implement, and optimize data pipelines using Snowflake, with a focus on Cortex AI capabilities.

• Extract, transform, and load (ETL) data from various sources into Snowflake, ensuring data integrity and accuracy.

• Implement Conversational AI solutions using Snowflake Cortex AI to facilitate data interaction through ChatBot agents.

• Collaborate with data scientists and AI developers to integrate predictive analytics and AI models into data workflows.

• Monitor and troubleshoot data pipelines to resolve data discrepancies and optimize performance.

• Utilize Snowflake%27s advanced features, including Snowpark, Streams, and Tasks, to enable data processing and analysis.

• Develop and maintain data documentation, best practices, and data governance protocols.

• Ensure data security, privacy, and compliance with organizational and regulatory guidelines.

Roles and Responsibilities:

• . Bachelor's degree in Computer Science, Data Engineering, or a related field.

• .experience in data engineering, with experience working with Snowflake.

• . Proven experience in Snowflake Cortex AI, focusing on data extraction, chatbot development, and Conversational AI.

• . Strong proficiency in SQL, Python, and data modeling.

• . Experience with data integration tools (e.g., Matillion, Talend, Informatica).

• . Knowledge of cloud platforms such as AWS, Azure, or GCP.

• . Excellent problem-solving skills, with a focus on data quality and performance optimization.

• . Strong communication skills and the ability to work effectively in a cross-functional team.

• Proficiency in using DBT%27s testing and documentation features to ensure the accuracy and reliability of data transformations.

• Understanding of data lineage and metadata management concepts, and ability to track and document data transformations using DBT%27s lineage capabilities.

• Understanding of software engineering best practices and ability to apply these principles to DBT development, including version control, code reviews, and automated testing.

• Should have experience building data ingestion pipeline.

• Should have experience with Snowflake utilities such as SnowSQL, SnowPipe, bulk copy, Snowpark, tables, Tasks, Streams, Time travel, Cloning, Optimizer, Metadata Manager, data sharing, stored procedures and UDFs, Snowsight.

• Should have good experience in implementing CDC or SCD type 2

• Proficiency in working with Airflow or other workflow management tools for scheduling and managing ETL jobs.

• Good to have experience in repository tools like Github/Gitlab, Azure repo

Qualifications/Minimum qualifications

• B.E./ Masters in Computer Science, Information technology, or Computer engineering or any equivalent degree with good IT experience and relevant of working experience as a Sr. Data Engineer with DBT+Snowflake skillsets

• Skill Matrix:

• DBT (Core or Cloud), Snowflake, AWS/Azure, SQL, ETL concepts, Airflow or any orchestration tools, Data Warehousing concepts