Job
Description
Role Overview: We are seeking a highly skilled and motivated Senior Data Engineer to join our growing data team. In this role, you will be responsible for designing, building, and maintaining scalable and robust data pipelines and infrastructure, leveraging Snowflake, DBT, and cloud platforms (AWS, Azure, or GCP). You will play a crucial role in enabling data-driven decision-making by ensuring the availability, quality, and performance of our data assets. Responsibilities: Data Pipeline Development and Management: Design, develop, and maintain efficient and scalable data pipelines for ingesting, transforming, and loading data from various sources into Snowflake. Implement data quality checks and monitoring to ensure data accuracy and reliability. Optimize data pipelines for performance and cost-effectiveness. Automate data pipeline deployments and monitoring using CI/CD principles. Snowflake Expertise: Develop and optimize Snowflake schemas, tables, and views for efficient data storage and retrieval. Implement Snowflake best practices for data warehousing and data lake architectures. Optimize Snowflake queries and performance tuning. Manage Snowflake security and access controls. DBT Implementation: Design and implement data transformations using DBT for data modeling and analytics. Develop and maintain DBT models, tests, and documentation. Implement data quality checks and validation using DBT. Utilize DBT for data lineage and dependency management. Cloud Platform Expertise (AWS, Azure, or GCP): Leverage cloud services for data storage, processing, and infrastructure management. Design and implement cloud-based data solutions using services such as [Specific cloud services relevant to your company, e.g., AWS S3, Azure Data Lake Storage, GCP Cloud Storage, AWS Glue, Azure Data Factory, GCP Dataflow]. Manage cloud infrastructure and resources for data pipelines and warehousing. Implement cloud security best practices. Collaboration and Communication: Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver solutions. Communicate technical concepts and solutions effectively to both technical and non-technical audiences. Participate in code reviews and contribute to team knowledge sharing. Document data pipelines and data models. Performance and Optimization: Monitor pipeline performance and implement optimizations where necessary. Identify and resolve bottlenecks within the data infrastructure. Implement best practices for data storage and retrieval to increase efficiency. Required Skills and Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 5+ years of experience as a Data Engineer. Strong proficiency in SQL and data warehousing concepts. Extensive experience with Snowflake. Strong experience with DBT for data transformation and modeling. Experience with at least one major cloud platform (AWS, Azure, or GCP). Experience with data pipeline development and automation. Proficiency in scripting languages such as Python. Experience with version control systems (e.g., Git). Strong problem-solving and analytical skills. Excellent communication and collaboration skills. Experience with CI/CD pipelines.
