Principal Data Engineer

Role Overview: As a Data Engineering Lead at Pattern, you will be responsible for leading the design, development, and optimization of enterprise-scale data pipelines and architectures to drive data-driven decision-making. You will architect efficient data tables and schemas to enhance query performance, ensuring seamless integration with cross-functional teams. Key Responsibilities: - Design and implement high-performance data pipelines using ETL processes, batch/streaming frameworks (e.g., Apache Spark, Airflow), and cloud platforms (AWS). - Optimize data ingestion, transformation, and storage workflows to meet scalability, reliability, and latency requirements. - Architect database schemas and dimensional models (e.g., star/snowflake) to maximize query performance and reduce latency. - Implement indexing strategies, partitioning, and materialized views to optimize data retrieval. - Partner with data scientists, product managers, and engineers to align data infrastructure with business needs. - Establish data governance frameworks, ensuring compliance with security, privacy, and quality standards. - Mentor junior engineers and foster best practices in data engineering. - Stay ahead of industry trends (e.g., AI/ML integration, real-time analytics) and advocate for emerging technologies. - Lead technical strategy for data infrastructure, balancing innovation with operational stability. Qualification Required: - 8+ years of experience in data engineering, with a focus on ETL, data modeling, and cloud-based architectures. - Proficiency in SQL, Python, and tools like Spark and Snowflake. - Strong knowledge of database design, dimensional modeling, and query optimization. - Proven ability to lead teams, mentor engineers, and drive technical initiatives. - Experience translating business requirements into scalable technical solutions. Company Details: Pattern is an equal opportunity employer that celebrates diversity and is committed to creating an inclusive environment for all employees. The core values of Pattern include being Data Fanatics, Partner Obsessed, a Team of Doers, and Game Changers. Role Overview: As a Data Engineering Lead at Pattern, you will be responsible for leading the design, development, and optimization of enterprise-scale data pipelines and architectures to drive data-driven decision-making. You will architect efficient data tables and schemas to enhance query performance, ensuring seamless integration with cross-functional teams. Key Responsibilities: - Design and implement high-performance data pipelines using ETL processes, batch/streaming frameworks (e.g., Apache Spark, Airflow), and cloud platforms (AWS). - Optimize data ingestion, transformation, and storage workflows to meet scalability, reliability, and latency requirements. - Architect database schemas and dimensional models (e.g., star/snowflake) to maximize query performance and reduce latency. - Implement indexing strategies, partitioning, and materialized views to optimize data retrieval. - Partner with data scientists, product managers, and engineers to align data infrastructure with business needs. - Establish data governance frameworks, ensuring compliance with security, privacy, and quality standards. - Mentor junior engineers and foster best practices in data engineering. - Stay ahead of industry trends (e.g., AI/ML integration, real-time analytics) and advocate for emerging technologies. - Lead technical strategy for data infrastructure, balancing innovation with operational stability. Qualification Required: - 8+ years of experience in data engineering, with a focus on ETL, data modeling, and cloud-based architectures. - Proficiency in SQL, Python, and tools like Spark and Snowflake. - Strong knowledge of database design, dimensional modeling, and query optimization. - Proven ability to lead teams, mentor engineers, and drive technical initiatives. - Experience translating business requirements into scalable technical solutions. Company Details: Pattern is an equal opportunity employer that celebrates diversity and is committed to creating an inclusive environment for all employees. The core values of Pattern include being Data Fanatics, Partner Obsessed, a Team of Doers, and Game Changers.