Sr. Snowflake Data Engineer (with Pyspark)

Role Overview: As a Senior Data Engineer with 10-12 years of experience, you will play a crucial role in building scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake. You will lead the design, implementation, and optimization of application data stores using PostgreSQL, DynamoDB, and advanced SQL. Your strong programming skills in SQL, Python, and PySpark will be essential for efficient data processing. Additionally, you will work with structured (CSV, SQL), semi-structured (JSON, XML), and unstructured (PDF, logs) data formats. Key Responsibilities: - Design and build scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake. - Lead the implementation and optimization of application data stores using PostgreSQL, DynamoDB, and advanced SQL. - Utilize strong programming skills in SQL, Python, and PySpark for efficient data processing. - Work with structured (CSV, SQL), semi-structured (JSON, XML), and unstructured (PDF, logs) data formats. - Design low-latency, high-throughput pipelines using Spark and cloud-native tools. - Implement CI/CD, automation, schema versioning, and enforce data security in production environments. - Ensure adherence to best practices for performance, scalability, and compliance. Qualifications Required: - Proven expertise in building scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake. - Strong programming skills in SQL, Python, and PySpark for efficient data processing. - Deep knowledge of Azure and AWS data ecosystems, including Microsoft Fabric and distributed computing frameworks. Additional Details: N/A,