Sr. Snowflake Data Engineer (Pyspark)

As a Senior Data Engineer with 10-12 years of experience, your role will involve leading the design, implementation, and optimization of application data stores using various technologies such as PostgreSQL, DynamoDB, and advanced SQL. Your expertise in building scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake will be crucial for the success of the projects. Key Responsibilities: - Build scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake. - Lead the design, implementation, and optimization of application data stores using PostgreSQL, DynamoDB, and advanced SQL. - Utilize strong programming skills in SQL, Python, and PySpark for efficient data processing. - Work with structured (CSV, SQL), semi-structured (JSON, XML), and unstructured (PDF, logs) data formats. - Design low-latency, high-throughput pipelines using Spark and cloud-native tools. - Implement CI/CD, automation, schema versioning, and enforce data security in production environments. - Ensure adherence to best practices for performance, scalability, and compliance. Qualifications Required: - Proven expertise in building scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake. - Strong programming skills in SQL, Python, and PySpark. - Deep knowledge of Azure and AWS data ecosystems. - Experience working with structured, semi-structured, and unstructured data formats. - Familiarity with Microsoft Fabric and distributed computing frameworks. This job offers an opportunity to work with cutting-edge technologies in a dynamic environment where you can contribute to the success of the projects by implementing best practices for performance, scalability, and compliance.,