Freelance Senior Data Engineer – Snowflake to Redshift Migration (Python)

Company Description

ThreatXIntel is a startup focused on providing innovative and affordable cyber security services tailored to the needs of businesses and organizations. Specializing in areas such as cloud security, web and mobile security testing, and DevSecOps, our team of professionals works to protect digital assets against potential cyber threats. With a proactive approach to security, we continuously monitor and address vulnerabilities in client environments. Our mission is to deliver exceptional cyber security solutions, enabling organizations to focus on growth while maintaining peace of mind.

Role Description

We are looking for a Senior Data Engineer with strong Python skills to support a large-scale data warehouse migration from Snowflake to AWS Redshift. The consultant will be responsible for schema conversion, data migration, validation, performance optimization, and post-migration stabilization. This role requires deep hands-on experience with AWS migration tools and modern data engineering practices.

Key Responsibilities

• Perform schema conversion from Snowflake to Amazon Redshift using AWS Schema Conversion Tool (SCT)
• Execute data migration using AWS DMS, Redshift COPY commands, and S3-based ingestion patterns
• Validate data type mappings, encoding, and partitioning strategies for Redshift compatibility and performance
• Develop automation scripts for migration, validation, and cutover using Python, SQL, or shell scripting
• Build data validation and reconciliation frameworks to ensure data accuracy and completeness post-migration
• Collaborate with DBT and reporting teams to align transformations, data lineage, and downstream models
• Optimize Amazon Redshift performance including distribution keys, sort keys, compression, and workload management
• Troubleshoot and resolve migration issues related to schemas, data loads, or query performance
• Document migration processes, mappings, validation results, and operational runbooks
• Support post-migration stabilization and performance tuning activities

Required Skills and Experience

• 7+ years of experience in data engineering or database development
• Strong hands-on experience with Snowflake and Amazon Redshift (including RA3 nodes and Redshift Spectrum)
• Proven experience using AWS Schema Conversion Tool (SCT) and AWS DMS
• Strong proficiency in SQL and Python for automation and data engineering tasks
• Solid understanding of data modeling, data warehouse optimization, and performance tuning
• Experience working with DBT and modern data stack tools
• Familiarity with S3-based data pipelines and data validation frameworks
• Experience with Git and CI/CD pipelines
• Strong problem-solving and communication skills

Preferred Skills

• AWS certifications such as Data Analytics Specialty, Database Specialty, or Solutions Architect
• Experience with large-scale data migration or data platform modernization initiatives
• Familiarity with AWS Glue, Lambda, and CloudWatch for integrated data workflows
• Experience with infrastructure-as-code tools such as Terraform or CloudFormation