Senior AWS Data Engineer

Job Description: You are sought after for the position of Senior AWS Data Engineer, requiring someone with a robust background in data engineering principles and hands-on proficiency in constructing scalable data pipelines and assets. Your expertise in Python, PySpark, and AWS services, coupled with a keen eye for performance optimization and contemporary software engineering practices, will be instrumental in this role. Key Responsibilities: - Design, develop, and optimize data pipelines utilizing PySpark and Spark SQL. - Refactor legacy codebases to enhance clarity, maintainability, and performance. - Implement test-driven development (TDD) principles by crafting unit tests to ensure the reliability of your code. - Troubleshoot intricate issues, such as performance bottlenecks, concurrency hurdles, and logical errors. - Leverage Python best practices, libraries, and frameworks effectively. - Manage code versioning and artifact repositories, employing tools like Git and JFrog Artifactory. - Collaborate with various AWS services like S3, EC2, Lambda, Redshift, CloudFormation, etc. - Offer architectural insights and articulate the advantages and trade-offs of AWS components. Requirements: - You must possess 78 years of practical experience with PySpark and Python, including Boto3 and relevant libraries. - Demonstrated capability in crafting clean, optimized, and scalable code in Spark SQL and PySpark. - Sound grasp of version control systems, preferably Git, and familiarity with JFrog Artifactory. - Profound understanding of AWS services and architecture, with the ability to elucidate service benefits and use cases. - Background in modernizing legacy systems and implementing clean code practices. - Strong debugging skills, enabling you to identify and rectify complex bugs and performance issues. - Familiarity with TDD principles and composing comprehensive unit tests.,