AWS Data Architect

We are looking for a highly skilled Big Data Architect with expertise in AWS, Kafka, Debezium, and Spark. This role presents an exciting opportunity to play a critical role in optimizing query performance, data synchronization, disaster recovery solutions, and simplifying reporting workflows. The ideal candidate should have hands-on experience with a variety of AWS-native services, big data processing frameworks, and CI/CD integrations to drive impactful system and performance enhancements. Key Responsibilities: - **Query Optimization**: Enhance performance and responsiveness by optimizing complex queries for OE dashboards and OE activity dashboards. Collaborate with database teams to fine-tune Oracle and SQL queries for optimal system performance. - **Data Synchronization & Streaming (Debezium/Kafka)**: Troubleshoot and resolve data synchronization issues within the Data Lake environment. Utilize Debezium and Kafka to implement efficient solutions for real-time data streaming and ensure data integrity. - **Spark Batch Jobs Performance Tuning**: Focus on optimizing Spark batch jobs to reduce execution time significantly through performance tuning and best practices. - **Disaster Recovery (DR) Solution Design & Implementation on AWS**: Implement disaster recovery strategies using AWS-native technologies like DMS, Kinesis, and other streaming solutions. Design and deploy a resilient DR solution for business continuity. - **System Enhancements & Minor Improvements**: Identify opportunities for minor adjustments to improve system performance. Implement solutions to increase efficiency and reduce bottlenecks. - **Reporting Simplification & CI/CD Integration**: Migrate reporting solutions to AWS Athena for improved performance and scalability. Optimize Athena implementation in DEV/CIT AWS environments for maximum efficiency. Replace legacy CI/CD jobs related to Dremio with Athena for streamlined operations. Required Skills & Qualifications: - 12+ years of experience in Big Data architecture and engineering - Extensive expertise in AWS services such as DMS, Kinesis, Athena, Glue, Lambda, S3, EMR, Redshift, etc. - Hands-on experience with Debezium and Kafka for real-time data streaming - Expertise in Spark optimization and strong SQL and Oracle query optimization skills - Experience with Big Data frameworks like Hadoop, Spark, Hive, Presto, and Athena - Proven background in CI/CD automation and integrating AWS services with DevOps pipelines - Exceptional problem-solving abilities and the capacity to work effectively in an Agile environment Preferred Skills (Nice to Have): - Experience with migrating from Dremio to Athena for streamlined data processing - Familiarity with cloud-native DR solutions on AWS - Strong analytical skills and the ability to document and implement performance improvements Skills Required: Data Architecture, AWS, Spark, SQL About UST: UST is a global digital transformation solutions provider that partners with clients from design to operation, embedding innovation and agility into their organizations. With over 30,000 employees in 30 countries, UST aims to make a real impact through transformation, touching billions of lives in the process.,