Data Engineer (AWS MSK Specialist)

As a Data Engineer specializing in AWS Managed Streaming for Apache Kafka (MSK), your role will involve designing, building, and maintaining high-performance data pipelines. Working in collaboration with data architects and software engineers, your focus will be on ensuring seamless data integration and streaming capabilities using AWS MSK. Your key responsibilities will include: - Designing, developing, and managing real-time data pipelines utilizing AWS MSK - Integrating AWS MSK with various data sources and consumers - Monitoring and optimizing AWS MSK cluster performance and costs - Collaborating with DevOps to ensure optimal performance and uptime - Implementing security best practices for data streaming and storage - Documenting technical processes and best practices - Utilizing AWS services like S3, Glue, and Lambda for data processing and integration To excel in this role, you should meet the following qualifications: - Hold a Bachelor's degree in computer science, Engineering, or a related field - Possess 3-5 years of experience working with AWS MSK or Apache Kafka in a production environment - Demonstrate a strong understanding of distributed systems, real-time data processing, and cloud-based data streaming - Have proficiency in programming languages such as Python - Exhibit experience with AWS services like S3, Glue, and Lambda - Showcase excellent problem-solving and debugging skills Additionally, the following skills would be advantageous: - Experience with containerization technologies like Docker and Kubernetes - Familiarity with other messaging systems like RabbitMQ or Apache Pulsar - Experience with data serialization formats like Avro, Protobuf, or JSON - Knowledge of Snowflake or other cloud-based data warehousing solutions Joining our team will provide you with the opportunity to work in a dynamic environment where your expertise in AWS MSK and data engineering will be valued and further developed.,