Senior Data Engineer

You are a talented and experienced software development consultant specializing in Agile Delivery, working closely with the client to add real value to their products. In this role, you will work as a polyglot alongside great, like-minded individuals, continuously learning new technologies and implementing best practices. As a Data Engineer with proven senior-level experience, you will be responsible for creating data pipelines for ingest, transformation, and storage of data at scale, as well as setting up analytics clusters. Your main responsibilities will include collaborating with the client and the team to determine data processing and access needs, creating and supporting highly available data pipelines and storage solutions, and delivering software using agile practices such as pair programming, TDD, and CI/CD. You will also be involved in automating data infrastructure and deployments, advocating agile practices to the client organization, mentoring client team members, and enhancing the team and clients" data capabilities. To excel in this role, you should have significant experience in working on data pipelines, platforms, and projects, as well as developing solutions in one or more of the main Cloud providers such as AWS, GCP, or Azure. A strong background in at least one programming language, preferably Python, is essential. Additionally, you should be proficient in working with various databases, data warehouses, and distributed file systems in production environments, and have expertise in SQL, data modeling, and data processing at scale. Knowledge of GDPR and data security, as well as experience with source control and infrastructure as code tools like Git, Github, Terraform, and Ansible, are also required. Moreover, you should be able to support data scientists or business intelligence teams in data preparation, mentoring, and support functions, and possess excellent collaboration and communication skills to effectively work with globally distributed teams, even in remote settings. If you are passionate about leveraging your skills and expertise to drive impactful data solutions and contribute to the success of the client organization, this role offers a rewarding opportunity for professional growth and development. For more information, please reach out to rajeshwari.vh@careerxperts.com.,