Senior Data Engineer

Company Description

Speridian Technologies is a global business and technology solutions provider specializing in Digital Transformation to help clients modernize their businesses. Headquartered in Albuquerque, NM, with regional offices across the globe, Speridian has been a leader in CRM, ERP, HR, and CX Innovation since 2003. We partner with industry leaders such as Oracle, Microsoft, Salesforce, Amazon, UiPath, Evolution AI, and Adobe to deliver Next-Gen Solutions. Our proven track record is built on a unique, industry-based 3-pillar approach: business focus, technology-fueled innovation, and delivery excellence. Speridian has a global talent pool, with a presence in the Americas, Middle East, Singapore, India, and Pakistan.

Role Description

Senior Data Engineer

Education

•Degree, Post graduate in Computer Science or related field (or equivalent industry experience)

Experience

Minimum 4+ years of development and design experience in Java/Scala with Flink, Beam (or spark streaming) and Kafka.

Experience in RTS

Extensive coding experience and knowledge in Event driven and streaming architecture

Experience in JVM tuning for performance

Knowledge on Containerisation using Docker and Kubernetes.

Working knowledge on Caching systems, with particular experience using Redis will be preferable. [Nice to have]

Linux OS configuration and use, including shell scripting.

Good hands on experience with design patterns and their implementation.

Well versed with CI/CD principles (GitHub, Jenkins etc.), and actively involved in solving, troubleshooting issues in distributed services ecosystem

Experience working with SQL and NoSQL databases

Familiar with Distributed services resiliency and monitoring in a production environment.

Experience in designing, building, testing and implementing security systems – including identifying security design gaps in existing and proposed architectures and recommend changes or enhancements.

Responsible for adhering to established policies, following best practices, developing and possessing an in-depth understanding of exploits and vulnerabilities, resolving issues by taking the appropriate corrective action.

Knowledge on security controls designing Source and Data Transfers including CRON, ETLs, and JDBC-ODBC scripts.

Understand basics of Networking including DNS, Proxy, ACL, Policy and troubleshooting

High level knowledge of compliance and regulatory requirements of data including but not limited to encryption, anonymization, data integrity, policy control features in large scale infrastructures

Understand data sensitivity in terms of logging, events and in memory data storage– such as no card numbers or personally identifiable data in logs.

Implements wrapper solutions for new/existing components with no/minimal security controls to ensure compliance to bank standards.

Experience in Agile methodology.