Officer - Data Engineer - C11 - Hybrid - CHENNAI

This is a data engineer position - a programmer responsible for the design, development implementation and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable, and secure manner in coordination with the Data & Analytics team.

The overall objective is defining optimal solutions to data collection, processing, and warehousing. Must be a Spark Java development expertise in big data processing, Python and Apache spark particularly within banking & finance domain. He/She designs, codes and tests data systems and works on implementing those into the internal infrastructure.

Responsibilities:

• Ensuring high quality software development, with complete documentation and traceability
• Develop and optimize scalable Spark Java-based data pipelines for processing and analyzing large scale financial data
• Design and implement distributed computing solutions for risk modeling, pricing and regulatory compliance
• Ensure efficient data storage and retrieval using Big Data
• Implement best practices for spark performance tuning including partition, caching and memory management
• Maintain high code quality through testing, CI/CD pipelines and version control (Git, Jenkins)
• Work on batch processing frameworks for Market risk analytics
• Promoting unit/functional testing and code inspection processes
• Work with business stakeholders and Business Analysts to understand the requirements
• Work with other data scientists to understand and interpret complex datasets

Qualifications:

• 5- 8 Years of experience in working in data eco systems.
• 4-5 years of hands-on experience in Hadoop, Scala, Java, Spark, Hive, Kafka, Impala, Unix Scripting and other Big data frameworks.
• 3+ years of experience with relational SQL and NoSQL databases: Oracle, MongoDB, HBase
• Strong proficiency in Python and Spark Java with knowledge of core spark concepts (RDDs, Dataframes, Spark Streaming, etc) and Scala and SQL
• Data Integration, Migration & Large Scale ETL experience (Common ETL platforms such as PySpark/DataStage/AbInitio etc.) - ETL design & build, handling, reconciliation and normalization
• Data Modeling experience (OLAP, OLTP, Logical/Physical Modeling, Normalization, knowledge on performance tuning)
• Experienced in working with large and multiple datasets and data warehouses
• Experience building and optimizing ‘big data’ data pipelines, architectures, and datasets.
• Strong analytic skills and experience working with unstructured datasets
• Ability to effectively use complex analytical, interpretive, and problem-solving techniques
• Experience with Confluent Kafka, Redhat JBPM, CI/CD build pipelines and toolchain – Git, BitBucket, Jira
• Experience with external cloud platform such as OpenShift, AWS & GCP
• Experience with container technologies (Docker, Pivotal Cloud Foundry) and supporting frameworks (Kubernetes, OpenShift, Mesos)
• Experienced in integrating search solution with middleware & distributed messaging - Kafka
• Highly effective interpersonal and communication skills with tech/non-tech stakeholders.
• Experienced in software development life cycle and good problem-solving skills.
• Excellent problem-solving skills and strong mathematical and analytical mindset
• Ability to work in a fast-paced financial environment

Education:

• Bachelor’s/University degree or equivalent experience in computer science, engineering, or similar domain

-

Job Family Group:

-

Job Family:

-

Time Type:

-

Most Relevant Skills

-

Other Relevant Skills

-

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.