This is a
data engineer position
- a programmer responsible for the design, development implementation and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable, and secure manner in coordination with the Data & Analytics team.The overall objective is defining optimal solutions to data collection, processing, and warehousing. Must be a Spark Java development expertise in big data processing, Python and Apache spark particularly within banking & finance domain. He/She designs, codes and tests data systems and works on implementing those into the internal infrastructure.
Responsibilities:
- Ensuring high quality software development, with complete documentation and traceability
- Develop and optimize scalable Spark Java-based data pipelines for processing and analyzing large scale financial data
- Design and implement distributed computing solutions for risk modeling, pricing and regulatory compliance
- Ensure efficient data storage and retrieval using Big Data
- Implement best practices for spark performance tuning including partition, caching and memory management
- Maintain high code quality through testing, CI/CD pipelines and version control (Git, Jenkins)
- Work on batch processing frameworks for Market risk analytics
- Promoting unit/functional testing and code inspection processes
- Work with business stakeholders and Business Analysts to understand the requirements
- Work with other data scientists to understand and interpret complex datasets
Qualifications:
- 5- 8 Years of experience in working in data eco systems.
- 4-5 years of hands-on experience in Hadoop, Scala, Java, Spark, Hive, Kafka, Impala, Unix Scripting and other Big data frameworks.
- 3+ years of experience with relational SQL and NoSQL databases: Oracle, MongoDB, HBase
- Strong proficiency in Python and Spark Java with knowledge of core spark concepts (RDDs, Dataframes, Spark Streaming, etc) and Scala and SQL
- Data Integration, Migration & Large Scale ETL experience (Common ETL platforms such as PySpark/DataStage/AbInitio etc.) - ETL design & build, handling, reconciliation and normalization
- Data Modeling experience (OLAP, OLTP, Logical/Physical Modeling, Normalization, knowledge on performance tuning)
- Experienced in working with large and multiple datasets and data warehouses
- Experience building and optimizing ‘big data’ data pipelines, architectures, and datasets.
- Strong analytic skills and experience working with unstructured datasets
- Ability to effectively use complex analytical, interpretive, and problem-solving techniques
- Experience with Confluent Kafka, Redhat JBPM, CI/CD build pipelines and toolchain – Git, BitBucket, Jira
- Experience with external cloud platform such as OpenShift, AWS & GCP
- Experience with container technologies (Docker, Pivotal Cloud Foundry) and supporting frameworks (Kubernetes, OpenShift, Mesos)
- Experienced in integrating search solution with middleware & distributed messaging - Kafka
- Highly effective interpersonal and communication skills with tech/non-tech stakeholders.
- Experienced in software development life cycle and good problem-solving skills.
- Excellent problem-solving skills and strong mathematical and analytical mindset
- Ability to work in a fast-paced financial environment
Education:
- Bachelor’s/University degree or equivalent experience in computer science, engineering, or similar domain
------------------------------------------------------
Job Family Group:
Technology------------------------------------------------------
Job Family:
Data Architecture------------------------------------------------------
Time Type:
Full time------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
