2 Spark Java Jobs

This is a data engineer position where you will be responsible for the design, development, implementation, and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable, and secure manner in coordination with the Data & Analytics team. Your overall objective will be to define optimal solutions to data collection, processing, and warehousing. You must have expertise in Spark Java development for big data processing, as well as proficiency in Python and Apache Spark, particularly within the banking & finance domain. Your role will involve designing, coding, and testing data systems, and implementing them into the internal infrastructure. Responsibilities: - Ensure high-quality software development with complete documentation and traceability - Develop and optimize scalable Spark Java-based data pipelines for processing and analyzing large-scale financial data - Design and implement distributed computing solutions for risk modeling, pricing, and regulatory compliance - Ensure efficient data storage and retrieval using Big Data - Implement best practices for Spark performance tuning including partition, caching, and memory management - Maintain high code quality through testing, CI/CD pipelines, and version control (Git, Jenkins) - Work on batch processing frameworks for Market risk analytics - Promote unit/functional testing and code inspection processes - Collaborate with business stakeholders and Business Analysts to understand the requirements - Work with other data scientists to understand and interpret complex datasets Qualifications: - 5-8 years of experience in working in data ecosystems - 4-5 years of hands-on experience in Hadoop, Scala, Java, Spark, Hive, Kafka, Impala, Unix Scripting, and other Big data frameworks - 3+ years of experience with relational SQL and NoSQL databases: Oracle, MongoDB, HBase - Strong proficiency in Python and Spark Java with knowledge of core spark concepts (RDDs, Dataframes, Spark Streaming, etc.), Scala, and SQL - Data Integration, Migration & Large Scale ETL experience - Data Modeling experience - Experience working with large and multiple datasets and data warehouses - Experience building and optimizing big data pipelines, architectures, and datasets - Strong analytic skills and experience working with unstructured datasets - Experience with Confluent Kafka, Redhat JBPM, CI/CD build pipelines, and toolchain - Experience with external cloud platforms such as OpenShift, AWS & GCP - Experience with container technologies and supporting frameworks - Experience in integrating search solutions with middleware & distributed messaging - Kafka - Excellent interpersonal and communication skills with tech/non-tech stakeholders - Experience in software development life cycle and good problem-solving skills - Strong mathematical and analytical mindset - Ability to work in a fast-paced financial environment Education: - Bachelors/University degree or equivalent experience in computer science, engineering, or similar domain If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity, review Accessibility at Citi. View Citi's EEO Policy Statement and the Know Your Rights poster.,

This is a data engineer position where you will be responsible for designing, developing, implementing, and maintaining data flow channels and data processing systems to support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable, and secure manner in coordination with the Data & Analytics team. Your main objective will be to define optimal solutions for data collection, processing, and warehousing, particularly within the banking & finance domain. You must have expertise in Spark Java development for big data processing, Python, and Apache Spark. You will be involved in designing, coding, and testing data systems and integrating them into the internal infrastructure. Your responsibilities will include ensuring high-quality software development with complete documentation, developing and optimizing scalable Spark Java-based data pipelines, designing and implementing distributed computing solutions for risk modeling, pricing, and regulatory compliance, ensuring efficient data storage and retrieval using Big Data, implementing best practices for Spark performance tuning, maintaining high code quality through testing, CI/CD pipelines, and version control, working on batch processing frameworks for Market risk analytics, and promoting unit/functional testing and code inspection processes. You will also collaborate with business stakeholders, Business Analysts, and other data scientists to understand and interpret complex datasets. Qualifications: - 5-8 years of experience in working in data ecosystems - 4-5 years of hands-on experience in Hadoop, Scala, Java, Spark, Hive, Kafka, Impala, Unix Scripting, and other Big data frameworks - 3+ years of experience with relational SQL and NoSQL databases such as Oracle, MongoDB, HBase - Strong proficiency in Python and Spark Java with knowledge of core Spark concepts (RDDs, Dataframes, Spark Streaming, etc.), Scala, and SQL - Data integration, migration, and large-scale ETL experience - Data modeling experience - Experience building and optimizing big data pipelines, architectures, and datasets - Strong analytic skills and experience working with unstructured datasets - Experience with various technologies like Confluent Kafka, Redhat JBPM, CI/CD build pipelines, Git, BitBucket, Jira, external cloud platforms, container technologies, and supporting frameworks - Highly effective interpersonal and communication skills - Experience with software development life cycle Education: - Bachelors/University degree or equivalent experience in computer science, engineering, or a similar domain This is a full-time position in the Data Architecture job family group within the Technology sector.,