Data Engineer III

Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express.

We are building a banking platform that enables teams to create products at scale and powers digital channels to deliver compelling experiences in a dramatically faster fashion.

We look primarily for people who are passionate about solving business problems through innovation and engineering practices. You will be required to apply your depth of knowledge and expertise to all aspects of the software development lifecycle, as well as partner continuously with your many stakeholders daily to stay focused on common goals. We embrace a culture of experimentation and constantly strive for improvement and learning. We welcome diverse perspectives and people who are not afraid to challenge assumptions.

Our mission is to accelerate the pace of financial innovation and build new financial products for American Express. Our platform streamlines the process of launching and iterating financial products.

Responsibilities:

• Develops and tests software, including ongoing refactoring of code & drives continuous improvement in code structure & quality.
• Functions as a core member of an Agile team driving User story analysis & elaboration, design and development of software applications, testing & builds automation tools.
• Designs, codes, tests, maintains, and documents data applications.
• Takes part in reviews of own work and reviews of colleagues' work.
• Defines test conditions based on the requirements and specifications provided.
• Partner with the product teams to understand business data requirements, identify data needs and data sources to create data architecture
• Documents data requirements / data stories and maintains data models to ensure flawless integration into existing data architectures
• Leads multiple tasks effectively - progresses work in parallel
• Adapts to change quickly and easily

• Handles problems and acts on own initiative without being prompted
  

  Qualifications:

• Must have demonstrated proficiency and experience in the following tools and technologies: Pyspark, Apache Airflow, BigQuery, GCP DataProc, GCP Dataflow
• Must have Banking Domain Knowledge: Money Movement, Zelle, ACH, Intraday , Data contribution agreements with EWS
• Experienced with following technology stack
  • Python Object Oriented Programming
  • Python Built in libraries: JSON, Base64, logging, os, etc.
  • Python: Poetry and dependency management
  • Asynchronous Reactive Micro services utilizing Fast API
  • Firm foundational understanding of Distributed Storage and Distributed Compute
  • Py spark framework: DataFrames (Aggregation, Windowing techniques), Spark SQL
  • Cornerstone Data Ingestion Process, Cornerstone Business Metadata management, Interactive Analytics using YellowBrick,
  • Hyperdrive JSON schema development, CStreams Realtime event ingestion pipeline using Kafka,
  • Event Engine Management
  • Test Driven Development
• Working Knowledge of following tools and technologies:
  • Data Governance Toolset: Collibra, Manta
  • REST APIs Specifications using Swagger
  • Development Tool Central, XLR, Jenkins
  • Docker Image creation, Containers, PODs
  • Hydra Cloud Deployment and Troubleshooting
  • Logging using Amex Enterprise Logging Framework
• Analytical and problem-solving skills
• Technical fluency - ability to clearly describe tradeoffs to technical and non- technical audiences alike to help support product decisions.
• Highly organized with strong prioritization skills and outstanding written and verbal communication - you are great at research and documenting your learnings.
• A bachelor's degree.