Job
Description
Dynamic Yield, a Mastercard company, is seeking a Data Engineer II to join our Data Engineering & Analytics team. We're focused on developing robust data and analytics solutions from vast datasets collected across various consumer-focused businesses. In this role, you'll be instrumental in creating high-performance algorithms, leveraging cutting-edge analytical techniques, and designing intuitive workflows. Your work will empower users to derive actionable insights from big data. You'll work with large-scale datasets and front-end visualizations to unlock the true value of big data, supporting business needs through innovative, data-driven solutions. The Role As a Data Engineer II, you will: Platform Evolution: Drive the evolution of data and services platforms with a strong emphasis on data engineering and data science, ensuring impactful advancements in data quality, scalability, and efficiency. Data Generation & Curation: Develop and fine-tune methods and algorithms to generate precise, high-quality data at scale. This includes creating and maintaining feature stores, analytical stores, and curated datasets to enhance data integrity and usability. Complex Problem Solving: Solve complex data challenges involving multi-layered datasets and optimize the performance of existing data pipelines, libraries, and frameworks. Support & Resolution: Provide support for deployed data applications and analytical models, identifying data issues and guiding resolutions. Data Governance: Ensure proper data governance policies are followed by implementing or validating data lineage, quality checks, classification, and other relevant measures. Data Integration: Integrate diverse data sources, including real-time, streaming, batch, and API-based data, to enrich platform insights and drive data-driven decision-making. Tool Experimentation: Experiment with new tools to streamline the development, testing, deployment, and running of our data pipelines. Best Practices: Develop and enforce best practices for data engineering, including coding standards, code reviews, and documentation. Security & Privacy: Ensure data security and privacy compliance, implementing robust measures to protect sensitive data. Global Collaboration: Communicate, collaborate, and work effectively within a global environment. All About You Education: Bachelor's degree in Computer Science, Software Engineering, or a related field. Data Engineering Experience: Extensive hands-on experience in Data Engineering, including implementing multiple end-to-end data warehouse projects in Big Data environments. Programming & Frameworks: Proficiency in application development frameworks ( Python, Java/Scala ) and data processing/storage frameworks ( Hadoop, Spark, Kafka ). Orchestration Tools: Experience in developing data orchestration workflows using tools such as Apache NiFi, Apache Airflow , or similar platforms to automate and streamline data pipelines. Performance Tuning: Experience with performance tuning of database schemas, databases, SQL, ETL jobs, and related scripts. Agile Environment: Experience working in Agile teams. Data-Driven Application Development: Experience in developing data-driven applications and data processing workflows/pipelines, and/or implementing machine learning systems at scale using Java, Scala, or Python . This includes all phases such as data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting analytics. Cloud Integration: Experience in developing integrated cloud applications with services like Azure, Databricks, AWS, or GCP . Analytical Skills: Excellent analytical and problem-solving skills, with the ability to analyze complex data issues and develop practical solutions. Communication & Collaboration: Strong communication and interpersonal skills, with the ability to collaborate effectively with and facilitate activities across cross-functional, geographically distributed teams and stakeholders.
