Job
Description
Data and ML Platform engineering employs new-age technologies such as Distributed Computing constructs, Real Time model predictions, Deep Learning, Accelerated Compute (GPU); scalable feature stores Cassandra, MySQL, Elastic Search, Solr, Aerospike; scalable programming constructs in, Python and ML Frameworks (TensorFlow, Pytorch, etc). Roles and Responsibilities Drive the data architecture, data modelling, design, and implementation of data applications using standard open source big data tech stack, Data Warehouse / MPP databases and distributed systems. Gather business and functional requirements from external and/or internal users, and translate requirements into technical specifications to build robust, scalable, supportable solutions. Participate and drive the full development lifecycle. Build the Standards and best practices around a Common Data Model and Architecture, Data Governance, Data Quality and Security for multiple business areas across Myntra. Collaborate with platform, product and other engineering and business teams to evangelise those Standards for adoption across the org. Mentor data engineers at various levels of seniority by doing their design and code reviews, providing constructive and timely feedback on code quality, design issues, technology choices with performance and scalability being critical drivers. Manage resources on multiple technical projects and ensure schedules, milestones, and priorities are compatible with technology and business goals. Setting up best practices to help the team achieve the above and constantly thinking about improving the technology use are your responsibilities. Driving the adoption of these best practices around coding, design, quality, performance in your team.Stay abreast of the technology industry, market trends in the field of data architecture and development. Demonstrates understanding of data lifecycle (data modelling, processing, data quality, data evolution) and underlying tech stacks (Hadoop, Spark, MPP). Drives setting data architecture standards encompassing complete data life cycle (ingestion, modelling, processing, consumption, change management, quality, anomaly detection). Challenge the status quo and propose innovative ways to process, model, consume data when it comes to tech stack choices or design principles. Implementation of long term technology vision for your team. Active participant in technology forums; represent Myntra in external forums. Qualifications & Experience 12 - 15 years of experience in software development 5+ years of development and / or DBA experience in Relational Database Management Systems[RDBMS] (MySql, SQLServer, etc.) 8+ years of hands-on experience in implementation and performance tuning MPP databases (Microsoft SQL DW, AWS Redshift, Teradata, Vertica, etc.) Experience designing database environments, analyzing production deployments, and making recommendations to optimize performance Problem solving skills for complex & large scale data applications problems. Technical Breadth Exposure to a wide variety of problem spaces, technologies in data e.g. real-time and batch data processing, options in commercial vs open source tech stack. Hands-on experience with Enterprise Data Warehouse and Big data storage and computation frameworks like OLAP Systems, MPP (SQL DW, Redshift, Oracle RAC, Teradata, Druid), Hadoop Compute (MR, Spark, Flink, Hive). Awareness of pitfalls & use cases for a large variety of solutions. Ability to drive capacity planning, performance optimization and large-scale system integrations. Expertise in designing, implementing, and operating stable, scalable, solutions to flow data from production systems into analytical data platforms (big data tech stack + MPP) and into end-user facing applications for both real-time and batch use cases. Data modelling skills (relational, multi-dimensional) and proficiency in one of the programming languages preferably Java, Scala or Python. Drive design and development of automated monitoring, alerting, self healing (restartability / graceful failures) features while building the consumption pipelines. Mentoring skills Be the technical mentor to your team. B. Tech. or higher in Computer Science or equivalent required.
