Sr Software Engineer

5 years

0 Lacs

Posted:5 days ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

About The Role

Uber's data infrastructure is composed of a wide variety of compute engines, scheduling/execution solutions, and storage solutions. Compute engines such as Apache Spark™, Presto®, Apache Hive™, Neutrino, Apache Flink®, etc., allow Uber to run petabyte-scale operations on a daily basis. Further, scheduling and execution engines such as Piper (Uber's fork of Apache Airflow™), Query Builder (user platform for executing compute SQLs), Query Runner (proxy layer for execution of workloads), and exist to allow scheduling and execution of compute workloads. Finally, a significant portion of storage is supported by HDFS, Google Cloud Storage (GCS),Apache Pinot™, ElasticSearch®, etc. Each engine supports thousands of executions, which are owned by multiple owners and sub-teams.With such a complex and diverse big data landscape operating at petabyte-scale and around a million applications/queries running each day, it's imperative to provide the stakeholders a holistic view of the right performance and resource consumption insights.DataCentral, is a comprehensive platform that provides users with essential insights into big data applications and queries. It empowers data platform users by offering detailed information on workflows and apps, improving productivity by reducing debugging time and improving the cost efficiency by providing detailed resource efficiency insightsAs an engineer in the Data Central Team, you will be solving some of the most complex problems in Observability and efficiency of Distributed Data Systems at Uber scale.

What You'll Do

  • Work with Uber data science and engineering teams to improve Observability of Batch Data use-cases at Uber.
  • Leverage knowledge of spark internals to dramatically help improve customer's Spark job performance.
  • Design and implement AI based solutions to improve the application debuggability.
  • Design and implement algorithms to optimize Resource consumption without impacting reliability
  • Design and develop prediction and forecasting models to proactively predict system degradations and failures
  • Work with multiple partner teams within and Uber and build cross-functional solutions in a collaborative work environment.
  • Work with the community to upstream Uber's contributions to open source and also keep our internal fork up to date

What You'll Need

  • Bachelor's degree in Computer Science or related field.
  • 5+ years of experience building large scale distributed software systems.
  • Solid understanding of Java for backend / systems software development.
  • MS / PhD in Computer Science or related field.
  • Experience managing production systems with a strong availability SLA.
  • Experience working with Apache Spark or similar analytics technologies.
  • Experience working with large scale distributed systems, HDFS / Yarn.
  • Experience working with SQL Compiler, SQL Plan / Runtime Optimization.
  • Experience working with Kubernetes

Mock Interview

Practice Video Interview with JobPe AI

Start Java Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Java Skills

Practice Java coding challenges to boost your skills

Start Practicing Java Now
Uber logo
Uber

Technology, Information and Internet

San Francisco California

RecommendedJobs for You

Noida, Uttar Pradesh, India