Data Engineer

Data Engineer
Airtel is looking for a Data Engineer to join the Data Platform team who can help develop and deploy data pipelines at a huge scale of ~5B daily events and concurrency of 500K users. The platform is built on a cloud-native modern data stack (AWS/GCP), enabling real-time reporting and deep data exploration from first principles. Experience: 2- 5 Years Job Location: Gurgaon

Responsibilities: Create and maintain a robust, scalable, and optimized data pipeline. Handle TBs of data daily across Xtelify's music and video streaming platforms. Extract and consume data from live systems to analyze and operate in a 99.999% SLA Big Data environment. Build and execute data mining and modeling activities using agile development techniques. Solve problems creatively and communicate effectively across teams. Build infrastructure for optimal data extraction, transformation, and loading from diverse sources. Drive internal process improvements: automation, data delivery optimization, infrastructure redesign, etc. Manage both structured and unstructured datasets using SQL and NoSQL databases. Understand and leverage cloud computation models to enhance delivery and deployment of solutions. Drive innovation in areas like analytics pipelines, streaming systems, and machine learning data applications. Desired Profile: B.E / B.Tech / M.E / M.Tech / M.S in Computer Science or Software Engineering from a premier institute. 2+ years of hands-on experience with Spark and Scala. Strong grasp of data structures and algorithms. Proficient with Hadoop, Pig, Hive, Storm, and SQL. Programming experience in at least one language: Java, Scala, or Python. Solid experience in big data infrastructure, distributed systems, dimensional modeling, query processing & optimization, and relational databases. Own end-to-end delivery of data modules/features. Good to Have: Exposure to NoSQL solutions like Cassandra, MongoDB, CouchDB, Postgres, Redis, Elasticsearch. Experience with Kafka and Spark Streaming/flink. Familiarity with OLAP engines like Apache Druid or Apache Pinot. Enthusiasm to explore and learn modern data architectures and streaming systems.