Job
Description
About Company : Our client is a global IT, consulting, and business process services company headquartered in Bengaluru, India. It offers end-to-end IT services, including application development, infrastructure management, and digital transformation. They serves clients across industries such as banking, healthcare, retail, energy, and manufacturing. It specializes in modern technologies like cloud computing, AI, data analytics, and cybersecurity. The company has a strong global presence, operating in over 66 countries. Our client employs more than 250,000 people worldwide. It is known for helping enterprises modernize their IT infrastructure and adopt agile practices. Their division includes consulting, software engineering, and managed services. The company integrates automation and AI into its services to boost efficiency and innovation. Job Title: Data Engineer(Python,Spark, Scala) · Location: Hyderabad(Hybrid) · Experience: 5+ yrs · Job Type : Contract to hire. · Notice Period:- Immediate joiners. Mandatory Skills: We are seeking a skilled and motivated Data Engineer with hands-on experience in Python, Apache Spark, and Scala to join our fast-growing data engineering team. The ideal candidate will be responsible for building scalable data pipelines, optimizing data processing frameworks, and supporting advanced analytics and machine learning workflows. Key Responsibilities: Design, build, and maintain large-scale, distributed data processing systems using Apache Spark. Develop robust ETL/ELT pipelines in Python and Scala to ingest, clean, and transform data from multiple sources. Collaborate with data scientists, analysts, and software engineers to support data infrastructure needs. Ensure high performance and reliability of data workflows by implementing best practices in code and system design. Monitor and troubleshoot data jobs, identify bottlenecks, and propose solutions for optimization. Work with structured and unstructured data, using tools and technologies like Hadoop, Hive, Parquet, Kafka, etc. Participate in code reviews and contribute to a culture of continuous improvement and knowledge sharing. Required Qualifications: 5+ years of experience in data engineering or big data development. Strong proficiency in Python for data processing and automation. Solid experience with Apache Spark (RDD, DataFrame, SparkSQL, and performance tuning). Proficient in Scala for building Spark applications. Hands-on experience with distributed data systems (Hadoop, HDFS, Hive). Experience with version control (e.g., Git), CI/CD, and cloud platforms (AWS, Azure, or GCP) is a plus. Strong understanding of data modeling, data warehousing, and data quality best practices. Preferred Skills: Experience with streaming data (Spark Streaming, Kafka). Knowledge of containerization tools like Docker and orchestration with Kubernetes. Familiarity with workflow orchestration tools (Airflow, Luigi). Exposure to machine learning pipelines and MLOps concepts. Education: Bachelorâs or Masterâs degree in Computer Science, Engineering, or a related field. Show more Show less
