Job
Description
About Client Hiring for One of Our Multinational Corporations! Job Title: Data Engineer (Scala, Spark, Hadoop) Developer Location: Bangalore Job Type: Full Time WORK FROM OFFICE Job Summary: We are seeking a talented and motivated Data Engineer with strong expertise in Scala , Apache Spark , and Hadoop to join our growing team. As a Data Engineer, you will be responsible for building, optimizing, and maintaining scalable data pipelines, data processing systems, and data storage solutions. The ideal candidate will be passionate about working with big data technologies and developing innovative solutions for processing and analyzing large datasets. Key Responsibilities: Design, develop, and implement robust data processing pipelines using Scala , Apache Spark , and Hadoop frameworks. Develop ETL processes to extract, transform, and load large volumes of structured and unstructured data into data lakes and data warehouses. Work with large datasets to optimize performance, scalability, and data quality. Collaborate with cross-functional teams, including Data Scientists, Analysts, and DevOps, to deliver end-to-end data solutions. Ensure data processing workflows are automated, monitored, and optimized for efficiency and cost-effectiveness. Troubleshoot and resolve data issues, ensuring data integrity and quality. Work on the integration of various data sources into the Hadoop ecosystem and ensure effective data management. Develop and implement best practices for coding, testing, and deployment of data processing pipelines. Document and maintain clear and comprehensive technical documentation for data engineering processes and systems. Stay up-to-date with the latest industry trends, tools, and technologies in the data engineering and big data ecosystem. Required Skills and Qualifications: Proven experience as a Data Engineer, Data Developer, or similar role working with Scala , Apache Spark , and Hadoop . Strong knowledge of big data processing frameworks such as Apache Spark, Hadoop, HDFS, and MapReduce. Experience with distributed computing and parallel processing techniques. Solid experience with ETL processes and working with relational and NoSQL databases (e.g., MySQL, MongoDB, Cassandra, etc.). Proficiency in SQL for querying large datasets. Strong experience with data storage technologies such as HDFS , Hive , HBase , or Parquet . Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data-related services (e.g., S3, Redshift, BigQuery). Experience with workflow orchestration tools such as Airflow , Oozie , or Luigi . Knowledge of data warehousing , data lakes , and data integration patterns. Familiarity with version control tools such as Git and CI/CD pipelines. Strong problem-solving skills and the ability to debug complex issues. Excellent communication skills and the ability to collaborate with different teams. Preferred Skills: Experience with streaming data technologies like Kafka , Flink , or Kinesis . Familiarity with data visualization tools (e.g., Tableau, Power BI) and reporting. Knowledge of machine learning models and working with Data Science teams. Experience working in an Agile/Scrum environment. Degree in Computer Science, Engineering, Mathematics, or a related field. Why Join Us? Be a part of an innovative and dynamic team working on cutting-edge data engineering technologies. Opportunities for growth and career advancement in the data engineering domain. Competitive salary and benefits package. Flexible work arrangements and a supportive work environment Srishty Srivastava Black and White Business Solutions Pvt.Ltd. Bangalore,Karnataka,India. Direct Number:8067432456 srishty.srivastava@blackwhite.in |www.blackwhite.in
