4.0 - 8.0 years

0 - 0 Lacs

guwahati, assam

On-site

As a Senior Data Engineer specialized in Scala, you will be responsible for leading Spark 3.X, SCALA, Delta lake implementation, and streaming solution implementation for IOT in Spark streaming. Your expertise in Kafka is essential for this role. Any prior experience with MFG BI, DWH, and Datalake implementation will be considered a bonus. This position offers the flexibility of working from home in India and requires a total experience of 10+ years with at least 4-5 years in Scala. The role is permanent under us with an annual salary range of INR 20-25 LPA. The notice period for this position is immediate to 30 days, and the interview process consists of 2 or 3 rounds. Key Responsibilities: - Understand the factories, manufacturing process, data availability, and avenues for improvement. - Collaborate with engineering, manufacturing, and quality teams to identify problems solvable using the acquired data in the data lake platform. - Define necessary data and collaborate with connectivity engineers and users to collect the data. - Develop and maintain optimal data pipeline architecture. - Assemble large, complex datasets meeting functional and non-functional business requirements. - Identify, design, and implement process improvements, automate manual processes, and optimize data delivery for scalability. - Work on data preparation, data deep dive, and help engineering, process, and quality teams understand process/machine behavior closely using available data. - Deploy and monitor solutions. - Collaborate with data and analytics experts to enhance functionality in data systems. - Work alongside Data Architects and data modeling teams. Skills / Competencies: - Solid knowledge of the business vertical with experience in solving use cases in manufacturing or similar industries. - Ability to apply cross-industry learning to enhance manufacturing processes. - Strong problem scoping, solving, and quantification skills. - Proficient in working with unstructured datasets and building data transformation processes. - Experience with message queuing, stream processing, and scalable big data data stores. - Skilled in data mining and data wrangling techniques for analytical dataset creation. - Proficient in building and optimizing big data pipelines, architectures, and datasets. - Adaptive mindset to address data challenges and drive desired outcomes. - Experience with Spark, Delta, CDC, NiFi, Kafka, relational SQL, NoSQL databases, and query languages. - Proficiency in object-oriented languages such as Scala, Java, C++. - Knowledge of visualization tools like PowerBI, Tableau for data presentation. - Ability to analyze data, generate findings, insights through exploratory data analysis. - Strong understanding of data transformation and connection across various data types. - Proficient in numerical, analytical skills, and identifying data acquisition opportunities. - Experience in enhancing data quality, reliability, building algorithms, and prototypes. - Ability to optimize existing frameworks for better performance. If you have the requisite expertise in Scala, Spark, and data engineering, and are keen to work on cutting-edge solutions for manufacturing processes, this role offers an exciting opportunity to make a significant impact in the domain.,

Posted 3 days ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.