Job
Description
Job Title: Data Engineering Department: NPCI Data Analytics Overview:- NPCI is looking for Data Engineers, who are well versed with the Big data technologies. The candidate will be working on-premises handling the massive data, that will be served to the data science and data modelling team. He / She is responsible for data availability in the existing data pipelines also should be able to establish and make the data available in the new pipelines including specified criteria using latest tools and technologies in Data Engineering. Should be able to understand, analyze the requirement and well versed in providing the data using file or dashboards. Key Responsibilities:- Understanding the requirement and design the data pipelines to make the data available in the data lake for the further processing and usage for the downstream systems Take responsibility for 24/7 data availability, by implementing reliable, fault tolerant system and maintainability and availability of the data. Good knowledge on the data base including the tables, schema, view and executing the SQL complex queries based on the business requirements and providing the required output for the relevant stakeholders. Write Python, Scala scripts for different types of data filtering, Data conversion, Data cleaning. Should have good knowledge on the Bigdata concepts and the Hadoop eco systems. Should be a quick learner and capable of easily adopting to the new technologies. Good knowledge on Data Warehouse, Data Lake and Data mart. Candidate Profile: - Experience in building the data pipeline and data flow. Experience in handing the data Wrangling, data mining, data processing etc Experience in NOSQL data base. Working experience or knowledge on DBT, Dagster, Trino SQL / Hive SQL, MINio, S3 , Superset will be an advantage. Experience in Stream processing like Kafka , Spark and Flink. Experience in different data engineering tools and their configurations. Should be able to write code in different languages that includes Python, Scala , Java and basic working knowledge in Linux etc. Should take complete ownership on the tasks implementing the above tasks. Should be good in BI, visualizing technics, displaying the data insights appropriately to the end user and have good experience in the dashboards like Tableau, Superset etc. Job Description: - NPCI Data Analytics team is looking for data engineers who can make the data available from various products, from different sources, do different processing and store in different storage, in different formats that will be useful to the Data scientists team and other stakeholders to solve the business problems and also to analyze and improve their product performance. Should be able to quickly understand different products of NPCI and their domain terms, and their business flow so as to understand their requirements, pain points and help them by proving their relevant data that will be useful and utilized for their business. Should have strong business understanding, analytical and problem-solving skills and good programming knowledge.
