Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Jobs

Interviews

Home
>
Jobs in pune
>
NCSI Technologies
>
Senior Data Engineer

Senior Data Engineer

NCSI Technologies

7 - 12 years

11 - 21 Lacs

pune

Posted:4 hours ago| Platform:

Apply

Skills Required

big data administration hive cloudera analytical scala pyspark data warehousing bitbucket azure devops sql cloud applications spark design flume jenkins hbase data lake communication skills python github rdbms data analytics microsoft azure impala bamboo field kafka sqoop aws

Work Mode

Work from Office

Job Type

Full Time

Job Description

Essential for this role:
Education and Qualifications: • Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent. • Minimum seven plus years of experience in data analytics field • Experience with Azure/AWS Databricks • Experience in building and optimizing data pipelines, architectures and data sets • Excellent experience in Scala or Python, PySpark and SQL • Ability to troubleshoot and optimize complex queries on the Spark platform • Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques • Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS • Hands on experience in performance tuning and optimizing code running in Databricks environment • Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment Technical / Professional Skills : • Excellent communication skills / should interact directly with the customer • Azure/AWS Databricks • Python / Scala / Spark / PySpark • HIVE / HBase / Impala / Parquet • Sqoop, Kafka, Flume • Very Strong SQL and RDBMS • Airflow • Jenkins / Bamboo • Github / Bitbucket • Nexus • Knowledge on Delta Live Tables (DLT) will be an added advantage Good to have: • Relevant accredited certifications for Azure, AWS, Cloud Engineering, and/or Databricks • Data Engineering expertise in Big Data Management using Cloudera, Python, Hive, Scala, Data Warehouse, Data Lake, AWS, Azure
Roles and Responsibilities
1 Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks
2 Ensure data pipelines run smoothly and efficiently 3 Developing scalable and re-usable frameworks for ingesting of data sets 4 Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times 5 Working with event based / streaming technologies to ingest and process data 6 Working with other members of the project team to support delivery of additional project components (API interfaces, Search) 7 Evaluating the performance and applicability of multiple tools against customer requirements 8 Technically competent in Cloud and Databricks to provide technical advice to the team and to be involved in issue resolution 9 Provide on-call support and afterhours/weekend support as needed 10 Fulfill Service Requests related to Data Analytics platform 11 Lead and drive optimization and continuous improvement initiatives 12 Play gate-keeping role and conduct technical review of the changes as part of release management 13 Understand various data security standards and adhere to the required data security controls in the platform 14 Lead the design, development, and deployment of advanced data pipelines and analytical workflows on the Databricks Lakehouse platform. 15 Collaborate with data scientists, engineers, and business stakeholders to build and scale end-to-end data solutions. 16 Own architectural decisions and ensure adherence to data governance, security, and compliance requirements. 17 Mentor a team of data engineers, providing technical guidance and career development. Implement CI/CD practices for data engineering pipelines using tools like Azure DevOps, GitHub Actions, or Jenkins.

More Jobs at NCSI Technologies

Test Lead

Pune

6 - 9 yrs

INR 8 - 18 Lacs

Domain Architect

Hyderabad, Pune, Bengaluru

8.0 - 12.0 yrs

INR 7 - 15 Lacs

Elastic Search Engineer

Pune

2.0 - 4.0 yrs

INR 5 - 10 Lacs

Engineer, Unix/Linux System

Pune

2.0 - 4.0 yrs

INR 5 - 14 Lacs

Senior Engineer, Unix/Linux System

Pune

5.0 - 8.0 yrs

INR 7 - 11 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

NCSI Technologies

Information Technology

Innovation City

Login to

Please Verify Your Phone or Email

Confirm Action

Search

Profile

Upskill and Grow with AI

Senior Data Engineer