Data Engineer

Prosperspective HR Solutions

0 years

0 Lacs

Pune/Pimpri-Chinchwad Area

Posted:1 week ago| Platform:

Apply

Skills Required

data azure aws databricks development support maintenance stability security audit drive optimization efficiency analytics scala python pyspark spark design storage tuning code hive sqoop kafka sql jenkins github

Work Mode

On-site

Job Type

Full Time

Job Description

About the Role

This role is accountable for running day to day operations of Data Platform in Azure / AWS Databricks. Data Engineer is accountable for ongoing Development, Enhancement support and maintenance data availability and data quality, performance enhancement and stability of the system.

1. Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks

2. Ensure data pipelines run smoothly and efficiently

3. Adherence to security, regulatory and audit control guidelines

4. Drive optimization, continuous improvement and efficiency

Essential for this role

Minimum 5yrs of experience in data analytics field
Experience with Azure/AWS Databricks
Experience in building and optimizing data pipelines, architectures and data sets
Excellent experience in Scala or Python, PySpark and SQL
Ability to troubleshoot and optimize complex queries on the Spark platform
Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques
Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS
Hands on experience in performance tuning and optimizing code running in Databricks environment
Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment

Technical / Professional Skills

at least 3

Azure/AWS Databricks.
Python / Scala / Spark / PySpark.
HIVE / HBase / Impala / Parquet.
Sqoop, Kafka, Flume.
SQL and RDBMS.
Airflow.
Jenkins / Bamboo.
Github / Bitbucket.
Nexus.

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.