Data Engineer

IDESLABS

5 - 7 years

4 - 8 Lacs

pune maharashtra india

Posted:5 hours ago| Platform: Foundit logo

Apply

Skills Required

airflow

Work Mode

On-site

Job Type

Full Time

Job Description

5+ years of relevant experience in the field of Data Engineering
Advance skills in big data technologies like Hadoop, Python, Spark, SQL.
Must have experience building data APIs.
Bachelor s in Computer Science or related disciplines
Knowledge of Data Structures and Algorithm.
Strong Python programming skills with ability to implement OOPs and functional programming. Knowledge of Scala/Java would be plus.
Strong knowledge on RDBMS and NoSQL databases with the ability to implement them from scratch. Knowledge of Graph databases will be a plus.
Strong expertise in building & optimizing data pipelines, architectures, and data sets.
Experience working with different file formats like Parquet, ORC, Avro, RC, etc.
Experience with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.
Knowledge and experience of using orchestration frameworks like Airflow, Oozie, Luigi, etc.
Experience using Spark, and building jobs using Python/Scala/Java.
Experience or Knowledge building stream processing platforms using Spark Streaming, Storm, etc. Knowledge of Kafka/Flink+Beam would be plus.
Knowledge of building REST API end points for data consumption.
Experience in building scalable data pipelines for both real time and batch using best practices in data modeling, ETL/ELT processes utilizing varioud technologies such as Spark, Kafka, Presto, SAP HANA, Airflow, informatica.
Perform Data analysis using Python, complex SQLs, and other tools.
Perform root cause analysis of issues from platform standpoint on Kubernetes, Containers, Hadoop, Spark, Hive, Presto
Excellent oral and written communication is a must.
Preferred
Masters in Computer Science or related disciplines
Experience building self-service tools for analytics would be plus.
Knowledge of ELK stack would be a plus.
Knowledge of implementing CI/CD on the pipelines is a plus.
Knowledge of Containerization (Docker/Kubernetes) will be plus.
Experience working with one of the popular Public Cloud based platforms is preferred

More Jobs at IDESLABS

Looking For Intimidate Joiners For Netscaler

Bengaluru

6 - 11 yrs

INR 10 - 20 Lacs

SMG Professional

Bengaluru

3.0 - 8.0 yrs

INR 6 - 9 Lacs

Frontend React JS Developer

Hyderabad

5.0 - 10.0 yrs

INR 8 - 13 Lacs

PMO

Gurugram

6.0 - 11.0 yrs

INR 3 - 6 Lacs

Immediate Openings on SAP Modules PAN India (Contract) For ideslabs

Hyderabad, Pune, Bengaluru

6.0 - 10.0 yrs

INR 15 - 30 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.