Senior Data Engineer

Cynosure Corporate Solutions

6 - 11 years

37 - 95 Lacs

chennai

Posted:17 hours ago| Platform:

Apply

Skills Required

hive pyspark etl python scala hadoop spark

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Requirements:

Participate in the customer system design meetings and collect the functional/technical requirements.
Buildupdatapipelines for consumption by the data science team.
Skillful in ETL processes and tools.
Clear understanding and experience with Python and PySpark or Spark and SCALA, with HIVE, Airflow, Impala, and Hadoop and RDBMS architecture.
Experience in writing Python programs and SQL queries.
Experience in SQL Query tuning.
Experienced in Shell Scripting (Unix/Linux).
Build and maintain data pipelines in Spark/Pyspark with SQL and Python or SCALA.
Knowledge of cloud (Azure/AWS/GCP, etc.) technologies is additional.
Good to have knowledge of Kubernetes, CI/CD concepts, Apache Kafka
Suggest and implement best practices in data integration.
Guide the QA team in defining system integration tests as needed.
Split the planned deliverables into tasks and assign them to the team.
Needs to Maintain/Deploy the ETL code and follow the Agile methodology
Needs to work on optimization wherever applicable.
Good oral, written, and presentation skills.

Preferred Qualifications:

Degree in Computer Science, IT, or similar field; a Masters is a plus.
Hands-onexperience with Python and Pyspark
Hands-on experience with Spark and SCALA.
Great numerical and analytical skills.
Working knowledge of cloud platforms such as MS Azure, AWS, etc.
Technical expertise with data models, data mining, and segmentation techniques.

Key traits:

Should have excellent communication skills.
Should be motivated and willing to work as part of a team.
Should be able to collaborate and coordinate in a remote environment.
Be a problem solver and be proactive to solve the challenges that come his way

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.