Interesting Job Opportunity: Data Engineer - Data Engineer

Deqode

0 years

0 Lacs

Bhopal Madhya Pradesh India

Posted:6 days ago| Platform:

Apply

Skills Required

data analysis model development software engineering mathematics databricks pyspark spark sql unity optimization gcp relational scripting python linux pipeline workflow management airflow git testing code design documentation reporting analytics collaborative communication

Work Mode

On-site

Job Type

Full Time

Job Description

Responsibilities Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation. Work closely with analysts/data scientists to understand impact to the downstream data models. Write efficient and well-organized software to ship products in an iterative, continual release environment. Contribute to and promote good software engineering practices across the team Communicate clearly and effectively to technical and non-technical audiences. Minimum Qualifications University or advanced degree in engineering, computer science, mathematics, or a related field Strong hands-on experience in Databricks using PySpark and Spark SQL (Unity Catalog, workflows, Optimization techniques) Experience with at least one cloud provider solution (GCP preferred) Strong experience working with relational SQL databases. Strong experience with object-oriented/object function scripting language : Python. Working knowledge in any transformation tools, DBT preferred. Ability to work with Linux platform. Strong knowledge of data pipeline and workflow management tools (Airflow) Working knowledge of Git hub /Git Toolkit Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation Experience creating Data pipelines that prepare data for ingestion & consumption appropriately. Experience in maintaining and optimizing databases/filesystems for production usage in reporting, analytics. Working in a collaborative environment and interacting effectively with technical and non-technical team members equally well. Good verbal and written communication skill (ref:hirist.tech) Show more Show less