AI Engineering - Databricks

Responsibilities :

Responsibilities:

Design and build modern data pipelines and data streams.

Move/Transform data across layers (Bronze, Silver, Gold) using ADF, Python, and PySpark.

Develop and maintain data pipelines and data streams.

Work with stakeholders to understand their data needs and provide solutions.

Collaborate with other teams to ensure data quality and consistency.

Develop and maintain data models and data dictionaries.

Develop and maintain ETL processes.

Develop and maintain data quality checks.

Develop and maintain data governance policies and procedures.

Develop and maintain data security policies and procedures.

Provide technical guidance and mentorship to junior data engineers

Additional Responsibilities:

A day in the life of an Infoscion

As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction.

You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain.

You will participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews and unit test plan reviews.

You will lead and guide your teams towards developing optimized high quality code deliverables, continual knowledge management and adherence to the organizational guidelines and processes.

You would be a key contributor to building efficient programs/ systems and if you think you fit right in to help our clients navigate their next in their digital transformation journey, this is the place for you! If you think you fit right in to help our clients navigate their next in their digital transformation journey, this is the place for you! Knowledge of more than one technology

Basics of Architecture and Design fundamentals

Knowledge of Testing tools

Knowledge of agile methodologies

Understanding of Project life cycle activities on development and maintenance projects

Understanding of one or more Estimation methodologies, Knowledge of Quality processes

Basics of business domain to understand the business requirements

Analytical abilities, Strong Technical Skills, Good communication skills

Good understanding of the technology and domain

Ability to demonstrate a sound understanding of software quality assurance principles, SOLID design principles and modelling methods

Awareness of latest technologies and trends

Excellent problem solving, analytical and debugging skills

Technical and Professional Requirements:

Requirements:

Experience in Databricks, SQL, PySpark, Spark, Python, and Azure Data Factory (ADF).

Experience in designing, developing, and maintaining data pipelines and data streams.

Experience in moving/transforming data across layers (Bronze, Silver, Gold) using ADF, Python, and PySpark.

Experience in working with stakeholders to understand their data needs and provide solutions.

Experience in collaborating with other teams to ensure data quality and consistency.

Experience in developing and maintaining data models and data dictionaries.

Experience in developing and maintaining ETL processes.

Experience in developing and maintaining data quality checks.

Experience in developing and maintaining data governance policies and procedures. Experience in developing and maintaining data security policies and procedures.