Proficient in designing, developing and testing code to build data infrastructure and pipeline processes for batch and streaming data in the public (or private) cloud environment such as AWS, Azure, GCP.
Familiar with leading practices around large scale data processing, data security and access control.
Responsible for developing, provisioning, securing, monitoring and maintaining the data pipelines to support multiple enterprise and consumer applications in various environments such as production, staging, testing and development environments in an automated manner.
Collaborate with Engineering teams to determine the infrastructure that is required to support the data pipelines.
Monitor health, security and costs of the provisioned data pipelines and take corrective actions as required.
Define, capture, monitor and share key performance and health metrics for the data infrastructure for different applications and environments.
Review and analyze logs related to data pipeline and infrastructure performance for incident management.

An engineer in this role, should:

Collaborate with engineering teams to assist in their automated deployment on the provisioned infrastructure.
Provide guidance on alternative infrastructure architectures as the cloud technologies evolve.
Experiment and run proof of concepts on newer infrastructure technologies available in the cloud to suggest improvements.

Skills:

Strong software development practices with emphasis on delivering high quality code.
Good understanding of cloud infrastructure and services on one or more public cloud platforms to be able select suitable services for data processing.
Strong skills in database management and querying languages such as SQL, MQL
Strong experience in one or more of the following databases PostgresQL, My SQL, SQL
Strong experience in Analytical Databases/Platform like RedShift, Azure Synapse, Snowflake, Spark, Databricks.
Experience with one or more programming languages Python,Java, Scala, C++, JavaScript
Familiarity with AWS data technologies such as AWS Glue, Athena, DynamoDB
Experience with design, developing, testing and maintaining data pipelines and infrastructure

More Jobs at Divit Technologies

Sr Frontend Engineer

chennai, bengaluru

13.0 - 18.0 yrs

INR 45 - 50 Lacs

Java Full Stack + Test Automation Engineer

chennai, bengaluru

6.0 - 8.0 yrs

INR 8 - 10 Lacs

Oracle and Axiom Developer

chennai, bengaluru

8.0 - 10.0 yrs

INR 25 - 30 Lacs

DevOps Engineer

chennai, bengaluru

7.0 - 8.0 yrs

INR 5 - 9 Lacs

Universe Integration Lead

chennai, bengaluru

10.0 - 15.0 yrs

INR 15 - 19 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Java Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.