Jobs
Interviews

Data Engineer - Data & Performance

12 years

5 - 8 Lacs

bengaluru

Posted:3 hours ago| Platform: GlassDoor logo

Apply

Skills Required

data technology design development analytics spark hive redshift aws engineering strategies integrity layers automate devsecops integration analyze apache support monitoring debugging drive service communication certification programming sql python github query writing informatica etl presentation consulting resolve cutting

Work Mode

On-site

Job Type

Part Time

Job Description

Bengaluru, Karnataka, India
Job Type
Full Time
About the Role
RESPONSIBILITIES:
This is a strong technology and solution delivery role, accountable for the successful design, development, and delivery of Data Engineer solutions on the corporate Data Platform. The duties and responsibilities of this job are to prepare data and make it available in an efficient and optimized format for consumer analytics, BI, or data science applications. It requires to work with current technologies used by IntelDS and in particular Spark, Presto, Hive and RedShift on AWS environment. This includes:
  • Responsible for the delivery of the development activities he/she owns including Data Engineering standards, patterns and best practices.
  • Work with team to ensure standards and patterns are followed.
  • Determines organizational strategies for data integrity validation processes.
  • Design and develop new data ingestion patterns into IntelDS raw and/or unified data layers based on the requirements and needs for connecting new data sources or for building new data objects. Working in ingestion patterns allow to automate the data pipelines.
  • Participate to and apply DevSecOps practices by automating the integration and delivery of data pipelines in a cloud environment. This can include the design and implementation of end-to-end data integration tests and/or CI/CD pipelines.
  • Analyze existing data models, identify and implement performance optimizations for data ingestion and data consumption. The objective is to accelerate data availability within the platform and to consumer applications. Target technologies are Apache Spark, Apache Presto and RedShift.
  • Support client applications in connecting and consuming data from the platform, and ensure they follow our guidelines and best practices.
  • Participate in the monitoring of the platform and debugging of detected issues and bugs.
  • Partner with the Data modelers to drive improvements and enhancements to the current data landscape and future strategy.
  • Identify opportunities for uses of those technologies to enhance current or anticipated information systems and business goals needs
  • Evaluates existing applications that could address client requirements and makes recommendations from complex projects.
  • Assists in the development of Data Engineering-related Service Level Agreements.
  • Communicates risks and ensures understanding of these risks.
  • Strong analytical & problem-solving skills
  • Strong verbal and business communication skills
  • Skill in identifying data issues and anomalies during the analysis
  • Strong business acumen & demonstrated an aptitude for analytics that incite action

QUALIFICATIONS: - EDUCATION : B.E / B TECH/MASTERS IN COMPUTER SCIENCE, ELECTRONICS, RELEVANT TECHNICAL CERTIFICATION Technology Skills and Project Experience:
  • 12 years of experience in modeling and business system designs.
  • 10 years hands on experience as Data Engineer/Architect with proven experience on Big Data and Data Lakes on AWS
  • 7 years of experience in below Technologies
o Scripts/Programming language: SQL, Python, PROC, Windows Functions
o GitHub o AWS: S3, RedShift, Glue o Big Data: Spark, Hive, Presto
  • Need to be strong in SQL query writing and procedures
  • Experience in Informatica ETL development will be an added advantage
  • Must have proven experience with STAR and SNOWFLAKE schema techniques.
  • Proven track record as a Data Engineer in delivering successful business intelligence developments with complex data sources.
  • Strong analytical skills and enjoys solving complex technical problems.

Business / Soft Skills:
  • Must have solid presentation, communication (on complex technical solutions) and inter-personal skills.
  • Ability to work effectively with globally dispersed stakeholders from business.
  • Ability to manage multiple priorities in a fast-paced environment.
  • A data-driven mindset, Ability to clearly communicate complex business problems and technical solutions.
  • Ability to manage and make decisions about competing priorities and resources.
  • Ability to delegate where appropriate
  • Must be a strong team player/leader

Requirements
RESPONSIBILITIES:
This is a strong technology and solution delivery role, accountable for the successful design, development, and delivery of Data Engineer solutions on the corporate Data Platform. The duties and responsibilities of this job are to prepare data and make it available in an efficient and optimized format for consumer analytics, BI, or data science applications. It requires to work with current technologies used by IntelDS and in particular Spark, Presto, Hive and RedShift on AWS environment. This includes:

  • Responsible for the delivery of the development activities he/she owns including Data Engineering standards, patterns and best practices.
  • Work with team to ensure standards and patterns are followed.
  • Determines organizational strategies for data integrity validation processes.
  • Design and develop new data ingestion patterns into IntelDS raw and/or unified data layers based on the requirements and needs for connecting new data sources or for building new data objects. Working in ingestion patterns allow to automate the data pipelines.
  • Participate to and apply DevSecOps practices by automating the integration and delivery of data pipelines in a cloud environment. This can include the design and implementation of end-to-end data integration tests and/or CI/CD pipelines.
  • Analyze existing data models, identify and implement performance optimizations for data ingestion and data consumption. The objective is to accelerate data availability within the platform and to consumer applications. Target technologies are Apache Spark, Apache Presto and RedShift.
  • Support client applications in connecting and consuming data from the platform, and ensure they follow our guidelines and best practices.
  • Participate in the monitoring of the platform and debugging of detected issues and bugs.
  • Partner with the Data modelers to drive improvements and enhancements to the current data landscape and future strategy.
  • Identify opportunities for uses of those technologies to enhance current or anticipated information systems and business goals needs
  • Evaluates existing applications that could address client requirements and makes recommendations from complex projects.
  • Assists in the development of Data Engineering-related Service Level Agreements.
  • Communicates risks and ensures understanding of these risks.
  • Strong analytical & problem-solving skills
  • Strong verbal and business communication skills
  • Skill in identifying data issues and anomalies during the analysis
  • Strong business acumen & demonstrated an aptitude for analytics that incite action

QUALIFICATIONS: - EDUCATION : B.E / B TECH/MASTERS IN COMPUTER SCIENCE, ELECTRONICS, RELEVANT TECHNICAL CERTIFICATION
Technology Skills and Project Experience:
  • 12 years of experience in modeling and business system designs.
  • 10 years hands on experience as Data Engineer/Architect with proven experience on Big Data and Data Lakes on AWS
  • 7 years of experience in below Technologies
o Scripts/Programming language: SQL, Python, PROC, Windows Functions
o GitHub
o AWS: S3, RedShift, Glue
o Big Data: Spark, Hive, Presto
  • Need to be strong in SQL query writing and procedures
  • Experience in Informatica ETL development will be an added advantage
  • Must have proven experience with STAR and SNOWFLAKE schema techniques.
  • Proven track record as a Data Engineer in delivering successful business intelligence developments with complex data sources.
  • Strong analytical skills and enjoys solving complex technical problems.

Business / Soft Skills:
  • Must have solid presentation, communication (on complex technical solutions) and inter-personal skills.
  • Ability to work effectively with globally dispersed stakeholders from business.
  • Ability to manage multiple priorities in a fast-paced environment.
  • A data-driven mindset, Ability to clearly communicate complex business problems and technical solutions.
  • Ability to manage and make decisions about competing priorities and resources.
  • Ability to delegate where appropriate
  • Must be a strong team player/leader
About the Company
Cigres Technologies Private Limited is a technology consulting and services company that focuses on helping clients resolve their significant digital problems and enabling radical digital transformation using multiple technologies on premise or in the cloud. The company was founded with the goal of leveraging cutting-edge technology to deliver innovative solutions to clients across various industries.

More Jobs at Cigres Technologies

Blackline System Administrator

bengaluru, karnataka

7.0 - 7.0 yrs
|

Salary: Not disclosed

Scrum Master

bengaluru, karnataka

8.0 - 8.0 yrs
|

Salary: Not disclosed

Service Desk Analyst

bengaluru

Experience: Not specified

|

INR 1 - 6 Lacs

Salesforce Data Analyst

bengaluru

Experience: Not specified

|

INR 2 - 6 Lacs

QA Engineer

bengaluru

3.0 - 5.0 yrs
|

INR 5 - 8 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Google Play App Store
coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Cigres Technologies

RecommendedJobs for You

Data Engineer - Data & Performance

Cigres Technologies

bengaluru

Data Engineer - Data & Performance

Cigres Technologies

bengaluru