Data Engineer

Hello,

Greeting from Quess Corp!!

Hope you are doing well

we have job opportunity with one of our client

Designation_ Data Engineer

Location – Gurugram

Experience – 8yrs to 15 Yrs

Qualification – Graduate / PG ( IT)

Skill Set – Data Engineer, Python, AWS, SQL

Essential capabilities

• Enthusiasm for technology, keeping up with latest trends
• Ability to articulate complex technical issues and desired outcomes of system enhancements
• Proven analytical skills and evidence-based decision making
• Excellent problem solving, troubleshooting & documentation skills
• Strong written and verbal communication skills
• Excellent collaboration and interpersonal skills
• Strong delivery focus with an active approach to quality and auditability
• Ability to work under pressure and excel within a fast-paced environment
• Ability to self-manage tasks
• Agile software development practices

Desired Experience

• Hands on in SQL and its Big Data variants (Hive-QL, Snowflake ANSI, Redshift SQL)
• Python and Spark and one or more of its API (PySpark, Spark SQL, Scala), Bash/Shell scripting
• Experience with Source code control - GitHub, VSTS etc.
• Knowledge and exposure to Big Data technologies Hadoop stack such as HDFS, Hive, Impala, Spark etc, and cloud Big Data warehouses - RedShift, Snowflake etc.
• Experience with UNIX command-line tools.
• Exposure to AWS technologies including EMR, Glue, Athena, Data Pipeline, Lambda, etc
• Understanding and ability to translate/physicalise Data Models (Star Schema, Data Vault 2.0 etc)

Essential Experience

It is expected that the role holder will most likely have the following qualifications and experience

• 4-11 years technical experience (within financial services industry preferred)
• Technical Domain experience (Subject Matter Expertise in Technology or Tools)
• Solid experience, knowledge and skills in Data Engineering, BI/software development such as ELT/ETL, data extraction and manipulation in Data Lake/Data Warehouse/Lake House environment.
• Hands on programming experience in writing Python, SQL, Unix Shell scripts, Pyspark scripts, in a complex enterprise environment
• Experience in configuration management using

  Ansible/Jenkins/GIT

• Hands on cloud-based solution design, configuration and development experience with Azure and AWS
• Hands on experience of using AWS Services - S3,EC2, EMR, SNS, SQS, Lambda functions, Redshift
• Hands on experience Of building Data pipelines to ingest, transform on Databricks Delta Lake platform from a range of data sources - Data bases, Flat files, Streaming etc..
• Knowledge of Data Modelling techniques and practices used for a Data Warehouse/Data Mart application.
• Quality engineering development experience (CI/CD – Jenkins, Docker)
• Experience in Terraform, Kubernetes and Docker
• Experience with Source Control Tools – Github or BitBucket
• Exposure to relational Databases - Oracle or MS SQL or DB2 (SQL/PLSQL, Database design, Normalisation, Execution plan analysis, Index creation and maintenance, Stored Procedures) , PostGres/MySQL
• Skilled in querying data from a range of data sources that store structured and unstructured data
• Knowledge or understanding of Power BI (Recommended)

Key Accountabilities

• Design, develop, test, deploy, maintain and improve software
• Develop flowcharts, layouts and documentation to identify requirements & solutions
• Write well designed & high-quality testable code
• Produce specifications and determine operational feasibility
• Integrate software components into fully functional platform
• Apply pro-actively & perform hands-on design and implementation of best practice CI/CD
• Coaching & mentoring of other Service Team members
• Develop/contribute to software verification plans and quality assurance procedures
• Document and maintain software functionality
• Troubleshoot, debug and upgrade existing systems, including participating in DR tests
• Deploy programs and evaluate customer feedback
• Contribute to team estimation for delivery and expectation management for scope.
• Comply with industry standards and regulatory requirements