Senior Data Engineer

Themathcompany

3 - 7 years

20 - 25 Lacs

bengaluru

Posted:-1 days ago| Platform:

Apply

Skills Required

functional version control unit testing data warehousing dbms sql software development life cycle data modeling spark stakeholder management leadership design etl cloud computing real-time data processing programming communication skills

Work Mode

Work from Office

Job Type

Full Time

Job Description

Responsible for designing, storing, processing, and maintaining of large-scale data and related infrastructure
Can drive multiple projects both from operational and technical standpoint
Ideate and build PoV or PoC for new product that can help drive more business
Responsible for defining, designing, and implementing data engineering best practices, strategies, and solutions
Is an Architect who can guide the customers, team, and overall organization on tools, technologies, and best practices around data engineering
Lead architecture discussions, align with business needs, security, and best practices
Has strong conceptual understanding of Data Warehousing and ETL, Data Goverce and Security, Cloud Computing, and Batch & Real Time data processing
Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
Understanding of Medallion architecture pattern
Has worked on at least one cloud platform
Has worked as data architect and executed multiple end-end data engineering project
Has extensive knowledge of different data architecture designs and data modelling concepts
Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes

Required Tech Stack -

Strong proficiency in SQL
Experience working on any of the three major cloud platforms i.e., AWS/Azure/GCP
Working knowledge of an ETL and/or orchestration tools like IICS, Talend, Matillion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc
Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.)
Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.
Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java)
Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables)
Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, etc.
Has worked on Kafka and real-time streaming
Has strong execution knowledge of data architecture design patterns (lambda vs kappa architecture, data harmonization, customer data platforms, etc.)
Has worked on code and SQL query optimization
Strong knowledge of version control systems like Git to manage source code repositories and designing CI/CD pipelines for continuous delivery
Has worked on data and networking security (RBAC, secret management, key vaults, vnets, subnets, certificates)

Required Non-Tech Stack-

Strong problem-solving skills
Able to analyze problems and quickly develop effective and actionable strategies
Displays excellent leadership qualities and can effectively lead teams
Proficient in written and verbal communication
Demonstrates exceptional stakeholder management abilities
Exercises independent judgment to determine the best course of action within established practices and procedures
Shows proficiency in providing technical guidance to junior team members

More Jobs at Themathcompany

Senior Power BI Consultant

Bengaluru

4 - 6 yrs

INR 10 - 15 Lacs

Sr. Research Specialist

Bengaluru

6.0 - 9.0 yrs

INR 10 - 14 Lacs

AI Delivery Lead

Bengaluru

11.0 - 14.0 yrs

INR 50 - 55 Lacs

Data Science Manager

Bengaluru

5.0 - 8.0 yrs

INR 25 - 35 Lacs

Product Engineer - II (QA)

Bengaluru

2.0 - 4.0 yrs

INR 11 - 15 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.