Senior Data Engineer

Job Title: - Manager/Sr Manager – ETL - Pyspark

Requisition ID:

Job Location: -Pune

Job Summary: -

Job Responsibilities: -

• Working experience in building Data Lake, DWH architecture using Databricks platform
• Engage with Client to participate in requirement gathering, Status update on work, UAT and be the key partner in the overall engagement
• Participates in ETL Design using any python framework of new or changing mappings and workflows with the team and prepares technical specifications
• Crafts ETL Mappings, Mapplets, Workflows, Worklets using Informatica PowerCenter
• Write complex SQL queries with performance tuning and optimization
• Should be able to handle task independently and lead the team
• Responsible for unit testing, integration testing and UAT as and when required
• Good communication Skills
• Coordinate with cross-functional teams to ensure project objectives are met.
• Collaborate with data architects and engineers to design and implement data models.
• Managing projects in fast paced agile ecosystem and ensuring quality deliverables within stringent timelines
• Responsible for Risk Management, maintaining the Risk documentation and mitigations plan.
• Drive continuous improvement in a Lean/Agile environment, implementing DevOps delivery approaches encompassing CI/CD, build automation and deployments.
• Communication & Logical Thinking – Demonstrates strong analytical skills, employing a systematic and logical approach to data analysis, problem-solving, and situational assessment. Capable of effectively presenting and defending team viewpoints, while securing buy-in from both technical and client stakeholders.
• Handle Client Relationship – Manage client relationship and client expectations independently. Should be able to deliver results back to the Client independently. Should have excellent communication skills.

Job Requirements: -

• Should have 7+ years of working experience in ETL & Data Warehousing
• Advanced knowledge of PySpark/python, pandas, numpy frameworks.
• Minimum 4 years of extensive experience in design, build and deployment of Spark/Pyspark - for data integration.
• Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations
• Create Spark jobs for data transformation and aggregation
• Spark query tuning and performance optimization - Good understanding of different file formats (ORC, Parquet, AVRO) to optimize queries/processing and compression techniques.
• Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
• Experience in Modular Programming & Robust programming methodologies
• ETL knowledge and have done ETL development using any pyspark/Python framework
• Advanced SQL knowledge
• Ability to perform multiple task in continually changing environment
• Worked with Redshift/Synapse/Snowflake in the past Preferable.
• Good understanding and experience in the SDLC phases like the Requirements Specification, Analysis, Design, Implementation, Testing, Deployment and Maintenance

Qualification: -

• BE/ B. Tech/ /M Tech/MBA

Must have Skills: -

• Expertise in pharma commercial domain
• Proficiency in ETL using PySpark
• Strong experience in data warehousing

Skills that give you an edge: -

• Experience in AWS or Azure cloud and its service offerings
• Excellent interpersonal/communication skills (both oral/written) with the ability to communicate at various levels with clarity & precision

We will provide– (Employee Value Proposition)

• Offer an inclusive environment that encourages diverse perspectives and ideas
• Delivering challenging and unique opportunities to contribute to the success of a transforming organization
• Opportunity to work on technical challenges that may impact across geographies
• Vast opportunities for self-development: online Axtria Institute, knowledge sharing opportunities globally, learning opportunities through external certifications
• Sponsored Tech Talks & Hackathons
• Possibility of relocating to any Axtria office for short and long-term projects
• Benefit package:

-Health benefits

-Retirement benefits

-Paid time off

-Flexible Benefits

-Hybrid /FT Office/Remote

Axtria is an equal-opportunity employer that values diversity and inclusiveness in the workplace.

Who we are

Axtria 14 years journey

Axtria, Great Place to Work

Life at Axtria

Axtria Diversity