Job Title:
Senior Data Engineer
Location:
Pune
Experience:
8+ yrs exp
I. Job Summary
We are seeking a
motivated and experienced Senior
Data Engineer
to join our dynamic Data Platform team. The ideal candidate will have
8+ years of professional experience
.This role is accountable for running day to day operations of Data Platform in Azure / AWS Databricks NCS. Data Engineer is accountable for ongoing Development, Enhancement support and maintenance data availability and data quality, performance enhancement and stability of the system.
- Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks
- Ensure data pipelines run smoothly and efficiently
- Adherence to security, regulatory and audit control guidelines
- Drive optimization, continuous improvement and efficiency
II. Roles & Responsibilities
- Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks
- Ensure data pipelines run smoothly and efficiently
- Developing scalable and re-usable frameworks for ingesting of data sets
- Integrating the end to end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
- Working with event based / streaming technologies to ingest and process data
- Working with other members of the project team to support delivery of additional project components (API interfaces, Search)
- Evaluating the performance and applicability of multiple tools against customer requirements
- Technically competent in Cloud and Databricks to provide technical advice to the team and to be involved in issue resolution
- Provide on-call support and afterhours/weekend support as needed
- Fulfill Service Requests related to Data Analytics platform
- Lead and drive optimization and continuous improvement initiatives
- Play gate-keeping role and conduct technical review of the changes as part of release management
- Understand various data security standards and adhere to the required data security controls in the platform
- Lead the design, development, and deployment of advanced data pipelines and analytical workflows on the Databricks Lakehouse platform.
- Collaborate with data scientists, engineers, and business stakeholders to build and scale end-to-end data solutions.
- Own architectural decisions and ensure adherence to data governance, security, and compliance requirements.
- Mentor a team of data engineers, providing technical guidance and career development.
- Implement CI/CD practices for data engineering pipelines using tools like Azure DevOps, GitHub Actions, or Jenkins.
III. Years Of Experience
The ideal candidate will have
8+ years of professional experience.
IV. Educational Qualification & Certifications (Optional)
Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent.
V. Skill Set Required
Must have:
Minimum seven plus years of experience in data analytics field
- Experience with Azure/AWS Databricks
- Experience in building and optimizing data pipelines, architectures and data sets
- Excellent experience in Scala or Python, PySpark and SQL
- Ability to troubleshoot and optimize complex queries on the Spark platform
- Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques
- Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS
- Hands on experience in performance tuning and optimizing code running in Databricks environment
- Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment
- Azure/AWS Databricks
- Python / Scala / Spark / PySpark
- HIVE / HBase / Impala / Parquet
- Sqoop, Kafka, Flume
- SQL and RDBMS
- Airflow
- Jenkins / Bamboo
- Github / Bitbucket
- Nexus
About SMC Squared
At SMC Squared, people are the foundation of our success. We help leading US and multinational companies accelerate digital transformation by building and managing Global Capability Centres with fully integrated, strategically aligned, and valued teams. Our people-first approach creates opportunities for talented professionals to grow, contribute, and thrive, while enabling organizations to scale intelligently.As part of Hexaware Technologies—a global leader in IT, BPO, and consulting—we deliver greater value through enhanced digital capabilities, automation platforms, and a shared commitment to people-centric transformation. Join us and be part of a purpose-driven company where your ideas matter, your work makes a global impact, and your career can reach new heights.Join us and be part of a purpose-driven company where your ideas matter, your work has a global impact, and your career can reach new heights.
EEO Statement
SMC Squared is an equal opportunity employer. We encourage applications from candidates of all backgrounds and experiences.If you are a creative individual with a passion for design and a drive to contribute to impactful projects, we invite you to apply for this exciting opportunity.
