Posted:1 week ago|
Platform:
Work from Office
Full Time
Design, implement, and maintain secure and scalable data ingestion pipelines from a wide variety of source systems, including SAP, SalesForce, SharePoint, APIs, and (legacy) manufacturing platforms.
Build and enhance metadata-driven services that enable discoverability, access governance, and operational transparency of enterprise data.
Main Tasks:
Build and maintain Python- or Scala-based extraction services (e.g., Debezium Server, custom APIs, rclone)
Implement CDC, delta, and event-based patterns.
Establish, operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal.o
Integrate with systems such as Salesforce, SharePoint, and other API- or file-based endpoints.
Establish and maintain a business-friendly, web-accessible data catalog application, with dataset profiles, metadata, and usability features.
Integrate dataset discoverability, preview/exploration options, and lineage information using Unity Catalog as a backend metadata system.
Perform design reviews with Cybersecurity.
Ensure documentation and compliance for all interfaces and data ingress points.
Collaborate closely with IT and business users to translate requirements into scalable technical patterns.
Serve as technical escalation point for complex source integration.
Define and implement a multi-layered data quality framework, including unit-level, integration-level, and cross-pipeline validation rules.
Establish centralized and version-controlled storage of DQ rules, with integration into orchestration and CI/CD pipelines.
Degree in Computer Science, Data Engineering, or a related field. Azure or Databricks certification is a plus.
5 8 years in data engineering, with hands-on experience in ingesting structured and unstructured enterprise data into modern cloud platforms.
Proven implementation of source system ingestion frameworks, metadata automation, and compliance-controlled interfaces.
Not required; however, experience mentoring junior developers or leading implementation workstreams is a plus; contributes to engineering standards and code quality improvement initiatives.
Comfortable working across geographies and time zones; collaborates effectively with global teams and enterprise stakeholders.
Continental
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
Practice Python coding challenges to boost your skills
Start Practicing Python Now
bengaluru
15.0 - 16.0 Lacs P.A.
hyderabad
2.0 - 5.0 Lacs P.A.
4.0 - 8.0 Lacs P.A.
bengaluru
36.0 - 84.0 Lacs P.A.
pune
10.0 - 15.0 Lacs P.A.
chennai, coimbatore
15.0 - 30.0 Lacs P.A.
pune, chennai, coimbatore
25.0 - 32.5 Lacs P.A.
7.0 - 17.0 Lacs P.A.
chennai
0.5 - 0.5 Lacs P.A.
20.0 - 27.5 Lacs P.A.
