Senior Software Engineer - Data Engineer

About the job

About Us

• Design and implement domain-oriented, federated data architectures with clear APIs, SLAs, and ownership.
• Build self-serve data infrastructure platforms for discovery, access, and lifecycle management.
• Implement automated governance and compliance policies at scale.
• Lead development of high-performance pipelines, data catalogs, and monitoring systems.
• Integrate data from enterprise systems, cloud-native services, external APIs, IoT, and unstructured sources.
• Utilize MCP Server for orchestration and workflow control of pipelines and compliance processes.
• Mentor engineers and drive adoption of data-as-a-product thinking across teams.
  

Core Qualifications

• Bachelor's degree in computer science, Engineering, or related field.
• 5-10 years in data engineering with increasing leadership responsibilities.
• 3+ years with distributed data architectures and multi-domain ecosystems.
• Strong expertise in Python, SQL, and distributed computing (Spark, Flink, Kafka).
• Experience with AWS (or multi-cloud), hands on experience on AWS services, setting up Containerize applications through IAC Tools like Terraform, CDK.
• Familiarity with data catalog/discovery tools (e.g., DataHub, Collibra), schema evolution, and data marketplaces.
• Hands-on experience with GRC-focused systems, MCP Server, policy automation and AI technology.
  

Preferred Skills

• Knowledge of real-time analytics, MLOps, and self-healing pipelines.
• Experience in regulated industries and with privacy/regulatory standards (GDPR, CCPA).
• Exposure to emerging data technologies (e.g., Data Mesh platforms, decentralized protocols).
  

What Diligent Offers You

• Creativity is ingrained in our culture. We are innovative collaborators by nature. We thrive in exploring how things can be differently both in our internal processes and to help our clients
• We care about our people. Diligent offers a flexible work environment, global days of service, comprehensive health benefits, meeting free days, generous time off policy and wellness programs to name a few
• We have teams all over the world. We may be headquartered in New York City, but we have office hubs in Washington D.C., Vancouver, London, Galway, Budapest, Munich, Bengaluru, Singapore, and Sydney.
• Diversity is important to us. Growing, maintaining and promoting a diverse team is a top priority for us. We foster and encourage diversity through our Employee Resource Groups and provide access to resources and education to support the education of our team, facilitate dialogue, and foster understanding.
  

work onsite at least 50% of the time.