Director of Data Engineering [T500-20958]

About Arch:

Arch Capital Group Ltd. (Arch) is a leading global insurer with operations in more than a dozen countries. We write insurance, reinsurance and mortgage insurance on a worldwide basis, and our customers value us as an innovative partner and dependable risk manager with decades of fresh ideas and solid results. Part of the S&P 500, Arch has the size and capital position to remain a market-leading specialty insurer.

About Global Services India:

Arch Global Services India, established in 2025, is the newest part of Arch’s Global Service Organization. With offices in Trivandrum and Pune, Arch Global Services India is strategically positioned to provide support to Arch Insurance, Reinsurance and Mortgage divisions globally. At Arch Global Services India, we collaborate with teams across the globe to develop and implement solutions that deliver exceptional results for our clients worldwide.

The Position – Head/Director of Data Engineering:

Location:

Employment Type:

About the Role:

We are looking for an accomplished Data Engineering leader (Director) to lead our enterprise-wide data engineering function from our Hyderabad office. This leader will be responsible for architecting scalable, secure, and compliant data platforms while embedding operational excellence and 'shift-left' practices into the organization. The role will drive continuous optimization of data operations, ensuring proactive monitoring, faster issue resolution, and resilience at scale.

Key Responsibilities:

Strategic Leadership:

• Define and execute the enterprise data engineering roadmap aligned with business and technology priorities.
• Build, mentor, and lead a high-performing team of data engineers, platform specialists and support engineers.
• Partner with global leaders/counterparts to scale data platforms as critical enterprise assets.

Data Platform & Architecture:

• Architect resilient, high-availability data ecosystems across data lakes, warehouses, and streaming pipelines.
• Optimize data pipelines for cost efficiency, performance, and reliability across hybrid and cloud environments.
• Drive adoption of automation-first principles, reusable components, and standardized frameworks.

Operational Excellence & Shift-Left Practices:

• Embed testing, monitoring, and governance early in the development lifecycle to minimize production issues.
• Drive 'shift-left operations' by empowering data engineers with self-service monitoring, observability, and automated quality checks.
• Establish enterprise-wide monitoring, incident management, and root cause analysis frameworks for production data pipelines.
• Optimize operations by reducing manual interventions, automating data quality checks, and building proactive alerting systems.
• Build a scalable L2/L3 support model to ensure seamless 24/7 operations with strong SLAs.

Governance, Security & Compliance:

• Implement enterprise-wide data governance, lineage, cataloging, and access controls.
• Ensure regulatory compliance with GDPR, HIPAA, SOX, and other enterprise standards.
• Partner with InfoSec and compliance teams to embed data security at every stage ('security by design').

Collaboration & Delivery:

• Partner with analytics, BI, AI/ML, and application teams to deliver high-quality, governed data products.
• Manage global vendor/partner ecosystems to optimize tools, platforms, and operations.
• Foster a culture of collaboration, innovation, and accountability across the data function.

Required Qualifications:

• 15+ years of experience, with 7+ years in data engineering leadership roles within enterprise-scale organizations.
• Proven ability to operationalize and optimize enterprise data platforms with 'shift-left' principles.

Expertise in:

• Cloud & Storage:

  Azure Cloud, Azure Blob Storage

• Data Warehousing:

  Snowflake, SQL Server, Azure Synapse

• Data Integration & Pipelines:

  Azure Data Factory (ADF), StreamSets, Qlik Replicate, Astronomer (Airflow)

• Data Visualization & Analytics:

  Power BI

• Governance & MDM:

  Master Data Management, Collibra, Alation, Data Governance frameworks

• Big Data & Streaming:

  Spark, Kafka, Flink, Hadoop
• Strong skills in SQL, data modeling, and building scalable, automated ETL/ELT pipelines.
• Excellent stakeholder management, leadership, and communication skills.

Preferred Qualifications:

• Experience leading enterprise-scale data operations in Fortune 500 environments.
• Familiarity with data observability, FinOps/cost optimization, and proactive reliability engineering.
• Exposure to AI/ML data pipelines and advanced analytics use cases.
• Advanced degree in Computer Science, Engineering, or related field.

What We Offer:

• Opportunity to build and scale a global data engineering hub in Hyderabad.
• Exposure to next-gen enterprise platforms with a focus on operational excellence and shift-left practices.
• Competitive compensation and benefits.
• Collaborative, innovative, and transformation-driven enterprise culture.