Senior Principal Consultant - Data Architect

Role Overview:

Data Architect

Required Skills and Experience:

• 8+ years of experience in data architecture or engineering roles, with at least 3+ years specializing in cloud-based big data solutions.
• Hands-on expertise with

  Databricks

  on

  Azure

  or

  AWS

  .
• Deep understanding of

  Delta Lake

  ,

  medallion architecture

  (bronze/silver/gold zones), and data governance tools (e.g., Unity Catalog, Purview).
• Strong experience migrating large datasets and batch/streaming pipelines from on-prem to Databricks.
• Expertise with Spark (PySpark/Scala) at scale and optimizing Spark jobs.
• Familiarity with ingestion from RDBMS (Oracle, SQL Server) and legacy Hadoop ecosystems.
• Proficiency in orchestration tools (Databricks Workflows, Airflow, Azure Data Factory, AWS Glue Workflows).
• Strong understanding of cloud-native services for storage, compute, security, and networking.

Preferred Qualifications:

• Databricks Certified Data Engineer or Architect.
• Azure/AWS cloud certifications.
• Experience with real-time/streaming ingestion (Kafka, Event Hubs, Kinesis).
• Familiarity with data quality frameworks (e.g., Deequ, Great Expectations).

Key Responsibilities:

• Define and design cloud-native data architecture on Databricks using Delta Lake, Unity Catalog, and related services.
• Develop migration strategies for moving on-premises data workloads (Oracle, Hadoop, Exadata, etc.) to Databricks on Azure/AWS.
• Architect and oversee data pipelines supporting ingestion, curation, transformation, and analytics in a multi-layered (bronze/silver/gold) model.
• Lead data modeling, schema design, performance optimization, and data governance best practices.
• Collaborate with data engineering, platform, and security teams to build production-ready solutions.
• Create standards for ingestion frameworks, job orchestration (e.g., workflows, Airflow), and data quality validation.
• Support cost optimization, scalability design, and operational monitoring frameworks.
• Guide and mentor engineering teams during the build and migration phases.

Attributes for Success:

• Ability to lead architecture discussions with technical and business stakeholders.
• Passion for modern cloud data architectures and continuous learning.
• Pragmatic and solution-driven approach to migrations.