AWS | Databricks Data Architect

Job Description

Experience 10+ Years. Must have architect experience. Data Architect : 1.AWS : PySpark with experience in architecting high throughput data lakes Glue ecosystem – Glue Jobs, Data Catalog, Bookmarks, Schema Registry, and Crawlers. Workflows – Airflow or Step Functions, or Glue Workflow Data Modeling – including medallion architecture, star schema (for architect) Understanding of Parquet and Iceberg file formats – including optimizing, partitioning and snapshotting (for architect) Good understanding of Data lineage, Data Governance, and DQ and AWS services around the same like Glue DQ, Deequ, GE, Lake Formation Redshift – Nice to have (since it is part of their existing infrastructure) Understanding of CDC and respective tools like Debezium, DMS AWS Certified Data Engineer Preferred OR 2. Databricks : PySpark with experience in architecting/building high throughput data lakes Workflows Autoloaders Understanding of Parquet and Iceberg/Hudi file formats – including optimizing, partitioning and snapshotting (for architect). Should understand DLTs Good understanding of Data lineage, Data Governance, and DQ including Unity Catalog Understanding of CDC and respective tools like Debezium, DMS Databricks certified engineer preferred