Job Summary
We are looking for an experienced Data Architect with strong expertise in Databricks, cloud data platforms, and modern data engineering technologies. The ideal candidate will be responsible for designing, developing, and implementing scalable, secure, and high-performance data architectures to support enterprise analytics and business intelligence initiatives.
The candidate should have strong hands-on experience in data modelling, ETL/ELT pipelines, advanced SQL, Python, and cloud technologies, along with the ability to lead technical discussions and provide architectural guidance.
Key Responsibilities
- Design and implement scalable, reliable, and high-performance data architecture solutions.
- Define and establish enterprise data architecture strategies, standards, and best practices.
- Lead the design and development of modern data platforms using Databricks.
- Develop and optimize ETL/ELT pipelines for large-scale data processing and integration.
- Design data models, data warehouses, data lakes, and Lakehouse architectures.
- Leverage Apache Spark and Python for distributed data processing and transformation.
- Implement data integration solutions across multiple sources and cloud platforms.
- Ensure data quality, consistency, security, governance, and compliance across data systems.
- Optimize SQL queries, data pipelines, and storage solutions to improve performance and cost efficiency.
- Collaborate with data engineers, business analysts, solution architects, and stakeholders to translate business requirements into technical solutions.
- Establish CI/CD practices, version control, and deployment strategies for data engineering workflows.
- Provide technical leadership, mentor engineering teams, and drive architectural best practices.
Required Skills & Qualifications
- 10+ years of experience in Data Architecture, Data Engineering, and Analytics.
- Strong expertise in Databricks, including its architecture and capabilities.
- Deep knowledge of ETL/ELT processes, data modelling, and advanced SQL.
- Proficiency in at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform (GCP).
- Hands-on experience with Python, Apache Airflow, Apache Spark, and Git.
- Strong understanding of data warehousing, data lakes, Lakehouse architecture, and distributed computing.
- Experience designing and implementing scalable data pipelines and data integration frameworks.
- Knowledge of Delta Lake, data partitioning, performance tuning, and optimization techniques.
- Familiarity with data governance, metadata management, data security, and access control.
- Knowledge of version control systems (Git), CI/CD pipelines, and DevOps practices.
- Excellent analytical, problem-solving, stakeholder management, and communication skills.
Preferred Candidate Profile
- Candidates with a strong background in designing and implementing enterprise-scale data architecture solutions.
- Experience in migrating legacy data platforms to modern cloud-based data platforms.
- Practical experience with Databricks Lakehouse, Delta Lake, Unity Catalog, and Delta Live Tables (DLT).
- Exposure to cloud-native data services such as AWS Glue, Azure Data Factory, Azure Data Lake Storage, or Google Cloud Dataflow.
- Experience implementing data governance frameworks, data lineage, and metadata management solutions.
- Understanding of data security frameworks, encryption, role-based access control (RBAC), and regulatory compliance.
- Experience working in Agile environments and collaborating with cross-functional teams.
- Candidates with Databricks certifications or relevant cloud certifications will be an added advantage.
- Ability to independently drive architecture decisions, communicate technical solutions, and provide guidance to engineering teams.