Data Architect

Job Title: Data Architect What are my responsibilities? As a Data Architect, you will: Design & develop technical solutions to integrate disparate information and create meaningful insights for business using big-data architectures. Build and analyze large structured and unstructured datasets on scalable cloud infrastructures. Develop prototypes and proof of concepts using multiple data sources and big-data technologies. Process, manage, extract, and cleanse data to apply data analytics effectively. Design and develop scalable end-to-end data pipelines for batch and stream processing. Stay current with the Data Analytics landscape, exploring new technologies, techniques, tools, and methods. Inspire enthusiasm for using modern data technologies to solve problems and deliver business value. Qualification: Bachelor’s or Master’s in Computer Science & Engineering, or equivalent. Professional Degree in Data Engineering/Analytics is desirable. Experience Level: Minimum 8 years in software development, with at least 2-3 years hands-on experience in Big Data/Data Engineering. Desired Knowledge & Experience: Data Engineer - Big Data Developer: Spark: Spark 3.x, RDD/DataFrames/SQL, Batch/Structured Streaming; knowledge of internals like Catalyst/Tungsten/Photon. Databricks: Workflows, SQL Warehouses/Endpoints, DLT, Pipelines, Unity, Autoloader. IDE & Tools: IntelliJ/PyCharm, Git, Azure DevOps, GitHub Copilot. Testing: pytest, Great Expectations. CI/CD: Yaml Azure Pipelines, Continuous Delivery, Acceptance Testing. Big Data Design: Lakehouse/Medallion Architecture, Parquet/Delta, Partitioning, Distribution, Data Skew, Compaction. Languages: Python/Functional Programming (FP). SQL: TSQL, Spark SQL, HiveQL. Storage: Data Lake, Big Data Storage Design. Additional Helpful Skills: Data Pipelines: ADF, Synapse Pipelines, Oozie, Airflow. Languages: Scala, Java. NoSQL: Cosmos DB, MongoDB, Cassandra. Cubes: SSAS (ROLAP, HOLAP, MOLAP), AAS, Tabular Model. SQL Server: TSQL, Stored Procedures. Hadoop Stack: HDInsight, MapReduce, HDFS, YARN, Oozie, Hive, HBase, Ambari, Ranger, Atlas, Kafka. Data Catalogs: Azure Purview, Apache Atlas, Informatica. Big Data Architect: Expertise: Mastery of technologies, languages, and methodologies mentioned in Data Engineer – Big Data Developer. Mentorship: Mentor and educate developers on relevant technologies and methodologies. Architecture Styles: Lakehouse, Lambda, Kappa, Delta, Data Lake, Data Mesh, Data Fabric, Data Warehouses (e.g., Data Vault). Application Architecture: Microservices, NoSQL, Kubernetes, Cloud-native solutions. Experience: Proven track record across multiple technology generations (Data Warehouse → Hadoop → Big Data → Cloud → Data Mesh). Certification: Architect certifications such as Siemens Certified Software Architect or iSAQB CPSA. Required Soft Skills & Other Capabilities: Excellent communication skills to convey technical concepts to non-technical stakeholders. Strong attention to detail with a proven ability to solve complex business problems. Initiative and resilience to experiment with new ideas. Effective planning and organizational skills. Collaborative mindset for sharing ideas and developing solutions. Ability to work independently and in a global team environment.