Big Data Architect

Job Summary:

We are looking for a skilled Big Data Engineer to design, develop, and maintain large-scale data processing systems that enable our organization to analyze vast amounts of data efficiently. The ideal candidate will have expertise in big data technologies, data pipelines, and distributed computing to support business intelligence, analytics, and machine learning initiatives.

Key Responsibilities:

• Design, build, and maintain scalable big data pipelines and architectures to process structured and unstructured data.
  
• Develop and optimize ETL/ELT workflows to ingest, clean, transform, and load data from various sources.
  
• Work with technologies such as Hadoop, Spark, Kafka, Hive, HBase, and related ecosystems.
  
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
  
• Implement data storage solutions using data lakes, data warehouses, and cloud platforms (AWS, Azure, GCP).
  
• Ensure data quality, security, and governance throughout the data lifecycle.
  
• Monitor system performance and troubleshoot issues in big data environments.
  
• Stay updated with emerging big data technologies and best practices.
  
• Document architecture designs, data flow diagrams, and technical specifications.
  

Qualifications

• Bachelors degree in Computer Science, Information Technology, Engineering, or related field.
  
• 3+ years of experience in big data engineering or related roles.
  
• Proficiency with big data frameworks like Hadoop, Apache Spark, Kafka, Hive, or Flink.
  
• Strong programming skills in Java, Scala, or Python.
  
• Experience with SQL and NoSQL databases (e.g., Cassandra, HBase, MongoDB).
  
• Knowledge of cloud data platforms and services (AWS EMR, Azure HDInsight, Google BigQuery).
  
• Familiarity with data modeling, data warehousing, and ETL tools.
  
• Understanding of distributed computing and parallel processing concepts.
  
• Strong problem-solving and analytical skills.
  

Preferred:

• Experience with containerization (Docker, Kubernetes) and DevOps tools.
  
• Knowledge of data governance, data security, and compliance standards.
  
• Experience with machine learning pipelines and AI integration.
  
• Certifications in big data technologies or cloud platforms.