Job Description

This role is ideal for someone with strong technical skills in cloud computing, data engineering, and analytics, who is passionate about working with cutting-edge technologies in GCP to build robust and scalable data solutions. Key Responsibilities: Data Architecture and Design: Design and implement scalable, reliable, and high-performance data pipelines on Google Cloud. Define and implement data architecture strategies to store, process, and analyze large datasets efficiently. Create optimized schemas and ensure data structures meet business requirements. Data Pipeline Development: Build and maintain ETL (Extract, Transform, Load) pipelines using tools like Google Cloud Dataflow , Apache Beam , and Cloud Dataproc . Work with Google Cloud Storage (GCS), BigQuery , and Pub/Sub for data ingestion, storage, and analysis. Automate and orchestrate data workflows using tools such as Apache Airflow or Google Cloud Composer . Data Processing and Transformation: Develop and manage batch and real-time data processing solutions. Transform raw data into useful formats for analysis or machine learning models using BigQuery , Dataflow , or Dataproc . Collaboration: Collaborate with data scientists, analysts, and other stakeholders to understand business needs and deliver data solutions. Provide data support for machine learning and AI models, ensuring the data is clean, structured, and properly ingested. Optimization and Monitoring: Monitor and optimize the performance of data pipelines, ensuring minimal downtime and efficient use of resources. Troubleshoot data issues and resolve bottlenecks in the pipeline or storage systems Required Skills and Qualifications: Experience with GCP: Strong experience with GCP services like BigQuery, Google Cloud Storage (GCS), Dataflow, Dataproc, Pub/Sub, Cloud Composer, and Cloud Functions. Programming Languages: Proficient in languages such as Python, Java, or Scala for developing data pipelines and processing. ETL Tools: Experience with ETL frameworks, tools like Apache Beam, Airflow, or Cloud Data Fusion. Data Modeling and Warehousing: Understanding of data modeling, relational databases, and data warehousing concepts. SQL and NoSQL Databases: Strong proficiency in SQL, with experience in data analysis using BigQuery or other relational databases. Familiarity with NoSQL databases is a plus. Cloud Infrastructure: Knowledge of cloud architecture and infrastructure best practices on GCP. Data Analytics and BI Tools: Experience working with data visualization tools like Google Data Studio, Tableau, or Looker is a plus. DevOps Practices: Experience with CI/CD pipelines, version control systems (e.g., Git), and automated testing. Preferred Skills: Experience with containerized environments, including Docker and Kubernetes. Familiarity with machine learning tools like AI Platform on GCP. Ability to manage large datasets efficiently and design solutions that scale with growing data volumes. Education and Certifications: A bachelors or masters degree in Computer Science, Information Technology, Data Science, or a related field. Google Cloud Certified Professional Data Engineer or other relevant certifications are often preferred.