We are seeking an experienced Data Engineer, skilled in building and managing data orchestration pipelines within cloud-native environments. The ideal candidate will have extensive experience with Kubernetes, Airflow, Python, modern observability tools (Grafana + Prometheus) and Google Cloud Platform (GCP). You will be responsible for designing, developing, and maintaining data pipelines that support NLP and LLM models, ensuring data quality, scalability, and reliability. Key Responsibilities: Design and Develop Data Pipelines: Create, manage, and optimize data collection and processing pipelines using Airflow, Kubernetes (GKE), and GCP to handle large volumes of text-based social media data. Cloud Infrastructure Management: Implement and maintain cloud infrastructure on GCP, ensuring high availability, scalability, and security of data processing environments. Data Integration: Develop robust data integration solutions to aggregate data from various social media platforms and other sources, ensuring data consistency and reliability. NLP and LLM Model Support: Work closely with data scientists and machine learning engineers to support the deployment and maintenance of NLP and LLM models in production. Database Management: Design, manage, and optimize databases for storage and retrieval of large-scale text data, ensuring efficient data access and query performance. Version Control: Utilize Git for version control and collaboration on codebases, ensuring best practices in code management and deployment. Performance Tuning: Monitor and improve the performance of data pipelines, identifying and resolving bottlenecks and inefficiencies. Documentation: Maintain comprehensive documentation for all data engineering processes, ensuring transparency and knowledge sharing within the team. Collaboration: Work collaboratively with cross-functional teams, including data scientists, product managers, and other stakeholders, to understand data requirements and deliver solutions that meet business needs. Requirements: Airflow on GKE Production Experience DAG authoring, Helm/Terraform cluster provisioning, autoscaling (KEDA/HPA/GKE Autopilot), and CI/CD of DAGs. Observability & Monitoring Vision Hands-on dashboarding in Grafana , metrics via Prometheus/Cloud Monitoring, and definition of SLA/SLOs for pipelines. Python Expertise Advanced Python for data processing, custom Airflow operators/hooks/sensors, and familiarity with Airflow 2.x. GCP Core Services Daily use of BigQuery, Cloud Storage, Pub/Sub, Secret Manager, IAM/Workload Identity, VPC-SC; infrastructure as code with Terraform. Database & SQL Skills Proficiency with relational databases (PostgreSQL) Git & DevOps Practices Branching strategies, code reviews, automated testing, and GitOps-style deployments. Preferred / Bonus: Prior experience supporting large-scale NLP or LLM workloads. Familiarity with social-media APIs (Twitter/X, Reddit, TikTok, Meta). GCP Professional Data Engineer or Cloud DevOps Engineer certification.