We are seeking an experienced Data Engineer, skilled in building and managing data orchestration pipelines within cloud-native environments. The ideal candidate will have extensive experience with Kubernetes, Airflow, Python, modern observability tools (Grafana + Prometheus) and Google Cloud Platform (GCP). You will be responsible for designing, developing, and maintaining data pipelines that support NLP and LLM models, ensuring data quality, scalability, and reliability. Key Responsibilities: Design and Develop Data Pipelines: Create, manage, and optimize data collection and processing pipelines using Airflow, Kubernetes (GKE), and GCP to handle large volumes of text-based social media data. Cloud Infrastructure Management: Implement and maintain cloud infrastructure on GCP, ensuring high availability, scalability, and security of data processing environments. Data Integration: Develop robust data integration solutions to aggregate data from various social media platforms and other sources, ensuring data consistency and reliability. NLP and LLM Model Support: Work closely with data scientists and machine learning engineers to support the deployment and maintenance of NLP and LLM models in production. Database Management: Design, manage, and optimize databases for storage and retrieval of large-scale text data, ensuring efficient data access and query performance. Version Control: Utilize Git for version control and collaboration on codebases, ensuring best practices in code management and deployment. Performance Tuning: Monitor and improve the performance of data pipelines, identifying and resolving bottlenecks and inefficiencies. Documentation: Maintain comprehensive documentation for all data engineering processes, ensuring transparency and knowledge sharing within the team. Collaboration: Work collaboratively with cross-functional teams, including data scientists, product managers, and other stakeholders, to understand data requirements and deliver solutions that meet business needs. Requirements: Airflow on GKE Production Experience DAG authoring, Helm/Terraform cluster provisioning, autoscaling (KEDA/HPA/GKE Autopilot), and CI/CD of DAGs. Observability & Monitoring Vision Hands-on dashboarding in Grafana , metrics via Prometheus/Cloud Monitoring, and definition of SLA/SLOs for pipelines. Python Expertise Advanced Python for data processing, custom Airflow operators/hooks/sensors, and familiarity with Airflow 2.x. GCP Core Services Daily use of BigQuery, Cloud Storage, Pub/Sub, Secret Manager, IAM/Workload Identity, VPC-SC; infrastructure as code with Terraform. Database & SQL Skills Proficiency with relational databases (PostgreSQL) Git & DevOps Practices Branching strategies, code reviews, automated testing, and GitOps-style deployments. Preferred / Bonus: Prior experience supporting large-scale NLP or LLM workloads. Familiarity with social-media APIs (Twitter/X, Reddit, TikTok, Meta). GCP Professional Data Engineer or Cloud DevOps Engineer certification.
Roles and Responsibilities 1. Conceptualize / Prepare / Check GA drawings and layout drawings of equipment as per orders received / client requirement 2. Prepare detailed fabrication / manufacturing drawings with bill of materials of ducting, fans, scrubbers, dust collectors etc. Also, prepare / check and release specification data sheets for Bought Out items like motors, bag / cartridge, RAV, explosion vent, expansion joints etc 3. Prepare / check layout and assembly drawings 4. Standardization of drawings 5. Prepare / check nesting drawings for laser cutting of sheets and plates as required 6. Coordination with shopfloor and construction sites during manufacturing, testing and commissioning of equipment 7. Site visits for preparing design proposal, layout drawings and data collection
Looking for someone with basic internet and Excel skills Work from home with international clients. What You Will Do: • Log in & perform actions on different accounts • Download and format reports • Search for info online & organize it neatly Flexi working Work from home