Job
Description
As a Software Development Engineer III at Hiver, you will be responsible for building and scaling backend systems that support both traditional workflows and AI-driven features. Here is what you will be doing: - Own low-latency, cost-efficient AI endpoints and data pipelines. - Integrate intelligent workflows into existing services with measurable impact. - Mentor engineers and elevate engineering standards. In this role, you will focus on various tasks including: - Making the architecture scalable to accommodate the growing traffic and AI workloads. - Building frameworks to monitor, enhance, and optimize AI-backed systems. - Enhancing the reliability, latency, and performance of traditional and AI services. - Designing and managing APIs in both monolith and microservices environments. - Developing event-driven systems using Kafka/RabbitMQ for high-volume pipelines. - Implementing AI components such as model serving, inference/generation APIs, retrieval/RAG, embeddings, rerankers, and vector stores. - Establishing evaluation and guardrails including test sets, canaries, A/B testing, drift detection, content safety, and fallback chains. - Creating secure storage and processing solutions for large-scale structured/unstructured data while enforcing data contracts. - Taking ownership of observability through tracing, metrics, feature flags, model/version routing, SLOs, and error budgets. - Debugging production issues across services and layers, leading incident response and postmortems. - Collaborating with AI/ML engineers and data scientists to operationalize models and notebooks. - Optimizing cost and latency through caching, token budgets, autoscaling, and hardware placement. What we are looking for in a candidate: - Strong experience in scaling backend/distributed systems and microservices. - Proficiency in concurrency, along with a deep understanding of reliability, performance, and resiliency. - Experience with event-driven architecture using Kafka/RabbitMQ and handling high-volume data pipelines. - Hands-on experience with SQL, knowledge of NoSQL/caches, and familiarity with vector databases. - Exposure to production model-serving techniques like embeddings, RAG, real-time inference APIs, or evaluation harnesses. - Proficiency in one or more of Go, Java, or Python, with a focus on writing high-quality, maintainable code. - Experience in cloud deployment (preferably AWS; GCP/Azure is acceptable), working with containers, CI/CD, and infrastructure as code. - Understanding of security and privacy fundamentals regarding PII and content safety. - Nice to have skills include familiarity with Triton/TorchServe/vLLM, quantization, OpenTelemetry, pgvector/OpenSearch/Pinecone, and feature-flag platforms. - Prior experience collaborating with AI/ML teams from research to production is desirable, along with a proven track record of scaling systems for over 5 years and owning production services.,
