Data Engineer

Key Responsibilities:

Generative AI Development

generative AI

or Llama-Index.

Apply prompt engineering techniques to design effective queries and ensure

optimal LLM responses for diverse use cases.

Master advanced LLM functionalities, including prompt optimization,

hyperparameter tuning, and response caching.

Implement Retrieval-Augmented Generation (RAG) workflows by integrating

vector databases like Pinecone, Weaviate, Supabase, or PGVector for efficient

similarity searches.

Work with embeddings and build solutions that leverage similarity search for

personalized query resolution.

Explore and process multimodal data, including image and video understanding

and generation.

Integrate observability tools for monitoring and evaluating LLM performance to

ensure system reliability.

Backend Engineering

Python

FastAPI, Django, or Flask.

Design and implement RESTful APIs for seamless communication between

systems and services.

Optimize database performance with relational databases (PostgreSQL, MySQL)

and integrate vector databases (Pinecone, PGVector, Weaviate, Supabase) for

advanced AI workflows.

Implement asynchronous programming and adhere to clean code principles for

maintainable, high-quality code.

Seamlessly integrate third-party SDKs and APIs, ensuring robust interoperability

with external systems.

Develop backend pipelines for handling multimodal data processing, and

supporting text, image, and video workflows.

Manage and schedule background tasks with tools like Celery, cron jobs, or

equivalent job queuing systems.

Leverage containerization tools such as Docker for efficient and reproducible

deployments.

Cloud Proficiency: Familiarity with platforms like AWS, GCP, or Azure, including

serverless applications and VM setups.

Frontend Basics: Understanding of HTML, CSS, and optionally JavaScript

frameworks like Angular or React for better collaboration with frontend teams.

Certifications:

technologies are a plus.