Backend Data & Integration Engineer

4 years

2 - 3 Lacs

Posted:1 week ago| Platform: GlassDoor logo

Apply

Work Mode

On-site

Job Type

Part Time

Job Description

Location : Kochi

Employment Type : Full Time

Work Mode : Hybrid

Experience : 4-8 yrs

Job Code : BEO-5090

Posted Date : 02/09/2025


Job Description

Responsibilities

Role objective: Build data pipelines (crawling/parsing, deduplication/delta, embeddings) and connect external systems and interfaces.
Tasks
  • Development of crawling/fetch pipelines (API-first; playwright/requests only where permitted)
  • Parsing/normalization of job postings & CVs, deduplication/delta logic (seen hash, repost heuristics)
  • Embeddings/similarity search (controlling Azure OpenAI, vector persistence in pgvector)
  • Integrations: HR4YOU (API/webhooks/CSV import), SerpAPI, BA job board, email/SMTP
  • Batch/stream processing (Azure Functions/container jobs), retry/off, dead-letter queues
  • Telemetry for data quality (freshness, duplicate rate, coverage, cost per 1,000 items)
  • Collaboration with FE for exports (CSV/Excel, presigned URLs) and admin configuration

Desired Candidate Profile


  • 4+ years of end/data engineering experience
  • Python (FastAPI, pydantic, httpx/requests, Playwright/Selenium), solid TypeScript for smaller services/SDKs
  • Azure: Functions/Container Apps or AKS jobs, Storage/Blob, Key Vault, Monitor/Log Analytics
  • Messaging: Service Bus/Queues, idempotence & exactly-once semantics, pragmatic approach
  • Databases: PostgreSQL, pgvector, query design & performance tuning
  • Clean ETL/ELT patterns, testability (pytest), observability (OpenTelemetry)

Nice-to-haves
  • NLP/IE experience (spaCy/regex/rapidfuzz), document parsing (pdfminer/textract)
  • Experience with license/ToS-compliant data retrieval, captcha/anti-bot strategies (legally compliant)
  • Working method: API-first, clean code, trunk-based development, mandatory code reviews
  • Tools/stack: GitHub, GitHub Actions/Azure DevOps, Docker, pnpm/Turborepo (Monorepo), Jira/Linear, Notion/Confluence
  • On-call/support: rotating, "you build it, you run it"

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You