Data Engineer

Design, build, and maintain data pipelines, ETL/ELT workflows, and scalable microservices.

Development of complex web scraping (playwright) and realtime pipelines (Kafka/Queues/Flink).

Develop end-to-end microservices with backend (Java 5+, Python 5+, Spring Batch 2+) and frontend (React or any).

Deploy, publish, and operate services in AWS ECS/Fargate using CI/CD pipelines (Jenkins, GitOps).

Architect and optimize data storage models in SQL (MySQL, PostgreSQL) and NoSQL stores.

Implement web scraping and external data ingestion pipelines.

Enable Databricks and PySpark-based workflows for large-scale analytics.

Build advanced data search capabilities (fuzzy matching, vector similarity search, semantic retrieval).

Apply ML techniques (scikit-learn, classification algorithms, predictive modeling) to data-driven solutions.

Implement observability, debugging, monitoring, and alerting for deployed services.

Create Mermaid sequence diagrams, flowcharts, and dataflow diagrams to document system architecture and workflows.

Drive best practices in fullstack data service development, including architecture, testing, and documentation.

2+ years of experience as a Data Engineer or a Software Backend engineering role.

Strong programming skills in Python, Scala, or Java.

Hands-on experience with HBase or similar NoSQL columnar stores.

Hands-on experience with distributed data systems like Spark, Kafka, or Flink.

Proficient in writing complex SQL and optimizing queries for performance.

Experience building and maintaining robust ETL/ELT pipelines in production.

Familiarity with workflow orchestration tools (Airflow, Dagster, or similar).

Understanding of data modeling techniques (star schema, dimensional modeling, etc.).

Familiarity with CI/CD pipelines (Jenkins or similar).

Ability to visualize and communicate architectures using Mermaid diagrams.

Experience working with Databricks, dbt, Terraform, or Kubernetes.

Familiarity with streaming data pipelines or real-time processing.

Exposure to data governance frameworks and tools.

Experience supporting data products or ML pipelines in production.

Strong understanding of data privacy, security, and compliance best practices.

Data with purpose:

Modern tooling:

Collaborative culture: