Senior AI/Big Data Engineer

• Design and implement

  large-scale, fault-tolerant data pipelines on OCI, using services like

  OCI Data Integration

  ,

  OCI Data Flow (Apache Spark)

  ,

  Object Storage

  , and

  Autonomous Database

  .
• Build and manage

  streaming data architectures

  using tools such as

  OCI GoldenGate

  ,

  Apache Kafka

  , and

  Spark/Flink Streaming

  .
• Enforce standards and automation across the

  entire data lifecycle

  , including schema evolution, dataset migration, and deprecation strategies.
• Improve platform

  resilience, data quality, and observability

  with advanced monitoring, alerting, and automated data governance.
• Serve as a

  technical leader

  , mentoring junior engineers, reviewing designs and code, and promoting engineering best practices.
• Collaborate cross-functionally with ML engineers, platform teams, and data scientists to integrate data services with AI/ML workloads.
• Partner in

  AI pipeline enablement

  , ensuring Lakehouse services efficiently support model training, feature engineering, and real-time inference.

Minimum Qualifications

• Bachelor's or Master's degree in

  Computer Science

  ,

  Engineering

  , or related technical field.
• 4-6 year's experience designing and building

  cloud-based data pipelines

  and

  distributed systems

  .
• Proficiency in at least one core language:

  Python

  ,

  Java

  , or

  Scala

  .
• Familiar with lakehouse formats (Iceberg, Delta, Hudi), file formats (Parquet, ORC, Avro), and streaming platforms (Kafka, Kinesis).
• Strong understanding of distributed systems fundamentals:

  partitioning

  ,

  replication

  ,

  idempotency

  ,

  consensus protocols

  .

Engineering & Infrastructure

• 5+ years building

  distributed systems

  or

  production-grade data platforms

  in the cloud.
• Strong coding proficiency in

  Python

  ,

  Java

  , or

  Scala

  , with an emphasis on performance and reliability.
• Expertise in

  SQL and PLSQL

  , data modeling, and query optimization.
• Proven experience with

  cloud-native architectures

  -especially

  OCI

  , AWS, Azure, or GCP.

Lakehouse & Streaming Mastery

• Deep knowledge of

  modern lakehouse/table formats

  :

  Apache Iceberg

  ,

  Delta Lake

  , or

  Apache Hudi

  .
• Production experience with

  big data compute engines

  :

  Spark

  ,

  Flink

  , or

  Trino

  .
• Skilled in

  real-time streaming

  and event-driven architectures using

  Kafka

  ,

  Flink

  ,

  GoldenGate

  , or

  Streaming

  .
• Experience managing

  data lakes

  , catalogs, and metadata governance in large-scale environments.

AI/ML Integration

• Hands-on experience enabling

  ML pipelines

  : from data ingestion to model training and deployment.
• Familiarity with

  ML frameworks

  (e.g.,

  PyTorch

  ,

  XGBoost

  ,

  scikit-learn

  ).
• Understanding of

  modern ML architectures

  : including

  RAG

  ,

  prompt chaining

  , and

  agent-based workflows

  .
• Awareness of

  MLOps practices

  , including model versioning, feature stores, and integration with AI pipelines.

DevOps & Operational Excellence

• Deep understanding of

  CI/CD

  , infrastructure-as-code (IaC), and release automation using tools like

  Terraform

  ,

  GitHub Actions

  , or

  CloudFormation

  .
• Experience with

  Docker

  ,

  Kubernetes

  , and

  cloud-native container orchestration

  .
• Strong focus on

  testing, documentation

  , and system observability (Prometheus, Grafana, ELK stack).
• Comfortable with

  cost/performance tuning

  , incident response, and data security standards (IAM, encryption, auditing).

Preferred Qualifications

• Experience with

  Oracle's cloud-native tools

  :

  OCI Data Integration

  ,

  Data Flow

  ,

  Autonomous Database

  ,

  GoldenGate

  ,

  OCI Streaming

  .
• Experience with

  query engines

  like

  Trino

  or

  Presto

  , and tools like

  dbt

  or

  Apache Airflow

  .
• Familiarity with

  data cataloging

  ,

  RBAC/ABAC

  , and enterprise data governance frameworks.
• Exposure to

  vector databases

  and

  LLM tooling

  (embeddings, vector search, prompt orchestration).
• Solid understanding of

  data warehouse design principles

  , star/snowflake schemas, and ETL optimization.

Soft Skills & Team Expectations

• Proven ability to lead technical initiatives

  independently end-to-end

  .
• Comfortable working in

  cross-functional teams

  and mentoring junior engineers.
• Excellent

  problem-solving skills

  , design thinking, and attention to operational excellence.
• Passion for

  learning emerging data and AI technologies

  and sharing knowledge across teams.

Career Level - IC3