Senior Data Engineer - Data Acquisition

About the Job

Senior Data Engineer – Data Acquisition (ODS)

You will be responsible for designing, developing, and maintaining robust data acquisition frameworks that support large-scale analytics and operational data systems.

Job Title

Senior Data Engineer – Data Acquisition (ODS)

Skills

Kafka, Python, Spark, Databricks, SQL, Snowflake, Azure Data Factory (ADF), XML, JSON, Scala (Basic)

Experience Required :

Location:

Notice Period:

• Design, develop, and maintain

  data ingestion and transformation pipelines

  in an Azure-based environment.
• Work with

  Kafka

  for real-time data streaming and message queuing.
• Build scalable

  ETL/ELT pipelines

  using

  Databricks (PySpark/Scala)

  and integrate with

  Snowflake

  for analytics workloads.
• Develop and optimize SQL queries and stored procedures for performance and data accuracy.
• Utilize

  Azure Data Factory (ADF)

  to orchestrate workflows and integrate with various data sources.
• Work with structured and semi-structured data formats such as

  XML, JSON, Parquet, and CSV

  .
• Collaborate with data architects, analysts, and business teams to deliver high-quality, reliable data pipelines.
• Ensure code quality, version control, and data security best practices are followed.
• Contribute to data architecture design discussions and provide technical recommendations for system improvements.

Key Responsibilities

• End-to-end development of

  data acquisition solutions

  for operational and analytical systems.
• Implement

  data transformation logic

  using Spark and Python.
• Optimize performance of large-scale data pipelines in Databricks/Snowflake.
• Handle real-time and batch data ingestion via Kafka and ADF.
• Conduct data validation, profiling, and documentation of technical specifications.

Qualification & Certification

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• Microsoft Azure or Snowflake certification (preferred).
• Strong understanding of modern data engineering best practices.

Good to Have

• Experience with

  DevOps tools

  and CI/CD pipelines for data deployment.
• Exposure to

  data modeling

  and

  data warehouse design

  .
• Familiarity with other streaming platforms or data orchestration tools.

About InfoBeans

InfoBeans is a global digital transformation and product engineering company, enabling businesses to thrive through innovation, agility, and cutting-edge technology solutions. With over 1,700 team members across the globe, we specialize in custom software development, enterprise solutions, cloud, AI/ML, UX, automation, and digital transformation services.

“Creating WOW!”

🔗 https://infobeans.ai/