Home
Jobs
Companies
Resume
Virtusa logo

Data Engineer

5 years

0 Lacs

Andhra Pradesh India

Posted:1 week ago| Platform: Linkedin logo

Apply

Skills Required

data design python orchestration automation apache airflow management analytics reporting kafka sqs processing collaboration engineering optimization governance security compliance integrity documentation etl scripting messaging aws database azure gcp agile development programming scala java containerization docker kubernetes learning integration communication

Work Mode

On-site

Job Type

Full Time

Job Description

Responsibilities Design and Develop Scalable Data Pipelines: Build and maintain robust data pipelines using Python to process, transform, and integrate large-scale data from diverse sources. Orchestration and Automation: Implement and manage workflows using orchestration tools such as Apache Airflow to ensure reliable and efficient data operations. Data Warehouse Management: Work extensively with Snowflake to design and optimize data models, schemas, and queries for analytics and reporting. Queueing Systems: Leverage message queues like Kafka, SQS, or similar tools to enable real-time or batch data processing in distributed environments. Collaboration: Partner with Data Science, Product, and Engineering teams to understand data requirements and deliver solutions that align with business objectives. Performance Optimization: Optimize the performance of data pipelines and queries to handle large scales of data efficiently. Data Governance and Security: Ensure compliance with data governance and security standards to maintain data integrity and privacy. Documentation: Create and maintain clear, detailed documentation for data solutions, pipelines, and workflows. Qualifications Required Skills: 5+ years of experience in data engineering roles with a focus on building scalable data solutions. Proficiency in Python for ETL, data manipulation, and scripting. Hands-on experience with Snowflake or equivalent cloud-based data warehouses. Strong knowledge of orchestration tools such as Apache Airflow or similar. Expertise in implementing and managing messaging queues like Kafka, AWS SQS, or similar. Demonstrated ability to build and optimize data pipelines at scale, processing terabytes of data. Experience in data modeling, data warehousing, and database design. Proficiency in working with cloud platforms like AWS, Azure, or GCP. Strong understanding of CI/CD pipelines for data engineering workflows. Experience working in an Agile development environment, collaborating with cross-functional teams. Preferred Skills Familiarity with other programming languages like Scala or Java for data engineering tasks. Knowledge of containerization and orchestration technologies (Docker, Kubernetes). Experience with stream processing frameworks like Apache Flink. Experience with Apache Iceberg for data lake optimization and management. Exposure to machine learning workflows and integration with data pipelines. Soft Skills Strong problem-solving skills with a passion for solving complex data challenges. Excellent communication and collaboration skills to work with cross-functional teams. Ability to thrive in a fast-paced, innovative environment. Show more Show less

More Jobs at Virtusa

Gen AI Senior Developer/Lead

Chennai, Bengaluru, Hyderabad

6 - 9 yrs
|

INR 22 - 37 Lacs

ETL Testing

Bengaluru, Gurgaon, Mumbai (All Areas)

5 - 9 yrs
|

INR 9 - 17 Lacs

Pega UI(React / Angular) Developer

Bengaluru, Hyderabad, Mumbai (All Areas)

5 - 9 yrs
|

INR 7 - 17 Lacs

Java Full stack

Chennai, Pune

6 - 9 yrs
|

INR 8 - 12 Lacs

Eagle Platform Engineer

Chennai, Pune, Hyderabad

7 - 12 yrs
|

INR 9 - 14 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Data Interview Now

My Connections Virtusa

Download Chrome Extension (See your connection in the Virtusa )

chrome image
Download Now
Virtusa
Virtusa
www.virtusa.com

Information Technology and Services

Southborough

20,000+ Employees

3452 Jobs

    Key People

  • Kris Canekeratne

    Chairman and CEO

  • Sanjay Singh

    President and COO

RecommendedJobs for You

Emtec Inc. logo

Data Engineer

Emtec Inc.

Kolkata, Mumbai, New Delhi, Hyderabad, Pune, Chennai, Bengaluru

MAN Truck & Bus logo

Data Engineer

MAN Truck & Bus

Pune

Arise Global Services logo

Data Engineer

Arise Global Services

Kochi

Proclink Consulting Services LLP logo

Data Engineer

Proclink Consulting Services LLP

Hyderabad, Ahmedabad

Varite logo

Data Engineer

Varite

Hyderabad, Chennai, Bengaluru

Vestas logo

Data Engineer

Vestas

Chennai, Tamil Nadu, India

NuWare logo

Data Engineer

NuWare

Bengaluru

Euphoric Thought Technologies Pvt Ltd logo

Data Engineer

Euphoric Thought Technologies Pvt Ltd

Hyderābād

Wipro Limited logo

Data Engineer

Wipro Limited

Hyderābād

Capgemini logo

Data Engineer

Capgemini

Mumbai