Senior Data Engineer

4 - 8 years

0 Lacs

Posted:23 hours ago| Platform: Shine logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

As a Database Architect, your role involves designing and architecting robust data pipelines for structured, semi-structured, and unstructured data. You will be responsible for developing and managing databases (RDBMS, NoSQL, data lakes) to ensure data accessibility and integrity. Key Responsibilities: - Implementing efficient ETL processes using tools like PySpark and Hadoop to prepare data for analytics and AI use cases. - Optimizing database performance through query tuning, indexing, and caching strategies using Azure and GCP caching databases. - Monitoring, maintaining, and troubleshooting database infrastructure to ensure high availability and performance. - Building and maintaining CI/CD pipelines and managing YML files for efficient code integration and deployment. - Leveraging Docker for containerized deployment and managing database and pipeline processes with Docker commands. - Ensuring solutions follow best practices in system design, focusing on trade-offs, security, performance, and efficiency. - Collaborating with engineering teams to design scalable solutions for large-scale data processing. - Staying updated with the latest database technologies and implementing best practices for database design and management. Qualifications: - 4+ years of experience in database architecture and optimization. - Proven expertise in RDBMS, NoSQL, and semi-structured databases (e.g., MySQL, PostgreSQL, MongoDB). - Proficiency in programming languages for database integration and optimization (Python preferred). - Strong knowledge of distributed data processing tools like PySpark and Hadoop. - Hands-on experience with AWS services for data storage and processing, including S3. - Strong familiarity with Redis for caching and query optimization. - Experience with Docker for containerized deployment and writing CI/CD pipelines using YML files. - Ability to collaborate effectively with engineering teams to design scalable data solutions. - Strong problem-solving skills and attention to detail.,

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
UST logo
UST

IT Services and IT Consulting

Aliso Viejo CA

RecommendedJobs for You

pune, maharashtra, india