Job
Description
Company Overview Viraaj HR Solutions is a leading talent management firm dedicated to providing exceptional human resource services for businesses in various sectors. Our mission is to connect companies with the best talent while fostering a supportive and inclusive workplace culture. We value innovation, integrity, and excellence, ensuring that our clients thrive in a competitive market. Role Overview We are seeking a skilled PySpark Data Engineer to join our dynamic team in India. The ideal candidate will have a strong background in data engineering, with expertise in processing large volumes of data using PySpark. You will play a crucial role in designing and implementing data pipelines and ensuring efficient data storage and retrieval. This position requires strong analytical skills and the ability to work collaboratively in a fast-paced environment. Role Responsibilities Design, develop, and maintain robust data pipelines using PySpark. Implement ETL processes for data extraction, transformation, and loading. Work with structured and unstructured data sources and ensure data quality. Optimize data processing workflows for better performance and efficiency. Collaborate with data scientists and analysts to understand data requirements. Create and manage data warehousing solutions for end-user access. Integrate data from various sources into a unified platform. Monitor and troubleshoot data pipeline performance issues. Conduct data validation and cleansing to maintain data accuracy. Document processes and workflows for team transparency. Stay updated on new technologies and trends in data engineering. Participate in code reviews and ensure best practices in data engineering. Assist in technical designs and architecture for data solutions. Provide technical support and training to junior staff. Contribute to strategic discussions around data architecture and management. Qualifications Bachelor's degree in Computer Science, Engineering, or related field. Proven experience as a Data Engineer or similar role. Strong proficiency in PySpark and the Spark ecosystem. Hands-on experience with SQL and database management systems. Familiarity with Hadoop and big data technologies. Experience with cloud platforms like AWS, Azure, or Google Cloud. Understanding of data modeling and database design concepts. Proficient in Python for data manipulation and analysis. Strong analytical and troubleshooting skills. Excellent communication and teamwork abilities. Knowledge of data governance and compliance practices. Experience in implementing ETL and data warehousing solutions. Able to work in a fast-paced environment and manage multiple priorities. Familiarity with version control systems (e.g., Git). Willingness to learn and adapt to new technologies. This is an exciting opportunity to work with a forward-thinking company that values expertise and innovation in the field of data engineering. If you are passionate about data and eager to contribute to transformative projects, we encourage you to apply today. Skills: big data technologies,etl,database management systems,aws glue,problem-solving skills,google cloud,version control systems,sql,data modeling,aws,azure,pyspark,data engineer,data warehousing,data governance,python,git,hadoop Show more Show less
