Job
Description
At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries. If your passion for data analytics solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering! We are looking for someone to work from office location - Vikroli / Churchgate, Mumbai. Responsibilities: Design, develop, and maintain data pipelines and workflows using PySpark to process large volumes of data efficiently. Implement data ingestion, transformation, and storage solutions to support business requirements. Develop and optimize Python scripts and algorithms for data manipulation, analysis, and modeling tasks. Design and manage database systems, including schema design, indexing, and performance optimization. Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs. Perform data quality assessment, validation, and cleansing to ensure accuracy and integrity of data. Monitor and troubleshoot data pipeline issues, performance bottlenecks, and system failures. Stay up-to-date with emerging technologies and best practices in data engineering and analytics. Qualification: Bachelor's degree in Computer Science, Engineering, or related field. Should have a minimum of 3 years of experience in the field of Database or Data Analytics Minimum of 3 years of experience in data engineering, with expertise in Spark, Python, and database technologies. Experience with PySpark for data processing and analysis tasks. Proficiency in Python programming for scripting and automation. Solid understanding of database concepts and hands-on experience with SQL and NoSQL databases. Experience with data modeling, schema design, and ETL processes. Strong problem-solving skills and ability to work independently and collaboratively in a fast-paced environment. Excellent communication and interpersonal skills, with the ability to effectively communicate technical concepts to non-technical stakeholders. Good to have experience with a cloud platforms. Show more Show less
