Python Web Scraper

As a Python Web Scraper at HiLabs in Bengaluru, Karnataka, you will play a crucial role in designing and building scalable web scraping solutions using Python/PySpark. Your responsibilities will include developing enterprise-grade scraping services, working with structured and unstructured data, implementing data validation processes, and collaborating with cross-functional teams to ensure smooth data flow between systems. Your technical expertise in Python, web scraping tools and libraries, PySpark, cloud-based platforms, and databases will be essential for optimizing data workflows and ensuring data accuracy, consistency, and availability. **Responsibilities:** - Design and build scalable, reliable web scraping solutions using Python/PySpark. - Develop enterprise-grade scraping services that are robust, fault-tolerant, and production-ready. - Work with large volumes of structured and unstructured data; parse, clean, and transform as required. - Implement robust data validation and monitoring processes to ensure accuracy, consistency, and availability. - Write clean, modular code with proper logging, retries, error handling, and documentation. - Automate repetitive scraping tasks and optimize data workflows for performance and scalability. - Optimize and manage databases (SQL/NoSQL) for efficient data storage, retrieval, and manipulation. - Analyze and identify relevant data sources for business needs. - Collaborate with data scientists, analysts, and engineers to integrate data across systems. **Desired Profile:** - Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. - 2-4 years of experience in web scraping, data crawling, or data. - Proficiency in Python with web scraping tools and libraries (e.g., Beautiful Soup, Scrapy, Selenium). - Basic working knowledge of PySpark, Apache Airflow, and EMR. - Experience with cloud-based platforms (AWS, Google Cloud, Azure) and cloud-native data tools. - Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). - Understanding of data governance, security best practices, and data privacy regulations. - Familiarity with version control systems like Git. If you are seeking a dynamic, fast-paced environment where you can leverage your expertise in web scraping and data manipulation to drive innovation in healthcare technology, HiLabs offers a collaborative workplace with competitive compensation, comprehensive benefits, and opportunities for professional growth and development. Thank you for considering a career with HiLabs, an equal opportunity employer dedicated to fostering diversity and inclusion in the workplace. For more information, visit our website at [hilabs.com](https://www.hilabs.com/privacy).,