448 Scrapy Jobs - Page 16

Hi, With reference to your profile on job portal we would like to share an opportunity with you for one of our Gurgaon Based client for Gurgaon location. Please find below the details regarding same: Location: Remote/WFH Experience: 6-10 Years Title: Manager-Data Engineer (Web Scraping) Notice Period: Only Immediate Joiner - 30 Days Max Job Responsibilities Technical Skills Required: Proficiency in Python and SQL/Database skills is required. Must have strong expertise in using Pandas library (Python). Experience with web technologies (HTML/JS, APIs, etc.) is essential. Should have a good understanding of tools such as Scrapy, BeautifulSoup, and Selenium. Responsible for reviewing and approvi...

Role : Senior Python Locations : Teynampet, Chennai/Kk Nagar, from Office : (1pm to 10pm) Monday of Interview : : 3+ Skills : 3+ years of Strong experience in Python & 2 years in Django Web framework Experience or Knowledge in implementing various Design Patterns Good Understanding of MVC framework & Object-Oriented Programming Experience in PGSQL / MySQL and MongoDB Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc. Experience developing in a Linux environment, GIT & Agile methodology Good to have knowledge in any one of the JavaScript frameworks: jQuery, Angular, ReactJS Good to have experience in implementing charts...

Roles and Responsibilities Design, develop, test, and deploy scalable web applications using Python frameworks such as Django or Flask. Collaborate with cross-functional teams to identify requirements and implement solutions that meet business needs. Ensure high-quality code by writing unit tests, integrating with databases like MongoDB, and good to have implementing data science concepts using Numpy, Pandas, Scrapy etc. Participate in Agile development methodologies to deliver projects on time and within budget. Troubleshoot issues related to application performance optimization and scalability. Desired Candidate Profile 3-7 years of experience in software development with expertise in Pyth...

Job Designation - Data Scraping Location - Ahmedabad Years of Experience - 3+ years We are seeking a skilled and motivated Web Crawler to join our team. The ideal candidate will have 3 to 5 years of experience in developing and maintaining robust web scraping solutions. You will be responsible for designing, implementing, and optimizing web crawlers to extract valuable data from diverse online sources. This role requires a strong understanding of web technologies, data handling, and problem-solving skills. Responsibilities: •Design, develop, and maintain efficient and scalable web crawlers using Python, Mozenda, etc. •Utilize web scraping frameworks such as Scrapy, Beautiful Soup, or Seleniu...

We are accepting applications for experienced Data Engineer with a strong background in data scraping, cleaning, transformation, and automation. The ideal candidate will be responsible for building robust data pipelines, maintaining data integrity, and generating actionable dashboards and reports to support business decision-making. Key Responsibilities: Develop and maintain scripts for scraping data from various sources including APIs, websites, and databases. Perform data cleaning, transformation, and normalization to ensure consistency and usability across all data sets. Design and implement relational and non-relational data tables and frames for scalable data storage and analysis. Build...

About the Role: We are looking for a skilled Python Developer with strong expertise in web scraping and data extraction. You will be responsible for designing and maintaining scalable scraping systems, handling large volumes of data, and ensuring the accuracy and integrity of data from various online sources. Responsibilities: Develop and maintain Python scripts for scraping structured and unstructured data from websites and APIs. Build robust, scalable, and efficient scraping solutions using libraries such as BeautifulSoup, Scrapy, Selenium, or Playwright. Monitor and optimize scraping performance and manage data pipelines. Handle website structure changes, anti-bot protections, and CAPTCHA...

We are looking for a skilled Python Developer with strong expertise in web scraping and data extraction. You will be responsible for designing and maintaining scalable scraping systems, handling large volumes of data, and ensuring data accuracy and integrity from various online sources. Responsibilities: Develop and maintain Python scripts for scraping structured and unstructured data from websites and APIs. Build robust, scalable, and efficient scraping solutions using libraries such as BeautifulSoup, Scrapy, Selenium, or Playwright. Monitor and optimize scraping performance and manage data pipelines. Handle website structure changes, anti-bot protections, and CAPTCHA bypassing when necessa...

About the Role: We are looking for a skilled Python Developer with strong expertise in web scraping and data extraction. You will be responsible for designing and maintaining scalable scraping systems, handling large volumes of data, and ensuring the accuracy and integrity of data from various online sources. Responsibilities: Develop and maintain Python scripts for scraping structured and unstructured data from websites and APIs. Build robust, scalable, and efficient scraping solutions using libraries such as BeautifulSoup, Scrapy, Selenium, or Playwright. Monitor and optimize scraping performance and manage data pipelines. Handle website structure changes, anti-bot protections, and CAPTCHA...

Job Summary: We are looking for a skilled Python Developer with strong expertise in web scraping and data extraction. You will be responsible for designing and maintaining scalable scraping systems, handling large volumes of data, and ensuring data accuracy and integrity from various online sources. Responsibilities: Develop and maintain Python scripts for scraping structured and unstructured data from websites and APIs. Build robust, scalable, and efficient scraping solutions using libraries such as BeautifulSoup, Scrapy, Selenium, or Playwright. Monitor and optimize scraping performance and manage data pipelines. Handle website structure changes, anti-bot protections, and CAPTCHA bypassing...

Data Science and AI Developer **Job Description:** We are seeking a highly skilled and motivated Data Science and AI Developer to join our dynamic team. As a Data Science and AI Developer, you will be responsible for leveraging cutting-edge technologies to develop innovative solutions that drive business insights and enhance decision-making processes. **Key Responsibilities:** 1. Develop and deploy machine learning models for predictive analytics, classification, clustering, and anomaly detection. 2. Design and implement algorithms for data mining, pattern recognition, and natural language processing. 3. Collaborate with cross-functional teams to understand business requirements and translat...

ABOUT THE PYTHON DATA ENGINEER ROLE: We are looking for a skilled Python Data Engineer to join our team and work on building high-performance applications and scalable data solutions. In this role, you will be responsible for designing, developing, and maintaining robust Python-based applications, optimizing data pipelines, and integrating various APIs and databases. This is more than just a coding role—it requires strategic thinking, creativity, and a passion for data-driven decision-making to drive results and innovation. KEY RESPONSIBILITIES: Develop, test, and maintain efficient Python applications. Design, develop, and maintain ETL pipelines for efficient data extraction, transformation...

About the Role: We are looking for a hands-on Data Engineer to join our team and take full ownership of scraping pipelines and data quality. You'll be working on data from 60+ websites involving PDFs, processed via OCR and stored in MySQL/PostgreSQL. You’ll build robust, self-healing pipelines and fix common data issues (missing fields, duplication, formatting errors). Responsibilities: Own and optimize Airflow scraping DAGs for 60+ sites Implement validation checks, retry logic, and error alerts Build pre-processing routines to clean OCR'd text Create data normalization and deduplication workflows Maintain data integrity across MySQL and PostgreSQL Collaborate with ML team for downstream AI...

Job Title: AI/ML + Backend Developer (SEO Automation & Technical Implementation) Location: Remote (APAC preferred) | Full-Time or Contract 🔹About the Role We’re looking for a 3-7 years experienced technical powerhouse who blends backend engineering, AI/ML experience, and hands-on SEO implementation skills. This hybrid role will support our mission to scale intelligent SEO operations and automate key parts of our publishing workflows. You’ll build custom tools and systems that integrate machine learning, backend development, and SEO performance logic — from programmatic content generation and internal linking engines to technical audits, schema injection, and Google Search Console automations...

Note: Please don't apply if you do not have at least 3 years of Scrapy experience. We are seeking a highly experienced Web Scraping Expert specialising in Scrapy-based web scraping and large-scale data extraction. This role is focused on building and optimizing web crawlers, handling anti-scraping measures, and ensuring efficient data pipelines for structured data collection. The ideal candidate will have 5+ years of hands-on experience developing Scrapy-based scraping solutions, implementing advanced evasion techniques, and managing high-volume web data extraction. You will collaborate with a cross-functional team to design, implement, and optimize scalable scraping systems that deliver hig...

Job Title: Data Automation Intern Location: Remote Duration: 3-6 months Type: Unpaid Internship About CollegePur: CollegePur is a remote educational consultancy dedicated to guiding students through their academic journeys. We focus on providing insightful resources and personalized support to help students make informed decisions about their education and careers. Overview: We are seeking motivated Python Programming Interns to join our team. In this role, you will develop automated solutions to extract and organize information from public online sources. This experience will provide exposure to real-world data workflows and foundational concepts in data engineering and automation. Key Resp...

Hiring: Python Developer (Web Scraping) – Mohali Location Location : Mohali Email : hr@baselineitdevelopment.com Contact : 9888122266 Experience : 1 Year Required Skills : Strong knowledge of Python. Hands-on experience in Web Scraping using tools like BeautifulSoup, Scrapy, or Selenium. Understanding of APIs and data parsing. Familiarity with data cleaning and automation scripts. Basic knowledge of databases (MySQL/PostgreSQL) is a plus. Good problem-solving skills and ability to write clean, efficient code Roles & Responsibilities : Develop and maintain web scraping scripts to extract data from various websites. Work with large datasets and automate data collection processes. Ensure data a...

Be a part of India’s largest and most admired news network! Network18 is India's most diversified Media Company in the fast growing Media market. The Company has a strong Heritage and we possess a strong presence in Magazines, Television and Internet domains. Our brands like CNBC, Forbes and Moneycontrol are market leaders in their respective segments. The Company has over 7,000 employees across all major cities in India and has been consistently managed to stay ahead of the growth curve of the industry. Network 18 brings together employees from varied backgrounds under one roof united by the hunger to create immersive content and ideas. We take pride in our people, who we believe are the ke...

Summary To enhance user profiling and risk assessment, we are building web crawlers to collect relevant user data from third-party sources, forums, and the dark web. We are seeking a Senior Web Crawler & Data Extraction Engineer to design and implement these data collection solutions. Job Responsibilities Design, develop, and maintain web crawlers and scrapers to extract data from open web sources, forums, marketplaces, and the dark web. Implement data extraction pipelines that aggregate, clean, and structure data for fraud detection and risk profiling. Use Tor, VPNs, and other anonymization techniques to safely crawl the dark web while avoiding detection. Develop real-time monitoring soluti...

About The Role Grade Level (for internal use): 10 The Team : As a member of the Data Transformation - Cognitive Engineering team you will work on building and deploying ML powered products and capabilities to power natural language understanding, data extraction, information retrieval and data sourcing solutions for S&P Global Market Intelligence and our clients. You will spearhead deployment of AI products and pipelines while leading-by-example in a highly engaging work environment. You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative. What’s In It For You Be a part of a global company and build solutions at enterprise scale Lead a highly skil...

About The Role Grade Level (for internal use): 10 The Team : As a member of the Data Transformation - Cognitive Engineering team you will work on building and deploying ML powered products and capabilities to power natural language understanding, data extraction, information retrieval and data sourcing solutions for S&P Global Market Intelligence and our clients. You will spearhead deployment of AI products and pipelines while leading-by-example in a highly engaging work environment. You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative. What’s In It For You Be a part of a global company and build solutions at enterprise scale Lead a highly skil...

About The Role Grade Level (for internal use): 10 The Team : As a member of the Data Transformation - Cognitive Engineering team you will work on building and deploying ML powered products and capabilities to power natural language understanding, data extraction, information retrieval and data sourcing solutions for S&P Global Market Intelligence and our clients. You will spearhead deployment of AI products and pipelines while leading-by-example in a highly engaging work environment. You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative. What’s In It For You Be a part of a global company and build solutions at enterprise scale Lead a highly skil...

We Are Hiring! Join Our Team as a Senior Python Developer Role: Senior Python Developer Work Locations: Teynampet, Chennai/Kk Nagar, Madurai Work from Office: (1pm to 10pm) Monday - Friday Mode of Interview: In-Person Experience : 4+ Years Hands-on software development skills, deep technical expertise across the entire software delivery process. Forward-thinking skilled individual. Structured, organized, and a good communicator. Write reusable, Testable, and Efficient code. Required Skills: - 3+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programm...

Responsibilities: Collaborate with cross-functional teams on project requirements & deliverables. Design, develop, test & maintain Python apps using Django/Flask & AWS/Azure Cloud.

We are looking for a Python Developer who has expertise in web scraping and backend development. The ideal candidate should be proficient in Python frameworks, data extraction techniques, and API integration.

Hiring Web Scraping Interns Location: Ahmedabad Internship Duration: 3 to 5 months Stipend: Based on performance (optional) Qualification: BCA / B.Sc (Computer Science) / B.Tech / BE (IT/CS) Final year students or recent graduates are preferred Skills Required: Basic knowledge of Python Familiarity with web scraping libraries such as BeautifulSoup, Selenium, or Scrapy Understanding of HTML and CSS; JavaScript knowledge is a plus Ability to work independently and in a team Willingness to learn and take ownership of tasks What You Will Learn: Real-world web scraping techniques Handling dynamic pages, proxies, captchas, and data cleaning Working on client-based projects with proper documentatio...