448 Scrapy Jobs - Page 2

Collect, cleanse, and validate data from multiple sources Design ETL processes and pipelines Build data acquisition systems Use APIs and web scraping tools for automated data collection Develop Python-based APIs to support data lakes Work from home

Experience required: 2–4 years of professional software engineering experience. Budget for the role - Max Up to Rs. 15 LPA 100% remote Opportunity Role Overview: You will design, build, and operate software for data collection and processing at scale. The role is hands‑on, with emphasis on clean design, reliability, and performance. Key Responsibilities: · Develop and maintain Python applications for crawling, parsing, enrichment, and processing of large datasets. · Build and operate data workflows (ETL/ELT), including validation, monitoring, and error‑handling. · Work with SQL and NoSQL (plus vector databases/data lakes) for modeling, storage, and retrieval. · Contribute to system design us...

Role & responsibilities Application Development: Design and build web-scale applications using Python (Django/Flask/FastAPI) and modern frontend frameworks (React / Angular / Vue). API Development: Develop and integrate RESTful and GraphQLAPIs. Implement responsive, intuitive, and interactive user interfaces. Build scalable and secure backend systems with optimized database queries and caching strategies. Create and manage relational (PostgreSQL/MySQL) and NoSQL (MongoDB/Redis) databases. Deploy applications on AWS / Azure / GCP, including containerization (Docker, Kubernetes). Write clean, reusable, and maintainable code following industry best practices. Ensure system performance, security...

Building and maintaining advanced web scraping and automation scripts across platforms (LinkedIn, Facebook, Twitter, Discord). Handle CAPTCHA, proxy, IP rotation, and anti-blocking. Ensure deployable, scalable, and monitored solutions using Docker.

Role Overview: Stats Perform, the market leader in sports tech, provides trusted sports data to global organizations in sports, media, and broadcasting. By leveraging AI technologies and machine learning, Stats Perform offers coaches, teams, professional bodies, and media channels worldwide access to top-notch data, content, and insights to enhance fan engagement and sports interactions. Key Responsibilities: - Design, develop, and maintain scalable Python applications for data extraction, transformation, and automation. - Build and optimize web scraping pipelines using Scrapy and Beautiful Soup. - Integrate AI/ML models into applications for enhanced automation and data insights. - Collabor...

All Jobs Senior Data Engineer at CivicDataLab Pvt Ltd Senior Data Engineer Apply Posted on July 22, 2024 | Closed on November 19, 2024 CivicDataLab Pvt Ltd New Delhi Full TIme Job Description At CivicDataLab, we are building robust and automated tools focusing on data analytics, encompassing data curation, data cleansing, data standardization, and sophisticated data wrangling. These products would support our projects across different sectors – public finance, digital public goods, climate change etc. We are also standardizing open datasets to adhere to the Data Catalog Vocabulary (DCAT) metadata standards, making them searchable and more user-friendly. To assist us with this endeavor, we ar...

Job Summary We are seeking a highly skilled and experienced Senior Data Scraping Engineer to design, develop, and orchestrate robust web scraping frameworks. The ideal candidate will have 8-10 years of experience in ethical web scraping, including navigating login-protected websites, solving CAPTCHAs, and managing proxies or third-party services. You will be responsible for building scalable, efficient, and compliant scraping pipelines using industry-standard programming languages and tools, ensuring data integrity and adherence to legal and ethical guidelines. Key Responsibilities Framework Development: Design and implement end-to-end web scraping frameworks to extract structured data from ...

In this internship role with a full-time opportunity, you will be working as a Data Engineer specializing in Web Data Scraping at Stylumia, located in Bengaluru with some work-from-home flexibility. Your key responsibilities will include: - Developing and maintaining web scraping scripts using tools like Scrapy, BeautifulSoup, and Puppeteer/Playwright. - Working with SQL/NoSQL databases to store, process, and query data. - Optimizing scrapers for efficiency, reliability, and scale. - Ensuring data accuracy, validation, and cleaning. - Collaborating with engineering and data teams to integrate scraped data into downstream systems. - Exploring and implementing the latest scraping tools and tec...

Position: Data Mining Analyst Please Note: Only Candidates with 2-3 years of experience should apply. This is a Delhi-based position and work from office only! Work Location: Sector 23 Dwarka, Delhi We are seeking a skilled Data Mining Analyst with expertise in automating data extraction processes from web platforms. The ideal candidate will be experienced in Python, Selenium, Pandas, SQL, and APIs, with the ability to design and implement efficient and scalable data scraping systems. If you have a passion for working with data and a solid understanding of web technologies, we want to hear from you! Key Responsibilities: Design, develop, and maintain robust web scraping solutions to extract ...

Job Description: We are seeking a highly skilled and passionate Python Developer to join our team. The ideal candidate should have strong expertise in backend development using Django, DRF, or FastAPI, along with extensive experience in advanced web scraping and hands-on knowledge in AI/LLM model training . Key Responsibilities: Develop and maintain RESTful APIs using Django, DRF, or FastAPI . Perform very advanced web scraping using tools like BeautifulSoup, Scrapy, and Selenium . Example platforms : Amazon, LinkedIn, Twitter, job portals, e-commerce sites, and real estate listings. Example content : Product data, job listings, user reviews, pricing data, dynamic JS-rendered content, etc. I...

Job Description: We are seeking a highly skilled and passionate Python Developer to join our team. The ideal candidate should have strong expertise in backend development using Django, DRF, or FastAPI, along with extensive experience in advanced web scraping and hands-on knowledge in AI/LLM model training . Key Responsibilities: Develop and maintain RESTful APIs using Django, DRF, or FastAPI . Perform very advanced web scraping using tools like BeautifulSoup, Scrapy, and Selenium . Example platforms : Amazon, LinkedIn, Twitter, job portals, e-commerce sites, and real estate listings. Example content : Product data, job listings, user reviews, pricing data, dynamic JS-rendered content, etc. I...

Experience: Fresher Qualification: Bachelor’s in Computer Science / IT or related field What You’ll Do Build & maintain data pipelines and ETL processes Work on web scraping using Selenium / Playwright / Scrapy Clean & transform data with Python & Pandas Collaborate with team using Git & Docker Get exposure to AWS Cloud What We’re Looking For Strong Python skills Familiarity with data handling & web scraping tools Basic knowledge of Docker, Git & Cloud (AWS preferred) Eagerness to learn and grow Why Join Us? Hands-on experience with real data projects Mentorship from experienced engineers Friendly & collaborative work culture Job Type: Full-time Pay: ₹15,000.00 - ₹20,000.00 per month Benefit...

About the role: We are seeking a highly motivated and detail-oriented AI Data Collection Expert to join our innovative AI/ML team. You will be instrumental in the end-to-end process of gathering, annotating, and curating high-quality datasets that serve as the foundation for our cutting-edge AI and machine learning models. This role requires a meticulous eye for detail, a proactive approach to problem-solving, and a commitment to data integrity and quality. Key responsibilities: Data sourcing and acquisition: Collaborate with AI/ML researchers and data scientists to understand data requirements and execute comprehensive data acquisition strategies. Dataset curation and management: Gather, cu...

As a Data Science and AI Developer, you will be part of a dynamic team where you will leverage cutting-edge technologies to develop innovative solutions that drive business insights and enhance decision-making processes. Your key responsibilities will include: - Developing and deploying machine learning models for predictive analytics, classification, clustering, and anomaly detection. - Designing and implementing algorithms for data mining, pattern recognition, and natural language processing. - Collaborating with cross-functional teams to understand business requirements and translate them into technical solutions. - Utilizing advanced statistical techniques to analyze complex datasets and...

Technopark, Trivandrum / Infopark, Cochin, Kerala Posted on 23.01.2025 What’s important to us: Techversant is seeking an experienced Python Team Lead to oversee our development team, drive best practices, and ensure the successful delivery of high-quality software solutions. The ideal candidate will have a strong background in Python development, excellent leadership skills, and a passion for mentoring and guiding team members. Job Description Python Team Lead with more than 7+ years of experience. Expert in Django, Flask. No day is the same – you will create technical designs that support varying client requirements. Should be capable of evaluating project requirements and preparing estimat...

As a Web Scraping Architect, your role involves designing, implementing, and maintaining web scraping processes to efficiently and accurately gather data from various online sources. **Key Responsibilities:** - Lead and provide expertise in scraping at scale for e-commerce marketplaces. - Identify relevant websites and online sources for data scraping. - Work closely with the team to understand data requirements and objectives. - Develop and implement effective strategies to extract data from targeted websites using appropriate tools, libraries, or frameworks. - Create and maintain web scraping scripts to extract required data, ensuring optimized, reliable code that can handle website struct...

Relu Consultancy is seeking a Data Extraction Engineer with expertise in Python (Selenium). In this role, you will design, implement, and maintain robust data scraping solutions that drive our projects forward. This is your chance to contribute to cutting-edge initiatives while enjoying a work schedule tailored to your needs. Job Title: Data Extraction Engineer Location: Bhopal, MP Job Type: Full-Time Responsibility: 1. Work on web scraping or data extraction through Selenium/Scrapy or other frameworks and related libraries. 2. Working knowledge in various DBSs, message queues & web Restful APIs. 3. Design, build, and maintain high-performance, reusable, and reliable Python code. 4. Ensure t...

As a Senior Product Engineer located in Bengaluru, your primary responsibility will be product engineering, design, research, and development. You will utilize analytical skills and excellent communication to ensure smooth workflows and successful project outcomes. Key Responsibilities: - Help build a quality product adhering to best practices and coding standards as a full-stack engineer - Work on both the back-end and front-end of systems, developing fully-fledged platforms - Play a key role in the development of the core product, working directly with the Senior Engineers and Product team - Understand, evaluate, and deliver quality work products based on requirements - Be responsible and ...

As a Python Quality Engineer at Tiger Analytics, you will be responsible for contributing to Python-based automation initiatives, including AI/ML model testing and framework development. You will have the opportunity to design frameworks, develop proof of concepts, and collaborate effectively with various teams, such as business consultants, data scientists, engineers, and application developers. Key Responsibilities: - Design frameworks, develop Proof of concepts, and accelerators, and execute effectively/independently. - Design and implement automation for Data, Web, and Mobile applications. - Participate in Demos, Planning, etc., with the rest of the team. - Collaborate with business cons...

Job Category: Job Type: Job Location: Salary: Years of Experience: Job Title: Senior Python Developer Web Scraping & Automation Location: Ahmedabad Job Type: Full-time Working Days: 5 Days a Week Role Overview We are seeking an experienced Senior Python Developer with proven expertise in Scrapy (must-have) and strong skills in web scraping and automation. The ideal candidate will design, develop, and optimize large-scale scraping solutions that power data-driven decision-making. Key Responsibilities Design, develop, and maintain scalable web scraping frameworks using Scrapy (mandatory). Work with additional libraries/tools such as BeautifulSoup, Selenium, Playwright, Requests, etc. Implement...

The Company Serving the People Who Serve the People Granicus is driven by the excitement of building, implementing, and maintaining technology that is transforming the Govtech industry by bringing governments and its constituents together. We are on a mission to support our customers with meeting the needs of their communities and implementing our technology in ways that are equitable and inclusive. Granicus has consistently appeared on the GovTech 100 list over the past 5 years and has been recognized as the best companies to work on BuiltIn. Over the last 25 years, we have served 5,500 federal, state, and local government agencies and more than 300 million citizen subscribers power an unma...

JOB DESCRIPTION Role : Data Engineer Job location : Chennai About Us At I, Saras, we develop AI applications for students, teachers and professionals. Why you should apply for I, Saras? Work in emerging tech like AI, ML and NLP. Funded by Startup India Scheme for being an innovative startup. Incubated at IIM Lucknow. Role Overview Build and maintain data infrastructure for AI-powered educational applications, enabling seamless data flow and intelligent features for students and teachers. Key Responsibilities Technical Documentation Understand business requirements from product docs (PRD and FSD) Translate them into Technical Specification Documents (TSD) Create comprehensive data architectur...

We are looking for an enthusiastic and proactive Python Developer with core Python expertise and hands-on experience in Generative AI (GenAI) to join our development team. Experience Required: 2-3 Years Mode of Work: On-Site Only (Mohali, Punjab) Mode of Interview : Face to Face( On-Site) Contact for Queries: +91-9872993778 (Mon–Fri, 11 AM – 6 PM) Note: This number will be unavailable on weekends and public holidays. Key Responsibilities: Backend Development: Assist in the development of clean, efficient, and scalable Python applications to meet business needs. Generative AI Experience: Working knowledge and experience in building applications using GenAI technologies is mandatory. API Integ...

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com Leadership Define and execute the overall visio...

Job Title: Python Automation Developer Location: Gurugram, Haryana, India (On-site/Hybrid) 🚀 We're looking for a Python Automation Developer to join our innovative team in Gurugram! Are you passionate about turning complex manual processes into elegant, automated solutions? Do you excel at building robust systems that handle data with precision and speed? If you're a Python expert with a love for problem-solving, we want to hear from you! This is a fantastic opportunity to take ownership of critical automation projects, work with a talented cross-functional team, and make a direct impact on business efficiency and growth. Your Impact & Responsibilities: 💻 Architect & Build: Design, develop, ...