83 Scrapy Jobs - Page 2

We are seeking an experienced and motivated Data Scraper / Lead Generator to join our fast-growing team in Mumbai. The ideal candidate will have a strong background in generating leads through web scraping and online research, specifically targeting the Europe, UK, USA and other international markets . Key Responsibilities: Conduct in-depth online research to identify potential leads in targeted geographies Use advanced web scraping tools and techniques to extract accurate contact and business data from various sources. Validate and verify collected data to ensure quality and relevance. Maintain and manage a structured database of leads for outreach and tracking. Collaborate closely with the sales and marketing teams to deliver a steady pipeline of high-quality leads. Stay up to date with industry trends, tools, and best practices in data scraping and lead generation. Requirements: Proven experience in data scraping lead generation , especially in international markets (UK preferred) . Proficiency in web scraping tools and methods (e.g., Python/BeautifulSoup, Scrapy, Octoparse, or similar). Strong attention to detail, organizational skills, and data accuracy. Ability to manage time efficiently and handle multiple tasks. Excellent communication and coordination skills. Preferred: Immediate availability or short notice period. Show more Show less

The Team : As a member of the Data Transformation - Cognitive Engineering team you will work on building and deploying ML powered products and capabilities to power natural language understanding, data extraction, information retrieval and data sourcing solutions for S&P Global Market Intelligence and our clients. You will spearhead deployment of AI products and pipelines while leading-by-example in a highly engaging work environment. You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative. Whats in it for you: Be a part of a global company and build solutions at enterprise scale Lead a highly skilled and technically strong team (including leadership) Contribute to solving high complexity, high impact problems Build production ready pipelines from ideation to deployment Responsibilities: Design, Develop and Deploy ML powered products and pipelines Mentor a team of Senior and Junior data scientists ML Engineers in delivering large scale projects Play a central role in all stages of the AI product development life cycle, including: Designing Machine Learning systems and model scaling strategies Research & Implement ML and Deep learning algorithms for production Run necessary ML tests and benchmarks for model validation Fine-tune, retrain and scale existing model deployments Extend existing ML librarys and write packages for reproducing components Partner with business leaders, domain experts, and end-users to gain business understanding, data understanding, and collect requirements Interpret results and present them to business leaders Manage production pipelines for enterprise scale projects Perform code reviews & optimization for your projects and team Lead and mentor by example, including project scrums Technical Requirements: Proven track record as a senior lead ML engineer Expert proficiency in Python (Numpy, Pandas, Spacy, Sklearn, Pytorch/TF2, HuggingFace etc.) Excellent exposure to large scale model deployment strategies and tools Excellent knowledge of ML & Deep Learning domain Solid exposure to Information Retrieval, Web scraping and Data Extraction at scale Exposure to the following technologies - R-Shiny/Dash/Streamlit, SQL, Docker, Airflow, Redis, Celery, Flask/Django/FastAPI, PySpark, Scrapy Experience with SOTA models related to NLP and expertise in text matching techniques, including sentence transformers, word embeddings, and similarity measures Open to learning new technologies and programming languages as required A Masters PhD from a recognized institute in a relevant specialization Good to have: 6-7+ years of relevant experience in ML Engineering Prior substantial experience from the Economics/Financial industry Prior work to show on Github, Kaggle, StackOverflow etc.

Job Overview We are seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for maintaining and optimizing data pipelines, implementing robust data checks, and ensuring the accuracy and integrity of data flows. This role is critical in supporting data-driven decision-making processes, especially in the context of our insurance-focused business operations. Key Responsibilities Data Collection and Acquisition: Source Identification, Data Licensing and Compliance, Data Crawling/Collection Data Preprocessing and Cleaning: Data Cleaning, Text Tokenization, Normalization, Noise Filtering Data Transformation and Feature Engineering: Text Embedding, Text Augmentation, Handling Multilingual Data Data Pipeline Development: Scalable Pipelines, ETL Processes, Automation Data Storage and Management: Data Warehousing, Database Optimization, Version Control Collaboration with Data Scientists and ML Engineers: Data Accessibility, Support for Model Development, Data Quality Assurance Performance Optimization and Scaling: Efficient Data Handling, Distributed Computing Data Security and Privacy: Data Anonymization, Compliance with Regulations Documentation and Reporting: Data Pipeline Documentation, Reporting Candidate Profile 6 -10 years of relevant experience in data engineering tools Tools: Data Processing & Storage: Apache Spark, Apache Hadoop, Apache Kafka, Google BigQuery, AWS S3, Databricks Machine Learning Frameworks: TensorFlow, PyTorch, Hugging Face Transformers, scikit-learn Data Pipelines & Automation: Apache Airflow, Kubeflow, Luigi Version Control & Collaboration: Git, DVC (Data Version Control) Data Extraction: BeautifulSoup, Scrapy, APIs (RESTful, GraphQL) What We Offer EXL Analytics offers an exciting, fast paced and innovative environment, which brings together a group of sharp and entrepreneurial professionals who are eager to influence business decisions. From your very first day, you get an opportunity to work closely with highly experienced, world class analytics consultants. You can expect to learn many aspects of businesses that our clients engage in. You will also learn effective teamwork and time-management skills - key aspects for personal and professional growth Analytics requires different skill sets at different levels within the organization. At EXL Analytics, we invest heavily in training you in all aspects of analytics as well as in leading analytical tools and techniques. We provide guidance/ coaching to every employee through our mentoring program wherein every junior level employee is assigned a senior level professional as advisors. Sky is the limit for our team members. The unique experiences gathered at EXL Analytics sets the stage for further growth and development in our company and beyond Show more Show less

Role & responsibilities Develop and implement Python scripts for web scraping using Selenium WebDriver to extract relevant data from client websites. Clean, transform, and manipulate extracted data using Python libraries (e.g., Pandas, BeautifulSoup) for schema (Structured data) markup implementation Write well-documented, maintainable, and efficient Python code adhering to best practices. Collaborate with SEOs and the Director of SEO to understand client requirements and translate them into technical solutions. Stay up-to-date on the latest trends and developments in web scraping, schema (Structured data) markup, and SEO best practices. Assist with testing and debugging developed scripts to ensure accuracy of schema (Structured data) implementation without any error. Experience working in Automation through AI agents Experience working with machine learning and AI (Artificial Intelligence) integration using Python. Preferred candidate profile Having 4-5 years of working experience in Python programming. Strong understanding of Python syntax, data structures, Iterator, Generators, Exception Handling, File handling, OOPs, Data Structures, ORM and object-oriented programming concepts. Proficiency in using web scraping libraries like Selenium WebDriver and Beautiful Soup. Must be familiar with Web Frameworks like HTML, CSS, JavaScript, Django or Flasks. Good knowledge of machine learning & ML frameworks like NumPy, Pandas, Kera's, scikit-learn, PyTorch, TensorFlow or Microsoft Azure Machine Learning will be added advantage. Must be familiar with development tools like Jupyter Notebook, IDLE, PyCharm or VS Code. Must be familiar with Scrum methodology, CI/CD, Git, Branching/Merging and test-driven software development. Candidates worked in product-based companies will be preferred. Excellent analytical and problem-solving skills. Ability to work independently and as part of a team. Strong communication and collaboration skills. A passion for SEO and a desire to learn about schema (Structured data) markup. Familiarity with cloud platforms (AWS, GCP, Azure DevOps, Azure Blob Storage Explorer) Experience with API integration. Experience working with AI (Artificial Intelligence) integration with Python to automate SEO tasks with Google Gemini, GenAI (Generative AI) & ChatGPT 4. Experience working in Automation through AI agents Good verbal and written communication skills.

Job Title: Senior Python Developer Company: Darwix AI Location: Gurgaon (On-site) Type: Full-Time Experience: 3–8 years About Darwix AI Darwix AI is one of India’s fastest-growing AI startups, transforming enterprise sales with our GenAI-powered conversational intelligence and real-time agent assist suite. Our platform is used by high-growth enterprises across India, MENA, and Southeast Asia to improve sales productivity, personalize customer conversations, and unlock revenue intelligence in real-time. We are backed by marquee VCs, 30+ angel investors, and led by alumni from IITs, IIMs, and BITS with deep experience in building and scaling products from India for the world. Role Overview As a Senior Python Developer at Darwix AI, you will be at the core of our engineering team, leading the development of scalable, secure, and high-performance backend systems that support AI workflows, real-time data processing, and enterprise-grade integrations. This role requires deep technical expertise in Python, a strong foundation in backend architecture, and the ability to collaborate closely with AI, product, and infrastructure teams. You will take ownership of critical backend modules and shape the engineering culture in a rapidly evolving, high-impact environment. Key Responsibilities System Architecture & API Development Design, implement, and optimize backend services and microservices using Python frameworks such as FastAPI, Django, or Flask Lead the development of scalable RESTful APIs that integrate with frontend, mobile, and AI systems Architect low-latency, fault-tolerant services supporting real-time sales analytics and AI inference Data Pipelines & Integrations Build and optimize ETL pipelines to manage structured and unstructured data from internal and third-party sources Integrate APIs with CRMs, telephony systems, transcription engines, and enterprise platforms like Salesforce, Zoho, and LeadSquared Lead scraping and data ingestion efforts from large-scale, dynamic web sources using Playwright, BeautifulSoup, or Scrapy AI/ML Enablement Work closely with AI engineers to build infrastructure for LLM/RAG pipelines , vector DBs , and real-time AI decisioning Implement backend support for prompt orchestration , Langchain flows , and function-calling interfaces Support model deployment, inference APIs, and logging/monitoring for large-scale GenAI pipelines Database & Storage Design Optimize database design and queries using MySQL , PostgreSQL , and MongoDB Architect and manage Redis and Kafka for caching, queueing, and real-time communication DevOps & Quality Ensure continuous delivery through version control (Git), CI/CD pipelines, testing frameworks, and Docker-based deployments Identify and resolve bottlenecks related to performance, memory, or data throughput Adhere to best practices in code quality, testing, security, and documentation Leadership & Collaboration Mentor junior developers and participate in code reviews Collaborate cross-functionally with product, AI, design, and sales engineering teams Contribute to architectural decisions, roadmap planning, and scaling strategies Qualifications 4–8 years of backend development experience in Python, with a deep understanding of object-oriented and functional programming Hands-on experience with FastAPI , Django , or Flask in production environments Proven experience building scalable microservices, data pipelines, and backend systems that support live applications Strong command over REST API architecture , database optimization, and data modeling Solid experience working with web scraping tools , automation frameworks, and external API integrations Knowledge of AI tools like Langchain , HuggingFace , Vector DBs (Pinecone, Weaviate, FAISS) , or RAG architectures is a strong plus Familiarity with cloud infrastructure (AWS/GCP) , Docker, and containerized deployments Comfortable working in fast-paced, high-ownership environments with shifting priorities and dynamic problem-solving Show more Show less

Job Description Python Web scraping Chennai/Pune About The Job NIQ Digital Shelf is one of Europe’s fastest-growing companies in the Retail Analytics space. Every day, we collect and process over 60 billion data points from web and mobile sources to power real-time market insights. Our tools help major brands and retailers understand what's happening in their market, how they compare to competitors, and what actions to take. We’re now a team of 60+ Scrapers from over 12 nationalities, working together with engineering, data, product, operations, and customer success. As we scale globally, we’re looking for new talent to help push our technology and data collection efforts even further. You’ll join an engineering team that values curiosity, autonomy, and the ability to iterate fast. You’ll also collaborate with people across the business to make sure the right data ends up in the right hands, in the cleanest, smartest way possible. What You'll Work On Build and maintain efficient web crawlers to extract structured data from websites (e.g. product listings, prices, reviews). Write robust data pipelines to parse and clean messy web content. Deal with real-world challenges like JavaScript-heavy pages, anti-bot measures, and changing page structures. Work closely with product and operations to adjust scraping strategies when sites change or new data needs emerge Qualifications Must Have: 1–3 years of experience working with Python. Comfortable using tools like Scrapy, Python Requests, BeautifulSoup, Playwright/Selenium. You understand how to work with HTTP headers, cookies, session management, and are not afraid of network debugging. You adapt quickly and aren’t scared of messy problems. When something breaks, your instinct is to figure out why and fix it. You enjoy learning, asking questions, and building better tools — not just copying and pasting scripts. Nice to Have: Basic exposure to concepts like rotating proxies, user-agent spoofing, or using headless browsers (e.g., with Selenium or Playwright). Some hands-on practice scraping structured websites, while using scrapy of python requests and BeautifulSoup. A basic understanding of HTML structure, XPaths, or CSS selectors. Additional Information Enjoy a flexible and rewarding work environment with peer-to-peer recognition platforms. Recharge and revitalize with help of wellness plans made for you and your family. Plan your future with financial wellness tools. Stay relevant and upskill yourself with career development opportunities. Our Benefits Flexible working environment Volunteer time off LinkedIn Learning Employee-Assistance-Program (EAP) About NIQ NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population. For more information, visit NIQ.com Want to keep up with our latest updates? Follow us on: LinkedIn | Instagram | Twitter | Facebook Our commitment to Diversity, Equity, and Inclusion NIQ is committed to reflecting the diversity of the clients, communities, and markets we measure within our own workforce. We exist to count everyone and are on a mission to systematically embed inclusion and diversity into all aspects of our workforce, measurement, and products. We enthusiastically invite candidates who share that mission to join us. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class. Our global non-discrimination policy covers these protected classes in every market in which we do business worldwide. Learn more about how we are driving diversity and inclusion in everything we do by visiting the NIQ News Center: https://nielseniq.com/global/en/news-center/diversity-inclusion Show more Show less

Job Title: Python Developer – Web Scraping Specialist Experience: 1+ Years Location: Jaipur (Work from Office) Job Type: Full-Time Job Summary: We are seeking a detail-oriented and skilled Python Developer with expertise in Web Scraping to join our technology team. The ideal candidate will have at least 1 year of experience in Python programming and hands-on knowledge of web scraping tools and techniques. You will be responsible for designing and implementing efficient, scalable web crawlers to extract structured data from various websites and online platforms. Key Responsibilities: Design, build, and maintain web scraping scripts and crawlers using Python . Utilize tools such as BeautifulSoup , Selenium , and Scrapy to extract data from dynamic and static websites. Clean, structure, and store extracted data in usable formats (e.g., CSV, JSON, databases). Handle data parsing, anti-scraping measures, and ensure scraping compliance with website policies. Monitor and troubleshoot scraping tasks for performance and reliability. Collaborate with team members to understand data requirements and deliver accurate, timely results. Optimize scraping scripts for speed, reliability, and error handling. Maintain documentation of scraping processes and codebase. Required Skills: Solid programming skills in Core Python and data manipulation. Strong experience in Web Scraping using BeautifulSoup , Selenium , and Scrapy . Familiarity with HTTP protocols, request headers, cookies, and browser automation. Understanding of HTML, CSS, and XPath for parsing and navigating web content. Ability to handle and solve CAPTCHA and anti-bot mechanisms. Experience with data formats like JSON, XML, and CSV. Knowledge of version control tools like Git. Preferred Qualifications: Bachelor’s degree in Computer Science, IT, or a related field. Experience with task schedulers (e.g., CRON, Celery) for automated scraping. Knowledge of storing data in SQL or NoSQL databases. Familiarity with proxy management and user-agent rotation. Job Type: Full-time Pay: ₹7,000.00 - ₹35,000.00 per month Schedule: Day shift Ability to commute/relocate: Jaipur, Rajasthan: Reliably commute or planning to relocate before starting work (Required) Education: Bachelor's (Required) Experience: Python: 1 year (Required) beautiful soup or scrapy: 1 year (Required) Selenium: 1 year (Preferred) Location: Jaipur, Rajasthan (Preferred)

Role & responsibilities As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. You will be responsible for implementing scalable and resilient data extraction solutions, ensuring seamless data retrieval while working with proxy management, anti-bot bypass techniques, and data parsing. Optimizing scraping workflows for performance, reliability, and efficiency will be a key part of your role. Additionally, you will ensure that all extracted data maintains high quality and integrity. Preferred candidate profile We are seeking candidates with: Strong experience in Python and web scraping frameworks such as Scrapy, Selenium, Playwright, or BeautifulSoup. Knowledge of distributed web crawling architectures and job scheduling. Familiarity with headless browsers, CAPTCHA-solving techniques, and proxy management to handle dynamic web challenges. Experience with data storage solutions, including SQL, and cloud storage. Understanding of big data technologies like Spark and Kafka (a plus). Strong debugging skills to adapt to website structure changes and blockers. A proactive, problem-solving mindset and ability to work effectively in a team-driven environment.

Job Summary: We are looking for a passionate and quick-learning Python Developer (Fresher) to join our growing team in Mohali . The ideal candidate should have completed at least one internship and be familiar with Python programming , Odoo ERP , PostgreSQL , and Web Scraping techniques . This is a great opportunity to gain hands-on experience and grow your skills in a professional and supportive environment. Key Responsibilities: Assist in the development and customization of Odoo modules Support integration of third-party services with Odoo Write clean and efficient Python code for automation and backend logic Help with creating and managing PostgreSQL databases Perform basic web scraping using Python tools (e.g., BeautifulSoup, Scrapy) Participate in testing, debugging, and improving application performance Collaborate with senior developers and follow best coding practices Required Skills: Strong understanding of Python programming Basic knowledge of Odoo ERP (academic or internship level) Familiarity with PostgreSQL and database queries Exposure to web scraping tools like BeautifulSoup, Scrapy, or Selenium Good understanding of REST APIs and data formats (JSON/XML) Eagerness to learn and grow in a development role Eligibility Criteria: B.Tech/B.E., MCA, BCA, or related technical degree Must have completed at least one internship or project in Python/Odoo/web technologies Good communication and teamwork skills Perks & Benefits: Mentorship and learning opportunities Hands-on experience with real-world projects Positive work environment with growth potential 5-day work week Call us - 9888122266 Job Types: Full-time, Permanent Pay: From ₹12,000.00 per month Schedule: Day shift Monday to Friday Morning shift Supplemental Pay: Performance bonus Work Location: In person

About Us: LdotR is an online brand protection service company, offering businesses the right solution and services to protect, manage and benefit from their digital assets in the online space. We work across all digital platforms - Domains, Website, Social Media, Online Marketplaces, and App Stores to identify, assess and nullify brand infringements. About the Role: We are looking for an experienced Data Scraping Specialist to help us extract and structure data from leading social media platforms at scale. The ideal candidate will have hands-on expertise with scraping tools, APIs, and large-scale data processing. Key Responsibilities: Design and develop custom scraping solutions to extract public data from platforms like Instagram, Facebook, X (Twitter), LinkedIn, YouTube, etc. Handle large-scale scraping tasks with efficiency and resilience against rate-limiting and platform-specific restrictions. Clean, normalize, and structure the scraped data for analysis or downstream applications. Maintain scraping scripts to adapt to frequent platform changes. Ensure compliance with data protection policies and terms of service. Required Skills: Proficiency in Python and scraping libraries (e.g., Scrapy, BeautifulSoup, Selenium, Playwright). Experience with API integration (official or unofficial social media APIs). Familiarity with rotating proxies, headless browsers, and CAPTCHA-solving techniques. Strong understanding of data structuring formats like JSON, CSV, and databases (MongoDB, PostgreSQL, etc.). Experience with cloud-based scraping and storage solutions (AWS/GCP preferred). Good to Have: Knowledge of NLP or data analytics for social media sentiment or trend analysis. Understanding of GDPR and CCPA compliance. Prior work with third-party scraping platforms or browser automation tools. What We Offer: Opportunity to work on impactful, large-scale data projects. Flexible work arrangements. Competitive compensation based on experience and delivery.

Responsibilities:As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. Annual bonus Health insurance Provident fund

Job Description: We are seeking a creative and independent Web Crawler Developer to join our team Seattle based Construction Team. The ideal candidate will have a keen eye for detail, a passion for problem-solving, and the ability to think outside the box to develop sophisticated web scraping solutions. Responsibilities: - Design, implement, and maintain web crawlers that can effectively extract data from various websites . - Analyze web page structures and adapt crawlers to extract relevant information efficiently. - Monitor crawler performance and make necessary adjustments to ensure optimal data collection. - Work independently to identify new opportunities for data extraction and offer insightful recommendations. - Ensure compliance with legal and ethical standards for data scraping. - Collaborate with data analysts and other team members to understand data needs and improve data accuracy. - Keep up-to-date with the latest web scraping technologies and best practices Qualifications: - Strong experience with web scraping tools and frameworks (e.g., Scrapy, BeautifulSoup, Selenium, etc.). - Proficiency in programming languages such as Python, Java, or others relevant to web crawling. - Experience with handling and parsing different data formats like HTML, JSON, XML, etc. - Excellent problem-solving skills and the ability to think outside the box. - Ability to work independently and manage multiple tasks efficiently. - Solid understanding of web protocols (HTTP, HTTPS) and web technologies. - Familiarity with version control systems, preferably Git. - Knowledge of data privacy laws and ethical web scraping practices. Preferred: - Experience with cloud services like AWS or Azure for deploying and managing web crawlers. - Understanding of databases and data storage solutions. - Previous experience in a similar role or related projects. Job Type: Contractual / Temporary Contract length: 2 months Pay: ₹76,000.00 - ₹80,000.00 per month Benefits: Work from home Supplemental Pay: Performance bonus Expected Start Date: 03/06/2025

Job Title: Data Analyst (Python & Web Scraping Expert) Location : Netaji Subhash Place, Pitampura, New Delhi Department : Data Analytics / Share Recovery Job Overview: We are seeking a detail-oriented and results-driven Data Analyst to join our team. The ideal candidate will have expertise in Python programming, web scraping, and data analysis, with a focus on IEPF share recovery . The role involves collecting, processing, and analyzing data from multiple online sources, providing actionable insights to support business decision-making. Key Responsibilities: Data Scraping : Use Python and web scraping techniques to gather data from financial, regulatory, and shareholding-related websites for IEPF (Investor Education and Protection Fund) share recovery. Data Cleaning & Preprocessing : Clean, process, and structure raw data for analysis. Ensure data quality and integrity by identifying and correcting errors in datasets. Data Analysis & Visualization : Analyze large datasets to extract actionable insights regarding share recovery and trends in investor shareholding. Present findings through visualizations (e.g., graphs, dashboards). Reporting : Prepare and present detailed reports on share recovery patterns, trends, and forecasts based on analysis. Present findings to the management team to help drive business decisions. Automation & Optimization : Build and maintain automated web scraping systems to regularly fetch updated shareholding data, optimizing the data pipeline for efficiency. Collaboration : Work closely with business stakeholders to understand data requirements and deliver reports or visualizations tailored to specific needs related to IEPF share recovery. Required Skills & Qualifications: Technical Skills : Strong proficiency in Python for data analysis and automation. Expertise in web scraping using libraries such as BeautifulSoup , Selenium , and Scrapy . Experience with data manipulation and analysis using Pandas , NumPy , and other relevant libraries. Familiarity with SQL for data extraction and querying relational databases. Knowledge of data visualization tools like Matplotlib , Seaborn , or Tableau for presenting insights in an easy-to-understand format. Experience : Minimum of 2-3 years of experience as a Data Analyst or in a similar role, with a focus on Python programming and web scraping. Experience working with financial or investment data, particularly in areas such as IEPF , share recovery , or investor relations . Strong problem-solving skills with the ability to analyze complex datasets and generate actionable insights. Additional Skills : Strong attention to detail and ability to work with large datasets. Ability to work in a collaborative team environment. Familiarity with cloud platforms (e.g., AWS, Google Cloud) and data storage (e.g., databases, cloud data lakes) is a plus. Education : Bachelor’s or Master’s degree in Data Science , Computer Science , Statistics , Finance , or a related field. Soft Skills : Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders. Ability to prioritize tasks and manage multiple projects simultaneously. Strong organizational skills and time management. Preferred Skills: Experience working in the financial industry or understanding of regulatory frameworks (e.g., IEPF regulations and procedures). Familiarity with machine learning models and predictive analytics for forecasting share recovery trends. Ability to automate workflows and optimize existing data collection pipelines. Job Requirements: Comfortable working in a fast-paced environment. Ability to think critically and provide insights that drive strategic decisions. Must be self-motivated and capable of working independently with minimal supervision. Willingness to stay updated with the latest data analysis techniques and web scraping technologies. Job Type: Full-time Pay: ₹20,000.00 - ₹32,000.00 per month Schedule: Day shift Education: Bachelor's (Preferred) Experience: total work: 1 year (Required) Work Location: In person

Sigmoid enables business transformation using data and analytics, leveraging real-time insights to make accurate and fast business decisions, by building modern data architectures using cloud and open source. Some of the world’s largest data producers engage with Sigmoid to solve complex business problems. Sigmoid brings deep expertise in data engineering, predictive analytics, artificial intelligence, and DataOps. Sigmoid has been recognized as one of the fastest growing technology companies in North America, 2021, by Financial Times, Inc. 5000, and Deloitte Technology Fast 500. Offices: New York | Dallas | San Francisco | Lima | Bengaluru This role is for our Bengaluru office. Why Join Sigmoid? • Sigmoid provides the opportunity to push the boundaries of what is possible by seamlessly combining technical expertise and creativity to tackle intrinsically complex business problems and convert them into straight-forward data solutions. • Despite being continuously challenged, you are not alone. You will be part of a fast-paced diverse environment as a member of a high-performing team that works together to energize and inspire each other by challenging the status quo • Vibrant inclusive culture of mutual respect and fun through both work and play Roles and Responsibilities: • Convert broad vision and concepts into a structured data science roadmap, and guide a team to successfully execute on it. • Handling end-to-end client AI & analytics programs in a fluid environment. Your role will be a combination of hands-on contribution, technical team management, and client interaction. • Proven ability to discover solutions hidden in large datasets and to drive business results with their data-based insights • Contribute to internal product development initiatives related to data science. • Drive excellent project management required to deliver complex projects, including effort/time estimation. • Be proactive, with full ownership of the engagement. Build scalable client engagement level processes for faster turnaround & higher accuracy • Define Technology/ Strategy and Roadmap for client accounts, and guides implementation of that strategy within projects • Manage the team-members, to ensure that the project plan is being adhered to over the course of the project • Build a trusted advisor relationship with the IT management at clients and internal accounts leadership. Mandated Skills: • A B-Tech/M-Tech/MBA from a top tier Institute preferably in a quantitative subject • 14+ years of hands-on experience in applied Machine Learning, AI and analytics • Experience of scientific programming in scripting languages like Python, R, SQL, NoSQL, Spark with ML tools & Cloud Technology (AWS, Azure, GCP) • Experience in Python libraries such as numpy, pandas, scikit-learn, tensor-flow, scrapy, BERT etc. • Strong grasp of depth and breadth of machine learning, deep learning, data mining, and statistical concepts and experience in developing models and solutions in these areas • Expertise with client engagement, understanding complex problem statements, and offering solutions in the domains of Supply Chain, Manufacturing, CPG, Marketing etc. Desired Skills: ● Deep understanding of ML algorithms for common use cases in both structured and unstructured data ecosystems. ● Comfortable with large scale data processing and distributed computing ● Providing required inputs to sales, and pre-sales activities ● A self-starter who can work well with minimal guidance ● Excellent written and verbal communication skills Show more Show less

Key Responsibilities: Develop and maintain automated web scraping scripts using Python libraries such as Beautiful Soup, Scrapy, and Selenium. Optimize scraping pipelines for performance, scalability, and resource efficiency. Handle dynamic websites, CAPTCHA-solving, and implement IP rotation techniques for uninterrupted scraping. Process and clean raw data, ensuring accuracy and integrity in extracted datasets. Collaborate with cross-functional teams to understand data requirements and deliver actionable insights. Leverage APIs when web scraping is not feasible, managing authentication and request optimization. Document processes, pipelines, and troubleshooting steps for maintainable and reusable scraping solutions. Ensure compliance with legal and ethical web scraping practices, implementing security safeguards. Requirements: Education : Bachelors degree in Computer Science, Engineering, or a related field. Experience : 2+ years of Python development experience, with at least 1 year focused on web scraping. Technical Skills : Proficiency in Python and libraries like Beautiful Soup, Scrapy, and Selenium. Experience with regular expressions (Regex) for data parsing. Strong knowledge of HTTP protocols, cookies, headers, and user-agent rotation. Familiarity with databases (SQL and NoSQL) for storing scraped data. Hands-on experience with data manipulation libraries such as pandas and NumPy. Experience working with APIs and managing third-party integrations. Familiarity with version control systems like Git. Bonus Skills : Knowledge of containerization tools like Docker. Experience with distributed scraping solutions and task queues (e.g., Celery, RabbitMQ). Basic understanding of data visualization tools. Non-Technical Skills : Strong analytical and problem-solving skills. Excellent communication and documentation skills. Ability to work independently and collaboratively in a team environmen

Job Description Developing ETL Pipelines: Designing, developing, and maintaining scalable and adaptable data pipelines using Python or PySpark to facilitate the smooth migration of data from diverse data sources . Host these ETL pipelines in AWS EC2, AWS Glue or AWS EMR and store this data to cloud database services like Google BigQuery, AWS S3, Redshift, RDS, Delta Lake etc. This includes managing significant data migrations and ensuring seamless transitions between systems. Implementing Data Quality Check Framework: Establishing and executing data quality checks and validation pipelines using different tools like Python, PySpark, Athena or BigQuery, S3, Delta Lake to uphold the integrity and accuracy of our datasets. Creating Mechanisms for Generating ETL Migration Status Reports: Devising a framework to generate concise summary reports detailing data migration progress, alongside promptly alerting stakeholders to any failures within ETL pipelines. This ensures swift resolution of data discrepancies arising from pipeline failures. Implement this using standard SMTP, Python, AWS SNS, AWS SES, AWS S3, Delta Lake etc services. Data Transformations and Processing: Implementing various data encryption and decryption techniques using Python and PySpark libraries, in addition to generating insightful reports and analyses derived from processed data to aid in informed business decision-making. Development of APIs: Building APIs using frameworks such as Flask or Django, incorporating diverse authentication and authorization techniques to safeguard the exchange of data. Host these API’s on EC2 server using services like Gearman etc or Write API logics in lambda and host these API’s using API Gateway services of cloud. Code Versioning and Deployment: Leveraging GitHub extensively for robust code versioning, deployment of the latest code iterations, seamless transitioning between different code versions, and merging various branches to streamline development and code release processes. Automation: Designing and implementing code automation solutions to streamline and automate manual tasks effectively. Required Candidate Profile Soft Skills Must Have Demonstrates adept problem-solving skills to efficiently address complex challenges encountered during data engineering tasks. Exhibits clear and effective communication skills, facilitating seamless collaboration and comprehension across diverse teams and stakeholders. Displays proficiency in both independent and collaborative work dynamics, fostering productivity and synergy within a fast-paced team environment. Demonstrates a high level of adaptability to changing requirements, customer dynamics, and work demands. Self-motivated and responsible individual who takes ownership and initiative in tasks. Good To Have Demonstrates project management experience, offering valuable insights and contributions towards efficient project execution and delivery. Good Presentation skills Excellent customer handling skills. Technical Skills Proficiency in SQL (Structured Query Language) for querying and manipulating databases. Experience with relational database systems like MySQL, PostgreSQL, or Oracle and NoSQL databases like Mongo. Proficiency in object-oriented programming concepts such as encapsulation, inheritance, and polymorphism. Knowledge of data warehousing concepts and experience with data warehousing solutions like Amazon Redshift, Google BigQuery, or Snowflake. Experienced in developing ETL pipelines using Python, PySpark. Knowledge of Python libraries/frameworks like Pandas, NumPy, or Spark for data processing and analysis. Familiarity with big data processing frameworks like Apache Hadoop and Apache Spark for handling large-scale datasets and performing distributed computing. Knowledge of cloud-based services like AWS S3, AWS Glue, AWS EMR, AWS Lambda, Athena, Azure Data Lake, Google BigQuery, etc. Familiarity with version control systems like Git for managing codebase changes, collaborating with team members, and maintaining code quality. Experience with web scraping libraries and frameworks like BeautifulSoup, Scrapy, Puppeteer, Selenium, etc., is highly beneficial. Knowledge of regular expressions is useful for pattern matching and extracting specific data formats from text. Understanding of HTTP protocols and how web servers respond to requests, how to send requests to web servers, handle responses, and manage sessions and cookies is essential. Familiarity with XPath expressions or CSS selectors is important for targeting specific elements within the HTML structure. Required Experience The ideal candidate will have a minimum of 1-3 years of relevant experience in data engineering roles, with a demonstrated history of successfully developing and maintaining ETL pipelines, handling big data migrations, and ensuring data quality and validation. Must have excellent knowledge and programing capability using Python, PySpark working on any of the Cloud Platforms like AWS, Azure or Google. Role Industry Type: Engineering Functional Area: Data Engineering, Software Development, Automation Employment Type: Full Time, Permanent Role Category: System Design/Implementation Education : A minimum educational requirement is graduation. Here at Havas across the group we pride ourselves on being committed to offering equal opportunities to all potential employees and have zero tolerance for discrimination. We are an equal opportunity employer and welcome applicants irrespective of age, sex, race, ethnicity, disability and other factors that have no bearing on an individual’s ability to perform their job. Show more Show less

Job Description Data Engineer (Manager) - Web Scraping (Experience: 5 to 10 years) The Data Engineer specializing in web scraping will be responsible for designing, implementing, and maintaining automated systems to extract, process, and analyse data from various online sources. This role is critical for gathering valuable insights to support business decisions and strategies. Responsibilities ?Lead and manage a team of data engineers specializing in web scraping and data extraction. ?Design, develop, and maintain scalable web scraping pipelines and ETL processes. ?Collaborate with cross-functional teams to understand data requirements and deliver effective solutions. ?Ensure data quality, integrity, and security across all scraping systems. ?Optimize web scraping workflows for performance and efficiency. ?Evaluate and integrate new tools and technologies for web scraping and data processing. ?Develop and enforce best practices for web scraping, including compliance with legal and ethical standards. ?Provide mentorship and professional development opportunities for team members. Skills Required ?Proficiency in web scraping tools and frameworks (e.g., Scrapy, Beautiful Soup, Selenium). ?Strong programming skills in Python, Java, or similar languages. ?Experience with data storage solutions (SQL, NoSQL, cloud databases). ?Knowledge of APIs and data integration techniques. ?Familiarity with big data technologies (e.g., Hadoop, Spark). ?Leadership and team management skills. ?Excellent problem-solving and analytical abilities. Preferred Qualifications ?Bachelor?s or Master?s degree in Computer Science, Data Engineering, or related fields. ?Experience in handling large-scale data extraction projects. ?Knowledge of data governance and compliance regulations. Skills Required RoleData Engineer (Manager) - Web Scraping Industry TypeITES/BPO/KPO Functional AreaFinance/Accounts/Taxation Required EducationAny Graduates Employment TypeFull Time, Permanent Key Skills WEB SCRAPING TOOLS DATA PROCESSING DATABASE MANAGEMENT CLOUD PLATFORMS DATA VISUALIZATION Other Information Job CodeGO/JC/21526/2025 Recruiter NameSPriya Show more Show less

Job Title: Senior Python Developer Company: Darwix AI Location: Gurgaon (On-site) Type: Full-Time Experience: 3–8 years About Darwix AI Darwix AI is one of India’s fastest-growing AI startups, transforming enterprise sales with our GenAI-powered conversational intelligence and real-time agent assist suite. Our platform is used by high-growth enterprises across India, MENA, and Southeast Asia to improve sales productivity, personalize customer conversations, and unlock revenue intelligence in real-time. We are backed by marquee VCs, 30+ angel investors, and led by alumni from IITs, IIMs, and BITS with deep experience in building and scaling products from India for the world. Role Overview As a Senior Python Developer at Darwix AI, you will be at the core of our engineering team, leading the development of scalable, secure, and high-performance backend systems that support AI workflows, real-time data processing, and enterprise-grade integrations. This role requires deep technical expertise in Python, a strong foundation in backend architecture, and the ability to collaborate closely with AI, product, and infrastructure teams. You will take ownership of critical backend modules and shape the engineering culture in a rapidly evolving, high-impact environment. Key Responsibilities System Architecture & API Development Design, implement, and optimize backend services and microservices using Python frameworks such as FastAPI, Django, or Flask Lead the development of scalable RESTful APIs that integrate with frontend, mobile, and AI systems Architect low-latency, fault-tolerant services supporting real-time sales analytics and AI inference Data Pipelines & Integrations Build and optimize ETL pipelines to manage structured and unstructured data from internal and third-party sources Integrate APIs with CRMs, telephony systems, transcription engines, and enterprise platforms like Salesforce, Zoho, and LeadSquared Lead scraping and data ingestion efforts from large-scale, dynamic web sources using Playwright, BeautifulSoup, or Scrapy AI/ML Enablement Work closely with AI engineers to build infrastructure for LLM/RAG pipelines , vector DBs , and real-time AI decisioning Implement backend support for prompt orchestration , Langchain flows , and function-calling interfaces Support model deployment, inference APIs, and logging/monitoring for large-scale GenAI pipelines Database & Storage Design Optimize database design and queries using MySQL , PostgreSQL , and MongoDB Architect and manage Redis and Kafka for caching, queueing, and real-time communication DevOps & Quality Ensure continuous delivery through version control (Git), CI/CD pipelines, testing frameworks, and Docker-based deployments Identify and resolve bottlenecks related to performance, memory, or data throughput Adhere to best practices in code quality, testing, security, and documentation Leadership & Collaboration Mentor junior developers and participate in code reviews Collaborate cross-functionally with product, AI, design, and sales engineering teams Contribute to architectural decisions, roadmap planning, and scaling strategies Qualifications 4–8 years of backend development experience in Python, with a deep understanding of object-oriented and functional programming Hands-on experience with FastAPI , Django , or Flask in production environments Proven experience building scalable microservices, data pipelines, and backend systems that support live applications Strong command over REST API architecture , database optimization, and data modeling Solid experience working with web scraping tools , automation frameworks, and external API integrations Knowledge of AI tools like Langchain , HuggingFace , Vector DBs (Pinecone, Weaviate, FAISS) , or RAG architectures is a strong plus Familiarity with cloud infrastructure (AWS/GCP) , Docker, and containerized deployments Comfortable working in fast-paced, high-ownership environments with shifting priorities and dynamic problem-solving Bonus Prior experience in an early-stage SaaS startup or AI-first product environment Contributions to open-source Python projects or developer communities Experience working with real-time streaming systems (Kafka, Redis Streams, WebSockets) What We Offer Competitive fixed salary + performance-linked incentives Equity options for high-impact performers Opportunity to work on cutting-edge GenAI and SaaS products used by global enterprises Autonomy, rapid decision-making, and direct interaction with founders and senior leadership High-growth environment with clear progression toward Tech Lead or Engineering Manager roles Access to tools, compute, and learning resources to accelerate your technical and leadership growth Show more Show less

Join our dynamic team as a Web Scraping Engineer and play a crucial role in driving our data-driven strategies. As a key player, you will develop and maintain innovative solutions to automate data extraction, parsing, and structuring from various online sources. Your expertise will empower our business intelligence, market research, and decision-making processes. If you are passionate about automation, dedicated to ethical practices, and have a knack for solving complex problems, we want you! Key Responsibilities Design, implement, and maintain web scraping solutions to collect structured data from publicly available online sources and APIs Parse, clean, and transform extracted data to ensure accuracy and usability for business needs Store and organize collected data in databases or spreadsheets for easy access and analysis Monitor and optimize scraping processes for efficiency, reliability, and compliance with relevant laws and website policies Troubleshoot issues related to dynamic content, anti-bot measures, and changes in website structure Collaborate with data analysts, scientists, and other stakeholders to understand data requirements and deliver actionable insights Document processes, tools, and workflows for ongoing improvements and knowledge sharing Requirements Proven experience in web scraping, data extraction, or web automation projects Proficiency in Python or similar programming languages, and familiarity with libraries such as BeautifulSoup, Scrapy, or Selenium Strong understanding of HTML, CSS, JavaScript, and web protocols Experience with data cleaning, transformation, and storage (e.g., CSV, JSON, SQL/NoSQL databases) Knowledge of legal and ethical considerations in web scraping, with a commitment to compliance with website terms of service and data privacy regulations Excellent problem-solving and troubleshooting skills Ability to work independently and manage multiple projects simultaneously Preferred Qualifications Experience with cloud platforms (AWS, GCP, Azure) for scalable data solutions Familiarity with workflow automation and integration with communication tools (e.g., email, Slack, APIs) Background in market research, business intelligence, or related fields Skills: data extraction,data cleaning,beautifulsoup,business intelligence,web automation,javascript,web scraping,data privacy regulations,web protocols,selenium,scrapy,sql,data transformation,nosql,css,market research,automation,python,html Show more Show less

Location: Remote About the Role We are seeking an experienced Integration Developer with a strong background in Python and JavaScript frameworks . The ideal candidate will have a minimum of 1 year in software engineering, including at least 1 year dedicated to ethical web scraping and automation. This role offers flexibility with options for remote work and in-person collaboration. Requirements Bachelor’s degree in Computer Science, Information Technology, or a related field. Minimum of 1 year of professional experience in software engineering. At least 1 year of hands-on experience with web scraping and data extraction. Proficiency in Python and JavaScript, including frameworks such as Scrapy, BeautifulSoup, Puppeteer, or Playwright. Strong understanding of RESTful APIs and experience with API integration. Familiarity with handling anti-bot measures, including CAPTCHA solving and IP rotation. Experience with cloud platforms (e.g., AWS, Azure) and version control systems like Git. Excellent problem-solving skills and attention to detail. Key Responsibilities Develop and maintain automated web scraping solutions to extract structured data from various online platforms. Implement and manage API integrations to facilitate seamless data exchange between systems. Ensure compliance with ethical standards and legal guidelines in all web scraping activities . Collaborate with cross-functional teams to understand integration requirements and deliver scalable solutions. Monitor and optimize data pipelines for performance, reliability, and accuracy. Show more Show less

eGrove Systems Pvt Ltd is looking for Senior Python Developer to join its team of experts. Skill : Senior Python Developer. Exp : 4+Yrs. NP : Immediate to 15days. Location : Chennai/Madurai. Skills Requirement Hands-on software development skills, deep technical expertise across the entire software delivery process. Forward-thinking skilled individual. Structured, organized, and a good communicator. Write reusable, Testable, and Efficient code. Required Skills 3+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc. Experience developing in a Linux environment, GIT & Agile methodology. Good to have knowledge in any one of the JavaScript frameworks: jQuery, Angular, ReactJS. Good to have experience in implementing charts, graphs using various libraries. Good to have experience in Multi-Threading, REST API management. About Company eGrove Systems is a leading IT solutions provider specializing in eCommerce, enterprise application development, AI-driven solutions, digital marketing, and IT consulting services. Established in 2008, we are headquartered in East Brunswick, New Jersey, with a global presence. Our expertise includes custom software development, mobile app solutions, DevOps, cloud services, AI chatbots, SEO automation tools, and workforce learning systems. We focus on delivering scalable, secure, and innovative technology solutions to enterprises, start-ups, and government agencies. At eGrove Systems, we foster a dynamic and collaborative work culture driven by innovation, continuous learning, and teamwork. We provide our employees with cutting-edge technologies, professional growth opportunities, and a supportive work environment to thrive in their careers. (ref:hirist.tech) Show more Show less

Urgent Hiring – Senior Python Developer (Web Scraping) Location: Ahmedabad (Work from Office) Joining: Immediate Joiners Only Experience: 2+ Years (Mandatory) Are you passionate about web scraping and ready to take on exciting data-driven projects? Actowiz Solutions is urgently hiring a skilled Senior Python Developer to join our dynamic team in Ahmedabad! Key Skills We're Looking For: • Strong hands-on experience with Scrapy framework • Deep understanding of XPath/CSS selectors, middleware & pipelines • Experience handling CAPTCHAs, IP blocks, and JS-rendered content • Familiar with proxy rotation, user-agent switching, and headless browsers • Proficient in data formats: JSON, CSV, and databases • Hands-on with Scrapy Splash / Selenium • Good knowledge of Pandas, Docker, AWS, and Celery How to Apply: Send your resume to hr@actowizsolutions.com / aanchalg.actowiz@gmail.com Contact HR: 8200674053 / 8401366964 Or DM me directly! If you’re ready to join a fast-paced team and work on global data projects, we’d love to hear from you! Feel free to like, share, or tag someone who might be a fit! #PythonJobs #WebScraping #Scrapy #ImmediateJoiner #AhmedabadJobs #PythonDeveloper #DataJobs #ActowizSolutions Show more Show less

We are hiring Python Developer! Role: Sr.Python Developer Work Locations: Teynampet, Chennai/Kk Nagar, Madurai Work from Office: (1pm to 10pm) Monday - Friday Mode of Interview: In-Person Experience : 3+ Years Required Skills: - 3+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc., Experience developing in a Linux environment, GIT & Agile methodology. Good to have knowledge in any one of the JavaScript frameworks: jQuery, Angular, ReactJS. Good to have experience in implementing charts, graphs using various libraries. Good to have experience in Multi-Threading, REST API management. Interested candidates can send their resume to this mail id - dharshanamurthy.v@egrovesys.com Or WhatsApp - 9342768767 About Company eGrove Systems is a leading IT solutions provider specializing in eCommerce, enterprise application development, AI-driven solutions, digital marketing, and IT consulting services . Established in 2008 , we are headquartered in East Brunswick, New Jersey , with a global presence. Our expertise includes custom software development, mobile app solutions, DevOps, cloud services, AI chatbots, SEO automation tools, and workforce learning systems . We focus on delivering scalable, secure, and innovative technology solutions to enterprises, startups, and government agencies. At eGrove Systems, we foster a dynamic and collaborative work culture driven by innovation, continuous learning, and teamwork . We provide our employees with cutting-edge technologies, professional growth opportunities, and a supportive work environment to thrive in their careers. Show more Show less

We are accepting applications for experienced Data Engineer with a strong background in data scraping, cleaning, transformation, and automation. The ideal candidate will be responsible for building robust data pipelines, maintaining data integrity, and generating actionable dashboards and reports to support business decision-making. Key Responsibilities: Develop and maintain scripts for scraping data from various sources including APIs, websites, and databases. Perform data cleaning, transformation, and normalization to ensure consistency and usability across all data sets. Design and implement relational and non-relational data tables and frames for scalable data storage and analysis. Build automated data pipelines to ensure timely and accurate data availability. Create and manage interactive dashboards and reports using tools such as Power BI, Tableau, or similar platforms. Write and maintain data automation scripts to streamline ETL (Extract, Transform, Load) processes. Ensure data quality, governance, and compliance with internal and external regulations. Monitor and optimize the performance of data workflows and pipelines. Qualifications & Skills: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field. Minimum of 5 years of experience in a data engineering or similar role. Proficient in Python (especially for data scraping and automation), and strong hands-on experience with Pandas, NumPy , and other data manipulation libraries. Experience with web scraping tools and techniques (e.g., BeautifulSoup, Scrapy, Selenium). Strong SQL skills and experience working with relational databases (e.g., PostgreSQL, MySQL) and data warehouses (e.g., Redshift, Snowflake, BigQuery). Familiarity with data visualization tools like Power BI, Tableau, or Looker. Knowledge of ETL tools and orchestration frameworks such as Apache Airflow, Luigi, or Prefect . Experience with version control systems like Git and collaborative platforms like Jira or Confluence . Strong understanding of data security, privacy , and governance best practices. Excellent problem-solving skills and attention to detail. Preferred Qualifications: Experience with cloud platforms such as AWS, GCP, or Azure. Familiarity with NoSQL databases like MongoDB, Cassandra, or Elasticsearch. Understanding of CI/CD pipelines and DevOps practices related to data engineering. Show more Show less

Job Title: Python AI Backend Developer Pay Bracket: INR 6 – 10 LPA (commensurate with skills & experience) Location: Mumbai (Andheri W) Company: ZANG – the AI-powered e-commerce search engine About ZANG ZANG is re-imagining online shopping with a unified, AI-driven search experience that pinpoints the right product—at the right price—across every major marketplace. We’re an early-stage, venture-backed team that moves fast, ships often, and sweats the details. The Role We’re looking for a Python AI Backend Developer who can turn cutting-edge AI models and readily available AI services into production-ready, automated workflows. You’ll design the back-end engines that power our search, recommendation, and data-ingestion pipelines—so shoppers get relevant results in milliseconds. Key Responsibilities What you’ll own AI Workflow Engineering • Orchestrate end-to-end workflows (ingestion → processing → vector indexing → API) using Airflow, Prefect, or similar. • Combine open-source LLMs/embeddings (e.g., OpenAI, Hugging Face) with in-house models to deliver ranking, personalization, and semantic search. Backend Development • Design and scale Python micro-services (FastAPI/Falcon/Flask) that expose clean REST & gRPC endpoints. • Implement robust authentication, rate-limiting, and logging/observability. Data & Scraping Pipelines • Maintain modular scrapers for key e-commerce sites; handle rotating proxies, CAPTCHA, and dynamic pages. • Transform raw HTML into structured datasets ready for model training and search indexing. Storage & Retrieval • Optimise vector / NoSQL stores (Pinecone, Milvus, MongoDB, Elasticsearch) for low-latency reads and high-throughput writes. • Implement data versioning and retention policies. Collaboration & Reporting • Work closely with front-end, DevOps, and product teams to ship features on time. • Write clear technical docs and participate in code reviews. Must-Have Skills Expert-level Python (type hints, async IO, packaging, unit tests). Hands-on with web-scraping stacks: Scrapy / BeautifulSoup / Selenium / Playwright . Solid grasp of AI/ML & NLP concepts and libraries (PyTorch, TensorFlow, spaCy, transformers). Experience integrating third-party AI APIs (OpenAI, Cohere, Google Gemini, etc.). Production experience with REST/gRPC , containerisation (Docker), and CI/CD. Working knowledge of vector databases or search engines (e.g., Pinecone, Qdrant, Elasticsearch). Git-centric workflow and comfort with Agile/GitHub boards. Nice-to-Have Prior work on e-commerce or large-scale product-catalogue data. AWS stack (ECS, Lambda, S3, Step Functions) or equivalent cloud experience. Familiarity with recommendation systems, learning-to-rank, or reinforcement-learning-to-rank. Knowledge of orchestration tools (Airflow, Prefect, Dagster). What’s in It for You Competitive pay + stock options. The freedom to choose the best tools and ship production code from Day 1. A front-row seat in a high-growth startup solving a real consumer pain-point. A culture that values clear thinking, quick execution, and continuous learning. How to Apply Skip the queue—email your résumé (or GitHub/portfolio) to amit.letsbegin@gmail.com with the subject “Python AI Backend Developer – ZANG” . We’ll set up a quick call to dive deeper into the role and your work. If building scalable AI products is your idea of fun, we’d love to hear from you. Show more Show less