20+ Web Scraping Jobs in India

Responsibilities

·       Develop Python-based APIs using FastAPI and Flask frameworks.

·       Develop Python-based Automation scripts and Libraries.

·       Develop Front End Components using VueJS and ReactJS.

·       Writing and modifying Docker files for the Back-End and Front-End Components.

·       Integrate CI/CD pipelines for Automation and Code quality checks.

·       Writing complex ORM mappings using SQLAlchemy.

Required Skills:

·       Strong experience in Python development in a full stack environment is a requirement, including NodeJS, VueJS/Vuex, Flask, etc.

·       Experience with SQLAchemy or similar ORM frameworks.

·       Experience working with Geolocation APIs (e.g., Google Maps, Mapbox).

·       Experience using Elasticsearch and Airflow is a plus.

·       Strong knowledge of SQL, comfortable working with MySQL and/or PostgreSQL databases.

·       Understand concepts of Data Modeling.

·       Experience with REST.

·       Experience with Git, GitFlow, and code review process.

·       Good understanding of basic UI and UX principles.

·       Project excellent problem-solving and communication skills.

Role : Web Scraping Engineer

Experience : 2 to 3 Years

Job Location : Chennai

About OJ Commerce: 

OJ Commerce (OJC), a rapidly expanding and profitable online retailer, is headquartered in Florida, USA, with a fully-functional office in Chennai, India. We deliver exceptional value to our customers by harnessing cutting-edge technology, fostering innovation, and establishing strategic brand partnerships to enable a seamless, enjoyable shopping experience featuring high-quality products at unbeatable prices. Our advanced, data-driven system streamlines operations with minimal human intervention.

Our extensive product portfolio encompasses over a million SKUs and more than 2,500 brands across eight primary categories. With a robust presence on major platforms such as Amazon, Walmart, Wayfair, Home Depot, and eBay, we directly serve consumers in the United States.

As we continue to forge new partner relationships, our flagship website, www.ojcommerce.com, has rapidly emerged as a top-performing e-commerce channel, catering to millions of customers annually.

Job Summary:

We are seeking a Web Scraping Engineer and Data Extraction Specialist who will play a crucial role in our data acquisition and management processes. The ideal candidate will be proficient in developing and maintaining efficient web crawlers capable of extracting data from large websites and storing it in a database. Strong expertise in Python, web crawling, and data extraction, along with familiarity with popular crawling tools and modules, is essential. Additionally, the candidate should demonstrate the ability to effectively utilize API tools for testing and retrieving data from various sources. Join our team and contribute to our data-driven success!

Responsibilities:

• Develop and maintain web crawlers in Python.
• Crawl large websites and extract data.
• Store data in a database.
• Analyze and report on data.
• Work with other engineers to develop and improve our web crawling infrastructure.
• Stay up to date on the latest crawling tools and techniques.

Required Skills and Qualifications:

• Bachelor's degree in computer science or a related field.
• 2-3 years of experience with Python and web crawling.
• Familiarity with tools / modules such as
• Scrapy, Selenium, Requests, Beautiful Soup etc.
• API tools such as Postman or equivalent. 
• Working knowledge of SQL.
• Experience with web crawling and data extraction.
• Strong problem-solving and analytical skills.
• Ability to work independently and as part of a team.
• Excellent communication and documentation skills.

What we Offer

• Competitive salary

• Medical Benefits/Accident Cover

• Flexi Office Working Hours

• Fast paced start up

 JD / Skills Sets

1. Good knowledge on Python

2. Good knowledge on My-Sql, mongodb 

3. Design Pattern

4. OOPs

5. Automation

6. Web scraping

7. Redis queue

8. Basic idea of Finance Domain will be beneficial.

9. Git10. AWS (EC2, RDS, S3)

• Overall, 4-5 years of experience
• At least 3 years of HTML/CSS development experience is required
• Must have solid working experience on HTML5, CSS3, XML
• Should be an expert on XPath/Regex expressions for complex website navigations
• Working knowledge on core JavaScript
• Experience of mentoring the junior team members will be desirable
• Must have good communication skills

• Worked on Python, with knowledge of Scrapy framework & Beautiful Soup.
• Maintaining the running web crawlers full-stack application.
• Create more/better ways to crawl relevant information
• Builds and maintain new API integrations to support continuing increases in data volume and complexity
• Python Tech stack (Python library: Requests, Urllib, Beautifulsoup).
• Good communication skills.

Position Full-Time: Investment Engineering Associate

Specific Responsibilities

The Investment Engineering Associate’s main responsibilities will include supporting the investment team by building technology-based analytical tools. The Investment Engineering Associate will design and maintain workflows vital to the firm’s investment process - this ranges from creating adhoc analysis to information rich dashboard construction. Typical tasks will include web crawler scripting/collection, web application development, programming (including API integrations), and optimal database/library/query management.

Qualifications

• BS or MS in Computer Science, Engineering, or Data Science

• 2+ years of experience • Candidates should have extensive experience with several of the following languages: Python, Ruby, C/C++, Javascript, HTML5, CSS3, SQL, PHP, etc.

• The Investment Solutions Associate should be familiar with Office VBA • Characteristics of the Solutions Associate include: effective communication skills, attention to detail, and strong ethical values

Key deliverables for the Python Engineer would be to writing and testing code, debugging programs and integrating applications with third-party web services using server-side logic and work well in a team to build features, be a part of the core team

What will you do?

• You will write an effective code for modularity, responsiveness, scalability and overall performance
• Experiment with new & relevant technologies and tools, for designing and implementing robust applications
• Integrating user-facing elements with server-side logic into applications
•  
• Debugging applications to ensure low-latency and high-availability.
• Improving functionality of existing systems and accommodating various data storage solutions
• Assessing and prioritizing feature requests along with implementing security and data protection solutions
• Coordinating with internal teams to understand user requirements and provide technical solutions

We would love to hear from you if :

• You have 2+ years of experience as a Python engineer at a SaaS or technology company
• You have hands-on experience with various Python technology such as Flask, FastAPI, and the latest python frameworks
• Experience in web crawling and data scraping
• Strive to craft production-level scalable python code
• Familiarity with the deployment of python services
• Knowledge of AWS services and ORM (Object Relational Mapper)
• Good communication & written skills

• Brand new Macbook
• Stock Options
• Health Insurance
• Unlimited Sick Leaves
• Passion Fund (Invest in yourself or your passion project)
• Wind Down

• Design, build web crawlers to scrape data and URLs.
• Integrate the data crawled and scraped into our databases
• Create more/better ways to crawl relevant information
• Strong knowledge of web technologies (HTML, CSS, Javascript, XPath, Regex)
• Understanding of data privacy policies (esp. GDPR) and personally identifiable information
• Develop automated and reusable routines for extracting information from various data sources
• Prepare requirement summary and re-confirm with Operation team
• Translate business requirements into specific solutions
• Ability to relay technical information to non-technical users
• Demonstrate Effective problem solving and analytical skill
• Ability to pay attention to detail, pro-active, critical thinking and accuracy is essential
• Ability to work to deadlines and give realistic estimates

Skills & Expertise

• 2+ years of web scraping experience

• Experience with two or more of the following web scraping frameworks and tools: Selenium, Scrapy, Import.io, Webhose.io, ScrapingHub, ParseHub, Phantombuster, Octoparse, Puppeter, etc.
• Basic knowledge of data engineering (database ingestion, ETL, etc.) 
• Solution orientation and "can do" attitude - with a desire to tackle complex problems.

Job role:

As a data analyst, you will be responsible for compiling actionable insights from data and assisting program, sales and marketing managers build data-driven processes. Your role will involve driving initiatives to optimize for operational excellence and revenue.

Responsibilities:

• Ensure that data flows smoothly from source to destination so that it can be processed
• Utilize strong database skills to work with large, complex data sets to extract insights
• Filter and cleanse unstructured (or ambiguous) data into usable data sets that can be analyzed to extract insights and improve business processes
• Identify new internal and external data sources to support analytics initiatives and work with appropriate partners to absorb the data into new or existing data infrastructure
• Build tools for automating repetitive asks so that bandwidth can be freed for analytics
• Collaborate with program managers and business analysts  to help them come up with actionable, high-impact insights across product lines and functions
• Work closely with top management to prioritize information and analytic needs

Requirements:

• Excel
• Scrapping
• Medium Communication Skill (Spoken and written)
• SaaS industries knowledge
• Add Product
• Check Dodontdo
• Scrapping
• Work on Support Tickets (Resolve customers issue)

• Join the product research team to develop the leading Geodata and GeoAI platform in Hong Kong
• Be part of the product team to develop web front-end interface with a focus of data and AI application
• Develop automated software in Arical data and AI pipeline
• On job training for rapid product development cycle
• Will have to deliver reliable and maintainable code
• Use analytical skills effectively for hands on problem solving
• Translate UI/UX design wireframes to actual code
• Work with UI/UX designer

• Degree or above in Computer Science, Information Systems, or related fields
• Coursework in programming, data science, machine learning, AI and databases
• Strong organizational and project management skills
• Proficiency with client-side scripting and database technology

• Knowledge of one or more system programming languages (Python, C++, Java, etc)
• Knowledge of MongoDB and other NoSQL data stores such as Redis
• Proficiency with HTML, CSS, JavaScript, cross-browser optimization
• Proficiency with one or more front-end web development framework (Bootstrap, Angular, React)
• Knowledge of one or more web-scripting languages (PHP, Rudy, Node.js etc)
• Knowledge of web-scraping library (Beautiful Soup, web driver etc)
• Knowledge of API packaging and Docker deployment
• Knowledge of app development
• Knowledge of UI/UX design

• Work from home

• Architect new and optimize existing software codebases and systems used to crawl, launch, run, and monitor the Anakin family of app crawlers
• Deeply own the lifecycle of software, including rolling out to operations, managing configurations, maintaining and upgrading, and supporting end-users
• Configure and optimize the automated testing and deployment systems used to maintain over 1000+ crawlers across the company
• Analyze data and bugs that require in-depth investigations
• Interface directly with external customers including managing relationships and steering requirements

• Extremely effective, self-driven builder
• 2+ years of experience as a backend software engineer
• 2+ years of experience with Python
• 2+ years of experience with AWS services such as EC2, S3, Lambda, etc.
• Should have managed a team of software engineers
• Deep experience with network debugging across all OSI layers (Wireshark)
• Knowledge of networks or/and cybersecurity

• Broad understanding of the landscape of software engineering design patterns and principles
• Ability to work quickly and accurately in a highly stressful environment during removing bugs in run-time within minutes
• Excellent communicator, both written and verbal

• Must be available to work extended hours and weekends when needed to meet critical deadlines
• Must have an aversion to politics and BS. Should let his/her work speak for him/her.
• Must be comfortable with uncertainty. In almost all the cases, your job will be to figure it out.
• Must not be bounded to comfort zone. Often, you will need to challenge yourself to go above and beyond.

Roles and Responsibilities

• Apply knowledge set to fetch data from multiple online sources, cleanse it and build APIs on top of it
• Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrap, parse and store
• We're looking for people who will naturally take ownership of data products and who can bring a project all the way from a fast prototype to production.
• Integrating and maintaining Python services
• Developing robust microservices and applications

Desired Candidate Profile

• Strong relevant experience of at least 2-3 years.
• Strong coding experience in Python3.
• Should have good experience with Django and NodeJs.
• Proficient on modelling applications on both RDBMS and NOSQL databases
• Should have experience in web scraping
• Good understanding and hands-on with scheduling and managing tasks with cron
• Should have experience with microservice architecture
• Compile and analyze data, processes, and codes to troubleshoot problems and identify areas for improvement
• Should have experience in shell scripting, GIT and docker
• Writing unit tests for 100% code coverage.

• Does analytics to extract insights from raw historical data of the organization. 
• Generates usable training dataset for any/all MV projects with the help of Annotators, if needed.
• Analyses user trends, and identifies their biggest bottlenecks in Hammoq Workflow.
• Tests the short/long term impact of productized MV models on those trends.
• Skills - Numpy, Pandas, SPARK, APACHE SPARK, PYSPARK, ETL mandatory. 

1: proficient in python, flask, pandas, GitHub and AWS

2: good knowledge of databases both SQL and NoSQL

3:Strong experience in REST and SOAP APIs

4: Experience with working on scalable interactive web applications

5:Basic knowledge of JavaScript and Html

6: Automation and crawling tools and modules

7: Multithreading and Multiprocessing

8:Good Understanding of test-driven Development

9: Preferred exposure to finance domain

Company Introduction –

• Information Security & Data Analytics Series A funded company.
• Working in cutting edge technologies - Using AI for predictive intelligence and Facial Biometrics.
• Among Top 5 Cyber excellence companies globally (Holger Schulze awards)
• Bronze award for best startup of the year (Indian Express IT awards), only cyber Security Company in top 3.
• More than 100+ clients in India.

Job Description:-

Job Title: Python Developer

Key Requirements:-

• Mine data from structured and unstructured data sources.
• Extract data (text, images, and videos) from multiple documents in different formats.
• Extract information and intelligence from data.
• Extract data based on regular expressions.
• Collect data from structured RDBMS databases.
• Work closely with Project/Business/Research teams to provide mined data/intelligence for analysis.
• Should have strong exposure to core python skills like multiprocessing, multithreading, file handling, data structure like JSON, Data frames, and User Defined Data structure.
• Should have excellent knowledge of classes, file handling, memory manipulations.
• Strong Knowledge in Python.
• Strong exposure to frond end languages like CSS, JavaScript, Ajax etc.
• Should have exposure to requests, Frontera, scarpy-cluster, elastic-search, distributed computing tools like Kafka, Hbase, Redis, Zookeeper, restAPI.
• Should be familiar with *nix development environment.
• Knowledge of Django will be added advantage.
• Excellent knowledge on Web Crawling/Web scraping.
• Should have used scraping modules like Selenium, Scrapy, and Beautiful soup.
• Experience with text processing.
• Basics of databases. Good troubleshooting and debugging skills.

 
Education

   B.Tech, MCA, Computer Engineering

We are looking for a Node.js Developer responsible for managing the interchange of data between the server and the users. Your primary focus will be the development of all server-side logic, definition and maintenance of the central database, and ensuring high performance and responsiveness to requests from the front-end. You will also be responsible for integrating the front-end elements built by your co-workers into the application. Therefore, a basic understanding of front-end technologies is necessary as well.

Responsibilities

• Integration of user-facing elements developed by front-end developers with server side logic
• Writing reusable, testable, and efficient code
• Design and implementation of low-latency, high-availability, and performant applications
• Implementation of security and data protection
• Creating APIs and backend systems
• Scraping sites and creating backend dashboards for data management
• Experience with React.js and native is a plus but not required
• Backend infrastructure management and deployment
• Experience with MongoDB, Express is a plus.