11+ Web Scraping Jobs in Ahmedabad | Web Scraping Job openings in Ahmedabad

Position: Lead Python Developer

Location: Ahmedabad, Gujarat

The Client company includes a team of experienced information services professionals who are passionate about growing and enhancing the value of information services businesses. They provide support with talent, technology, tools, infrastructure and expertise required to deliver across the Data ecosystem. Position Summary We are seeking a skilled and experienced Backend Developer with strong expertise in TypeScript, Python, and web scraping. You will be responsible for designing, developing, and maintaining scalable backend services and APIs that power our data-driven products. Your role will involve collaborating with cross functional teams, optimizing system performance, ensuring data integrity, and contributing to the design of efficient and secure architectures.

Job Responsibility

● Design, develop, and maintain backend systems and services using Python and TypeScript.

● Develop and maintain web scraping solutions to extract, process, and manage large-scale data from multiple sources.

● Work with relational and non-relational

databases, ensuring high availability, scalability, and performance.

● Implement authentication, authorization, and security best practices across services.

● Write clean, maintainable, and testable code following best practices and coding standards.

● Collaborate with frontend engineers, data engineers, and DevOps teams to deliver robust solutions and troubleshoot, debug, and upgrade existing applications.

● Stay updated with backend development trends, tools, and frameworks to continuously improve processes.

● Utilize core crawling experience to design efficient strategies for scraping the data from different websites and applications.

● Collaborate with technology teams, data collection teams to build end to end technology-enabled ecosystems and partner in research projects to analyze the massive data inputs.

● Responsible for the design and development of web crawlers, able to independently solve various problems encountered in the actual development process.

● Stay updated with the latest web scraping techniques, tools, and industry trends to continuously improve the scraping processes.

Job Requirements

● 4+ years of professional experience in backend development with TypeScript and Python.

● Strong understanding of TypeScript-based server-side frameworks (e.g., Node.js, NestJS, Express) and Python frameworks (e.g., FastAPI, Django, Flask).

● Experience with tools and libraries for web scraping (e.g., Scrapy, BeautifulSoup, Selenium, Puppeteer)

● Hands-on experience with Temporal for creating and orchestrating workflows

● Proven hands-on experience in web scraping, including crawling, data extraction, deduplication, and handling dynamic websites.

● Proficient in implementing proxy solutions and handling bot-detection challenges (e.g., Cloudflare).

● Experience working with Docker, containerized deployments, and cloud environments (GCP or Azure).

● Proficiency with database systems such as MongoDB and Elastic Search.

● Hands-on experience with designing and maintaining scalable APIs.

● Knowledge of software testing practices (unit, integration, end-to-end).

● Familiarity with CI/CD pipelines and version control systems (Git).

● Strong problem-solving skills, attention to detail, and ability to work in agile environments.

● Great communication skills and ability to navigate in undirected situations.

Job Exposure:

● Opportunity to apply creative methods in acquiring and filtering the North American government, agencies data from various websites, sources

● In depth industry exposure on data harvesting techniques to build, scale the robust and sustainable model, using open-source applications ● Effectively collaboration with IT team to design the tailor-made solutions basis upon clients’ requirement

● Unique opportunity to research on various agencies, vendors, products as well as technology tools to compose a solution 

Job Location - Hyderabad & Ahmedabad 

What you'll do: 

• Perform complex application programming activities with an emphasis on mobile development:  Node.js, TypeScript, JavaScript, RESTful APIs and related backend frameworks 

• Assist in the definition of system architecture and detailed solution design that are scalable and extensible 
• Collaborate with Product Owners, Designers, and other engineers on different permutations to find the best solution possible 
• Own the quality of code and do your own testing. Write unit test and improve test coverage. 
• Deliver amazing solutions to production that knock everyone’s socks off 
• Mentor junior developers on the team 

What we’re looking for: 

• Amazing technical instincts. You know how to evaluate and choose the right technology and approach for the job. You have stories you could share about what problem you thought you were solving at first, but through testing and iteration, came to solve a much bigger and better problem that resulted in positive outcomes all-around. 
• A love for learning. Technology is continually evolving around us, and you want to keep up to date to ensure we are using the right tech at the right time. 
• A love for working in ambiguity—and making sense of it. You can take in a lot of disparate information and find common themes, recommend clear paths forward and iterate along the way. You don’t form an opinion and sell it as if it’s gospel; this is all about being flexible, agile, dependable, and responsive in the face of many moving parts. 
• Flexible and willing to accept change in priorities, as necessary 

• Demonstrable passion for technology (e.g., personal projects, open-source involvement)  
• Enthusiastic embrace of DevOps culture and collaborative software engineering 
• Ability and desire to work in a dynamic, fast paced, and agile team environment 
• Enthusiasm for cloud computing platforms such as AWS or Azure 
• Excellent written and verbal communications skills 

Preferred Qualifications:  

• Experience with Apache Cordova framework 
• Demonstrable knowledge of native coding background in iOS, Android 
• Experience developing and deploying applications within Kubernetes based containers  
• Experience in Agile and SCRUM development techniques 

Who You Are:

• You’ve built scalable, production-ready APIs in fast-paced environments, and are comfortable with the challenges that come with a growing startup.
• You understand event-driven architectures and have hands-on experience building real-time applications.
• You are fully capable of taking ownership of backend systems, from designing the database schema to writing efficient, maintainable code.
• You think beyond the code: optimizing performance, scalability, and security are as important as building new features.
• You’re comfortable troubleshooting complex issues, whether that’s API bottlenecks, database performance, or production issues.

Responsibilities:

• Design, implement, and manage a NestJS backend with a modular, scalable architecture that can easily handle real-time events and communication.
• Develop and maintain WebSocket services for real-time event-driven updates and ensure seamless communication across the system.
• Integrate RabbitMQ for reliable internal messaging, handling queues and ensuring event-driven workflows are efficient and fault-tolerant.
• Implement database management strategies using MongoDB, Redis, and Elasticsearch, ensuring efficient data handling, indexing, and optimization.
• Integrate with external APIs (JSON-RPC, XML-RPC, REST) to enhance system capabilities.
• Ensure high performance of the system by optimizing database queries, implementing caching strategies using Redis, and ensuring optimal indexing and data flow.
• Implement role-based authentication and authorization mechanisms using JWT, OAuth, and RBAC patterns within NestJS.
• Follow security best practices to ensure sensitive data is protected, credentials are stored securely, and the system is resilient to common vulnerabilities.
• Work collaboratively with cross-functional teams to ensure seamless integration between different services and technologies.

Must-Have Skills:

• Strong proficiency in Node.js and NestJS with TypeScript.
• Solid experience with MongoDB, Redis, and Elasticsearch for data storage and real-time data handling.
• In-depth knowledge of WebSockets and Socket.IO in NestJS Gateways, enabling real-time communication and updates.
• Experience with RabbitMQ for message queuing and asynchronous task handling.
• Strong understanding of authentication & authorization systems using JWT, OAuth, and RBAC (Role-Based Access Control).
• Expertise in optimizing APIs, including techniques for caching, improving database performance, and reducing latency.
• Familiar with API security best practices, including secure storage of credentials, encryption, and safeguarding sensitive data.

Key Skills:

• TypeScript, NestJS, WebSockets, Socket.io
• Redis, MongoDB, Elasticsearch, RabbitMQ
• API Optimization, JWT, OAuth, RBAC
• Real-time communication, Event-driven architecture

Requirements:

• At least 4+ years of experience as a Node.js developer.
• Must have knowledge of at least one card game with rules.
• Must have experience in multiplayer games with Socket.io
• Must have good communication skills in English.
• Superb interpersonal, communication, and collaboration skills.
• Exceptional analytical and problem-solving aptitude.
• Great organizational and time management skills.
• Knowledge of Game development life cycle.
• Must be willing to do hands-on coding where required and perform regular code reviews etc.
• Knowledge of JavaScript, TypeScript, NPMs, ExpressJS, MongoDB, Socket.io and Redis.
• Familiarity with AWS infrastructure.
• Basic knowledge of front-end technologies.
• Strong understanding of git.

We are looking for Sr. Node Js Developer

Experience - 3-5 years 

Location - Ahmedabad, Gujarat. (Work from Office)

5 Days working 

Should have good communication skills.

Requirements:

•API Development

• Mongodb or Firebase

•Designing and developing user interfaces using Angular OR JavaScript MV-VM/MVC frameworks

•Good understanding of AJAX and JavaScript, TypeScript Dom manipulation Techniques

•Experience with RESTful services

•Expert in any one of the modern JavaScript MV-VM/MVC frameworks (AngularJS, JQuery, ReactJS)

•Knowledge of Node.js, Express and frameworks available for it

•Understanding the nature of asynchronous programming and its quirks and workarounds

Roles and Responsibilities :

• Design, Develop, test and maintain software applications with MVC architectures, Microservices and scalable cloud-native apps.
• Perform hands-on coding and provide thoughtful code reviews
• Mentor and lead team members to continuously improve the quality and productivity of deliverables
• Demonstrate ability to work within an agile team as well as collaborate with many other teams on design and implementation strategies. Follow SCRUM Agile Development methodology
• Effective Communication Skills- An ability to express your ideas to the recipient & Co-ordinate and communicate with the cross-functional team along with the client's team
• Problem Solving, Hustler - Get Stuff Done, Self-Disciplined Open-minded, User-Focused, Adaptive
• Lean Principles & Analytical Thinking
• Project planning, timelines and milestones for execution and deliverables
• Participate in the Evaluation Review of your team member

Primary Skills

• Expert in Python with experience on Django.
• Expert in server programming, you should be able to implement complex business logic.
• Good understanding of the threading limitations of Python, and multi-process architecture.
• Good understanding of fundamental design principles behind a scalable application.
• Knowledge of event-driven programming in Python.
• Experience with python packages like Numpy, pandas, flask, Pyspark, pymongo, os, logging, re, pyyaml.
• Experience with swagger io module to expose API
• Strong unit test and debugging skills.
• At least 2 years of experience in RESTful API/HTTP/JSON`
• Knowledge of AWS cloud and python Boto lib.
• Proficient understanding of code versioning tools Like Git.
• Good understanding of Linux OS.

Secondary Skills:

• AWS Kinesis
• Web Sockets
• RAbbitMQ
• AWS Cloud formation

• At least 4 years of experience programming in Python.
• Develop application code and modules for business and technical requirements.
• Tune up design for maintainability, scalability and efficiency.
• Resolve and troubleshoot problems and complex issues.
• Perform unit tests and fix bugs.
• Integrate best qualitative practices in design and development aspects of programs.
• Ability to mentor subordinates.
• Excellent communication skills.