Cutshort logo
Our client provides data solutions for fraud prevention logo
Tech Lead - Web Scraping and Crawling
Our client provides data solutions for fraud prevention
Tech Lead - Web Scraping and Crawling
Purple Squirrel Consulting Services's logo

Tech Lead - Web Scraping and Crawling

at Our client provides data solutions for fraud prevention

4 - 10 yrs
₹10L - ₹20L / yr
Remote only
Skills
Web Scraping
Web crawling
Selenium
Selenium Web driver
Beautiful Soup
skill iconData Science
web crawler
skill iconJavascript
skill iconJava
skill iconPython
skill iconHTML/CSS
  • Manage individual projects priorities, deadlines, and deliverables

  • Gather and process raw data at scale (including writing scripts, web scraping, calling/create

    APIs, etc.) from the web / internet

  • Develop frameworks for automating and maintaining constant flow of data from multiple

    sources

  • Identify, analysis, design, and implement internal process improvements

  • Design and implement tooling upgrades to increase stability and data quality

  • Help team to fix issues that occur in test and production environments

  • Automate software development processes, including build, deploy, and test

  • Manage and guide the team members

REQUIRED QUALIFICATIONS:

  •   4+ years of web crawling/ scraping experience is a must

  •   Strong knowledge of scraping frameworks such as Scrapy, Beautiful Soup, HTQL, Jsoup, Web-

    Harvest and others

  • Excellent verbal, written, and interpersonal communication skills in English

  •   Good to have Experience of complex crawling (like captcha, Mobile OTP based crawling,

    bypassing proxy)

  •   Sound Knowledge in Bot Management Techniques

  •   Experience in various data extraction methods (like data extraction from PDF Files, web

    pages, etc)

  •   Good understanding of HTML DOM, CSS, Javascript, and RESTful web services

  •   Good to have understanding of AWS

  •   Experience with Linux

  •   Experience with Java / Python

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Wissen Technology
at Wissen Technology
4 recruiters
Sukanya Mohan
Posted by Sukanya Mohan
Hyderabad
3 - 8 yrs
Best in industry
Software Testing (QA)
Test Automation (QA)
Appium
Selenium
skill iconJava

Key Responsibilities:

 

- Design, develop, and execute automated test scripts for trading applications.

- Work with product owners and business analysts to understand and write the acceptance test cases.

- Collaborate with developers, product managers, and other stakeholders to understand requirements and create test plans.

- Perform regression, performance, and end to end testing to ensure software reliability.

- Identify, document, and track defects using appropriate tools and methodologies.

- Maintain and enhance existing test automation frameworks for both frontend and backend.

- Report on coverage, functionality, defect aging, closure reports to the stakeholders so that they know the stability of releases.

- Integrate automation cases into CI/CD pipelines

 

Qualifications:

 

- Bachelor’s degree in Computer Science, Engineering, or a related field.

- Proven 3+ Years experience in automation testing for web and backend applications.

- Strong knowledge of testing frameworks (e.g., Selenium, Cypress, JUnit, TestNG, Playwright).

- Experience with API testing tools (e.g., Postman, SoapUI, RestAssured).

- Familiarity with programming languages such as Java, Python, or JavaScript.

- Understanding of basic SQL queries to validate data in the databases

- Understanding of CI/CD processes and tools (e.g., Jenkins, GitLab CI).

- Strong analytical and problem-solving skills.

- Excellent communication and teamwork abilities.

- Prior experience with trading applications or core financial services related applications is a big plus

 

Read more
EnterpriseMinds
at EnterpriseMinds
2 recruiters
phani kalyan
Posted by phani kalyan
Pune
5 - 8 yrs
₹9L - ₹17L / yr
Splunk
skill iconPython
skill iconVisual Studio
Bitbucket
Apache Kafka
+2 more

Enterprise Minds, with core focus on engineering products, automation and intelligence, partners customers on the trajectory towards increasing outcomes, relevance, and growth.

Harnessing the power of Data and the forces that define AI, Machine Learning and Data Science, we believe in institutionalizing go-to-market models and not just explore possibilities.

We believe in a customer-centric ethic without and people-centric paradigm within. With a strong sense of community, ownership, and collaboration our people work in a spirit of co-creation, co-innovation and co-development to engineer next-generation software products with the help of accelerators.

Through Communities we connect and attract talent that shares skills and expertise. Through Innovation Labs and global design studios we deliver creative solutions.
We create vertical isolated pods which has narrow but deep focus. We also create horizontal pods to collaborate and deliver sustainable outcomes.

We follow Agile methodologies to fail fast and deliver scalable and modular solutions. We constantly self-asses and realign to work with each customer in the most impactful manner.

Pre-requisites for the Role

1.Job ID-EMSP0120PS

  1. Primary skill:
  • Splunk Development and Administration

 

  1. Secondary skill:

Python, Splunk DB connect, Visual Studio (C#), BitBucket, Kafka, Devops tools. 

  1. 4. Years of Experience: 5-8 Years
  2. Location:(Hybrid Model)-Pune
  3. Position-1
  4. Budget- - 5-6 years (Max up to 17 LPA) and 6-8 Years (Max up to 22 LPA)
  5. NP- Immediate

 

Primary Role & Responsibility:

As a software engineer, your daily work involves technically challenging applications and projects where your code makes a direct contribution to the further development and upkeep of our software suite and to its application in projects.

You should be able to create Splunk dashboards, apps and should have good understanding of source interfaces for Splunk.

You should have idea of onboarding of data from different sources like JSON, XML, syslog, errorlog files.

As a software engineer, we expect much more from you than just the ability to design and develop good software. We find it important that you possess an inherent drive to get the best out of yourself every day, that you are inquisitive and that you are not intimidated by situations which require you to branch off from the beaten track. You work together with colleagues in a SCRUM team. In addition, you have regular contact with other software teams, software architects, testers and end users. Good communication skills are therefore extremely important, as well as the ability to think pro-actively and suggest possible improvements. This gives you every opportunity to contribute your personal input and grow and develop within the department.

The often complex functionality of the software includes business logic, controls for logistical transport, communication with external computer systems, reporting, data analysis and simulation. This functionality is spread across various components. You design, program and test the software based on a design concept and a set of requirements. In some cases, you will have to personally formulate these requirements together with the (end) users and / or internal stakeholders. Learn more about the Software modular stack

  

Desired Profile & Experience: Knowledge of Kafka and experience with Java

  • Splunk Architecture, on-premise and cloud based deployment.
  • IoT edge.
  • Analytical skills and capabilities to understand how raw (unstructured) data needs to be transformed into processed information.
Read more
Arjh Tech Labs
Amit Pandey
Posted by Amit Pandey
Noida
1 - 3 yrs
₹2.4L - ₹3.6L / yr
skill iconVue.js
skill iconAngularJS (1.x)
skill iconAngular (2+)
skill iconReact.js
skill iconJavascript
+4 more

We are looking for an experienced Back-end developer to join our IT team. You will be responsible for the server side of our web applications. If you have excellent programming skills and a passion for developing applications or improving existing ones, we would like to meet you. As a Back-end developer, you’ll work closely with our engineers to ensure system consistency and improve user experience. Ultimately, you should be able to develop and maintain functional and stable web applications to meet our company’s needs.


Responsibilities-

  • Participate in the entire application lifecycle, focusing on coding and debugging
  • Write clean code to develop functional web applications
  • Troubleshoot and debug applications
  • Perform UI tests to optimize performance
  • Manage cutting-edge technologies to improve legacy applications
  • Gather and address technical and design requirements
  • Provide training and support to internal teams
  • Build reusable code and libraries for future use
  • Liaise with developers, designers and system administrators to identify new features
  • Follow emerging technologies
Read more
trivial works solutions pvt. ltd.
Somesh Parashar
Posted by Somesh Parashar
Remote, Lucknow
2 - 4 yrs
₹3L - ₹8L / yr
skill iconNodeJS (Node.js)
skill iconMongoDB
Mongoose
skill iconExpress
We are looking for Full Stack Developers(Nodejs/Reactjs) to join our team and deliver a streamlined user experience. We want a team member who strives to develop a first-class codebase using Nodejs and Reactjs best practices.

Requirements
  • Bachelor's degree in Computer Science or related field or equivalent experience
  • 2+ years of Full Stack development experience using NodeJs
  • Knowledge of relational databases (PostgreSQL, MySQL, etc. )
  • Experience designing and developing REST APIs
  • Strong analytical and communication skills
  • Eager to learn new technologies
  • Strong fundamental knowledge of object-oriented programming, data structures, algorithms, and performance
  • Hands-on experience with agile practices, including coding standards, reviews, build process, testing, and code control management.
Nice To Have
  • Infrastructure experience including AWS, Jenkins, Docker, Kubernetes and CI/CD configuration and deployment is ideal.
  • Our development team is looking for an experienced full-time Full Stack Developer to join our fast-growing organization.
  • As part of the development team, you will be maintaining, enhancing, designing, developing, testing, and documenting on both the client and server-side of our products.
  • The ideal candidate is an innovative problem solver, who has excellent communication and teamwork skills.
  • NodeJS, React JS
  • Docker, AWS
Read more
Dev Group of Institutions
Dev Group of Institutions
Agency job
via Dev Group of institutions by vishal singh
Delhi, Gurugram
2 - 5 yrs
₹3L - ₹6L / yr
ASP.NET
ASP.NET MVC
ms sql
skill iconGit
SVN
+1 more

Job Description

We are looking for .NET developers with following skill sets for building .NET applications.

Asp.Net with MVC 4.0+, Razor, Multi-threading, jQuery, Angular, SOAP API, Web API 2.0, DI Container, IOC, Bootstrap, Bootstrapper, HTML5, MS SQL

Primary responsibility will be to design and develop three layers of applications, and to coordinate with the rest of the team working on different layers of the infrastructure. A commitment to collaborative problem solving, sophisticated design, and quality product is essential.

Job Category            :  Software Engineer
Location                     : Delh

Experience                : 2 to 4 Years

Position Title: Senior Dot Net Developer

 

Key Responsibilities:

  • Individual/ Team.
  • Design, build, and maintain efficient, reusable, and reliable code
  • Integrate data storage solutions
  • Ensure the best possible performance, quality, and responsiveness of applications
  • Identify bottlenecks and bugs, and devise solutions to mitigate and address these issues
  • Help maintain code quality, organization, and automatization
  • Timely submission of reports as necessary and achieving monthly goals is critical to this role.

 

Competencies:

Should possess sound communication and conflict management skills. 


Experience and Knowledge Requirements:

At least 2 years working experience with MS SQL, Asp.Net Web Form & windows services are mandatory.

Qualification:

MCA / B. Tech/ M.Sc. IT

Required Skillset:

  • Strong knowledge of .NET framework
  • Strong understanding of object-oriented programming
  • Skill for writing reusable libraries
  • Familiar with various design and architectural patterns
  • Basic understanding of Common Language Runtime (CLR), its limitations, weaknesses, and workarounds
  • Experience in generics collections. Client-side libraries like jQuery & Angular 1.3
  • Proficient understanding of code versioning tools such as Git, SVN, and TFS
  • Solid working experience on MVC, Singleton, abstract factory, facade design patterns
Read more
Technology Service Company
Technology Service Company
Agency job
via Jobdost by Riya Roy
Remote only
5 - 10 yrs
₹10L - ₹25L / yr
Business Analysis
Business Analyst
Agile/Scrum
Agile Methodologies
Agile Environment
+5 more
 Responsibilities:
  • Evaluating business processes, anticipating requirements, uncovering areas for improvement, and developing and implementing solutions.
  • Leading ongoing reviews of business processes and developing optimization strategies.
  • Staying up-to-date on the latest process and IT advancements to automate and modernize systems.
  • Conducting meetings and presentations to share ideas and findings.
  • Performing requirements analysis.
  • Documenting and communicating the results of your efforts.
  • Effectively communicating your insights and plans to cross-functional team members and management.
  • Gathering critical information from meetings with various stakeholders and producing useful reports.
  • Working closely with clients, technicians, and managerial staff.
  • Providing leadership, training, coaching, and guidance to junior staff.
  • Allocating resources and maintaining cost efficiency.
  • Ensuring solutions meet business needs and requirements.
  • Performing user acceptance testing.
  • Managing projects, developing project plans, and monitoring performance.
  • Updating, implementing, and maintaining procedures.
  • Prioritizing initiatives based on business needs and requirements.
  • Serving as a liaison between stakeholders and users.
  • Managing competing resources and priorities.
  • Monitoring deliverables and ensuring timely completion of projects.

 Requirements:

  • A minimum of 5 years of experience in business analysis or a related field.
  • Exceptional analytical and conceptual thinking skills.
  • The ability to influence stakeholders and work closely with them to determine acceptable solutions.
  • Advanced technical skills.
  • Excellent documentation skills.
  • Fundamental analytical and conceptual thinking skills.
  • Experience creating detailed reports and giving presentations.
  • Competency in Microsoft applications including Word, Excel, and Outlook.
  • A track record of following through on commitments.
  • Excellent planning, organizational, and time management skills.
  • Experience leading and developing top-performing teams.
  • A history of leading and supporting successful projects.
Read more
Gurugram
5 - 9 yrs
₹3L - ₹10L / yr
Data entry
Market Research
Quantitative research
Team leadership
Team Management
Designation / Position Manager - Data Entry Information Services (Data Entry & Market Research)
Job Description
(Minimum 250 characters)*
• Formulate techniques for quality data collection to ensure   adequacy, accuracy, and legitimacy of data.

• Devise and implement efficient and secure procedures for data handling and analysis with attention to all technical aspects.

• Establish rules and procedures for data sharing with upper management, external stakeholders etc.

• Work closely with software engineering to define product requirements and coordinate resources Assist with reports and data extraction when needed.

• Monitor and analyze information and data systems and evaluate their performance to discover ways of enhancing them (new technologies, upgrades etc.)

• Ensure digital databases and archives are protected from security breaches and data losses.

• Troubleshoot data-related problems and authorize maintenance or modifications
Keywords
•Real Estate background only.

•Expert in Data & Market Research,

•Proficient in MS Office (Excel, Access, Word etc.)

•An analytical mindset with problem-solving skills.

• Excellent communication and collaboration skills.
Work Experience Required 8-12Years
CTC offered 5-8 LPA (As per experience)
Job Location(s) Gurgaon
Industry Type Real Estate
Functional Area  Data Entry, MIS, Ms Excel 
Qualification Any Graduate with Team handling experience of Team Leaders, MIS Executive & Data Entry Executives.
   
3 Mandatory Keywords Real Estate, Good in Team Handling, Good in Excel,
MIS,  Data Management & Market Reaseerch
Read more
Sequelstring Solutions and Consultancy Pvt Ltd
NCR (Delhi | Gurgaon | Noida)
2 - 7 yrs
₹3L - ₹7.5L / yr
skill iconPython
skill iconDjango
Basic Requirement
 
  • Should be a Backend Developer - Python only
  • Should be MCA/B.Tech IT
  • Should have Database Knowledge
  • Should have DJango Knowledge
  • Writing effective, scalable code
  • Good Communication
Read more
An Education Company
An Education Company
Agency job
via Multi Recruit by Gi de On Paramesh
Bengaluru (Bangalore)
3 - 4 yrs
₹4L - ₹5L / yr
skill icon.NET
MySQL
SQL
API
skill iconC#
+1 more
  • .NET Core API with C#
  • API calls with stored procedures calls to handle data
  • Implementation of Routes with multiple Controllers
  • Implementation of Centralized Error Handling
  • Three tier structure of Controller, Business and Data Access Layer
  • JSON handling    
 
Technical skills
 
  • TSql
  • Stored procedures
  • using Sql Studio
  • Error handling.
 
Read more
Genpact is one of the service based Company
Genpact is one of the service based Company
Agency job
via Vfind Solutions Pvt Ltd by Sameer Sam
Bengaluru (Bangalore)
3 - 6 yrs
₹3L - ₹5L / yr
skill iconNodeJS (Node.js)
skill iconReact.js
skill iconJavascript
skill iconPHP
skill iconHTML/CSS

equirements:

  • Should have 3.6 - 5 Years work experience.
  • Should have experience in Node. JS.
  • Should have experience in HTML, CSS.
  • Should have experience in JavaScript.
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos