Data Scraping - Python at Qube Research Technologies · Mumbai · 4 - 12 years · ₹30L - ₹70L / yr · Posted 21 Jan 2025

Data Scraping Specialist
Qube Research & Technologies (QRT) is a global quantitative and systematic investment manager, operating in all liquid asset classes across the world. We are a technology and data driven group implementing a scientific approach to investing. Combining data, research, technology and trading expertise has shaped QRT’s collaborative mindset which enables us to solve the most complex challenges. QRT’s culture of innovation continuously drives our ambition to deliver high quality returns for our investors.
Your future role within QRT
- Your core objective is to develop and maintain web scraping tools to build datasets for research and trading
- Contributing to integrate datasets from internal or external providers
- Manage the multiple and extensive dataset used in the QRT research and trading platform
- Monitoring fetching processes and data health
- Supporting users (Quants & Traders)
Your present skillset
- 4+ years’ experience in web scraping
- Advanced English, both written and verbal communication skills
- Polyglot developer, Python coding is important
- Software engineering best practices
- Ability to communicate and understand user needs
- Structured and unstructured data management expertise is a plus
- Knowledge of the financial data of equity/derivatives is a plus
- Intellectual curiosity to learn new technologies
- Capacity to work with autonomy within a global team

Similar jobs (10)
Responsibilities
- Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session or token-based login flows.
- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
- Integrate and manage complex workflows across multiple portals, APIs, and data sources.
- Contribute to architectural decisions, tooling, and best practices.
- Responsible for Code reviews and deployments.
Qualifications
- Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent professional experience
- 10+ years of experience in Python development, with a strong focus on automation and data extraction
- Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests
- In-depth experience handling multi-step authentication flows, including multi-factor authentication (MFA), CAPTCHA solving, and session or cookie management
- Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager
- Experience with asynchronous programming, headless browsers, and JavaScript-rendered content
- Solid understanding of web protocols (i.e. HTTP, HTTPS, cookies, headers), and the ability to reverse-engineer network calls and authentication mechanisms
- Comfortable working with APIs, JSON/XML, and data transformation
- Experience with CI or CD pipelines, Docker, and infrastructure as code (i.e. CloudFormation, Terraform)
- Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions
- Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal)
- Background in data engineering or ETL workflows is a plus
- Exposure to Python testing frameworks and writing unit or integration tests
About Us:
Datum builds market intelligence solutions for the retail industry. We transform public and proprietary data into actionable insights that help retail brands decide where to expand, compete, and grow. We are a small, fast-moving team that works closely with customers and
believes in shipping impactful products quickly.
About the Role
We're looking for a Full Stack Platform Engineer to build and own our end-to-end product ecosystem, including:
● Data acquisition through scalable web scraping and ETL pipelines
● Customer-facing analytics platform
● Geospatial analysis engine powering retail insights
You'll work across frontend, backend, data engineering, cloud infrastructure, and geospatial systems while collaborating directly with the founder and customers.
Key Responsibilities
● Build scalable web scrapers and ETL pipelines
● Develop customer-facing features using React/Next.js
● Design REST & WebSocket APIs
● Work with PostgreSQL/PostGIS and geospatial data
● Own deployment, CI/CD, monitoring, and cloud infrastructure
● Translate customer feedback into product features
Must-Have Skills
● 3–6 years of Full Stack development experience
● Python (FastAPI/Django)
● React or Next.js with JavaScript/TypeScript
● Web scraping using Scrapy, Playwright, Selenium, or BeautifulSoup
● PostgreSQL (PostGIS preferred)
● REST APIs, JWT/OAuth
● Docker, Git, AWS/GCP/Azure
● Understanding of proxy rotation and anti-bot techniques
Good to Have
● Mapbox, Leaflet, or Google Maps Platform
● Airflow, Redis, Kafka, or SQS
● Experience with large-scale scraping (Google Maps, Zomato, Justdial, etc.)
● Geospatial analytics or retail domain experience
● Startup experience with end-to-end ownership
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
Apply directly
You can apply directly at careers.ritually.ai.
About Ritually
We're using on-device AI to build a set of novel tools that help large enterprise customers map and automate the rituals that run their back office operations. We'll be operating from great co-working space in central Bengaluru.
About you
We're looking for a few incredible people in Bangalore to join our founding team from day 0. If you're super high ownership, want an opportunity to build with us from scratch, and love the craft of making, please reach out to us!
About us
We're Ari Winkleman (New York) and Rachel Bush (Denver). We previously built Involvio and exited to Cisco where we worked building enterprise AI products. We're already funded and have design partners.
The Role
Build our core data pipeline and intelligence layer from 0-1 alongside the founders, tackling critical technical challenges across the stack.
What You'll Do
- Design pipelines capturing high-volume system activity into structured, queryable data.
- Sessionize event logs, cluster sequences, and leverage LLMs to label and summarize processes.
- Build synthetic data generation pipelines and evaluation frameworks to measure accuracy and prevent regressions.
- Own data quality, privacy, and end-to-end feature delivery with founding leadership.
What You're Looking For
- 1–4 years in data/AI engineering or backend development focused on data systems.
- Strong proficiency in Python/TypeScript, SQL, and reliable data pipeline construction.
- Expert experience with LLMs, prompting, structured output, and managing non-determinism.
- Rigorous approach to evaluation, and data privacy, and navigating open-ended technical problems.
- Expertise leveraging AI-native coding workflows and building tools that accelerate development.
Nice to Have
- Process mining, event-log analysis, or workflow analytics experience.
- Advanced PostgreSQL (window functions, pgvector) and cloud infrastructure experience.
- Early-stage startup experience or a CS degree.
Candidates can apply directly at careers.ritually.ai.
1st virtual , 2nd round F2F -
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
Location - Bangalore / Pune
Exp - 4 to 10 years
Required
· 3–8 years of relevant experience in data science, AI engineering, or applied machine learning roles.
· Bachelor’s or master’s degree in computer science, Mathematics, Engineering, or a related quantitative field.
· System design: ability to design scalable, reliable, and maintainable systems, including service decomposition, APIs, caching, queues and event-driven architectures, and the trade-offs between latency, throughput, cost, and consistency.
· Strong foundation in data structures and algorithms, with the ability to analyse time and space complexity and choose efficient approaches for large-scale data processing.
· Experience building and operating data-intensive or data-driven applications, such as large-scale data pipelines, batch and streaming processing, or applications whose core features depend on high-volume data.
· Strong hands-on experience in Python.
· Solid foundation in machine learning, NLP, and statistical modeling, including data exploration and visualization.
· Hands-on experience or projects involving LLMs, LangChain, LangGraph, RAG, or agentic AI systems.
· Experience delivering AI/ML solutions into production environments.
· Understanding of text processing and unstructured data.
· Knowledge of software engineering best practices (TDD, refactoring, modular design, version control).
· System design: ability to design scalable, reliable, and maintainable systems, including service decomposition, APIs, caching, queues and event-driven architectures, and the trade-offs between latency, throughput, cost, and consistency.
· Strong foundation in data structures and algorithms, with the ability to analyze time and space complexity and choose efficient approaches for large-scale data processing.
· Experience building and operating data-intensive or data-driven applications, such as large-scale data pipelines, batch and streaming processing, or applications whose core features depend on high-volume data.
· Strong analytical, problem-solving, and communication skills.
· Self-motivated, proactive, and delivery-oriented mindset.
Key Responsibilities
Strong understanding of Machine Learning algorithms (supervised and unsupervised)
Hands-on experience with Deep Learning frameworks (TensorFlow, PyTorch, or similar)
Experience in NLP techniques and libraries (NLTK, spaCy, Hugging Face, etc.)
Solid knowledge of Statistics, probability, and data analysis methods
Proficiency in SQL for querying relational databases
Strong programming skills in Python (or similar languages)
Excellent communication skills with the ability to explain complex concepts simply
Required Qualifications
3+ years of hands-on experience as a Data Scientist or in a similar role.
Strong expertise in classical machine learning and regression modeling.
Solid understanding of statistics, including probability, distributions, hypothesis testing,
and correlation analysis.
Proficiency in Python with libraries such as: scikit-learn pandas NumPy
Experience working with structured/tabular data.
Strong problem-solving and analytical thinking skills.
Ability to clearly explain models and results to non-technical stakeholders.
We are hiring experienced Data Engineers for a permanent opportunity in Bangalore.
Job Description
Role: Data Engineer
Location: Bangalore
Experience: 6+ Years
Key Responsibilities:
- Design, develop, and maintain scalable data processing pipelines using Amazon EMR.
- Develop data engineering solutions using Python and Scala.
- Process and transform large volumes of structured and unstructured data.
- Build and optimize distributed data processing workflows for performance and scalability.
- Work with big data technologies and cloud-based data processing environments.
- Perform data integration, validation, transformation, and quality checks.
- Troubleshoot data pipeline issues and optimize processing performance.
- Collaborate with data architects, analysts, and development teams to deliver reliable data solutions.
Required Skills:
- Strong hands-on experience with Amazon EMR.
- Proficiency in Python and Scala.
- Experience in big data processing and data pipeline development.
- Good understanding of distributed computing and data transformation techniques.
- Experience in performance tuning and troubleshooting data processing jobs.
- Strong analytical, problem-solving, and communication skills.
About the Company:
WebMD Health Corp., an Internet Brands Company, is the leading provider of health information services, serving patients, physicians, health care professionals, employers, and health plans through our public and private online portals, mobile platforms, and health-focused publications. The WebMD Health Network includes WebMD Health, Medscape, Jobson Healthcare Information, prIME Oncology, MediQuality, Frontline, QxMD, Vitals Consumer Services, MedicineNet, eMedicineHealth, RxList, OnHealth, Medscape Education, and other owned WebMD sites. WebMD®, Medscape®, CME Circle®, Medpulse®, eMedicine®, MedicineNet®, theheart.org®, and RxList® are among the trademarks of WebMD Health Corp. or its subsidiaries. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.
For Company details, visit our website: www.webmd.com
Experience: 3+ years
Qualification: Master's degree in Computer Science, Statistics, Mathematics, or a related field.
Work Timings: 2 PM to 11 PM IST
Position Requirement:
- Write clean and efficient code in Python and/or R for data analysis, statistical modeling, and machine learning tasks.
- Apply statistical and machine learning techniques to analyze large datasets and extract meaningful patterns, trends, and insights. Develop and implement predictive models for various business applications.
- Formulate hypothesis tests and design experiments to validate assumptions, measure the impact of changes, and optimize decision-making processes.
- Demonstrate strong SQL skills for efficient data querying and manipulation using window functions.
- Work with large datasets and ensure data quality and accuracy.
- Experience with dashboarding tools like Tableau is a plus to visualize and present data-driven insights to both technical and non-technical stakeholders.
- Ability to communicate complex technical concepts and insights in a clear and concise manner to both technical and non-technical audiences. Engage with the leadership team and clients to understand business requirements and provide data-driven solutions.
- Work with real world case studies in Data Science and a chance to implement various modelling techniques. Opportunity to independently execute and lead analytical projects and assignments.
- Help solve some challenging Healthcare related digital marketing problems globally. Transform business question into data requirements; collect and merge the data; analyse the data, link it to the business reality and present the results
- Develop predictive models and machine learning algorithms to study change in physician prescribing behaviour as well as WebMD integrated campaign response behaviour. Build analysis to understand user engagement and behaviours across various WebMD products.
Technical Skills needed:
1.Experience with data manipulation in SQL environment is a must. Experience with BigQuery/Vertica data warehousing is good to have
- Experience with statistical and data manipulation tools such as Python or R is a must
- Need very good expertise in using Microsoft Excel, and Microsoft PowerPoint. Excellent presentation skills required
- Experience developing statistical models like hypothesis testing, regression models, classifications models, forecasting etc is a must
- Algorithm designing and implementation skills in R or Python is required
Requirements
- B.Tech/B.E. / MSc Statistics from premier institutes with
- Minimum 80% marks in 10th and 12th
- Minimum 75% marks in graduation
- 2-5 years of relevant work experience in analytics field
- Proven experience in data science, statistical modeling, and machine learning applications.
- Strong proficiency in Python and/or R programming languages.
- Solid understanding of statistical concepts, hypothesis testing, and experimental design
- Expertise in SQL for data querying and manipulation.
- Familiarity with dashboarding tools is a plus.
- Excellent presentation, communication and interpersonal skills.
SHARE YOUR RESUME : { pavithra.e @vysystems.com },
{ 8 9 2 5 8 0 2 4 5 7 }
Job Description – Data / Metadata Engineer
Position
Data / Metadata Engineer
Location
Pune
Experience
- 6–9 Years: Up to 28 LPA
- 10–17 Years: Up to 32 LPA
Notice Period
Immediate to 15 Days Joiners Preferred
Role Overview
We are looking for an experienced Data / Metadata Engineer with strong expertise in Data Engineering, Metadata Management, Data Modeling, Cloud technologies, Python/PySpark, and ETL/Data Pipelines.
The ideal candidate should have hands-on experience designing and developing data solutions, working with metadata and data models, and building scalable data pipelines on cloud platforms.
Mandatory Skills
- Strong experience in Data Engineering
- Hands-on experience in Metadata Management / Metadata Engineering
- Strong understanding of Data Modeling
- Experience with DDL and ER/Data Modeling
- Experience with Cloud technologies – Any Cloud Platform
- Strong programming experience in Python or PySpark
- Experience with ETL processes and Data Pipelines
- Good understanding of data architecture and data integration concepts
Key Responsibilities
- Design, develop, and maintain scalable data engineering solutions.
- Develop and manage data pipelines and ETL workflows.
- Work with metadata and data cataloging concepts.
- Create and maintain data models, DDLs, ER diagrams, and database structures.
- Develop data processing solutions using Python or PySpark.
- Work with cloud-based data platforms and services.
- Collaborate with Data Architects, Data Engineers, Developers, and Business teams.
- Ensure data quality, consistency, availability, and reliability across data pipelines.
- Troubleshoot and optimize existing data pipelines and data processes.
- Follow best practices for data engineering, modeling, and cloud-based implementations.
Candidate Profile
- Strong experience in Data Engineering + Metadata + Data Modeling.
- Good hands-on experience with Python/PySpark and ETL/Data Pipelines.
- Experience working with any major cloud platform.
- Strong understanding of databases, data structures, and data modeling concepts.
- Candidates who can join immediately or within 15 days are preferred.
Job Details
Role: Data / Metadata Engineer
Location: Pune
Experience: 6–17 Years
Budget: 28 LPA (6–9 Yrs) / 32 LPA (10–17 Yrs)
Notice Period: Immediate to 15 Day










