Python Developer - web scraping

About Us:

Datum builds market intelligence solutions for the retail industry. We transform public and proprietary data into actionable insights that help retail brands decide where to expand, compete, and grow. We are a small, fast-moving team that works closely with customers and

believes in shipping impactful products quickly.

About the Role

We're looking for a Full Stack Platform Engineer to build and own our end-to-end product ecosystem, including:

● Data acquisition through scalable web scraping and ETL pipelines

● Customer-facing analytics platform

● Geospatial analysis engine powering retail insights

You'll work across frontend, backend, data engineering, cloud infrastructure, and geospatial systems while collaborating directly with the founder and customers.

Key Responsibilities

● Build scalable web scrapers and ETL pipelines

● Develop customer-facing features using React/Next.js

● Design REST & WebSocket APIs

● Work with PostgreSQL/PostGIS and geospatial data

● Own deployment, CI/CD, monitoring, and cloud infrastructure

● Translate customer feedback into product features

Must-Have Skills

● 3–6 years of Full Stack development experience

● Python (FastAPI/Django)

● React or Next.js with JavaScript/TypeScript

● Web scraping using Scrapy, Playwright, Selenium, or BeautifulSoup

● PostgreSQL (PostGIS preferred)

● REST APIs, JWT/OAuth

● Docker, Git, AWS/GCP/Azure

● Understanding of proxy rotation and anti-bot techniques

Good to Have

● Mapbox, Leaflet, or Google Maps Platform

● Airflow, Redis, Kafka, or SQS

● Experience with large-scale scraping (Google Maps, Zomato, Justdial, etc.)

● Geospatial analytics or retail domain experience

● Startup experience with end-to-end ownership

role: data engineer

Python pyspark, SQL, data engineer

5+yrs

Bangalore/Hyderabad

immediate to 15days.

Strong Data Engineer (AWS) Profile

2

Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.

3

Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.

4

Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.

5

Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.

6

Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.

7

Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.

8

Mandatory (Education) – B.Tech / B.E./ M.Tech

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data processing solutions using Python.
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain data ingestion and integration workflows.
• Implement data quality, validation, monitoring, and error-handling processes.
• Develop and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
• Collaborate with data analysts, data scientists, software engineers, and business teams.
• Optimize data pipelines for performance, reliability, and scalability.
• Troubleshoot production data issues and ensure timely resolution.
• Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

• Python
• ETL
• SQL
• CI/CD
• DevOps
• Git / Version Control
• Strong problem-solving and debugging skills

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

9+yrs

Bangalore/Hyderabad

immediate to 15days.

1st virtual , 2nd round F2F -

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

We are looking for an experienced Software Engineer to join an AI engineering startup developing a document collection platform for accountants and professional services firms. 

Preference to candidates from Kerala, India.

The product eliminates the friction involved in gathering client files by automating document requests, centralising their collection, and organising incoming documents according to each organisation’s preferred folder structure.

The ideal candidate will be able to take ownership of work from start to finish, communicate clearly, and deliver high-quality solutions within tight timeframes.

What You’ll Work On

You’ll work with Python and Django daily, including models, views, templates, background jobs, and the wider product around them.

The frontend uses Django templates with HTMX and Alpine.js, built with Vite, TypeScript, and Tailwind CSS. The stack runs in Docker using PostgreSQL, Redis, RabbitMQ, and Celery.

You may also assist with ancillary projects, including custom integrations.

Project-based training will be provided.

Technology Stack

Backend: Python, Django, PostgreSQL, Celery, Redis and RabbitMQ

Frontend: Django Templates, HTMX, Alpine.js, Vite, TypeScript and Tailwind CSS

Infrastructure: Docker

Must Have

• Strong Python and Django skills
• Comfortable working with Docker
• Fluent written and spoken English
• Clear communication skills, including providing concise updates, asking honest questions, and writing information that others can act on
• Evidence of exceptional ability—not simply a list of tools, but something challenging you have built or solved

Preference will be given to candidates with at least three years of relevant professional experience.

Nice to Have

• Frontend experience with HTML, CSS and JavaScript
• Experience with HTMX, Alpine.js, TypeScript or Tailwind CSS
• Knowledge of PostgreSQL, Celery or pytest
• Experience with integrations, including APIs, OAuth and cloud storage
• Basic accounting knowledge

How to Apply

Please do not send a generic CV alone. Your application must include:

1. Evidence of exceptional ability: Describe a project, open-source contribution, production system or challenging problem you solved. Include a link to the repository, write-up or demo where possible. Focus on your personal contribution by detailing the specific parts of the project where you played a critical role and explaining precisely what you built or solved.
2. What you accomplished: Provide a short explanation of the outcome in your own words.
3. The hardest part: Explain the hardest part of the problem and how you dealt with it.
4. Your use of AI: Explain whether you use AI in your work and, if so, how you use it.
5. Your professional experience and interests: Include a brief paragraph summarising your professional experience and general interests.

Applications that do not include the above Croissant details above will not be considered.

What We Offer

• For the right candidate, salary will not be a constraint
• Project-based training
• A rewarding career with genuine opportunities for professional growth
• The opportunity to work on an innovative AI-driven product
• A remote, full-time position
• 
  

Job Details and Application Submission

Location: Remote

Employment Type: Full-time

Contract: One-year contract, with the possibility of extension based on satisfactory performance

Probationary Period: Six months

Preferred Experience: Three or more years

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

About LH2 AI LabsLH2 AI Labs is an applied research lab solving data platform and curation challenges for foundation model development. We serve every frontier AI lab with the mission of delivering the best data to power the best models.

Our customers are the ones building the foundation models themselves and our work sits directly in the loop of how those systems improve. This is a rare opportunity to join a company at a defining moment in AI.

About the role:

This is a full-time, hands-on role where you will own the core infrastructure and systems that enable us to discover customers' data landscape, handle sensitive data like names, address securely and to clean and catalog them into training-ready datasets.

That path covers connectors and on-prem agentic discovery components, secrets and PII scrubbing, scale and cost efficient ingestion and clearance for use in model training. You lead a small team of 3-4 engineers and raise the quality bar in your pod.

What you'll own

• Connectors and extraction - You'll build connectors for common SaaS and databases (Google Workspace, Slack, Postgres/MySQL, Jira, MongoDB) and for specific tools such as Razorpay, GreytHR, Keka, Zoho and LeadSquared. The rule is to build only where no good open-source connector or clean export exists.
• On-prem scanning - You'll build a CLI or agent that runs at the data owner's end to sample and estimate the value of their data without shipping all of it to us.
• Clearance pipeline - Secrets detection across full git history, fail-closed. PII redaction for code and conversational data, including Indian identifiers (PAN, Aadhaar, GSTIN, IFSC, UPI). Measurable recall and precision.
• Lineage - Every output record must trace back to its raw source, and every lot must be revocable.
• Delivery - Packaging, sampling for buyers, and supporting the supply and BD teams on data questions.

You have

• 8+ years in backend or data engineering, with at least 2 years leading up to 4 or more engineers.
• You've built ingestion or ETL pipelines that run in production.
• Hands-on work with sensitive data such as PII, financial or health data, including redaction, masking or access controls.
• The judgment to decide what to build and what to adopt.

Nice to have

• Experience with Presidio, gitleaks/TruffleHog, dlt or Airbyte.
• Knowledge of DPDP, GDPR, HIPAA or SOC 2.
• You've shipped software that runs in customers' environments.

This is not a pure people-management role. You'll write code, review everything, and personally own the hardest problems in the pod.