Python Developer Web Scraping.

About Us:

Datum builds market intelligence solutions for the retail industry. We transform public and proprietary data into actionable insights that help retail brands decide where to expand, compete, and grow. We are a small, fast-moving team that works closely with customers and

believes in shipping impactful products quickly.

About the Role

We're looking for a Full Stack Platform Engineer to build and own our end-to-end product ecosystem, including:

● Data acquisition through scalable web scraping and ETL pipelines

● Customer-facing analytics platform

● Geospatial analysis engine powering retail insights

You'll work across frontend, backend, data engineering, cloud infrastructure, and geospatial systems while collaborating directly with the founder and customers.

Key Responsibilities

● Build scalable web scrapers and ETL pipelines

● Develop customer-facing features using React/Next.js

● Design REST & WebSocket APIs

● Work with PostgreSQL/PostGIS and geospatial data

● Own deployment, CI/CD, monitoring, and cloud infrastructure

● Translate customer feedback into product features

Must-Have Skills

● 3–6 years of Full Stack development experience

● Python (FastAPI/Django)

● React or Next.js with JavaScript/TypeScript

● Web scraping using Scrapy, Playwright, Selenium, or BeautifulSoup

● PostgreSQL (PostGIS preferred)

● REST APIs, JWT/OAuth

● Docker, Git, AWS/GCP/Azure

● Understanding of proxy rotation and anti-bot techniques

Good to Have

● Mapbox, Leaflet, or Google Maps Platform

● Airflow, Redis, Kafka, or SQS

● Experience with large-scale scraping (Google Maps, Zomato, Justdial, etc.)

● Geospatial analytics or retail domain experience

● Startup experience with end-to-end ownership

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

1st virtual , 2nd round F2F -

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data processing solutions using Python.
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain data ingestion and integration workflows.
• Implement data quality, validation, monitoring, and error-handling processes.
• Develop and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
• Collaborate with data analysts, data scientists, software engineers, and business teams.
• Optimize data pipelines for performance, reliability, and scalability.
• Troubleshoot production data issues and ensure timely resolution.
• Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

• Python
• ETL
• SQL
• CI/CD
• DevOps
• Git / Version Control
• Strong problem-solving and debugging skills

We are looking for an experienced Software Engineer to join an AI engineering startup developing a document collection platform for accountants and professional services firms. 

Preference to candidates from Kerala, India.

The product eliminates the friction involved in gathering client files by automating document requests, centralising their collection, and organising incoming documents according to each organisation’s preferred folder structure.

The ideal candidate will be able to take ownership of work from start to finish, communicate clearly, and deliver high-quality solutions within tight timeframes.

What You’ll Work On

You’ll work with Python and Django daily, including models, views, templates, background jobs, and the wider product around them.

The frontend uses Django templates with HTMX and Alpine.js, built with Vite, TypeScript, and Tailwind CSS. The stack runs in Docker using PostgreSQL, Redis, RabbitMQ, and Celery.

You may also assist with ancillary projects, including custom integrations.

Project-based training will be provided.

Technology Stack

Backend: Python, Django, PostgreSQL, Celery, Redis and RabbitMQ

Frontend: Django Templates, HTMX, Alpine.js, Vite, TypeScript and Tailwind CSS

Infrastructure: Docker

Must Have

• Strong Python and Django skills
• Comfortable working with Docker
• Fluent written and spoken English
• Clear communication skills, including providing concise updates, asking honest questions, and writing information that others can act on
• Evidence of exceptional ability—not simply a list of tools, but something challenging you have built or solved

Preference will be given to candidates with at least three years of relevant professional experience.

Nice to Have

• Frontend experience with HTML, CSS and JavaScript
• Experience with HTMX, Alpine.js, TypeScript or Tailwind CSS
• Knowledge of PostgreSQL, Celery or pytest
• Experience with integrations, including APIs, OAuth and cloud storage
• Basic accounting knowledge

How to Apply

Please do not send a generic CV alone. Your application must include:

1. Evidence of exceptional ability: Describe a project, open-source contribution, production system or challenging problem you solved. Include a link to the repository, write-up or demo where possible. Focus on your personal contribution by detailing the specific parts of the project where you played a critical role and explaining precisely what you built or solved.
2. What you accomplished: Provide a short explanation of the outcome in your own words.
3. The hardest part: Explain the hardest part of the problem and how you dealt with it.
4. Your use of AI: Explain whether you use AI in your work and, if so, how you use it.
5. Your professional experience and interests: Include a brief paragraph summarising your professional experience and general interests.

Applications that do not include the above Croissant details above will not be considered.

What We Offer

• For the right candidate, salary will not be a constraint
• Project-based training
• A rewarding career with genuine opportunities for professional growth
• The opportunity to work on an innovative AI-driven product
• A remote, full-time position
• 
  

Job Details and Application Submission

Location: Remote

Employment Type: Full-time

Contract: One-year contract, with the possibility of extension based on satisfactory performance

Probationary Period: Six months

Preferred Experience: Three or more years

Data Engineer Hiring Post

🚨 Hiring: Data Engineer | PySpark + Python + SQL

We are looking for experienced Data Engineers to join our team!

🔹 Experience: 5 to 9 Years

🔹 Locations: Bangalore / Hyderabad

🔹 Interview Process:

• 1st Round – Virtual

• 2nd Round – Face-to-Face (Karat Test)

🔑 Key Skills:

✅ PySpark

✅ SQL

✅ Python

✅ ETL

📩 Interested candidates can share their updated resume.

#Hiring #DataEngineer #PySpark #Python #SQL #ETL #BangaloreJobs #HyderabadJobs #TechHiring #ImmediateHiring

Job Summary

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate should have strong expertise in SQL, Python, Linux, and modern data engineering practices to support data integration, transformation, and analytics.

Key Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines.
• Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
• Develop automation scripts using Python for data processing and workflow optimization.
• Work with Linux environments for deployment, monitoring, and troubleshooting.
• Ensure data quality, integrity, and reliability across data platforms.
• Collaborate with data analysts, software engineers, and business stakeholders to deliver data solutions.
• Monitor, troubleshoot, and optimize data pipelines for performance and scalability.
• Implement best practices for data security, governance, and documentation.

Required Skills

• Strong experience in Data Engineering concepts and ETL/ELT processes.
• Proficiency in SQL, including query optimization and database design.
• Strong programming skills in Python.
• Hands-on experience with Linux commands, shell scripting, and system administration basics.
• Experience with relational databases such as PostgreSQL, MySQL, SQL Server, or Oracle.
• Familiarity with Git/version control.
• Strong analytical and problem-solving skills.

Preferred Skills

• Experience with cloud platforms (AWS, Azure, or GCP).
• Knowledge of Apache Spark, Airflow, Kafka, or similar data engineering tools.
• Experience with data warehousing solutions and big data technologies.
• Understanding of CI/CD pipelines and containerization (Docker/Kubernetes).

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Relevant certifications in cloud or data engineering are an added advantage.

About LH2 AI LabsLH2 AI Labs is an applied research lab solving data platform and curation challenges for foundation model development. We serve every frontier AI lab with the mission of delivering the best data to power the best models.

Our customers are the ones building the foundation models themselves and our work sits directly in the loop of how those systems improve. This is a rare opportunity to join a company at a defining moment in AI.

About the role:

This is a full-time, hands-on role where you will own the core infrastructure and systems that enable us to discover customers' data landscape, handle sensitive data like names, address securely and to clean and catalog them into training-ready datasets.

That path covers connectors and on-prem agentic discovery components, secrets and PII scrubbing, scale and cost efficient ingestion and clearance for use in model training. You lead a small team of 3-4 engineers and raise the quality bar in your pod.

What you'll own

• Connectors and extraction - You'll build connectors for common SaaS and databases (Google Workspace, Slack, Postgres/MySQL, Jira, MongoDB) and for specific tools such as Razorpay, GreytHR, Keka, Zoho and LeadSquared. The rule is to build only where no good open-source connector or clean export exists.
• On-prem scanning - You'll build a CLI or agent that runs at the data owner's end to sample and estimate the value of their data without shipping all of it to us.
• Clearance pipeline - Secrets detection across full git history, fail-closed. PII redaction for code and conversational data, including Indian identifiers (PAN, Aadhaar, GSTIN, IFSC, UPI). Measurable recall and precision.
• Lineage - Every output record must trace back to its raw source, and every lot must be revocable.
• Delivery - Packaging, sampling for buyers, and supporting the supply and BD teams on data questions.

You have

• 8+ years in backend or data engineering, with at least 2 years leading up to 4 or more engineers.
• You've built ingestion or ETL pipelines that run in production.
• Hands-on work with sensitive data such as PII, financial or health data, including redaction, masking or access controls.
• The judgment to decide what to build and what to adopt.

Nice to have

• Experience with Presidio, gitleaks/TruffleHog, dlt or Airbyte.
• Knowledge of DPDP, GDPR, HIPAA or SOC 2.
• You've shipped software that runs in customers' environments.

This is not a pure people-management role. You'll write code, review everything, and personally own the hardest problems in the pod.

The Opportunity 

We are building the next generation of engineers at AvancerPI. You will work on real enterprise products and automation rather than isolated training assignments. Depending on your strengths, you may contribute to frontend, backend, APIs, automation, integrations, cloud-native engineering or AI-assisted capabilities. 

What You’ll Work On 

• Modern web applications and customer-facing product experiences. 
• Backend services and REST APIs. 
• Python-based automation and platform integrations. 
• Databases, data models and application state. 
• Terraform and Ansible based infrastructure automation. 
• Containers and Kubernetes-based applications. 
• Integration with enterprise infrastructure and cloud APIs. 
• AI/LLM, RAG and agentic workflows where they solve a real product or operational problem. 
• Testing, debugging, Git workflows and CI/CD. 
• Product features used by enterprise customers and internal engineering teams. 

What We’re Looking For 

• BE/BTech/MCA or equivalent technical education. 
• Strong programming fundamentals and problem-solving ability. 
• Working knowledge of at least one language such as Python, JavaScript/TypeScript, Java or similar. 
• Basic understanding of APIs, databases, Git and Linux. 
• Curiosity about cloud, infrastructure, automation, Kubernetes and AI. 
• Ability to learn unfamiliar technologies independently and ask good technical questions. 
• Evidence that you build things: academic projects, GitHub work, internships, hackathons or personal projects. 

What Matters More Than a Long Skills List 

Fundamentals + curiosity + learning speed + ownership + ability to turn an idea into working software. 

What You Can Expect to Learn 

• How enterprise products are designed, built, tested and released. 
• How software interacts with infrastructure, cloud and Kubernetes platforms. 
• How to write maintainable code and participate in design/code reviews. 
• How customer problems become product requirements and engineering decisions. 

For more information about the company, you can visit our website: https://avancerpi.com/about/