Python Developer Web Scraping.

About Us:

Datum builds market intelligence solutions for the retail industry. We transform public and proprietary data into actionable insights that help retail brands decide where to expand, compete, and grow. We are a small, fast-moving team that works closely with customers and

believes in shipping impactful products quickly.

About the Role

We're looking for a Full Stack Platform Engineer to build and own our end-to-end product ecosystem, including:

● Data acquisition through scalable web scraping and ETL pipelines

● Customer-facing analytics platform

● Geospatial analysis engine powering retail insights

You'll work across frontend, backend, data engineering, cloud infrastructure, and geospatial systems while collaborating directly with the founder and customers.

Key Responsibilities

● Build scalable web scrapers and ETL pipelines

● Develop customer-facing features using React/Next.js

● Design REST & WebSocket APIs

● Work with PostgreSQL/PostGIS and geospatial data

● Own deployment, CI/CD, monitoring, and cloud infrastructure

● Translate customer feedback into product features

Must-Have Skills

● 3–6 years of Full Stack development experience

● Python (FastAPI/Django)

● React or Next.js with JavaScript/TypeScript

● Web scraping using Scrapy, Playwright, Selenium, or BeautifulSoup

● PostgreSQL (PostGIS preferred)

● REST APIs, JWT/OAuth

● Docker, Git, AWS/GCP/Azure

● Understanding of proxy rotation and anti-bot techniques

Good to Have

● Mapbox, Leaflet, or Google Maps Platform

● Airflow, Redis, Kafka, or SQS

● Experience with large-scale scraping (Google Maps, Zomato, Justdial, etc.)

● Geospatial analytics or retail domain experience

● Startup experience with end-to-end ownership

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

Job Summary

We are looking for a Software Engineer with 3–5 years of experience in Python development, browser automation, Azure cloud technologies, and API development. The ideal candidate should have hands-on experience in building automation solutions, cloud-native applications, and backend services using Python while collaborating with cross-functional teams to deliver scalable and reliable enterprise solutions.

Key Responsibilities

* Design, develop, test, and maintain Python-based applications and automation solutions.

* Develop browser automation scripts for enterprise web applications using Python.

* Create robust and dynamic element selectors to handle changing web interfaces.

* Implement exception handling, retry mechanisms, logging, and monitoring in automation solutions.

* Develop solutions to handle authentication workflows and CAPTCHA scenarios where permitted.

* Build and maintain RESTful APIs using FastAPI or similar Python frameworks.

* Develop serverless applications using Azure Functions and integrate with Azure services.

* Design and implement integrations using Azure Logic Apps, Service Bus, Storage Accounts, and Key Vault.

* Deploy and manage applications using Azure Container Registry (ACR), Azure Container Apps, and Azure Container Instances (ACI).

* Configure and manage KEDA for event-driven application scaling.

* Develop and maintain frontend applications using React.js, Node.js, or similar modern JavaScript frameworks.

* Design and optimize database solutions using SQL Server, PostgreSQL, or Azure Cosmos DB.

* Troubleshoot production issues and implement permanent fixes.

* Participate in code reviews, testing, deployment, and release activities.

* Follow coding standards, security best practices, and application performance guidelines.

Required Skills

Python Development

* Strong experience in Python programming.

* Experience developing scalable backend applications and automation solutions.

* Experience with asynchronous programming and API integrations.

Browser Automation

* Hands-on experience in browser automation using Python libraries such as Playwright or Selenium.

* Experience creating dynamic selectors and handling dynamic web elements.

* Strong understanding of synchronization techniques, waits, retries, and exception handling.

* Experience implementing logging, monitoring, and debugging for automation solutions.

* Familiarity with CAPTCHA handling approaches and authentication workflows.

Azure Cloud

* Hands-on experience with Azure Functions.

* Experience with Azure Logic Apps.

* Experience using Azure Storage Accounts.

* Experience with Azure Service Bus.

* Experience with Azure Key Vault.

* Understanding of cloud-native application design and Azure security best practices.

Containerization & Deployment

* Experience with Azure Container Registry (ACR).

* Experience deploying applications to Azure Container Apps.

* Experience with Azure Container Instances (ACI).

* Experience configuring and managing KEDA for event-driven autoscaling.

* Familiarity with container-based deployments and application lifecycle management.

### API Development

* Experience building REST APIs using FastAPI or similar Python frameworks.

* Strong understanding of REST principles, JSON, authentication, and API integrations.

Databases

* Experience with SQL Server and/or PostgreSQL.

* Experience with Azure Cosmos DB.

* Ability to write efficient SQL queries and optimize database performance.

Frontend Development

* Experience with React.js, Node.js, or similar frontend technologies.

* Understanding of modern web application architecture and API consumption.

Additional Skills

* Experience with Git and version control.

* Strong debugging and analytical skills.

* Good communication and teamwork skills.

Preferred Skills

* Experience with OCR and document processing solutions.

* Experience with Azure AI or AI-based services.

* Experience with event-driven architectures.

* Experience working in Agile/Scrum environments.

* Experience with automation platforms such as UiPath is an added advantage.

Education

Bachelor's degree in Computer Science, Information Technology, Software Engineering, or a related field.

1st virtual , 2nd round F2F -

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data processing solutions using Python.
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain data ingestion and integration workflows.
• Implement data quality, validation, monitoring, and error-handling processes.
• Develop and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
• Collaborate with data analysts, data scientists, software engineers, and business teams.
• Optimize data pipelines for performance, reliability, and scalability.
• Troubleshoot production data issues and ensure timely resolution.
• Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

• Python
• ETL
• SQL
• CI/CD
• DevOps
• Git / Version Control
• Strong problem-solving and debugging skills

Data Engineer Hiring Post

🚨 Hiring: Data Engineer | PySpark + Python + SQL

We are looking for experienced Data Engineers to join our team!

🔹 Experience: 5 to 9 Years

🔹 Locations: Bangalore / Hyderabad

🔹 Interview Process:

• 1st Round – Virtual

• 2nd Round – Face-to-Face (Karat Test)

🔑 Key Skills:

✅ PySpark

✅ SQL

✅ Python

✅ ETL

📩 Interested candidates can share their updated resume.

#Hiring #DataEngineer #PySpark #Python #SQL #ETL #BangaloreJobs #HyderabadJobs #TechHiring #ImmediateHiring

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, Hadoop, and Apache Spark.
• Develop efficient data processing and transformation workflows for large datasets.
• Write, optimize, and maintain complex SQL queries and PL/SQL procedures.
• Perform data extraction, transformation, and loading (ETL) from multiple data sources.
• Work with structured and unstructured data to support analytics and reporting requirements.
• Optimize data pipelines and queries for performance, scalability, and reliability.
• Ensure data quality, consistency, security, and integrity across data platforms.
• Collaborate with cross-functional teams to understand business requirements and deliver data engineering solutions.
• Use Git, automation, and CI/CD practices to support development and deployment.

Required Skills

• 4.5–10 years of experience in software development or data engineering.
• Strong proficiency in AWS
• Hands-on experience with Hadoop and Apache Spark.
• Good knowledge of SQL and PL/SQL, including query optimization and stored procedures.
• Experience in ETL development, data processing, and data pipeline design.
• Understanding of data warehousing concepts and distributed computing.
• Strong problem-solving, debugging, and performance-tuning skills.

About LH2 AI LabsLH2 AI Labs is an applied research lab solving data platform and curation challenges for foundation model development. We serve every frontier AI lab with the mission of delivering the best data to power the best models.

Our customers are the ones building the foundation models themselves and our work sits directly in the loop of how those systems improve. This is a rare opportunity to join a company at a defining moment in AI.

About the role:

This is a full-time, hands-on role where you will own the core infrastructure and systems that enable us to discover customers' data landscape, handle sensitive data like names, address securely and to clean and catalog them into training-ready datasets.

That path covers connectors and on-prem agentic discovery components, secrets and PII scrubbing, scale and cost efficient ingestion and clearance for use in model training. You lead a small team of 3-4 engineers and raise the quality bar in your pod.

What you'll own

• Connectors and extraction - You'll build connectors for common SaaS and databases (Google Workspace, Slack, Postgres/MySQL, Jira, MongoDB) and for specific tools such as Razorpay, GreytHR, Keka, Zoho and LeadSquared. The rule is to build only where no good open-source connector or clean export exists.
• On-prem scanning - You'll build a CLI or agent that runs at the data owner's end to sample and estimate the value of their data without shipping all of it to us.
• Clearance pipeline - Secrets detection across full git history, fail-closed. PII redaction for code and conversational data, including Indian identifiers (PAN, Aadhaar, GSTIN, IFSC, UPI). Measurable recall and precision.
• Lineage - Every output record must trace back to its raw source, and every lot must be revocable.
• Delivery - Packaging, sampling for buyers, and supporting the supply and BD teams on data questions.

You have

• 8+ years in backend or data engineering, with at least 2 years leading up to 4 or more engineers.
• You've built ingestion or ETL pipelines that run in production.
• Hands-on work with sensitive data such as PII, financial or health data, including redaction, masking or access controls.
• The judgment to decide what to build and what to adopt.

Nice to have

• Experience with Presidio, gitleaks/TruffleHog, dlt or Airbyte.
• Knowledge of DPDP, GDPR, HIPAA or SOC 2.
• You've shipped software that runs in customers' environments.

This is not a pure people-management role. You'll write code, review everything, and personally own the hardest problems in the pod.

We are hiring experienced Data Engineers for a permanent opportunity in Bangalore.

Job Description

Role: Data Engineer

Location: Bangalore

Experience: 6+ Years

Key Responsibilities:

• Design, develop, and maintain scalable data processing pipelines using Amazon EMR.
• Develop data engineering solutions using Python and Scala.
• Process and transform large volumes of structured and unstructured data.
• Build and optimize distributed data processing workflows for performance and scalability.
• Work with big data technologies and cloud-based data processing environments.
• Perform data integration, validation, transformation, and quality checks.
• Troubleshoot data pipeline issues and optimize processing performance.
• Collaborate with data architects, analysts, and development teams to deliver reliable data solutions.

Required Skills:

• Strong hands-on experience with Amazon EMR.
• Proficiency in Python and Scala.
• Experience in big data processing and data pipeline development.
• Good understanding of distributed computing and data transformation techniques.
• Experience in performance tuning and troubleshooting data processing jobs.
• Strong analytical, problem-solving, and communication skills.