STAFF SOFTWARE ENGINEER

Django + Scraper Developer – Job Description

Position: Django + Scraper Developer

Job Summary

We are looking for a skilled Django + Web Scraping Developer with 3+ years of experience in developing scalable web applications using Django and building robust web scrapers. The ideal candidate should have strong Python knowledge, experience with scraping frameworks and APIs, and the ability to work with databases and third-party integrations.

Key Responsibilities

• Develop, maintain, and enhance web applications using Python and Django.
• Design and implement reliable web scraping solutions for extracting structured data from websites.
• Develop scrapers using tools such as Scrapy, Selenium, BeautifulSoup, Requests, or similar technologies.
• Handle dynamic websites, pagination, authentication, proxies, CAPTCHA-related challenges, and anti-bot mechanisms where legally and technically appropriate.
• Clean, validate, transform, and store scraped data in databases.
• Develop and integrate REST APIs and third-party APIs.
• Work with databases such as MySQL, PostgreSQL, or MongoDB.
• Optimize scraper performance, reliability, and data accuracy.
• Troubleshoot and fix issues related to scraping, Django applications, APIs, and databases.
• Write clean, reusable, and maintainable Python code.
• Collaborate with developers, project managers, and other stakeholders to understand project requirements.
• Perform testing, debugging, and performance optimization.

Required Skills

• 3+ years of professional experience in Python/Django development.
• Strong knowledge of Python and Django.
• Hands-on experience with Web Scraping / Data Extraction.
• Experience with Scrapy, Selenium, BeautifulSoup, Requests, or equivalent tools.
• Good understanding of HTML, CSS, JavaScript, DOM, and HTTP/HTTPS.
• Experience working with REST APIs and JSON.
• Good knowledge of SQL and databases such as MySQL/PostgreSQL.
• Understanding of Git/version control.
• Strong debugging and problem-solving skills.

Good to Have

• Experience with Celery, Redis, Docker, and Linux.
• Knowledge of cloud platforms such as AWS.
• Experience handling large-scale scraping projects.
• Understanding of proxy rotation, browser automation, and anti-bot techniques.
• Experience with data pipelines and ETL processes.

What We Offer

• Opportunity to work on challenging and real-world projects.
• Collaborative and growth-oriented work environment.
• Learning and professional development opportunities.
• Competitive salary based on skills and experience.

How to Apply

Interested candidates can share their updated resume along with their current salary, expected salary, notice period, and total experience.

Build backend systems that make complex workflows reliable, maintainable, and easier to evolve. You’ll work on production Python services, modernize existing applications, and deliver APIs and integrations used by real teams.

The role is focused on backend development using Python, Django, and FastAPI, with occasional contributions to frontend applications built with React or Next.js. You will work across API development, system integrations, data workflows, background processing, and the modernization of existing services.

You will help improve reliability and operational maturity using AWS, practical Site Reliability Engineering principles, and observability platforms such as Datadog, Grafana, CloudWatch, Prometheus, or similar tools.

This is a backend-first role for an engineer who understands that testing, security, reliability, and observability are core parts of production software development.

What you'll do

• Own the delivery and day-to-day reliability of clearly scoped backend features and APIs built with Python, Django, and FastAPI.
• Improve the maintainability of existing services by simplifying legacy workflows, business logic, data models, and integrations.
• Deliver secure, well-documented APIs that enable frontend applications, internal tools, and third-party integrations.
• Keep data workflows reliable through safe schema changes, migrations, backfills, validation, and failure recovery.
• Build background jobs and asynchronous workflows that handle retries, timeouts, rate limits, idempotency, and operational failures.
• Raise confidence in releases through automated tests, thoughtful code reviews, and production-readiness checks.
• Improve performance and reliability using profiling, query optimization, caching, monitoring, and practical fault handling.
• Make services easier to operate by maintaining useful logs, metrics, dashboards, alerts, and runbooks.
• Contribute to React or Next.js integrations and internal administration interfaces when needed to deliver complete product outcomes.
• Collaborate with product, frontend, platform, DevOps, SRE, and operations teammates to ship well-scoped solutions and document technical decisions.

What we're looking for

Required

• 3–5 years of professional software engineering experience, mainly in backend development.
• Good proficiency in Python and production experience with either Django or FastAPI, with willingness to learn the other.
• Practical experience designing REST APIs and integrating with third-party or internal services.
• Working knowledge of SQL, relational databases, an ORM such as Django ORM or SQLAlchemy, and database migrations.
• Experience writing automated tests and diagnosing defects or production issues.
• Good problem-solving, communication, documentation, and collaboration skills.

Preferred

• Familiarity with background jobs or asynchronous processing using Celery, SQS, RabbitMQ, Kafka, or similar tools.
• Experience deploying or supporting applications on AWS or another cloud platform.
• Familiarity with structured logging, metrics, dashboards, alerts, and observability tools.
• Exposure to React or Next.js integrations.
• Experience with Docker, Redis, webhooks, queues, or event-driven systems.
• Familiarity with CI/CD pipelines, infrastructure-as-code, secrets management, and least-privilege access.
• Experience helping with production incidents, performance testing, or safe deployment practices.

We’re hiring a remote, contract-based Backend & Infrastructure Engineer who can build and run production systems end-to-end.

You will build and scale high-throughput backend services in Golang and Python, operate ClickHouse-powered analytics at scale, manage Linux servers for maximum uptime, scalability, and reliability, and drive cost efficiency as a core engineering discipline across the entire stack.

What You Will Do:

Backend Development (Golang & Python)

• Design and maintain high-throughput RESTful/gRPC APIs — primarily Golang, Python for tooling and supporting services
• Architect for horizontal scalability, fault tolerance, and low-latency at scale
• Implement caching (Redis/Memcached), rate limiting, efficient serialization, and CI/CD pipelines

Scalable Architecture & System Design

• Design and evolve distributed, resilient backend architecture that scales without proportional cost increase
• Make deliberate trade-offs (CAP, cost vs. performance) and design multi-region HA with automated failover

ClickHouse & Analytical Data Infrastructure

• Deploy, tune, and operate ClickHouse clusters for real-time analytics and high-cardinality OLAP workloads
• Design optimal table engines, partition strategies, materialized views, and query patterns
• Manage cluster scaling, replication, schema migrations, and upstream/downstream integrations

Cost Efficiency & Cost Optimization

• Own cost optimization end-to-end: right-sizing, reserved/spot capacity, storage tiering, query optimization, compression, batching
• Build cost dashboards, budgets, and alerts; drive a culture of cost-aware engineering

Linux Server Management & Infrastructure

• Administer and harden Linux servers (Ubuntu, Debian, CentOS/RHEL) — patching, security, SSH, firewalls
• Manage VPS/bare-metal provisioning, capacity planning, and containerized workloads (Docker, Kubernetes/Nomad)
• Implement Infrastructure-as-Code (Terraform/Pulumi); optionally manage AWS/GCP as needed

Data, Storage & Scheduling

• Optimize SQL schemas and queries (PostgreSQL, MySQL); manage data archival, cold storage, and lifecycle policies
• Build and maintain cron jobs, scheduled tasks, and batch processing systems

Uptime, Reliability & Observability

• Own system uptime: zero-downtime deployments, health checks, self-healing infra, SLOs/SLIs
• Build observability stacks (Prometheus, Grafana, Datadog, OpenTelemetry); structured logging, distributed tracing, alerting
• Drive incident response, root cause analysis, and post-mortems

Required Qualifications:

Must-Have (Critical)

• Deep proficiency in Golang (primary) and Python
• Proven ability to design and build scalable, distributed architectures
• Production experience deploying and operating ClickHouse at scale
• Track record of driving measurable cost efficiency and cost optimization
• 5+ years in backend engineering and infrastructure roles

Also Required

• Strong Linux server administration (Ubuntu, Debian, CentOS/RHEL) — comfortable living in the terminal
• Proven uptime and reliability track record across production infrastructure
• Strong SQL (PostgreSQL, MySQL); experience with high-throughput APIs (10K+ RPS)
• VPS/bare-metal provisioning, Docker, Kubernetes/Nomad, IaC (Terraform/Pulumi)
• Observability tooling (Prometheus, Grafana, Datadog, OpenTelemetry)
• Cron jobs, batch processing, data archival, cold storage management
• Networking fundamentals (DNS, TCP/IP, load balancing, TLS)

Nice to Have

• AWS, GCP, or other major cloud provider experience
• Message queues / event streaming (Kafka, RabbitMQ, SQS/SNS)
• Data pipelines (Airflow, dbt); FinOps practices
• Open-source contributions; compliance background (SOC 2, HIPAA, GDPR)

What We Offer

• Remote, contractual role
• Flexible time zones (overlap for standups + incident coverage)
• Competitive contract compensation + equity
• Long-term engagement opportunity based on performance

Job Title: Senior Linux Kernel Engineer

Experience: 5–10 Years

Location: Bangalore / Chennai

Domain: Enterprise Linux / Kernel Development

Job Summary

We are seeking a highly skilled Senior Linux Kernel Engineer with deep expertise in kernel development, debugging, and performance optimization. The role involves working on enterprise-grade Linux distributions, kernel lifecycle management, security patching, and low-level hardware integration.

Key Responsibilities

1. Kernel Lifecycle & Maintenance

• Lead kernel upgrade strategies (e.g., LTS migrations such as 5.15 → 6.x) while ensuring stability and compatibility.
• Perform patch porting across kernel versions, resolving API and dependency conflicts.
• Track and mitigate security vulnerabilities by monitoring CVEs and upstream sources (e.g., LKML).
• Backport critical fixes to production kernels without impacting system stability.

2. Debugging & System Stability

• Act as an escalation point for kernel panics and system crashes.
• Perform post-mortem analysis using kdump, crash, and gdb.
• Debug early boot issues (UEFI, initramfs, kernel initialization).
• Conduct performance analysis using eBPF, ftrace, and perf to optimize system behavior.

3. Driver Development & Hardware Integration

• Design, develop, and maintain device drivers (network, storage, GPU, or character devices).
• Work closely with hardware through DMA, interrupts (MSI-X), and register-level programming.
• Maintain out-of-tree drivers using DKMS or similar frameworks.
• Ensure compatibility of drivers across kernel updates.

Required Technical Skills

• Programming: Strong expertise in C (mandatory) and C++
• Kernel Internals: Deep understanding of:
• Virtual File System (VFS)
• Memory Management (MMU, Paging)
• Process Scheduler
• Linux Networking Stack
• Debugging Tools:
• kdump, crash, gdb
• kprobes, trace-cmd, ftrace
• perf, valgrind
• Hardware debugging tools (JTAG, Serial Console)
• Build Systems:
• Kbuild, Makefiles
• Kernel packaging (RPM/Debian)
• Security:
• Experience with CVE patching and backporting
• Knowledge of SELinux/AppArmor
• Kernel hardening (FIPS, KSPP)

Preferred Skills

• Experience contributing to open-source kernel projects
• Familiarity with Linux Kernel Mailing List (LKML) workflows
• Exposure to enterprise Linux distributions (RHEL, Ubuntu, SUSE)
• Experience with performance tuning and system optimization at scale

1. Core Programming (C Language)

• Must have strong hands-on experience in C programming
• Comfortable with pointers, memory management, and low-level concepts

2. Kernel Internals Expertise

• Should have worked in at least one subsystem:
• VFS / File Systems
• Memory Management
• Scheduler / Networking

3. Debugging & Crash Analysis

• Experience handling kernel panics
• Hands-on with vmcore analysis tools

4. Security & Patching

• Understanding of CVE fixes and backporting

5. Driver Development

• Experience in writing or maintaining device drivers

6. Performance & Advanced Debugging

• Exposure to eBPF, ftrace, perf

7. Hardware-Level Understanding

• Knowledge of DMA, interrupts, hardware interaction

Soft Skills

• Strong analytical and problem-solving abilities
• Excellent communication skills
• Ability to work independently and in collaborative environments
• Quick learner with adaptability to new technologies

Strong Full stack/Backend engineer profile

Mandatory (Experience): Must have 2+ years of hands-on experience as a full stack developer (backend-heavy)

Mandatory (Backend Skills): Must have 1.5+ strong experience in Python, building REST APIs, and microservices-based architectures

Mandatory (Frontend Skills): Must have hands-on experience with modern frontend frameworks (React or Vue) and JavaScript, HTML, and CSS

Mandatory (AI): Must have hands on experience with using AI tools (eg: Claude, Cursor, GitHub Copilot, Codeium, Deepdcode) for coding.

Mandatory (Database Skills): Must have solid experience working with relational and NoSQL databases such as MySQL, MongoDB, and Redis

Mandatory (Cloud & Infra): Must have hands-on experience with AWS services including EC2, ELB, AutoScaling, S3, RDS, CloudFront, and SNS

Mandatory (DevOps & Infra): Must have working experience with Linux environments, Apache, CI/CD pipelines, and application monitoring

Mandatory (CS Fundamentals): Must have strong fundamentals in Data Structures, Algorithms, OS concepts, and system design

Mandatory (Company) : Product companies (Preferably Top product companies, AI native companies, B2B SaaS)

Mandatory (Stability): Must have atleast 2 years of experience in each of the previous companies (if less exp, then proper reason)

Mandatory (Note): Candidates who have owned end-to-end product development or worked on app development projects during their graduation will be highly preferred.

Mandatory (Note 2): The role offers a mix of work setups, including remote, Mumbai (in-office), and Bangalore (in-office) opportunities

• Leads more than one projects end-to-end and collaborates across functions. Drives planning, estimation and execution.
• Manages stakeholder expectations and offers scalable, reliable, performant and easy to maintain solutions
• Consistently delivers complex, well backed and bug-free products in time
• Consistently takes well thought technical/design decisions
• Develops expertise in more than one area and shares knowledge with others. able to mentor/train in areas which are new to them.
• Drives people to solve engineering challenges
• Enjoys high respect of Tech and other cross functional teams
• Demonstrates effective communication with project team, management and

internal/external clients as necessary.

• Surfaces both technical and non-technical team challenges and helps resolve them
• Champion for SDLC best practices and high quality standards

• Expert at in RoR, Golang, NodeJS or Python. Good to have exposure to ML.
• Must have experience in cloud computing
• Operates independently with almost no oversight
• Is able to apply domain expertise to think critically and make wise decisions for the

team, taking into account tradeoffs and constraints.

• Communicates tech decisions through design docs and tech talks
• Has delivered multiple projects with end-to-end engineering ownership
• Keeps track of new technology/tools and embraces them as necessary
• 7+ years of experience in product driven organization
• A Bachelors or Masters degree in engineering from a reputed institute (preferably

Company Introduction –

• Information Security & Data Analytics Series A funded company.
• Working in cutting edge technologies - Using AI for predictive intelligence and Facial Biometrics.
• Among Top 5 Cyber excellence companies globally (Holger Schulze awards)
• Bronze award for best startup of the year (Indian Express IT awards), only cyber Security Company in top 3.
• More than 100+ clients in India.

Job Description:-

Job Title: Python Developer

Key Requirements:-

• Mine data from structured and unstructured data sources.
• Extract data (text, images, and videos) from multiple documents in different formats.
• Extract information and intelligence from data.
• Extract data based on regular expressions.
• Collect data from structured RDBMS databases.
• Work closely with Project/Business/Research teams to provide mined data/intelligence for analysis.
• Should have strong exposure to core python skills like multiprocessing, multithreading, file handling, data structure like JSON, Data frames, and User Defined Data structure.
• Should have excellent knowledge of classes, file handling, memory manipulations.
• Strong Knowledge in Python.
• Strong exposure to frond end languages like CSS, JavaScript, Ajax etc.
• Should have exposure to requests, Frontera, scarpy-cluster, elastic-search, distributed computing tools like Kafka, Hbase, Redis, Zookeeper, restAPI.
• Should be familiar with *nix development environment.
• Knowledge of Django will be added advantage.
• Excellent knowledge on Web Crawling/Web scraping.
• Should have used scraping modules like Selenium, Scrapy, and Beautiful soup.
• Experience with text processing.
• Basics of databases. Good troubleshooting and debugging skills.

 
Education

   B.Tech, MCA, Computer Engineering