Data platform Engineer

We are looking for a Database Engineer (PostgreSQL) with 1-5 years of experience to manage, optimize, and maintain PostgreSQL databases. The ideal candidate will have expertise in PostgreSQL, SQL Optimization, Database Design, Performance Tuning, Data Migration, and Backup & Recovery.

Responsibilities:

• Design, develop, and maintain PostgreSQL databases.
• Optimize SQL queries and improve database performance.
• Perform database design, indexing, and schema management.
• Execute data migration, backup, recovery, and disaster recovery activities.
• Monitor database health and troubleshoot performance issues.
• Collaborate with development teams to ensure scalable database solutions.

Requirements:

• 1-5 years of PostgreSQL database experience.
• Strong SQL and performance tuning skills.
• Knowledge of database backup, recovery, and migration processes.

Job Summary

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate should have strong expertise in SQL, Python, Linux, and modern data engineering practices to support data integration, transformation, and analytics.

Key Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines.
• Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
• Develop automation scripts using Python for data processing and workflow optimization.
• Work with Linux environments for deployment, monitoring, and troubleshooting.
• Ensure data quality, integrity, and reliability across data platforms.
• Collaborate with data analysts, software engineers, and business stakeholders to deliver data solutions.
• Monitor, troubleshoot, and optimize data pipelines for performance and scalability.
• Implement best practices for data security, governance, and documentation.

Required Skills

• Strong experience in Data Engineering concepts and ETL/ELT processes.
• Proficiency in SQL, including query optimization and database design.
• Strong programming skills in Python.
• Hands-on experience with Linux commands, shell scripting, and system administration basics.
• Experience with relational databases such as PostgreSQL, MySQL, SQL Server, or Oracle.
• Familiarity with Git/version control.
• Strong analytical and problem-solving skills.

Preferred Skills

• Experience with cloud platforms (AWS, Azure, or GCP).
• Knowledge of Apache Spark, Airflow, Kafka, or similar data engineering tools.
• Experience with data warehousing solutions and big data technologies.
• Understanding of CI/CD pipelines and containerization (Docker/Kubernetes).

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Relevant certifications in cloud or data engineering are an added advantage.

What you'll need

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical field.
• 5+ years of professional data engineering experience.
• Experience designing and building cloud-native data solutions.
• Strong expertise with Google Cloud Platform, including BigQuery. Experience developing transformation frameworks using dbt.
• Strong SQL and Python programming skills.
• Experience with PostgreSQL or other relational databases.
• Experience orchestrating workflows using Apache Airflow or Cloud Composer.
• Experience implementing Infrastructure as Code using Terraform. Experience building CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or similar platforms.
• Experience developing scalable batch and streaming data pipelines. Strong problem-solving skills with the ability to balance scalability, reliability, and cloud cost optimization.

Preferred Qualifications

• Experience with Pub/Sub, Datastream, Dataflow, Cloud Storage, Cloud Functions, or Cloud Run.
• Experience building multi-tenant SaaS platforms.
• Experience implementing metadata-driven governance, lineage, and data quality frameworks.
• Experience supporting AI, machine learning, or customer-facing analytics platforms.
• Experience with Kubernetes and Docker.

Job Title: Senior Data Engineer

Position Summary

We are seeking a highly skilled Data Engineer with strong expertise in reporting infrastructure, data modeling, and analytics platforms. This role will focus on building scalable reporting solutions, optimizing data pipelines, and enabling data-driven decision-making across the organization. The ideal candidate will have deep experience with databases, data warehousing, and complex query development. While Ruby on Rails experience is beneficial, the primary emphasis is on data engineering and reporting systems.

Key Responsibilities

• Design, develop, and maintain reporting infrastructure and data platforms.
• Build and optimize complex SQL queries, data models, and reporting datasets.
• Develop scalable ETL/ELT pipelines and data integration processes.
• Design and maintain database schemas to support analytics and reporting requirements.
• Collaborate with business stakeholders to understand reporting needs and translate them into technical solutions.
• Implement data quality, governance, and performance best practices.
• Support and enhance enterprise reporting solutions and dashboards.
• Work with large-scale datasets in cloud data warehouse environments.
• Troubleshoot reporting issues and optimize database performance.
• Partner with engineering and product teams to establish sustainable data architecture.

Must Have: Technical Skills

• PostgreSQL
• Snowflake
• Advanced SQL
• Data Modeling
• Data Warehousing
• Reporting & Analytics Infrastructure
• ETL/ELT Development
• Query Performance Tuning

Success Profile:

The ideal candidate is a data-focused engineer who enjoys working with complex datasets, designing scalable reporting solutions, and building foundational infrastructure that empowers analytics and business intelligence teams. They should be comfortable navigating databases, optimizing queries, and translating business reporting needs into robust technical implementations.

Experience Range: 5-10 years

Role Type: Individual Contributor / Senior Engineer

Primary Focus: Reporting Infrastructure, Data Engineering, Data Warehousing, and Analytics Enablement.

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

About Us

We believe the future of software development is AI-native — where engineers operate at a higher level of abstraction and quality remains non-negotiable. 

Incubyte is a software craft consultancy where the “how” of building software matters as much as the “what”.  

We partner with companies of all sizes, from helping enterprises build, scale, and modernize to early-stage founders bring their ideas to life. 

Our engineers operate in an AI-native development model, using AI as a collaborator across the SDLC to accelerate development while upholding the discipline of software craftsmanship. Guided by Software Craftsmanship and Extreme Programming practices, we build reliable, maintainable, and scalable systems with speed, without compromising quality. If this way of building software resonates with you, we’d like to talk. 

Our Guiding Principles 

These principles define how we work at Incubyte. They are non-negotiable. 

Relentless Pursuit of Quality with Pragmatism 

  We build high-quality systems without losing sight of delivery. 

Extreme Ownership 

  We take responsibility end-to-end for decisions, execution, and outcomes. 

Proactive Collaboration 

  We collaborate closely, challenge each other, and solve problems together. 

Active Pursuit of Mastery 

  We continuously improve our craft and raise our bar. 

Invite, Give, and Act on Feedback 

We seek, give, and act on feedback to get better every day. 

Ensuring Client Success 

We act as trusted partners and focus on real outcomes, not just output. 

Job Description

This is a remote position.

Experience Level

2+ years of experience in SQL, Python, and Snowflake (or equivalent cloud data warehouse), Azure Cloud services.

Role Overview

If you're a Data Craftsperson who takes pride in clean, well-tested data solutions and believes in the principles of Extreme Programming, we'd love to meet you. At Incubyte, we're a DevOps organization where developers own the entire release cycle — you'll get hands-on experience across data engineering, analytics, cloud infrastructure, and direct client communication. This role sits primarily in data engineering (80%) with a meaningful analytics component (20%), supporting our client's data systems end-to-end.

What You'll Do

• Design, build, and maintain data pipelines and infrastructure using SQL and Python
• Work within Snowflake to build and optimize data models supporting business use cases
• Parse and process structured and semi-structured data (JSON, XML) from varied sources
• Diagnose issues across raw, intermediate, and summary tables
• Build SQL queries to support repeatable analytics use cases based on stakeholder requirements
• Investigate and resolve data quality issues, including time-sensitive or urgent ones
• Identify opportunities to consolidate models and maintain a single source of truth (SSOT)

Requirements

What We're Looking For

• 2+ years of experience with SQL and relational databases, with the ability to understand complex data relationships and transformations (required)
• 2+ years of experience with Python for data engineering tasks (required)
• Experience with Snowflake or an equivalent cloud data warehouse (required)
• Experience working with Snowflake Coco or any other AI tools(required) 
• Experience parsing JSON and XML data (a plus)
• A strong eye for data quality and attention to detail
• Knowledge of Git (required)
• Knowledge of Azure cloud services such as Azure Data Factory, Azure Blob Storage, and Azure SQL Database (required)
• Knowledge of data infrastructure/modeling tools like DBT, Fivetran (a plus)
• Experience with BI tools like Power BI(a plus, not core to this role)
• Knowledge of Docker, Linux, Shell/Bash, and virtualization technologies (a plus)
• Knowledge of SSIS packages (a plus)
• Familiarity with CI/CD methodologies

Benefits

Life at Incubyte  

We are a remote-first company with structured flexibility. Teams commit to shared rhythms during core hours, ensuring smooth collaboration while maintaining autonomy. Twice a year, we come together in person for a co-working sprint and once a year for a retreat - with all travel expenses covered. 

Our environment is built for crafters: pairing, refactoring, experimenting with AI, and pushing the boundaries of software excellence. We are all lifelong learners, and our work is our passion. 

Perks

• Dedicated learning & development budget. 
• Sponsorship for conference talks. 
• Comprehensive medical & term insurance.
• Employee-friendly leave policies. 
• Home Office fund 
• Medical Insurance

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.

4 - 10 years of experience in designing and buildingarchitecting highly resilient data platforms 

∙Strong knowledge of data engineering, architecture and data modeling 

∙Experience in platforms like Databricks and Snowflake 

∙Experience on building applications on cloud (AWS or Azure or Google Cloud) 

∙Strong analytical and problem-solving skills 

∙Prior experience in developing data or computation intensive (e.g. grid based) backend applications is an 

advantage 

∙OOP design skills with an understanding or at least personal interest towards the concepts of Functional 

Programming  

∙Willingness to understand and enhance other people’s code, being able to work in an environment where 

developers will oversee and work on wider components also dealing with older “legacy” code 

∙Strong programming skills (Java/ Scala / Python) skills with the willingness to pick up the other language if not 

already mastered at a sufficient level is important 

∙Spring knowledge is an advantage, but in general willingness to learn, work with and even enhance in-house 

developed frameworks is a must 

∙Prior experience in working with Git, Bitbucket, Jenkins, working with PR-s, using JIRA, following the Scrum Agile 

methodology is an advantage 

∙Prior knowledge of financial products is an advantage 

∙Bachelors or Masters in any relevant field of IT/Engineering area is an advantage  

Senior Database Administrator (DBA)

Database Reliability & Performance  •  Senior Individual Contributor

• Experience: 6–9 years in production database administration / engineering
• Level: Senior Individual Contributor (hands-on)
• Function: Data Platform — reliability, performance, and governance across relational and distributed SQL & NoSQL DBs.

About the Role

We are hiring a Senior Database Administrator to own the health, performance, and reliability of the data layer behind a high-scale, multi-tenant platform. Our environment spans several independent PostgreSQL database instances, a cache-first read tier, change-data-capture pipeline, analytics and archival stores. Should be capable of implementing NoSQL stores (Cassandra / ScyllaDB) & MySQL  end to end as we scale.

This is a hands-on senior IC role for someone who has run demanding production databases and wants full ownership: partitioning and maintenance, performance tuning, monitoring, uptime, disaster recovery, and the security and data-governance documentation that keeps a regulated, multi-tenant platform audit-ready. You will work closely with, and take direction from, engineering leadership — translating priorities into a well-run, well-documented data platform.

What You'll Own

PostgreSQL Depth

• Administer multiple production PostgreSQL instances (v16+), including schema/DDL review, migrations, and release coordination.
• Design and maintain table partitioning strategies (range/hash/time-based) and automate partition lifecycle — creation, retention windows, and safe drops.
• Own query and instance performance tuning: EXPLAIN/ANALYZE, indexing strategy, autovacuum/bloat management, connection pooling (e.g. PgBouncer), and parameter-group tuning.
• Execute zero-downtime schema changes using expand–migrate–contract / blue-green patterns; assess lock impact and avoid long ACCESS EXCLUSIVE locks on hot tables.
• Manage replica topology (writer/reader endpoints), replication lag, and read-scaling as load grows.
• Partner with app teams on migration tooling (e.g. Flyway) and paired forward/rollback scripts; enforce idempotent, reviewed, tested DDL.

NoSQL & Scale

• Data-model, deploy, and operate distributed NoSQL clusters — Cassandra and/or ScyllaDB — including partition/clustering key design, compaction strategy, repair, and node operations (bootstrap, decommission, scale-out).
• Lead the planned migration of high-volume payload tables from partitioned PostgreSQL to Cassandra/ScyllaDB: capacity planning, dual-write/backfill design, cutover, and validation.
• Tune distributed-store read/write paths, consistency levels, and cluster sizing for predictable latency at scale.
• Support the cache-first architecture (Redis) and change-data-capture / streaming pipelines (e.g. Kafka) that move data to analytics and cold-archival stores.

Operations, Reliability & DR

• Own database uptime and SLOs; build and maintain monitoring, alerting, and dashboards for the key signals (latency, replication lag, CPU/IO, connections, vacuum lag, error rates).
• Define and regularly test backup and disaster-recovery strategy — RPO/RTO targets, point-in-time recovery, restore drills, and cross-AZ / future multi-region readiness.
• Drive incident response for database-related issues: triage, mitigation, root-cause analysis, and follow-up actions.
• Write and maintain runbooks (partition maintenance, failover, archival, DR, migration procedures) and participate in an on-call rotation for the data layer.
• Establish production-change discipline: pre-flight and post-deploy verification, rollback criteria, and backup verification before destructive operations.

Security, Governance & Documentation

• Implement and maintain database access control — least-privilege roles, schema-scoped permissions, and row-level security (RLS) for multi-tenant isolation.
• Own data-governance documentation: PII maps, per-table retention policies, audit-readiness, and the controls behind privacy-regulation compliance (e.g. GDPR / DPDP-style erasure and legal-hold workflows).
• Author and maintain architecture decision records (ADRs), schema documentation, and monitoring/runbook docs so the data platform is legible to the wider team.
• Uphold encryption-at-rest/in-transit posture and safe handling of sensitive columns; keep the data layer aligned with the platform's compliance requirements.

Coding & Automation Bar (Required)

This role is automation-first, not click-ops. The right candidate treats repetitive database work as something to script away and can drop into application code when a problem demands it.

• Strong scripting in Python and Bash for automation of maintenance, backups, partition management, backfills, and health checks.
• Comfortable with infrastructure-as-code (e.g. Terraform) and configuration management (e.g. Ansible) for reproducible database infrastructure.
• Solid SQL and PL/pgSQL; able to read and debug application-layer database code and, worst case, write or patch application code (e.g. Python/Node/Java) to unblock a fix.
• Fluent with Git-based workflows, code review, and CI for database migrations and tooling.

Required Qualifications

• 6–9 years operating relational databases in demanding production environments, with deep PostgreSQL expertise (internals, partitioning, replication, performance tuning).
• Hands-on experience running at least one distributed NoSQL store in production — Cassandra or ScyllaDB strongly preferred (other wide-column/distributed stores considered).
• Proven ownership of backup/DR, monitoring, and incident response for production databases.
• Strong scripting/automation background (Python, Bash) and experience with IaC.
• Experience with managed cloud databases (AWS RDS or equivalent) and understanding of instance/parameter-group tuning, replicas, and failover.
• Experience designing for multi-tenancy and data isolation at scale.
• Clear written communication — able to produce runbooks, ADRs, and governance docs a team can rely on.

Nice to Have

• Experience migrating workloads from PostgreSQL to Cassandra/ScyllaDB (or similar relational-to-distributed migrations).
• Familiarity with Redis as a cache tier and with CDC / streaming pipelines (e.g. Kafka, Debezium).
• Exposure to Flyway (or Liquibase), pgTAP or similar schema testing, and PgBouncer.
• Working knowledge of privacy regulations (GDPR, India DPDP) and audit/compliance processes.

How You Work

• Ownership mindset — takes a directive and runs it to done, with sound judgment on trade-offs.
• Reliability-first and detail-oriented; disciplined about verification, rollback, and documentation.
• Collaborative in a high-performing engineering team; communicates clearly with developers and leadership.
• Pragmatic and scale-aware — designs for where the platform is going, not just where it is.

Data Engineer

Location: Bengaluru, India (Hybrid)

Employment Type: Full-time

Experience: 3-5 years

Role Overview  

What We’re Looking For:

• Bachelor’s degree in Computer Science/Engineering or equivalent experience required.
• Experience designing and shipping cloud services products.
• Experience driving and managing technical and architectural dependencies on AWS Cloud.
• A firm understanding of system architecture, cloud computing, PaaS/SaaS design principles, S3, DynamoDB, RDS mandatory.
• Experience in building or maintaining ETL processes and tools, i.e., AWS Glue or any open-source tool.
• Proven system-level design contribution to a current “Live” (in production / under daily high load) multi-region SaaS or PaaS offering.
• Proven experience with S3, DynamoDB, SQL, and AWS RDS services.
• Proficiency in programming languages such as Python.
• Strong analytical and problem-solving skills.
• 
  

Required Skills & Experience

• Experience with Python, SQL, and data visualization/exploration tools.
• Familiarity with the AWS ecosystem, specifically S3, DynamoDB, and RDS.
• Communication skills, especially for explaining technical concepts to nontechnical business leaders.
• Ability to work on a dynamic, research-oriented team that has concurrent projects.
• Experience in AWS cost optimization (Savings Plans, Reserved Instances, Spot Instances) and governance frameworks.
• Experience developing solutions using infrastructure orchestration tools (SSM, automation account, Ansible, etc.).
• Excellent leadership, stakeholder management, and communication skills.

What We Offer

• Work with some of the brightest minds in the emerging EV industry.
• Make a tangible impact in reducing carbon emissions and enabling sustainable energy.
• Freedom to suggest, implement, and innovate on systems, processes, and technologies.
• Daily ownership in a high-growth, challenging environment.
• Flexible work environment with hybrid schedules and virtualization options.
• Competitive pay and benefits including health coverage, innovative PTO program, and performance bonuses.