Sr. Spark Software Engineer

Role: Sr. Azure Data Engineer

Experience: 8–10 Years

Work Timings: 1:30 PM – 10:30 PM IST

Location: Bellandur Bengaluru (Work from Office)

Company: Chevron

Employment Type: 6- 12 months Contract

Role Overview

We are seeking an experienced Senior Data Engineer to design and deliver scalable cloud data solutions on Azure. The ideal candidate will have strong expertise in Databricks, PySpark, and modern data architectures, with exposure to energy domain standards like OSDU.

Key Responsibilities

• Architect and design robust Azure-based data solutions using Databricks, ADLS, and PaaS services
• Define and implement scalable data Lakehouse architectures aligned with OSDU standards
• Build and manage end-to-end data pipelines for batch and real-time processing using PySpark
• Establish data governance frameworks including metadata, lineage, security, and access control
• Implement DevOps best practices (CI/CD, Azure Pipelines, GitHub, automated deployments)
• Collaborate with stakeholders to translate business needs into technical solutions
• Develop and maintain architecture documentation, solution patterns, and standards
• Provide technical leadership and mentorship to engineering teams
• Optimize solutions for performance, cost, reliability, and security
• Ensure alignment with enterprise architecture and compliance standards
• Drive adoption of modular and reusable cloud data components

Required Skills & Qualifications

Core Technical Skills

• Azure Databricks, Apache Spark (PySpark), Delta Lake, Unity Catalog
• Azure Data Lake Storage (ADLS), Azure Data Factory, Synapse Analytics
• Strong experience in Python-based data engineering
• Data pipeline development (batch + real-time)

Architecture & Advanced Skills

• Data Lakehouse architecture and distributed systems
• Microservices, APIs, and integration frameworks
• OSDU (Open Subsurface Data Universe) or similar energy data models

DevOps & Tools

• CI/CD tools: Azure Pipelines, GitHub Actions
• Infrastructure as Code: Terraform or similar

Other Skills

• Data governance, security, compliance, and cost optimization
• Strong analytical and problem-solving skills
• Excellent communication and stakeholder management

We are looking for a Senior Backend Engineer with a strong background in building distributed, scalable, and high-performance backend systems. The ideal candidate has 6+ years of backend engineering experience, including 3+ years in microservices or distributed architecture, and deep expertise in Python, FastAPI, PostgreSQL, Redis, and async I/O patterns.

The candidate will own backend services end-to-end — from architecture and design to implementation, optimization, and production support — while working closely with cross-functional teams to deliver mission-critical systems.

Key Responsibilities

1. Backend Architecture & Development

• Architect, design, and implement backend services using Python (preferably FastAPI) with clean, maintainable, and scalable code.
• Develop microservice components, service communication layers, and distributed workflows.
• Build APIs and backend modules optimized for performance, fault tolerance, and low latency.

2. Distributed Systems & Scalability

• Design and maintain distributed systems capable of handling high throughput and large-scale operations.
• Implement inter-service communication patterns (REST, event-driven, message queues, pub/sub).
• Ensure system resiliency through redundancy, graceful degradation, and autoscaling strategies.

3. Asynchronous & Event-Driven Architecture

• Build async workflows, background tasks, and pipeline services leveraging asyncio or async frameworks.
• Develop event-driven components including producers, consumers, and event processors.
• Optimize I/O-heavy operations to maximize concurrency and throughput.

4. Data Storage, Pipelines & Caching

• Design relational schemas, write optimized SQL queries, and work with PostgreSQL performance tuning (indexes, partitions, query planning).
• Implement caching strategies, rate limiting, distributed locks, and session storage using Redis.
• Build data validation pipelines and compliance-driven data flows ensuring data correctness and auditability.

5. Security, Reliability & Observability

• Implement authentication, authorization, encryption, and secure API practices.
• Build monitoring, alerting, metrics, and tracing systems using modern observability tools.
• Diagnose production issues, identify bottlenecks, and drive long-term fixes and improvements.

6. Code Quality & Technical Leadership

• Conduct detailed code reviews and ensure engineering best practices across the team.
• Participate in technical roadmap planning, architectural discussions, and system design decisions.
• Mentor junior and mid-level engineers; drive a culture of technical excellence and ownership.

Required Qualifications

• 6+ years backend engineering experience.
• 3+ years building microservices or distributed systems.
• Strong expertise in Python and frameworks like FastAPI (or Flask/Django with async exposure).
• Deep understanding of async I/O, concurrency models, and asynchronous programming in Python.
• Strong experience with PostgreSQL (schema design, advanced queries, optimization).
• Hands-on experience with Redis for caching, queues, session management, and distributed patterns.
• Experience with event-driven architectures, message processing, or streaming systems.
• Solid understanding of system design, scalability principles, and API lifecycle management.
• Experience working with CI/CD pipelines, containerization (Docker), and cloud environments.

Are you a fun loving and passionate to be a part of Global Innovator team? Are you planning to grow your career, which enhance your skills in technology? A career at SmartDocs can open many doors for you in the world of technology. If you are looking for a company that is dedicated to your ideas, recognizes you for your unique competency & contributions and provides a fun, flexible and delightful work atmosphere.

Then, we are the right place to ignite your passion. We are totally committed about our employees, our Clients & Customers, our work culture and especially our technology. We are a flat organization where opportunities are provide based on talent and we always encourage new ideas of employees through collaboration and creativity.

We are seeking for smart-driven JAVA Programmer’s to join us. The candidate will work with the global product development team and subject matter experts. An ideal candidate must possess excellent business skills with outstanding Analytical & logical skills, professionalism, Intelligent and should have the zeal to learn.

Desirable Skills & Expertise:

• Master’s or bachelor’s degree in Engineering (CSE / IT / MCA / Software) are preferred.
• Possesses intellectual humility; smart-driven, creative and able to learn things from slipups, willing to raise others up.
• Excellent Logical & Analytical skills with integrated professionalism at all levels.
• Strong Knowledge in Java, Frame-works like Spring, Spring boot; J2EE technologies like Servlets, JSP and Web Application Server.
• Good knowledge in web technologies like HTML, Java Script, XML and CSS.
• Candidates must be team players, have a thirst for knowledge, the energy to work in a fast-paced environment and a desire to grow in an entrepreneurial company.

Key Responsibilities:

• The role is responsible for designing, coding and deploying of high-performance applications.
• Excellent interpersonal, Communication and effective organizational skills with solid technical skills.
• Should be able to communicate effectively with both technical and non-technical personnel.
• Excellent trouble shootings & problem-solving skills.
• Practical Knowledge with SDLC from requirement analysis through testing and deployment is a plus.
• Devising possible solutions to anticipated problems.
• Develop and maintain strong product knowledge.
• Guide the clients through various stages of the project and transition to support organization.
• Review existing business processes and participate in the Process Improvement Program.

JD:

• Demonstrable hands-on experience working with Azure technologies.
• Extensive experience with Azure DevOps and pipelines.
• Experience working with Jira and Github.
• Experience of Infrastructure as Code using Terraform to model cloud infrastructure.
• Practical experience of engineering best practices with continuous improvement.

Required

• HashiCorp Certified: Terraform Associate
• Microsoft Certified: Azure Fundamentals

Desirable

Microsoft Certified: Azure Administrator Associate

Location- Remote

Job Type: Full time on contract

Responsibilities:

• Develop features and modules independently based on the overall product architecture provided.
• Research and find solutions to challenging new problems; align with technology needs of the future.
• Fully grasp the product concept and think through challenging customer variations and use cases.
• Proactively seek to identify issues and resolve them at the stages of design, deployment and QA.
• Maintain quality and ensure responsiveness of applications.
• Demonstrate a strong sense of milestone-based performance and adherence to overall development timelines.

Requirements:

Experience - 2 to 5 years experience in Java Tech stack

• Strong hands-on experience in Java, Spring Boot, microservices and Rest API.
• Good understanding of general data privacy and information security.
• Exposure to cloud securities for SAAS application development would be an added advantage.
• A good understanding of all facets of building and delivering data-centric products is desirable.
• Knowledge of code versioning tools.

         3+ Yrs Senior Software Engineer - Java Developer 

EMPLOYER: Terra Pay

Headquartered in The Netherlands, TerraPay believes that the smallest payment deserves a borderless journey as safe as the largest. The company has been building an ever-expanding payments highway that empowers businesses to create transparent customer experiences with an uninterrupted, secure, and real-time global passage for every payment, however small or large. Registered and regulated across 25 global markets, TerraPay is a leading global partner to banks, mobile wallets, money transfer operators, merchants, and financial institutions, creating a more expansive and inclusive international financial ecosystem.

With access to payments infrastructure that spans the globe, their partners become beacons of the promise of global financial inclusion.

JOB DETAILS

• Seniority Level: Middle  level
• Industry: Financial Services
• Employment Type: Full-Time (In-Office)
• Job Functions: Information Technology
• Job Location: Bengaluru
• Experience: 3-6 Years

WHAT WILL BE YOUR RESPONSIBILITIES?

• Participate in a full software development lifecycle and write code that is maintainable, clear, and concise.
• Coding, troubleshooting and Multi-threaded applications.
• Producing and refactoring code without assistance.
• Internal team coordination and integration of back end 

SKILLS AND ABILITIES

• Java(J2EE) 
• JSON,SOAP/REST,XML
• Spring MVC, Spring Boot , Spring JDBC, Tomcat, Hibernate
• postgresql

WHAT ARE WE LOOKING FOR?

• Educational Qualification - B.Tech (Computer Science) /M.C.A
• 3-6  years of experience in working as software engineer - Java developer 
• Thorough understanding of coding and testing  
• Expertise with Java(J2EE)  
• Good technical level fluency (spoken and written) in English
• Understands TDD/Performance/Security/Smoke testing.
• Should have knowledge on Junit,SOAPUI,JMeter, postman & other Unit Testing framework.
• Understands feature toggling and version control.miliarity with code versioning tools 

WHAT'S IN IT FOR YOU?

• An opportunity to be a part of a fast-scaling startup in the AdTech space that offers unmatched work opportunities.
• Join us and be certain that you grow up with a growing  MNC 
• You will work in an employee friendly ambience 
• Pick up and drop facility at selected locations