Data Engineer

Job Details

- Job Title: Lead Software Engineer - Java, Python, API Development

- Industry: Global digital transformation solutions provider

- Domain - Information technology (IT)

- Experience Required: 8-10 years

- Employment Type: Full Time

- Job Location: Pune & Trivandrum/ Thiruvananthapuram

- CTC Range: Best in Industry

Job Description

Job Summary

We are seeking a Lead Software Engineer with strong hands-on expertise in Java and Python to design, build, and optimize scalable backend applications and APIs. The ideal candidate will bring deep experience in cloud technologies, large-scale data processing, and leading the design of high-performance, reliable backend systems.

Key Responsibilities

• Design, develop, and maintain backend services and APIs using Java and Python
• Build and optimize Java-based APIs for large-scale data processing
• Ensure high performance, scalability, and reliability of backend systems
• Architect and manage backend services on cloud platforms (AWS, GCP, or Azure)
• Collaborate with cross-functional teams to deliver production-ready solutions
• Lead technical design discussions and guide best practices

Requirements

• 8+ years of experience in backend software development
• Strong proficiency in Java and Python
• Proven experience building scalable APIs and data-driven applications
• Hands-on experience with cloud services and distributed systems
• Solid understanding of databases, microservices, and API performance optimization

Nice to Have

• Experience with Spring Boot, Flask, or FastAPI
• Familiarity with Docker, Kubernetes, and CI/CD pipelines
• Exposure to Kafka, Spark, or other big data tools

Skills

Java, Python, API Development, Data Processing, AWS Backend

Skills: Java, API development, Data Processing, AWS backend, Python,

Must-Haves

Java (8+ years), Python (8+ years), API Development (8+ years), Cloud Services (AWS/GCP/Azure), Database & Microservices

8+ years of experience in backend software development

Strong proficiency in Java and Python

Proven experience building scalable APIs and data-driven applications

Hands-on experience with cloud services and distributed systems

Solid understanding of databases, microservices, and API performance optimization

Mandatory Skills: Java API AND AWS

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Pune, Trivandrum

Level of skills and experience:

5 years of hands-on experience in using Python, Spark,Sql.

Experienced in AWS Cloud usage and management.

Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).

Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.

Experience with orchestrators such as Airflow and Kubeflow.

Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).

Fundamental understanding of Parquet, Delta Lake and other data file formats.

Proficiency on an IaC tool such as Terraform, CDK or CloudFormation.

Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

Cloudera Data Warehouse Hive team looking for a passionate senior developer to join our growing engineering team. This group is targeting the biggest enterprises wanting to utilize Cloudera’s services in a private and public cloud environment. Our product is built on open source technologies like Hive, Impala, Hadoop, Kudu, Spark and so many more providing unlimited learning opportunities. 

A Day in the Life

Over the past 10+ years, Cloudera has experienced tremendous growth making us the leading contributor to Big Data platforms and ecosystems and a leading provider for enterprise solutions based on Apache Hadoop. You will work with some of the best engineers in the industry who are tackling challenges that will continue to shape the Big Data revolution.  We foster an engaging, supportive, and productive work environment where you can do your best work. The team culture values engineering excellence, technical depth, grassroots innovation, teamwork, and collaboration.

You will manage product development for our CDP components, develop engineering tools and scalable services to enable efficient development, testing, and release operations.  You will be immersed in many exciting, cutting-edge technologies and projects, including collaboration with developers, testers, product, field engineers, and our external partners, both software and hardware vendors.

Opportunity:

Cloudera is a leader in the fast-growing big data platforms market. This is a rare chance to make a name for yourself in the industry and in the Open Source world. The candidate will responsible for Apache Hive and CDW projects. We are looking for a candidate who would like to work on these projects upstream and downstream. If you are curious about the project and code quality you can check the project and the code at the following link. You can start the development before you join. This is one of the beauties of the OSS world. 

https://hive.apache.org/" target="_blank">Apache Hive

Responsibilities:

• Build robust and scalable data infrastructure software

• Design and create services and system architecture for your projects 

• Improve code quality through writing unit tests, automation, and code reviews

• The candidate would write Java code and/or build several services in the Cloudera Data Warehouse. 

• Worked with a team of engineers who reviewed each other's code/designs and held each other to an extremely high bar for the quality of code/designs

• The candidate has to understand the basics of Kubernetes.

• Build out the production and test infrastructure. 

• Develop automation frameworks to reproduce issues and prevent regressions.

• Work closely with other developers providing services to our system.

• Help to analyze and to understand how customers use the product and improve it where necessary.  

Qualifications:

• Deep familiarity with Java programming language.

• Hands-on experience with distributed systems. 

• Knowledge of database concepts, RDBMS internals.

• Knowledge of the Hadoop stack, containers, or Kubernetes is a strong plus.  

• Has experience working in a distributed team. 

• Has 3+ years of experience in software development.

• Minimum 7 years of relevant work experience in similar roles.
• Hands-on experience developing and delivering scalable multi-tenant SaaS applications on AWS platform.
• In-depth knowledge of Spring, Spring Boot, Java, REST Web Services, SQL databases, microservices, GRAND stack, SQL and NoSQL databases.
• In-depth knowledge and experience developing and delivering scalable data lakes, data ingestion and processing pipelines, data access microservices.
• In-depth knowledge of AWS platform, tools and services, specifically AWS networking and security, Route53, API Gateway, ECS/Fargate, RDS; Java/Spring development; modern database and data processing technologies; DevOps; microservices architecture; container/Docker technology.
• Outstanding collaboration and communication skills. Ability to effectively collaborate with distributed team.
• Understand and practice agile development methodology.
• Prior experience working in a software product company.
• Prior experience with security product development.

Nice to Have:

• AWS Certified Developer certification is highly desired.
• Prior experience with Apache Spark and Scala.