Data Engineer

Job Type: Full Time

Shift: US Time Zone

About the Role:

We are seeking a talented Senior Python Developer with strong hands-on experience in building agentic systems (multi-agent systems). This role requires expertise in Python, PostgreSQL, and the ability to work on a dynamic platform that supports multi-agent architectures. You will be responsible for developing, enhancing, and maintaining platforms that involve complex agent-based interactions, while also supporting the database systems and overall infrastructure.

Key Responsibilities:

Design, implement, and maintain multi-agent systems to support complex interactions and decision-making processes.

Develop scalable and efficient Python code for integrating and managing agents and their communication layers.

Work extensively with PostgreSQL databases to design schemas, optimize queries, and manage data storage effectively.

Collaborate with cross-functional teams to integrate agentic systems into larger platforms and ensure their smooth operation.

Troubleshoot, debug, and optimize Python-based solutions and database interactions.

Write high-quality, maintainable, and well-documented code.

Provide technical support for the platform, ensuring its scalability and performance over time.

Mentor junior developers and provide technical guidance as needed.

Required Skills and Experience:

5+ years of strong hands-on experience in Python development.

Experience in building and deploying multi-agent systems or agent-based frameworks.

Proficiency in PostgreSQL, including schema design, query optimization, and database management.

Knowledge of agent-based modeling or similar paradigms is highly desirable.

Familiarity with various Python libraries for handling complex algorithms and system integration.

Experience in supporting and maintaining platforms or systems post-deployment.

Strong understanding of software development best practices and version control (Git).

Excellent problem-solving skills and the ability to think algorithmically.

Strong communication skills with the ability to explain complex technical concepts.

Preferred Qualifications:

A degree in Computer Science, Engineering, or a related field.

Experience with cloud platforms (AWS, Azure, etc.) is a plus.

Knowledge of agent communication languages (ACL) or other protocols related to multi-agent systems is beneficial.

Familiarity with Docker, Kubernetes, or other containerization tools.

Position : Software Engineer (Java Backend Engineer)

Experience : 4+ Years

📍 Location : Bangalore, India (Hybrid)

Mandatory Skills : Java 8+ (Advanced Features), Spring Boot, Apache Spark (Spark Streaming), SQL & Cosmos DB, Git, Maven, CI/CD (Jenkins, GitHub), Azure Cloud, Agile Scrum.

About the Role :

We are seeking a highly skilled Backend Engineer with expertise in Java, Spark, and microservices architecture to join our dynamic team. The ideal candidate will have a strong background in object-oriented programming, experience with Spark Streaming, and a deep understanding of distributed systems and cloud technologies.

Key Responsibilities :

• Design, develop, and maintain highly scalable microservices and optimized RESTful APIs using Spring Boot and Java 8+.
• Implement and optimize Spark Streaming applications for real-time data processing.
• Utilize advanced Java 8 features, including:
• Functional interfaces & Lambda expressions
• Streams and Parallel Streams
• Completable Futures & Concurrency API improvements
• Enhanced Collections APIs
• Work with relational (SQL) and NoSQL (Cosmos DB) databases, ensuring efficient data modeling and retrieval.
• Develop and manage CI/CD pipelines using Jenkins, GitHub, and related automation tools.
• Collaborate with cross-functional teams, including Product, Business, and Automation, to deliver end-to-end product features.
• Ensure adherence to Agile Scrum practices and participate in code reviews to maintain high-quality standards.
• Deploy and manage applications in Azure Cloud environments.

Minimum Qualifications:

• BS/MS in Computer Science or a related field.
• 4+ Years of experience developing backend applications with Spring Boot and Java 8+.
• 3+ Years of hands-on experience with Git for version control.
• Strong understanding of software design patterns and distributed computing principles.
• Experience with Maven for building and deploying artifacts.
• Proven ability to work in Agile Scrum environments with a collaborative team mindset.
• Prior experience with Azure Cloud Technologies.

Job Details

- Job Title: Lead Software Engineer - Java, Python, API Development

- Industry: Global digital transformation solutions provider

- Domain - Information technology (IT)

- Experience Required: 8-10 years

- Employment Type: Full Time

- Job Location: Pune & Trivandrum/ Thiruvananthapuram

- CTC Range: Best in Industry

Job Description

Job Summary

We are seeking a Lead Software Engineer with strong hands-on expertise in Java and Python to design, build, and optimize scalable backend applications and APIs. The ideal candidate will bring deep experience in cloud technologies, large-scale data processing, and leading the design of high-performance, reliable backend systems.

Key Responsibilities

• Design, develop, and maintain backend services and APIs using Java and Python
• Build and optimize Java-based APIs for large-scale data processing
• Ensure high performance, scalability, and reliability of backend systems
• Architect and manage backend services on cloud platforms (AWS, GCP, or Azure)
• Collaborate with cross-functional teams to deliver production-ready solutions
• Lead technical design discussions and guide best practices

Requirements

• 8+ years of experience in backend software development
• Strong proficiency in Java and Python
• Proven experience building scalable APIs and data-driven applications
• Hands-on experience with cloud services and distributed systems
• Solid understanding of databases, microservices, and API performance optimization

Nice to Have

• Experience with Spring Boot, Flask, or FastAPI
• Familiarity with Docker, Kubernetes, and CI/CD pipelines
• Exposure to Kafka, Spark, or other big data tools

Skills

Java, Python, API Development, Data Processing, AWS Backend

Skills: Java, API development, Data Processing, AWS backend, Python,

Must-Haves

Java (8+ years), Python (8+ years), API Development (8+ years), Cloud Services (AWS/GCP/Azure), Database & Microservices

8+ years of experience in backend software development

Strong proficiency in Java and Python

Proven experience building scalable APIs and data-driven applications

Hands-on experience with cloud services and distributed systems

Solid understanding of databases, microservices, and API performance optimization

Mandatory Skills: Java API AND AWS

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Pune, Trivandrum

Requirements

• 3+ years work experience with production-grade python. Contribution to open source repos is preferred
• Experience writing concurrent and distributed programs, AWS lambda, Kubernetes, Docker, Spark is preferred.
• Experience with one relational & 1 non-relational DB is preferred
• Prior work in the ML domain will be a big boost

What You’ll Do

• Help realize the product vision: Production-ready machine learning models with monitoring within moments, not months.
• Help companies deploy their machine learning models at scale across a wide range of use-cases and sectors.
• Build integrations with other platforms to make it easy for our customers to use our product without changing their workflow.
• Write maintainable, scalable performant python code
• Building gRPC, rest API servers
• Working with Thrift, Protobufs, etc.

• Build campaign generation services which can send app notifications at a speed of 10 million a minute
• Dashboards to show Real time key performance indicators to clients
• Develop complex user segmentation engines which creates segments on Terabytes of data within few seconds
• Building highly available & horizontally scalable platform services for ever growing data
• Use cloud based services like AWS Lambda for blazing fast throughput & auto scalability
• Work on complex analytics on terabytes of data like building Cohorts, Funnels, User path analysis, Recency Frequency & Monetary analysis at blazing speed
• You will build backend services and APIs to create scalable engineering systems.
• As an individual contributor, you will tackle some of our broadest technical challenges that requires deep technical knowledge, hands-on software development and seamless collaboration with all functions.
• You will envision and develop features that are highly reliable and fault tolerant to deliver a superior customer experience.
• Collaborating various highly-functional teams in the company to meet deliverables throughout the software development lifecycle.
• Identify and improvise areas of improvement through data insights and research.

• 2-5 years of experience in backend development and must have worked on Java/shell/Perl/python scripting.
• Solid understanding of engineering best practices, continuous integration, and incremental delivery.
• Strong analytical skills, debugging and troubleshooting skills, product line analysis.
• Follower of agile methodology (Sprint planning, working on JIRA, retrospective etc).
• Proficiency in usage of tools like Docker, Maven, Jenkins and knowledge on frameworks in Java like spring, spring boot, hibernate, JPA.
• Ability to design application modules using various concepts like object oriented, multi-threading, synchronization, caching, fault tolerance, sockets, various IPCs, database interfaces etc.
• Hands on experience on Redis, MySQL and streaming technologies like Kafka producer consumers and NoSQL databases like mongo dB/Cassandra.
• Knowledge about versioning like Git and deployment processes like CICD.

·      Core responsibilities to include analyze business requirements and designs for accuracy and completeness. Develops and maintains relevant product.

·      BlueYonder is seeking a Senior/Principal Architect in the Data Services department (under Luminate Platform ) to act as one of key technology leaders to build and manage BlueYonder’ s technology assets in the Data Platform and Services.

·      This individual will act as a trusted technical advisor and strategic thought leader to the Data Services department. The successful candidate will have the opportunity to lead, participate, guide, and mentor other people in the team on architecture and design in a hands-on manner. You are responsible for technical direction of Data Platform. This position reports to the Global Head, Data Services and will be based in Bangalore, India.

·      Core responsibilities to include Architecting and designing (along with counterparts and distinguished Architects) a ground up cloud native (we use Azure) SaaS product in Order management and micro-fulfillment

·      The team currently comprises of 60+ global associates across US, India (COE) and UK and is expected to grow rapidly. The incumbent will need to have leadership qualities to also mentor junior and mid-level software associates in our team. This person will lead the Data platform architecture – Streaming, Bulk with Snowflake/Elastic Search/other tools

Our current technical environment:

·      Software: Java, Springboot, Gradle, GIT, Hibernate, Rest API, OAuth , Snowflake

·      • Application Architecture: Scalable, Resilient, event driven, secure multi-tenant Microservices architecture

·      • Cloud Architecture: MS Azure (ARM templates, AKS, HD insight, Application gateway, Virtue Networks, Event Hub, Azure AD)

·      Frameworks/Others: Kubernetes, Kafka, Elasticsearch, Spark, NOSQL, RDBMS, Springboot, Gradle GIT, Ignite

Cloudera Data Warehouse Hive team looking for a passionate senior developer to join our growing engineering team. This group is targeting the biggest enterprises wanting to utilize Cloudera’s services in a private and public cloud environment. Our product is built on open source technologies like Hive, Impala, Hadoop, Kudu, Spark and so many more providing unlimited learning opportunities. 

A Day in the Life

Over the past 10+ years, Cloudera has experienced tremendous growth making us the leading contributor to Big Data platforms and ecosystems and a leading provider for enterprise solutions based on Apache Hadoop. You will work with some of the best engineers in the industry who are tackling challenges that will continue to shape the Big Data revolution.  We foster an engaging, supportive, and productive work environment where you can do your best work. The team culture values engineering excellence, technical depth, grassroots innovation, teamwork, and collaboration.

You will manage product development for our CDP components, develop engineering tools and scalable services to enable efficient development, testing, and release operations.  You will be immersed in many exciting, cutting-edge technologies and projects, including collaboration with developers, testers, product, field engineers, and our external partners, both software and hardware vendors.

Opportunity:

Cloudera is a leader in the fast-growing big data platforms market. This is a rare chance to make a name for yourself in the industry and in the Open Source world. The candidate will responsible for Apache Hive and CDW projects. We are looking for a candidate who would like to work on these projects upstream and downstream. If you are curious about the project and code quality you can check the project and the code at the following link. You can start the development before you join. This is one of the beauties of the OSS world. 

https://hive.apache.org/" target="_blank">Apache Hive

Responsibilities:

• Build robust and scalable data infrastructure software

• Design and create services and system architecture for your projects 

• Improve code quality through writing unit tests, automation, and code reviews

• The candidate would write Java code and/or build several services in the Cloudera Data Warehouse. 

• Worked with a team of engineers who reviewed each other's code/designs and held each other to an extremely high bar for the quality of code/designs

• The candidate has to understand the basics of Kubernetes.

• Build out the production and test infrastructure. 

• Develop automation frameworks to reproduce issues and prevent regressions.

• Work closely with other developers providing services to our system.

• Help to analyze and to understand how customers use the product and improve it where necessary.  

Qualifications:

• Deep familiarity with Java programming language.

• Hands-on experience with distributed systems. 

• Knowledge of database concepts, RDBMS internals.

• Knowledge of the Hadoop stack, containers, or Kubernetes is a strong plus.  

• Has experience working in a distributed team. 

• Has 3+ years of experience in software development.

• Minimum 7 years of relevant work experience in similar roles.
• Hands-on experience developing and delivering scalable multi-tenant SaaS applications on AWS platform.
• In-depth knowledge of Spring, Spring Boot, Java, REST Web Services, SQL databases, microservices, GRAND stack, SQL and NoSQL databases.
• In-depth knowledge and experience developing and delivering scalable data lakes, data ingestion and processing pipelines, data access microservices.
• In-depth knowledge of AWS platform, tools and services, specifically AWS networking and security, Route53, API Gateway, ECS/Fargate, RDS; Java/Spring development; modern database and data processing technologies; DevOps; microservices architecture; container/Docker technology.
• Outstanding collaboration and communication skills. Ability to effectively collaborate with distributed team.
• Understand and practice agile development methodology.
• Prior experience working in a software product company.
• Prior experience with security product development.

Nice to Have:

• AWS Certified Developer certification is highly desired.
• Prior experience with Apache Spark and Scala.