Data Engineer

Position : Software Engineer (Java Backend Engineer)

Experience : 4+ Years

📍 Location : Bangalore, India (Hybrid)

Mandatory Skills : Java 8+ (Advanced Features), Spring Boot, Apache Spark (Spark Streaming), SQL & Cosmos DB, Git, Maven, CI/CD (Jenkins, GitHub), Azure Cloud, Agile Scrum.

About the Role :

We are seeking a highly skilled Backend Engineer with expertise in Java, Spark, and microservices architecture to join our dynamic team. The ideal candidate will have a strong background in object-oriented programming, experience with Spark Streaming, and a deep understanding of distributed systems and cloud technologies.

Key Responsibilities :

• Design, develop, and maintain highly scalable microservices and optimized RESTful APIs using Spring Boot and Java 8+.
• Implement and optimize Spark Streaming applications for real-time data processing.
• Utilize advanced Java 8 features, including:
• Functional interfaces & Lambda expressions
• Streams and Parallel Streams
• Completable Futures & Concurrency API improvements
• Enhanced Collections APIs
• Work with relational (SQL) and NoSQL (Cosmos DB) databases, ensuring efficient data modeling and retrieval.
• Develop and manage CI/CD pipelines using Jenkins, GitHub, and related automation tools.
• Collaborate with cross-functional teams, including Product, Business, and Automation, to deliver end-to-end product features.
• Ensure adherence to Agile Scrum practices and participate in code reviews to maintain high-quality standards.
• Deploy and manage applications in Azure Cloud environments.

Minimum Qualifications:

• BS/MS in Computer Science or a related field.
• 4+ Years of experience developing backend applications with Spring Boot and Java 8+.
• 3+ Years of hands-on experience with Git for version control.
• Strong understanding of software design patterns and distributed computing principles.
• Experience with Maven for building and deploying artifacts.
• Proven ability to work in Agile Scrum environments with a collaborative team mindset.
• Prior experience with Azure Cloud Technologies.

Job Title: Python Developer (Full Time)

Location: Hyderabad (Onsite)

Interview: Virtual and Face to Face Interview (Last round)

Experience Required: 4 + Years  

Working Days: 5 Days

About the Role

We are seeking a highly skilled Lead Python Developer with a strong background in building scalable and secure applications. The ideal candidate will have hands-on expertise in Python frameworks, API integrations, and modern application architectures. This role requires a tech leader who can balance innovation, performance, and compliance while driving successful project delivery.

Key Responsibilities

1. Application Development

• Architect and develop robust, high-performance applications using Django, Flask, and FastAPI.

1. API Integration

• Design and implement seamless integration with third-party APIs (including travel-related APIs, payment gateways, and external service providers).

1. Data Management

• Develop and optimize ETL pipelines for structured and unstructured data using data lakes and distributed storage solutions.

1. Microservices Architecture

• Build modular, scalable applications using microservices principles for independent deployment and high availability.

1. Performance Optimization

• Enhance application performance through load balancing, caching, and query optimization to deliver superior user experiences.

1. Security & Compliance

• Apply secure coding practices, implement data encryption, and ensure compliance with industry security and privacy standards (e.g., PCI DSS, GDPR).

1. Automation & Deployment

• Utilize CI/CD pipelines, Docker/Kubernetes, and monitoring tools for automated testing, deployment, and production monitoring.

1. Collaboration

• Partner with front-end developers, product managers, and stakeholders to deliver user-centric, business-aligned solutions.

Requirements

Education

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Technical Expertise

• 4+ years of hands-on experience with Python frameworks (Django, Flask, FastAPI).
• Proficiency in RESTful APIs, GraphQL, and asynchronous programming.
• Strong knowledge of SQL/NoSQL databases (PostgreSQL, MongoDB) and big data tools (Spark, Kafka).
• Familiarity with Kibana, Grafana, Prometheus for monitoring and visualization.
• Experience with AWS, Azure, or Google Cloud, containerization (Docker, Kubernetes), and CI/CD tools (Jenkins, GitLab CI).
• Working knowledge of testing tools: PyTest, Selenium, SonarQube.
• Experience with API integrations, booking flows, and payment gateway integrations (travel domain knowledge is a plus, but not mandatory).

Soft Skills

• Strong problem-solving and analytical skills.
• Excellent communication, presentation, and teamwork abilities.
• Proactive, ownership-driven mindset with the ability to perform under pressure.

Roles and Responsibilities

About the Role-

Thinking big and executing beyond what is expected. The challenges cut across algorithmic problem solving, systems engineering, machine learning and infrastructure at a massive scale.

Reason to Join-

An opportunity for innovators, problem solvers & learners.  Working will be Innovative, empowering, rewarding & fun. Amazing Office, competitive pay along with excellent benefits package.

Requiremets and Responsibilities- (please read carefully before applying)

• The overall experience of 3-6 years in Java/Python Framework and Machine Learning.
• Develop Web Services, REST, XSD, XML technologies, Java, Python, AWS, API.
• Experience on Elastic Search or SOLR or Lucene -Search Engine, Text Mining, Indexing.
• Experience in highly scalable tools like Kafka, Spark, Aerospike, etc.
• Hands on experience in Design, Architecture, Implementation, Performance & Scalability, and Distributed Systems.
• Design, implement, and deploy highly scalable and reliable systems.
• Troubleshoot Solr indexing process and querying engine.
• Bachelors or Masters in Computer Science from Tier 1 Institutions

The Sr. Analytics Engineer would provide technical expertise in needs identification, data modeling, data movement, and transformation mapping (source to target), automation and testing strategies, translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective.

Understands and leverages best-fit technologies (e.g., traditional star schema structures, cloud, Hadoop, NoSQL, etc.) and approaches to address business and environmental challenges.

Provides data understanding and coordinates data-related activities with other data management groups such as master data management, data governance, and metadata management.

Actively participates with other consultants in problem-solving and approach development.

Responsibilities :

Provide a consultative approach with business users, asking questions to understand the business need and deriving the data flow, conceptual, logical, and physical data models based on those needs.

Perform data analysis to validate data models and to confirm the ability to meet business needs.

Assist with and support setting the data architecture direction, ensuring data architecture deliverables are developed, ensuring compliance to standards and guidelines, implementing the data architecture, and supporting technical developers at a project or business unit level.

Coordinate and consult with the Data Architect, project manager, client business staff, client technical staff and project developers in data architecture best practices and anything else that is data related at the project or business unit levels.

Work closely with Business Analysts and Solution Architects to design the data model satisfying the business needs and adhering to Enterprise Architecture.

Coordinate with Data Architects, Program Managers and participate in recurring meetings.

Help and mentor team members to understand the data model and subject areas.

Ensure that the team adheres to best practices and guidelines.

Requirements :

- Strong working knowledge of at least 3 years of Spark, Java/Scala/Pyspark, Kafka, Git, Unix / Linux, and ETL pipeline designing.

- Experience with Spark optimization/tuning/resource allocations

- Excellent understanding of IN memory distributed computing frameworks like Spark and its parameter tuning, writing optimized workflow sequences.

- Experience of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., Redshift, Bigquery, Cassandra, etc).

- Familiarity with Docker, Kubernetes, Azure Data Lake/Blob storage, AWS S3, Google Cloud storage, etc.

- Have a deep understanding of the various stacks and components of the Big Data ecosystem.

- Hands-on experience with Python is a huge plus

Cloudera Data Warehouse Hive team looking for a passionate senior developer to join our growing engineering team. This group is targeting the biggest enterprises wanting to utilize Cloudera’s services in a private and public cloud environment. Our product is built on open source technologies like Hive, Impala, Hadoop, Kudu, Spark and so many more providing unlimited learning opportunities. 

A Day in the Life

Over the past 10+ years, Cloudera has experienced tremendous growth making us the leading contributor to Big Data platforms and ecosystems and a leading provider for enterprise solutions based on Apache Hadoop. You will work with some of the best engineers in the industry who are tackling challenges that will continue to shape the Big Data revolution.  We foster an engaging, supportive, and productive work environment where you can do your best work. The team culture values engineering excellence, technical depth, grassroots innovation, teamwork, and collaboration.

You will manage product development for our CDP components, develop engineering tools and scalable services to enable efficient development, testing, and release operations.  You will be immersed in many exciting, cutting-edge technologies and projects, including collaboration with developers, testers, product, field engineers, and our external partners, both software and hardware vendors.

Opportunity:

Cloudera is a leader in the fast-growing big data platforms market. This is a rare chance to make a name for yourself in the industry and in the Open Source world. The candidate will responsible for Apache Hive and CDW projects. We are looking for a candidate who would like to work on these projects upstream and downstream. If you are curious about the project and code quality you can check the project and the code at the following link. You can start the development before you join. This is one of the beauties of the OSS world. 

https://hive.apache.org/" target="_blank">Apache Hive

Responsibilities:

• Build robust and scalable data infrastructure software

• Design and create services and system architecture for your projects 

• Improve code quality through writing unit tests, automation, and code reviews

• The candidate would write Java code and/or build several services in the Cloudera Data Warehouse. 

• Worked with a team of engineers who reviewed each other's code/designs and held each other to an extremely high bar for the quality of code/designs

• The candidate has to understand the basics of Kubernetes.

• Build out the production and test infrastructure. 

• Develop automation frameworks to reproduce issues and prevent regressions.

• Work closely with other developers providing services to our system.

• Help to analyze and to understand how customers use the product and improve it where necessary.  

Qualifications:

• Deep familiarity with Java programming language.

• Hands-on experience with distributed systems. 

• Knowledge of database concepts, RDBMS internals.

• Knowledge of the Hadoop stack, containers, or Kubernetes is a strong plus.  

• Has experience working in a distributed team. 

• Has 3+ years of experience in software development.

• Bachelor’s or master’s degree in Computer Engineering, Computer Science, Computer Applications, Mathematics, Statistics, or related technical field. Relevant experience of at least 3 years in lieu of above if from a different stream of education.

• Well-versed in and 3+ hands-on demonstrable experience with: ▪ Stream & Batch Big Data Pipeline Processing using Apache Spark and/or Apache Flink.
  ▪ Distributed Cloud Native Computing including Server less Functions
  ▪ Relational, Object Store, Document, Graph, etc. Database Design & Implementation
  ▪ Micro services Architecture, API Modeling, Design, & Programming

• 3+ years of hands-on development experience in Apache Spark using Scala and/or Java.

• Ability to write executable code for Services using Spark RDD, Spark SQL, Structured Streaming, Spark MLLib, etc. with deep technical understanding of Spark Processing Framework.

• In-depth knowledge of standard programming languages such as Scala and/or Java.

• 3+ years of hands-on development experience in one or more libraries & frameworks such as Apache Kafka, Akka, Apache Storm, Apache Nifi, Zookeeper, Hadoop ecosystem (i.e., HDFS, YARN, MapReduce, Oozie & Hive), etc.; extra points if you can demonstrate your knowledge with working examples.

• 3+ years of hands-on development experience in one or more Relational and NoSQL datastores such as PostgreSQL, Cassandra, HBase, MongoDB, DynamoDB, Elastic Search, Neo4J, etc.

• Practical knowledge of distributed systems involving partitioning, bucketing, CAP theorem, replication, horizontal scaling, etc.

• Passion for distilling large volumes of data, analyze performance, scalability, and capacity performance issues in Big Data Platforms.

• Ability to clearly distinguish system and Spark Job performances and perform spark performance tuning and resource optimization.

• Perform benchmarking/stress tests and document the best practices for different applications.

• Proactively work with tenants on improving the overall performance and ensure the system is resilient, and scalable.

• Good understanding of Virtualization & Containerization; must demonstrate experience in technologies such as Kubernetes, Istio, Docker, OpenShift, Anthos, Oracle VirtualBox, Vagrant, etc.

• Well-versed with demonstrable working experience with API Management, API Gateway, Service Mesh, Identity & Access Management, Data Protection & Encryption.

  Hands-on experience with demonstrable working experience with DevOps tools and platforms viz., Jira, GIT, Jenkins, Code Quality & Security Plugins, Maven, Artifactory, Terraform, Ansible/Chef/Puppet, Spinnaker, etc.

• Well-versed in AWS and/or Azure or and/or Google Cloud; must demonstrate experience in at least FIVE (5) services offered under AWS and/or Azure or and/or Google Cloud in any categories: Compute or Storage, Database, Networking & Content Delivery, Management & Governance, Analytics, Security, Identity, & Compliance (or) equivalent demonstrable Cloud Platform experience.

• Good understanding of Storage, Networks and Storage Networking basics which will enable you to work in a Cloud environment.

• Good understanding of Network, Data, and Application Security basics which will enable you to work in a Cloud as well as Business Applications / API services environment.