
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
at Global digital transformation solutions provider.
JOB DETAILS:
* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka
* Industry: Global digital transformation solutions provider
* Salary: Best in Industry
* Experience: 5-8 years
* Location: Hyderabad
Job Summary
We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.
Key Responsibilities
ETL Pipeline Development & Optimization
- Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
- Optimize data pipelines for performance, scalability, fault tolerance, and reliability.
Big Data Processing
- Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
- Ensure fault-tolerant, scalable, and high-performance data processing systems.
Cloud Infrastructure Development
- Build and manage scalable, cloud-native data infrastructure on AWS.
- Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.
Real-Time & Batch Data Integration
- Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
- Ensure consistency, data quality, and a unified view across multiple data sources and formats.
Data Analysis & Insights
- Partner with business teams and data scientists to understand data requirements.
- Perform in-depth data analysis to identify trends, patterns, and anomalies.
- Deliver high-quality datasets and present actionable insights to stakeholders.
CI/CD & Automation
- Implement and maintain CI/CD pipelines using Jenkins or similar tools.
- Automate testing, deployment, and monitoring to ensure smooth production releases.
Data Security & Compliance
- Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
- Implement data governance practices ensuring data integrity, security, and traceability.
Troubleshooting & Performance Tuning
- Identify and resolve performance bottlenecks in data pipelines.
- Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.
Collaboration & Cross-Functional Work
- Work closely with engineers, data scientists, product managers, and business stakeholders.
- Participate in agile ceremonies, sprint planning, and architectural discussions.
Skills & Qualifications
Mandatory (Must-Have) Skills
- AWS Expertise
- Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
- Strong understanding of cloud-native data architectures.
- Big Data Technologies
- Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
- Experience with Apache Spark and Apache Kafka in production environments.
- Data Frameworks
- Strong knowledge of Spark DataFrames and Datasets.
- ETL Pipeline Development
- Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
- Database Modeling & Data Warehousing
- Expertise in designing scalable data models for OLAP and OLTP systems.
- Data Analysis & Insights
- Ability to perform complex data analysis and extract actionable business insights.
- Strong analytical and problem-solving skills with a data-driven mindset.
- CI/CD & Automation
- Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
- Familiarity with automated testing and deployment workflows.
Good-to-Have (Preferred) Skills
- Knowledge of Java for data processing applications.
- Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
- Familiarity with data governance frameworks and compliance tooling.
- Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
- Exposure to cost optimization strategies for large-scale cloud data platforms.
Skills: big data, scala spark, apache spark, ETL pipeline development
******
Notice period - 0 to 15 days only
Job stability is mandatory
Location: Hyderabad
Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer
F2F Interview: 14th Feb 2026
3 days in office, Hybrid model.

Similar jobs
Strong Partner Sales Profile (Cloud / Enterprise Ecosystem)
Mandatory (Total Experience): Must have 1+years of B2B sales experience in Cloud / IT Services OR partner / enterprise solution sales exposure within BFSI, Manufacturing, or Pharmaceutical sectors
Mandatory (Experience 1 – Partner / Channel Sales): Must have experience working with channel partners, system integrators, resellers, or strategic alliances
Mandatory (Experience 2 – Partner-Led Deal Execution): Must have supported or driven partner-led sales cycles, including requirement alignment, proposal coordination, and deal closure
Mandatory (Communication & Stakeholder Management): Must be able to coordinate across partners, internal teams, and end customers effectively
Mandatory (Education Background): Must have a technical or IT-oriented education background —BE / BSc IT / MSc IT is mandatory
JOB TITLE: ASSOCIATE MANAGER - TELUGU PROGRAMMING
LOCATION: NAVI MUMBAI, INDIA (Remote)
RESPONSIBILITIES:
Assist in managing the daily Telugu programming and curation strategy. Working knowledge of
Kannada + Malayalam is a big plus
Basic data analysis and number crunching is an important part of this role
Work with the Programming Teams as well as other cross-functional teams to achieve various
organizational objectives
Act as a liaison between various departments to ensure the on time and orderly flow of work
Curate and maintain a substantial number of varied playlists based on different Moods, Genres
and Activities.
Think of creative ideas & concepts for Festivals, events & Trending Topics
Stay on top of new music releases
Help find ways to organize, facilitate, and improve general programming processes
Create and update departmental calendar and project status
Keep entire team informed on the current status of all jobs in progress via status meetings
Keep abreast of all work in the Programming Department and follow up on due dates that were
established for copy, artwork, playlists and other deliverables
REQUIREMENTS:
Bachelor’s/ Master’s degree
Proficient in Telugu – reading and understanding
3-5 years’ experience in the music industry, ideally programming for music services, working
with a broad range of music content
Basic data analysis and number crunching
Passionate about music and have deep knowledge of music culture (Telugu).
Highly organized, creative self-starter with experience working on multiple projects at the same
time
Care about attention to detail and organization, have a team spirit, strong work ethic, and a
sense of humor
Exceptional time management and prioritization skills
Out-of-the-box thinking and ideas that can significantly contribute to Company’s success.
BENEFITS AND PERKS:
We blur work and play, and you get all the perks of a global company. You will get to
work with a dynamic group of entrepreneurs, who are delivering results and working zealously to
make a difference in the way the world experiences music.
We are looking out for a technically driven "ML OPS Engineer" for one of our premium client
COMPANY DESCRIPTION:
Key Skills
• Excellent hands-on expert knowledge of cloud platform infrastructure and administration
(Azure/AWS/GCP) with strong knowledge of cloud services integration, and cloud security
• Expertise setting up CI/CD processes, building and maintaining secure DevOps pipelines with at
least 2 major DevOps stacks (e.g., Azure DevOps, Gitlab, Argo)
• Experience with modern development methods and tooling: Containers (e.g., docker) and
container orchestration (K8s), CI/CD tools (e.g., Circle CI, Jenkins, GitHub actions, Azure
DevOps), version control (Git, GitHub, GitLab), orchestration/DAGs tools (e.g., Argo, Airflow,
Kubeflow)
• Hands-on coding skills Python 3 (e.g., API including automated testing frameworks and libraries
(e.g., pytest) and Infrastructure as Code (e.g., Terraform) and Kubernetes artifacts (e.g.,
deployments, operators, helm charts)
• Experience setting up at least one contemporary MLOps tooling (e.g., experiment tracking,
model governance, packaging, deployment, feature store)
• Practical knowledge delivering and maintaining production software such as APIs and cloud
infrastructure
• Knowledge of SQL (intermediate level or more preferred) and familiarity working with at least
one common RDBMS (MySQL, Postgres, SQL Server, Oracle)
-Design and build advanced applications from scratch keeping scalability and performance in mind.
-You’ll be working on our P2P and cryptocurrency exchange platforms to build more features around it.
-Collaborating with various stakeholders to determine software requirements
being a financial product, you’ll be involved in building tools to make our user’s data more secure.
-You will be working on blockchain technologies
-You will help maintain code quality and organization.
-Should be able to work with juniors; help and guide them as needed.
-We expect you to learn about new software technologies, and proactively look for ways to make them better.
#You'll be a good fit if:
-You’ve got 3+ years of development experience and an expert in the Ruby and Rails framework.
-You’re a Hands-on expert who can code, architect/design systems, and understand building highly available and scalable microservices on cloud infrastructure.
-You have a very good understanding of Containers and Container Orchestration Technology such as Docker / Kubernetes and AWS Cloud Platform exposure.
-You’ve hands-on experience in using DevOps tools such as Newrelic, Jenkins, Grafana, ELK, etc
-You’re an expert in writing background tasks, test-driven development.
-You enjoy collaborating with the front-end developers, DevOps, and other team members to establish objectives and design more functional, cohesive codes to enhance the user experience.
-You’re committed to quality, and your code solves more problems than it creates ;)
NET Profile
DESIRED SKILLS
· Experience in Microsoft technologies (Microsoft Dot Net, C#, http://asp.net/" target="_blank">ASP.NET, MVC, Microsoft SQL, .NET Core, Web API, WCF, Bootstrap, JavaScript, Angular JS, Web API, Web services, Rest API)
· Design a complex architecture landscape involving API integrations.
· Hands-on experience in ASP.Net, C#, MVC, WCF, Restful services, Web API, AngularJS, Entity Framework
· Strong analytical & problem-solving skills
· Must have excellent oral & email communication skills.
· Strong analytical and problem-solving skills
· Must have a positive attitude and a yearning to get things done and demonstrate ownership of deliverables.
NICE TO HAVE SKILLS
· Knowledge in Microsoft Azure.
· Good Command on Agile Development & Azure DevOps.
· Knowledge in Angular / React / Node JS
· Mainly they are looking for:
C#
http://asp.net/" target="_blank">ASP.NET MVC,
AJAX,
LINQ,
jQuery,
WebApi.
Big Data Engineer/Data Engineer
What we are solving
Welcome to today’s business data world where:
• Unification of all customer data into one platform is a challenge
• Extraction is expensive
• Business users do not have the time/skill to write queries
• High dependency on tech team for written queries
These facts may look scary but there are solutions with real-time self-serve analytics:
• Fully automated data integration from any kind of a data source into a universal schema
• Analytics database that streamlines data indexing, query and analysis into a single platform.
• Start generating value from Day 1 through deep dives, root cause analysis and micro segmentation
At Propellor.ai, this is what we do.
• We help our clients reduce effort and increase effectiveness quickly
• By clearly defining the scope of Projects
• Using Dependable, scalable, future proof technology solution like Big Data Solutions and Cloud Platforms
• Engaging with Data Scientists and Data Engineers to provide End to End Solutions leading to industrialisation of Data Science Model Development and Deployment
What we have achieved so far
Since we started in 2016,
• We have worked across 9 countries with 25+ global brands and 75+ projects
• We have 50+ clients, 100+ Data Sources and 20TB+ data processed daily
Work culture at Propellor.ai
We are a small, remote team that believes in
• Working with a few, but only with highest quality team members who want to become the very best in their fields.
• With each member's belief and faith in what we are solving, we collectively see the Big Picture
• No hierarchy leads us to believe in reaching the decision maker without any hesitation so that our actions can have fruitful and aligned outcomes.
• Each one is a CEO of their domain.So, the criteria while making a choice is so our employees and clients can succeed together!
To read more about us click here:
https://bit.ly/3idXzs0" target="_blank">https://bit.ly/3idXzs0
About the role
We are building an exceptional team of Data engineers who are passionate developers and wants to push the boundaries to solve complex business problems using the latest tech stack. As a Big Data Engineer, you will work with various Technology and Business teams to deliver our Data Engineering offerings to our clients across the globe.
Role Description
• The role would involve big data pre-processing & reporting workflows including collecting, parsing, managing, analysing, and visualizing large sets of data to turn information into business insights
• Develop the software and systems needed for end-to-end execution on large projects
• Work across all phases of SDLC, and use Software Engineering principles to build scalable solutions
• Build the knowledge base required to deliver increasingly complex technology projects
• The role would also involve testing various machine learning models on Big Data and deploying learned models for ongoing scoring and prediction.
Education & Experience
• B.Tech. or Equivalent degree in CS/CE/IT/ECE/EEE 3+ years of experience designing technological solutions to complex data problems, developing & testing modular, reusable, efficient and scalable code to implement those solutions.
Must have (hands-on) experience
• Python and SQL expertise
• Distributed computing frameworks (Hadoop Ecosystem & Spark components)
• Must be proficient in any Cloud computing platforms (AWS/Azure/GCP) • Experience in in any cloud platform would be preferred - GCP (Big Query/Bigtable, Pub sub, Data Flow, App engine )/ AWS/ Azure
• Linux environment, SQL and Shell scripting Desirable
• Statistical or machine learning DSL like R
• Distributed and low latency (streaming) application architecture
• Row store distributed DBMSs such as Cassandra, CouchDB, MongoDB, etc
. • Familiarity with API design
Hiring Process:
1. One phone screening round to gauge your interest and knowledge of fundamentals
2. An assignment to test your skills and ability to come up with solutions in a certain time
3. Interview 1 with our Data Engineer lead
4. Final Interview with our Data Engineer Lead and the Business Teams
Preferred Immediate Joiners
Requirements:
- 3+ years’ experience under your belt developing performance-critical applications that run in a production environment using Node.Js
- Expertise in Database modeling, preferably relational DB, spotting DB bottlenecks, and writing optimized DB queries.
- Expertise in common Cloud technologies, AWS/GCP Knowledge of CI/CD tools and workflows.
- Knowledge in one or more of the following: NoSQL (especially, but not only Redis), Queueing system experience (Kafka, RabbitMQ, ActiveMQ, MSMQ)
- Good stakeholder management skills, with an ability to build trust and rapport with internal and external stakeholders
- It is good to have knowledge of Kubernetes and Docker and related deployment technologies.
Roles & Responsibilities:
- Write well-designed, testable, efficient, and high-quality code.
- Design (LLD & HLD) and architect technical solutions for the business problems of a large-scale system.
- Prepare and produce releases of software components.
- Support continuous improvement by investigating alternatives and technologies.
- Contribute to all phases of the development lifecycle
- We believe leadership comes through ownership and so we love people who take ownership of their work and lead by example
Android Developer JD @Tailwebs
We are looking for an Android Developer who possesses a passion for pushing
mobile technologies to the limits and will work with our team of talented engineers
to design and build the next generation of our mobile applications.
Responsibilities:
-
Design, develop, modify, and implement mobile applications using Android.
-
Implement creative and usable user interfaces.
-
Collaborate with cross-functional teams to define, design, and ship new features
-
Work on bug fixing and improving application performance.
-
Continuously discover, evaluate, and implement new technologies to maximize development
efficiency.
Required Technical Skills:
-
Object-Oriented Programming Skills
-
Strong in Core Java, and Android SDK implementation & good knowledge on Web Services,
Web APIs & JSON- Kotlin
-
Strong debugging & troubleshooting skills.
-
Good understanding of Android frameworks like Activity Manager, Resource Manager, Location
Manager, and Notification Manager
-
1 to 3 years, fresher’s with proven development skills considered.
Job Type: Full-time
Salary offered: INR 250,000 – INR 600,000
Team Manager – Direct Sales, Toppr.com Location: PAN India https://toppr.breezy.hr/
About Us
Toppr is an adaptive after-school learning app for classes 5th to 12th. Our vision is to make learning personalized using technology. We believe each child is unique with distinct learning needs.
We are building Toppr as an adaptive platform that can increase engagement, optimize learning and improve outcomes. We have the widest syllabus coverage in India with over 20 subjects, 20 boards and 50 exams. Over 30 lakh students use the learning app.
Toppr was started in 2013 by IIT Bombay alumni ZishaanHayath and HemanthGoteti and is backed by venture capital firms SAIF, Helion and Eight Roads.
If the future of education interests you, we would love to talk about what you can bring to our team.
About the Team
We are the face of Toppr for all our students and their parents.
We help them understand how to leverage Toppr platform for their learning We strive to take Toppr to every single household in India; and then the world.
Role & Responsibilities
- Provide the team with a vision and objectives; manage key performance indicators.
- Set the monthly, weekly, daily targets for the team and ensure that the team targets are achieved.
- Handling and guiding team of Academic Counselor to achieve monthly sales target.
- Maintain the sales report of the team.
- Motivating & mentoring team to achieve & exceed targets
- Design & develop business models as per the location and market situation.
- Conducting weekly reviews for performance & training.
- Identifying the areas of improvements & KPI’s
- Involve team in calling for negotiation and objection handling as and when required.
Team Managers at Toppr
- Are dynamic professionals who drive their team to exceed business targets
- Possess sound knowledge and understanding of consumer (students and parents) behavior
- Thrive in a fast paced ambiguous work environment
- Have excellent communication skills in English and the regional language
Qualification & Skills
- MBA or Graduate with at least 5 years B2C sales experience
- Minimum 1 years’ experience of managing a team of 10 – 20 sales executives
- Experience in to Data Analysis; proficient with using Excel









