
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
at Global digital transformation solutions provider.
JOB DETAILS:
* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka
* Industry: Global digital transformation solutions provider
* Salary: Best in Industry
* Experience: 5-8 years
* Location: Hyderabad
Job Summary
We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.
Key Responsibilities
ETL Pipeline Development & Optimization
- Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
- Optimize data pipelines for performance, scalability, fault tolerance, and reliability.
Big Data Processing
- Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
- Ensure fault-tolerant, scalable, and high-performance data processing systems.
Cloud Infrastructure Development
- Build and manage scalable, cloud-native data infrastructure on AWS.
- Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.
Real-Time & Batch Data Integration
- Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
- Ensure consistency, data quality, and a unified view across multiple data sources and formats.
Data Analysis & Insights
- Partner with business teams and data scientists to understand data requirements.
- Perform in-depth data analysis to identify trends, patterns, and anomalies.
- Deliver high-quality datasets and present actionable insights to stakeholders.
CI/CD & Automation
- Implement and maintain CI/CD pipelines using Jenkins or similar tools.
- Automate testing, deployment, and monitoring to ensure smooth production releases.
Data Security & Compliance
- Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
- Implement data governance practices ensuring data integrity, security, and traceability.
Troubleshooting & Performance Tuning
- Identify and resolve performance bottlenecks in data pipelines.
- Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.
Collaboration & Cross-Functional Work
- Work closely with engineers, data scientists, product managers, and business stakeholders.
- Participate in agile ceremonies, sprint planning, and architectural discussions.
Skills & Qualifications
Mandatory (Must-Have) Skills
- AWS Expertise
- Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
- Strong understanding of cloud-native data architectures.
- Big Data Technologies
- Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
- Experience with Apache Spark and Apache Kafka in production environments.
- Data Frameworks
- Strong knowledge of Spark DataFrames and Datasets.
- ETL Pipeline Development
- Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
- Database Modeling & Data Warehousing
- Expertise in designing scalable data models for OLAP and OLTP systems.
- Data Analysis & Insights
- Ability to perform complex data analysis and extract actionable business insights.
- Strong analytical and problem-solving skills with a data-driven mindset.
- CI/CD & Automation
- Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
- Familiarity with automated testing and deployment workflows.
Good-to-Have (Preferred) Skills
- Knowledge of Java for data processing applications.
- Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
- Familiarity with data governance frameworks and compliance tooling.
- Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
- Exposure to cost optimization strategies for large-scale cloud data platforms.
Skills: big data, scala spark, apache spark, ETL pipeline development
******
Notice period - 0 to 15 days only
Job stability is mandatory
Location: Hyderabad
Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer
F2F Interview: 14th Feb 2026
3 days in office, Hybrid model.

Similar jobs
Position: Customer Service Executive
Company: Technomine Group (UK Travel Operations)
About Us:
Technomine Group is a global outsourcing and operations partner supporting clients across the UK, US, and Canada. Our UK-based travel division delivers premium travel support and customer experience management. We are now welcoming fresh, enthusiastic minds to join us as Travel Desk Admin Executives for our UK travel process.
Role Overview:
An incredible opportunity for freshers to gain international exposure in travel operations! You’ll handle backend travel coordination, ensure smooth booking processes, assist with travel administration, and collaborate with UK teams for seamless operations.
Key Responsibilities:
- Manage travel bookings, reservations, and related documentation.
- Handle customer queries via email, chat, or phone.
- Maintain accurate travel records, itineraries, and invoices.
- Coordinate with UK counterparts to ensure process adherence.
- Work in rotational shifts to support UK operations.
Required Skills & Qualifications:
- Graduate in any discipline (Freshers welcome!)
- Good command over English (written & verbal)
- Basic computer proficiency (MS Office, emails, data entry)
- Strong attention to detail and willingness to learn
- Comfortable working in rotational shifts
What We Offer:
- 5 days working with rotational shifts
- Opportunity to work with a UK-based travel company
- Global exposure & career growth within Technomine Group
- Food provided, Leave Encashment, Provident Fund, Accommodation
Work Type: Full-time, Permanent
Location: Navrangpura, Ahmedabad, Gujarat (In-person role)
Excellent verbal & written communication skills.
Manage Email Process- Payment, Refunds, Cancellation etc.
Responsible to provide timely & effective resolutions to customers through emails, chat, etc.
Knowledge of payments and fintech would be plus
Perks and benefits
Ownership and Responsibility.Medical Insurance.
- Software Developer with minimum 2+ years of development experience in .net
- Write effective and reusable code using the MVVM framework
- Expertise in .net core, OOPS concept and .net coding guidelines
- Solid understanding of Object-Oriented design patterns
- Good in logical and problem solving skills
- Good analytical and debugging skills
- Ability to understand requirements
- Knowledge of unit testing and code analysis tools will be added advantage
- Knowledge of Azure services will be added advantage
- Knowledge of working with XML will be added advantage
Angular
- Looking for an Angular Developer who has deep knowledge of Angular practices and commonly used modules based on extensive work experience.
- Should be flexible to adapt quickly to new technologies as required.
- Hands on and implementation of Angular applications, directives, controllers, services
- Thorough understanding of the responsibilities of the platform, database, API, caching layer, proxies, and other web services used in the system
- Good experience on OOPS Design skills.
- Experience with consuming RESTful services
- Creating self-contained, reusable, and testable modules and components
- Should have good knowledge of the MVVM frameworks
- Exposure to Microsoft Azure is an added advantage
- Critical thinker and problem-solving skills
- Team player
- Great interpersonal and communication skills
- Should have strong programming knowledge in Angular (Angular v8+) with at least 3+ years of relevant experience
Backend Development - Intern
Converts into a full time position based on performance
A Backend Developer, for an early-stage Analytics startup who can help us translate Product specs into a high-performing and scalable product across platforms. Also help in improving functionality of product. End goal is to achieve product-market objectives
The candidate will have rich learning into what goes into developing, scaling a product from scratch. A chance to grow into a Backend Development Owner in medium-long term
Responsibilities
-
Own Back End Development closely with Product Owner
-
Design, implement and maintain product modules/sub-systems in your scope
-
Perform bug verification, release testing and support for assigned products.
-
Conduct unit testing and integration testing.
-
Make sure the delivered product matches the business requirements.
Expectations from candidate
-
Ability: Good at logical thinking, problem solving
-
Inclination: Curious and strong willingness to learn from problems, write good code
-
Knowledge: Basics of Back End Development (Some academic degree/certification
or Prior experience/Projects preferred)
-
Languages/Tool/Framework: Python, MongoDB, GraphQL, GoLang
-
Interest in product focused work in a fast-moving start-up environment
-
Attitude: We are looking for a person with an owner mindset who can partner with
us in building our product with a lot of commitment. We like people who are curious problem solvers and are interested in learning with us and working hard.
What we offer
-
Great culture focused on outcomes through learning and experimentation
-
Mentoring by experienced professionals from pedigreed backgrounds (IIT/IIMs)
-
Growth, ownership, and flexibility to achieve your goals.
Desired Profile:
- Excellent communication and problem-solving skills,
- Strong analytical skills Ability to turn business requirements into technical concepts and code,
- Minimum 3 - 6 years experience in software development in general with 3 years in complex web applications using SAP UI5,
- Software architecture knowledge of enterprise applications is a must
- Experience with Web - Technologies (SAP UI5, HTML5, CSS3, JS, REST - WS) is a must
- Experience with SAP Mobile Platform as well as SAP Web IDE is a plus
- Knowledge and experience with user experience and design concepts is a plus
ZYCUS OVERVIEW:
Zycus is looking out for Seasoned Sales professionals, with a Software sales experience of about 1-5 years, and hands-on experience in the generation of leads and new business opportunities. Headquartered in Princeton, the U.S. in 1998, Zycus has grown every day to be established as an organization which now is a leading global provider of complete Source-to-Pay suite of procurement performance solutions.
We develop cloud-based (SaaS) Source-to-Pay solutions for large global enterprises, and have successfully deployed about 200 solutions to over 1000 Global clients. Our spirit of innovation and our passion to help procurement create greater business impact are reflected among procurement solution deployments that we have undertaken over the years. We are proud to have as our clients, some of the best-of-breed companies across verticals like Manufacturing, Automotives, Banking and Finance, Oil and Gas, Food Processing, Electronics, Telecommunications, Chemicals, Health and Pharma, Education and more. With a team of 1200+ employees, we are present in India with 3 development centers at Bengaluru, Mumbai & Pune and offices in the U.S. (New Jersey, Atlanta, Chicago), U.K., Australia, Dubai, Netherland, and Singapore.
Know more about the LEADER of Gartner's 2013, 2015 & 2017 Magic Quadrant for Strategic Sourcing Application Suites and The Forrester Wave-: eProcurement, Q2 2017
MARKET SIZEUSD 5 Billion and grows tremendously during economic downturns
Roles & Responsibilities:-
- The incumbent will report to Manager/Sr. Manager - Business Development and will be Executive/Sr. Executive depending upon the relevance and no. of years of experience.-
- Collaborating with Regional Sales Managers to accelerate the influx of business opportunities in the assigned territory.
- Reaching out via Phone/Email/Social Media to connect with prospective customers.- Conducting secondary research into company financials, line of business, technology requirements, etc., to personalize prospecting efforts-
- Staying up to date with the latest procurement industry trends & best practices- Competitor analysis & positioning against them-
- Delivering first-level Corporate Overview Presentations- Pre-qualification of business opportunities (Gauging- Budget, Authority, Need & Timeline)-
- Strategizing cross-functional campaigns to be conducted by BD & Marketing Necessary
- Educational Qualification:
- Graduation is a must (B.E.and MBA referred)-
- Excellent verbal & written communication skills in English (Public Speaking background is preferable)-
- Soft Skills (Pitching, Probing, Objection Handling, Rapport Building)-
- Strong, in-depth analytical skill-
- Proficient in professional business conversational etiquette-
- Account Mapping & Social Selling ability-
- Networking & Team Management
- Work Location: Mumbai
- Education: Any Graduate/ Post Graduate
Benefits
- LOCATION: Mumbai
- COMPENSATION: Competitive
- WHY ZYCUS? :- Be a part of one of the fastest-growing product Companies in India-
- Come join a young, dynamic & enterprising team-
- Work on the latest technologies-
- Flexible working hours (As per business requirement).
Designation: Web consultant/ web caller/ Website sales executive/ CCE.
Process: International B2B website process.
- Fresher candiates can also apply for the Job.
- Fresher candidates with flueny in english communication required.
- At least 3 to 6 months Exp. required in web selling,
Knowledge in Website sales, SEO/ GLL sales preferred.
#seo #insidesales #voiceprocess #kolkata #internationalbpo #cce #webconsultant #kolkatajobs #websitesales #websiteprocess #seosales #b2bcampaign #searchengineoptimization #gllsales #websitecampaign
- Design, maintain and supply content for the organization's website.
- Formulate strategies to build a lasting digital connection with customers.
- A monitor company presence on social media.
- Launch advertisements to increase brand awareness.
- Bachelor's degree in Marketing or related field.
- Excellent understanding of digital marketing concepts.
- Experience with business to customer social media and content generation.
- Strong creative and analytical skills.
- Analytical Skills
- Social Media Marketing
- Marketing Campaigns
- Marketing Strategy
- Online Marketing
- Brand Awareness
- Content Strategy
- Content Management Systems (CMS)
- SEO
- Google Ads
- English Proficiency (Verbal and Written)
2. Experience of integrating any of the SDKs or third party libraries.
3. Knowledge of Android Software design patterns, Android/Media Frameworks.
4. Experience with NDK and JNI
5. Basic knowledge of C & C++ language/libraries (optional)








