Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
at Global digital transformation solutions provider.
Amazon Web Services (AWS)
Scala
Jenkins
Data Analytics
Data Science
Java
MongoDBJOB DETAILS:
* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka
* Industry: Global digital transformation solutions provider
* Salary: Best in Industry
* Experience: 5-8 years
* Location: Hyderabad
Job Summary
We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.
Key Responsibilities
ETL Pipeline Development & Optimization
- Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
- Optimize data pipelines for performance, scalability, fault tolerance, and reliability.
Big Data Processing
- Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
- Ensure fault-tolerant, scalable, and high-performance data processing systems.
Cloud Infrastructure Development
- Build and manage scalable, cloud-native data infrastructure on AWS.
- Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.
Real-Time & Batch Data Integration
- Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
- Ensure consistency, data quality, and a unified view across multiple data sources and formats.
Data Analysis & Insights
- Partner with business teams and data scientists to understand data requirements.
- Perform in-depth data analysis to identify trends, patterns, and anomalies.
- Deliver high-quality datasets and present actionable insights to stakeholders.
CI/CD & Automation
- Implement and maintain CI/CD pipelines using Jenkins or similar tools.
- Automate testing, deployment, and monitoring to ensure smooth production releases.
Data Security & Compliance
- Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
- Implement data governance practices ensuring data integrity, security, and traceability.
Troubleshooting & Performance Tuning
- Identify and resolve performance bottlenecks in data pipelines.
- Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.
Collaboration & Cross-Functional Work
- Work closely with engineers, data scientists, product managers, and business stakeholders.
- Participate in agile ceremonies, sprint planning, and architectural discussions.
Skills & Qualifications
Mandatory (Must-Have) Skills
- AWS Expertise
- Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
- Strong understanding of cloud-native data architectures.
- Big Data Technologies
- Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
- Experience with Apache Spark and Apache Kafka in production environments.
- Data Frameworks
- Strong knowledge of Spark DataFrames and Datasets.
- ETL Pipeline Development
- Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
- Database Modeling & Data Warehousing
- Expertise in designing scalable data models for OLAP and OLTP systems.
- Data Analysis & Insights
- Ability to perform complex data analysis and extract actionable business insights.
- Strong analytical and problem-solving skills with a data-driven mindset.
- CI/CD & Automation
- Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
- Familiarity with automated testing and deployment workflows.
Good-to-Have (Preferred) Skills
- Knowledge of Java for data processing applications.
- Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
- Familiarity with data governance frameworks and compliance tooling.
- Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
- Exposure to cost optimization strategies for large-scale cloud data platforms.
Skills: big data, scala spark, apache spark, ETL pipeline development
******
Notice period - 0 to 15 days only
Job stability is mandatory
Location: Hyderabad
Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer
F2F Interview: 14th Feb 2026
3 days in office, Hybrid model.
Shubham Vishwakarma
Full Stack Developer - Averlon
Similar jobs
Industry: Travel & Tourism only
Experience: 2–4 Years
Location: Malad, Mumbai (Work from Office)
Job Description:
We are hiring a Ticketing Operations (Travel Consultant) with hands-on experience
in flight bookings, reissues, refunds, and fare rules. The role requires strong
knowledge of GDS systems and airline procedures, supporting both B2B and B2C
ticketing operations.
Required Skills & Qualifications:
• 2–3 years of experience in travel consulting with focus on group bookings
(B2B) • 2 -3 years of hands-on experience of GDS systems -Amadeus, Galileo,
Sabre • Excellent communication and negotiation skills
NodeJS (Node.js)We are hiring a SAP BTP CAPM Consultant with a minimum of 6 years of experience in building applications using SAP’s Cloud Application Programming Model (CAPM). Strong hands-on experience with Node.js or Java in BTP, OData services, and deployment in Cloud Foundry is required. Experience with SAP HANA DB and BAS (Business Application Studio) is preferred.
.NET
C#Years of experience-3 to 5 Years
skills set-.net mvc,asp.net mvc,dot net mvc developer
AngularJS (1.x)
Angular (2+)
React.jsNodeJS (Node.js)
JavascriptRequirements:
1. 3 to 10 years of experience in full stack development.
2. Strong proficiency in Node.js and React.js, with a deep understanding of their ecosystems.
3. Solid understanding of web development principles, including RESTful APIs and responsive design.
Python
Django
Swagger
FlaskWe are looking for an experienced Senior API Developer with expertise in Python to join our talented development team. The candidate should have over 8 years of hands-on experience in designing and developing APIs, with a strong emphasis on performance and scalability. Proficiency with Swagger/OpenAPI specifications is required. This role is pivotal to our mission of delivering top-tier solutions that power our core business functions.
Key Responsibilities:
- API Development: Design, develop, and maintain high-quality APIs using Python frameworks such as Django Rest Framework or FastAPI.
- Documentation with Swagger/OpenAPI: Employ Swagger/OpenAPI tools to produce clear and thorough API documentation.
- Integration: Implement and manage integrations between APIs and various internal and external services.
- Optimize Performance: Monitor and enhance the performance of APIs to ensure fast response times and low latency.
- Team Collaboration: Work closely with other developers, product owners, and stakeholders to understand requirements and deliver optimal solutions.
- Code Quality: Uphold best practices in code quality, testing, and deployment. Participate in peer code reviews.
- Debugging and Support: Troubleshoot and resolve issues in development, test, and production environments.
- Continuous Learning: Stay informed about emerging technologies and methodologies in Python and API development.
Qualifications:
- Education: Bachelor's degree in Computer Science, Information Technology, or a related discipline.
- Experience: At least 8 years of professional experience in API development with a focus on Python.
Technical Skills:
- Proficiency in Python programming language.
- Strong experience with API design, development, and RESTful services.
- Expertise with Swagger/OpenAPI specifications.
- Familiarity with Python frameworks like Django, Flask, or FastAPI.
- Experience with ORMs such as SQLAlchemy or Django ORM.
- Knowledge of version control systems, particularly Git.
- Experience working with relational databases like PostgreSQL or MySQL.
Soft Skills:
- Excellent problem-solving and analytical skills.
- Strong verbal and written communication.
- Ability to collaborate effectively within a team and across departments.
Preferred Qualifications:
- Experience with cloud platforms like AWS, GCP, or Azure.
- Knowledge of microservices architecture and distributed systems.
- Familiarity with CI/CD pipelines and tools like Jenkins or GitLab CI.
- Understanding of asynchronous programming and event-driven architecture.
What We Offer:
- Competitive Compensation: Reflective of your experience and the value you bring.
- Growth Opportunities: Support for professional development, training, and certifications.
- Flexible Work Environment: Remote work options and flexible hours to promote work-life balance.
NodeJS (Node.js)
Express
PostgreSQL
Elastic SearchAmazon Web Services (AWS)Responsibilities :
- Knowledge of Node.js and frameworks available for it {such as NestJS, Express, StrongLoop, etc.}
- Experience in Database, Postgres Preferred.
- Integration of user-facing elements developed by front-end developers with server-side logic
- Writing reusable, testable, and efficient code
- Design and implementation of low-latency, high-availability, and performant applications
- Implementation of security and data protection
- Integration of data storage solutions
- Strong proficiency with JavaScript
BPT/MPT**
General Task**
New patient assessments, creating care plans, doing regular follow-ups.
Help accident victims, medical patients, and chronically disabled people maintain and improve their physical well-being by designing, implementing, and supervising custom treatment plans.
Supervise patients' health progress and make adjustments to treatment plans as needed.
Help patients maintain or restore flexibility, strength, and range of motion
Help people affected by injury, illness or disability through movement and exercise, manual therapy, education and advice.
They maintain health for people of all ages, helping patients to manage pain and prevent disease.
Maintaining electronic medical records and sharing with other specialties or HOD when required.
Developing protocols for assessment and management.
Maintaining patient database and service details.
Constantly updating knowledge and practical skills in the field of rehabilitation.
Training customer care and sales and marketing team regarding the product.
Assist in implementing and standardizing treatment and assessment protocols.
Role and Skill of Physiotherapy **
Willingness to work for a Home Care environment.
Need to focus on restoring physical function.
Need to be sensitive to the patients' needs and disabilities and must have excellent communication and motivational skills.
Create positive impact in the prevention of subsequent acute events and support the individual’s ability to live independently through targeted interventions aimed at improving balance, strength, coordination and function.
Should personally also be physically fit enough to properly demonstrate the activities in their treatment plans and must possess strong organizational skills.
Should possess either of the areas of expertise: pain management, sports injury management, paediatric therapy, post-operative therapy, arthritis management, neuro or posture correction.
The conditions to be treated are back pain, neck pain, shoulder pain, knee pain, arthritic pain, slipped disc and any other kind of sprains.
Creating an impact on overall patient experience by
Targeting the recovery of physical function of stroke patients through low risk activities resulting in high patient satisfaction.
Providing a service which reduces the risk for poor health outcomes, increases daily living and personal activity.
PHP
CodeIgniter
jQuery- Developing and maintaining dynamic websites and web applications
- Ensure foolproof performance of the deliverable
- Coordinate with co-developers and other related departments.
- Experience own and work on end to end on an application.
- Set Milestones and deadline.
- Fine-tuning and resolve performance issues
- Mentor colleagues and code walk through.
Must have Skills:
PHP, OOP, codeigniter, Jquery, MySQL
Location: Job Location : Senapati Bapat Marg, Elphinstone Road.
LeadershipJavascriptPythonPHPAngularJS (1.x)
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon