Cutshort logo
Inncircles  logo
Sr. Big Data Engineer
Sr. Big Data Engineer
Inncircles 's logo

Sr. Big Data Engineer

Gangadhar M's profile picture
Posted by Gangadhar M
3 - 5 yrs
Best in industry
Hyderabad
Skills
PySpark
Spark
skill iconPython
ETL
Amazon EMR
Amazon S3
SQL
NOSQL Databases
athena
Apache Kafka
Apache Spark
skill iconMongoDB


We are looking for a highly skilled Sr. Big Data Engineer with 3-5 years of experience in

building large-scale data pipelines, real-time streaming solutions, and batch/stream

processing systems. The ideal candidate should be proficient in Spark, Kafka, Python, and

AWS Big Data services, with hands-on experience in implementing CDC (Change Data

Capture) pipelines and integrating multiple data sources and sinks.


Responsibilities

  • Design, develop, and optimize batch and streaming data pipelines using Apache Spark and Python.
  • Build and maintain real-time data ingestion pipelines leveraging Kafka and AWS Kinesis.
  • Implement CDC (Change Data Capture) pipelines using Kafka Connect, Debezium or similar frameworks.
  • Integrate data from multiple sources and sinks (databases, APIs, message queues, file systems, cloud storage).
  • Work with AWS Big Data ecosystem: Glue, EMR, Kinesis, Athena, S3, Lambda, Step Functions.
  • Ensure pipeline scalability, reliability, and performance tuning of Spark jobs and EMR clusters.
  • Develop data transformation and ETL workflows in AWS Glue and manage schema evolution.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable and high-quality data solutions.
  • Implement monitoring, logging, and alerting for critical data pipelines.
  • Follow best practices for data security, compliance, and cost optimization in cloud environments.


Required Skills & Experience

  • Programming: Strong proficiency in Python (PySpark, data frameworks, automation).
  • Big Data Processing: Hands-on experience with Apache Spark (batch & streaming).
  • Messaging & Streaming: Proficient in Kafka (brokers, topics, partitions, consumer groups) and AWS Kinesis.
  • CDC Pipelines: Experience with Debezium / Kafka Connect / custom CDC frameworks.
  • AWS Services: AWS Glue, EMR, S3, Athena, Lambda, IAM, CloudWatch.
  • ETL/ELT Workflows: Strong knowledge of data ingestion, transformation, partitioning, schema management.
  • Databases: Experience with relational databases (MySQL, Postgres, Oracle) and NoSQL (MongoDB, DynamoDB, Cassandra).
  • Data Formats: JSON, Parquet, Avro, ORC, Delta/Iceberg/Hudi.
  • Version Control & CI/CD: Git, GitHub/GitLab, Jenkins, or CodePipeline.
  • Monitoring/Logging: CloudWatch, Prometheus, ELK/Opensearch.
  • Containers & Orchestration (nice-to-have): Docker, Kubernetes, Airflow/Step
  • Functions for workflow orchestration.


Preferred Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
  • Experience in large-scale data lake / lake house architectures.
  • Knowledge of data warehousing concepts and query optimisation.
  • Familiarity with data governance, lineage, and cataloging tools (Glue Data Catalog, Apache Atlas).
  • Exposure to ML/AI data pipelines is a plus.


Tools & Technologies (must-have exposure)

  • Big Data & Processing: Apache Spark, PySpark, AWS EMR, AWS Glue
  • Streaming & Messaging: Apache Kafka, Kafka Connect, Debezium, AWS Kinesis
  • Cloud & Storage: AWS (S3, Athena, Lambda, IAM, CloudWatch)
  • Programming & Scripting: Python, SQL, Bash
  • Orchestration: Airflow / Step Functions
  • Version Control & CI/CD: Git, Jenkins/CodePipeline
  • Data Formats: Parquet, Avro, ORC, JSON, Delta, Iceberg, Hudi
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Inncircles

Founded :
2019
Type :
Product
Size :
100-500
Stage :
Profitable

About

Inncircles Arena – a Modern Construction Management Platform, is a composed one stop solution for construction management that is highly configurable, easily adaptable, data centric while leveraging advanced artificial intelligence and cloud technologies to ramp up construction company's digital transformation initiatives. It empowers construction teams to go beyond the tactical to strategic. Through the platform, residential, industrial and infrastructure construction companies gain real-time insights into their projects, a unified communication and collaboration space, powerful configuration capabilities to integrate with existing or legacy tools and technologies. Detailed products and solution packages allow for planning, scheduling, assigning, tracking, and analyzing construction work packages from pre-planning stage to handover. Solving Construction Business Challenges through Technology • Work Management: Elevate efficiency and transform project management by scheduling works, tracking progress while gaining real-time insights. • Quality Management: Ensure projects meet the required quality standards by performing regular inspections and issue tracking. • Safety Management: Record and monitor safety incidents in real-time, schedule inspections, and store pertinent safety documentation. • Advanced Work Package (AWP): Build package of works, schedule work packages and monitor progress. • Data and Analytics: Gain profound insights of projects through visually stunning dashboards that bring your data to life. • Mobile Application: Offer mobile accessibility to enable field teams to access information, submit updates, and communicate in real-time, improving overall efficiency. • Integrations: Foster connectivity in the construction eco system with Arenas potent integration capabilities. • Pre-Construction Planning: Drive project planning, budgeting, estimating, bidding, and collaboration during the pre-construction phase. • Procurement: Create procurement packages
Read more

Company social profiles

linkedin

Similar jobs

Amplifai India Private Limited
Pune
3 - 6 yrs
₹10L - ₹15L / yr
Analytical Skills
Communication Skills
Requirement Analysis

Job Description

Product Analyst, Amplifai Pune

Amplifai transforms AI potential into measurable business value, guiding organizations from strategic planning to execution. With deep expertise in AI product development, technical architecture, regulatory compliance, and commercialization, we deliver secure, ethical, and high-performing solutions. Having co-founded one of Europe’s most innovative AI companies, our team drives unparalleled growth for clients through cutting-edge technologies like GPT tools, AI agents, and modern frameworks. Join our new Pune office to shape the future of AI-driven innovation!

Amplifai is seeking a dynamic Product Analyst to join our newly established Pune office and play a key role in turning business needs into reality. As a Product Analyst, you’ll bridge the gap between clients, UX designers, developers, and QA teams, ensuring seamless translation of ideas into high-quality, defect-free software. This role offers an exciting opportunity to work with AI technologies like GPT tools and AI agents, contributing to product descriptions, specifications, and test case creation in a fast-paced, collaborative environment.

 Key Responsibilities:             

·      Gather, analyse, and prioritize business requirements, translating them into clear user stories and acceptance criteria for UX, development, and QA teams.

·      Collaborate with stakeholders to align on project goals, ensuring solutions meet client needs and technical feasibility.

·      Leverage GPT tools and AI agents to streamline product descriptions, requirement specifications, and test case creation.

·      Facilitate Scrum ceremonies (sprint planning, daily stand-ups, reviews, and retrospectives) as a Scrum Master, fostering Agile best practices

·      Manage and prioritize the product backlog, ensuring clarity and readiness for upcoming sprints.

·      Conduct requirement analysis and support user acceptance testing (UAT) to ensure defect-free deliverables.

·      Track project progress, identify risks, and resolve roadblocks to ensure timely delivery.

·      Communicate effectively with clients and internal teams, presenting solutions and progress updates with clarity.

·      Maintain comprehensive documentation of requirements, user stories, and sprint metrics.

·      Identify opportunities to enhance Agile processes, team collaboration, and product outcomes through continuous improvement

 

Required Skills & Experience:

·      Familiarity with AI technologies (e.g., GPT tools, AI agents) or a strong willingness to learn.

·      Strong analytical and problem-solving skills with a keen attention to detail.

·      Excellent communication and presentation skills, with the ability to engage clients and cross-functional teams.

·      3+ years of experience in product analysis, requirement gathering, or Scrum activities.

·      Basic understanding of software testing and defect tracking processes.

·      Exposure to Agile methodologies and Scrum frameworks.

·      Team player with strong interpersonal skills and a collaborative mindset.

·      Bachelor’s degree in business, IT, Computer Science, or a related field (MBA is a plus).

Preferred Qualifications

  • Experience with automation tools for test case creation or requirement management (e.g., Jira, Confluence, or similar).
  • Familiarity with cloud platforms (e.g., Azure) or modern development frameworks.

 

 

Read more
EaseMyTrip.com
at EaseMyTrip.com
1 recruiter
Madhu Sharma
Posted by Madhu Sharma
Noida
0.6 - 1 yrs
₹1L - ₹2.5L / yr
Human Resources (HR)
HR Intern
HR Operations
PF
esic
+4 more

We’re looking for a HR Executive(Female) – Operations to join our growing team in Noida


✨ What you’ll do:

Welcome & onboard new joiners

Manage employee documentation

Support insurance onboarding

Handle smooth exit formalities


✨ What we’re looking for:

6 months – 1 year experience in HR

Freshers with HR internship experience are welcome!


Should have basic knowledge of PF/ESIC, HR operarions.


💰 Salary: ₹19,500 CTC in Hand 15000 Monthly

📍 Location: Noida-Sec-63

Shift- 10 to 7 PM 6 Days.

Preferred who can Join Immediately


If you’re passionate about HR and eager to grow, this could be your next big step!

Read more
Quenzo Tech
Aaron Diaz
Posted by Aaron Diaz
Remote only
0 - 50 yrs
₹10L - ₹20L / yr
Data Visualization
skill iconData Analytics
skill iconData Science

We are seeking an experienced Data Scientist to join our data-driven team. As a Data Scientist, you will work with large datasets, apply advanced analytics techniques, and build machine learning models to provide actionable insights that drive business decisions. You will collaborate with various teams to translate complex data into clear recommendations and innovative solutions.

Key Responsibilities:

  • Analyze large datasets to identify trends, patterns, and insights that can inform business strategy.
  • Develop, implement, and maintain machine learning models and algorithms to solve complex problems.
  • Work closely with stakeholders to understand business objectives and translate them into data science tasks.
  • Preprocess, clean, and organize raw data from various sources for analysis.
  • Conduct statistical analysis and build predictive models to support data-driven decision-making.
  • Create data visualizations and reports to communicate findings clearly and effectively to both technical and non-technical teams.
  • Design experiments and A/B testing to evaluate business initiatives.
  • Ensure the scalability and performance of data pipelines and machine learning models.
  • Collaborate with engineering teams to integrate data science solutions into production systems.
  • Continuously stay updated with the latest developments in data science, machine learning, and analytics technologies.


Read more
Paytm
at Paytm
41 recruiters
Anuj Kanojia
Posted by Anuj Kanojia
Noida, Bengaluru (Bangalore), Pune
3 - 8 yrs
₹10L - ₹40L / yr
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
Data Structures
+4 more
Requirements:
3+ years of server-side application development experience with Java, Spring boot, Microservice.
Deeply technical with a track record of successful delivery.
An entrepreneurial spirit combined with strong program and product management skills .
Excellent written and verbal communication skills with the ability to present complex plans and design.
Excellent judgment, organizational, and problem-solving skills.
Excellent design and architecture design.
Data driven decision making.
 
Preferred Qualifications : Bachelor's/Master's Degree in Computer Science or equivalent
 
Skills that will help you succeed in this role: 
Language: Java/Node DB: RDBMS, Messaging: Kafka/RabbitMQ, Caching: Redis/Aerospike, Micro services, AWS.
Good experience in scaling, performance tuning & optimization at both API and storage layers  
 
Read more
Startup-E-Learning
Startup-E-Learning
Agency job
via Merito by Gaurav Bhosle
Remote
7 - 14 yrs
₹30L - ₹45L / yr
DevOps
skill iconKubernetes
skill iconDocker
skill iconAmazon Web Services (AWS)
Windows Azure
+1 more
Hello

Senior DevOps Engineer (8-12 yrs Exp)
Job Description:
We are looking for an experienced and enthusiastic DevOps Engineer. As our new DevOps
Engineer, you will be in charge of the specification and documentation of the new project
features. In addition, you will be developing new features and writing scripts for automation
using Java/BitBucket/Python/Bash.
Roles and Responsibilities:
• Deploy updates and fixes
• Utilize various open source technologies
• Need to have hands on experience on automation tools like Docker / Jenkins /
Puppet etc.
• Build independent web based tools, micro-services and solutions
• Write scripts and automation using Java/BitBucket/Python/Bash.
• Configure and manage data sources like MySQL, Mongo, Elastic search, Redis etc
• Understand how various systems work
• Manage code deployments, fixes, updates and related processes.
• Understand how IT operations are managed
• Work with CI and CD tools, and source control such as GIT and SVN.
• Experience with project management and workflow tools such as Agile, Redmine,
WorkFront, Scrum/Kanban/SAFe, etc.
• Build tools to reduce occurrences of errors and improve customer experience
• Develop software to integrate with internal back-end systems
• Perform root cause analysis for production errors
• Design procedures for system troubleshooting and maintenance
Requirements:
• More than six years of experience in a DevOps Engineer role (or similar role);
experience in software development and infrastructure development is a mandatory.
• Bachelor’s degree or higher in engineering or related field
• Proficiency in deploying and maintaining web applications
• Ability to construct and execute network, server, and application status monitoring
• Knowledge of software automation production systems, including code deployment
• Working knowledge of software development methodologies
• Previous experience with high-performance and high-availability open source web
technologies
• Strong experience with Linux-based infrastructures, Linux/Unix administration, and
AWS.
• Strong communication skills and ability to explain protocol and processes with team
and management.
• Solid team player.
Read more
Syscort Technologies
at Syscort Technologies
2 recruiters
Ajinkya Kalantri
Posted by Ajinkya Kalantri
Pune, Aurangabad, Thiruvananthapuram
3 - 8 yrs
₹5L - ₹20L / yr
SDET
Selenium
Selenium Web driver
Cucumber
Test Automation (QA)
+4 more

Job Description

 

We are looking for a Senior Test Engineer who will work closely with the project team members and ensure the quality of the delivery.

 

Roles and Responsibilities

 

  • Develop automated tests for product validation (regression and integration).
  • Develop and execute test cases, scripts, plans, and procedures (manual and automated).
  • Integrate and execute automated tests in CI/CD environments
  • Troubleshoot, and improve existing automation scripts
  • Act as a lead team member as the team grows, providing support and training to other team members
  • Execute test cases against various software applications and document the results
  • Perform ad hoc testing of new and existing features to ensure they meet design requirements, quality,

    and usability standards

  • Follow-up with and document resolution of bugs
  • Maintain test cases for manual and automated tests
  • Communicate defects to stakeholders in a clear and concise manner

 

Technical Skills

 

  • Minimum 3 - 5 years experience in a diverse technology environment in a Quality Assurance role
  • Experience developing testing strategies and test plans
  • Experience creating and executing automated end-to-end tests, using tools such as Selenium,

    Cucumber, or other automation frameworks

  • Exposure to ‘Leapwork’ is good to have. Expertise on Java, OOPS Concepts
  • Good understanding of software development lifecycles
  • Ability to work in multiple application environments

Strong data analysis, data verification, and problem-solving abilities

 

 

 

You should have

 

  • Fluency in written and communication skills in English
  • Strong ability to work collaboratively in a team environment
  • Excellent organizational skills including, management of multiple assignments, setting personal goals and

    targets, and document management

  • Excellent focus on detail through adherence to quality assurance best practices
Read more
Innoviti Payment Solutions
Prajwal P M
Posted by Prajwal P M
Bengaluru (Bangalore)
3 - 8 yrs
₹4L - ₹6L / yr
Recruitment
Recruitment/Talent Acquisition
Sourcing
Negotiation

Role and responsibilities:

  • Develop recruitment goals and objectives.
  • Develop sustainable recruiting strategy based on our goals and needs.
  • Source passive candidates
  • Search resume databases for the most fit candidates
  • Communicate with Hiring Managers
  • Determine the effectiveness and success of current recruiting plans and strategies.
  • Write and proofread job descriptions.
  • Recommend new sources for active and passive candidate recruiting.
  • Build talent networks to find qualified active and passive candidates.
  • Use multi-channel approach to recruit.
  • Evaluate which sources bring best candidates.
  • Identify key recruiting KPIs.
  • Plan and implement a Recruitment Marketing and Employer Branding strategy to attract high quality applicants.
  • Review applicants to evaluate if they meet the position requirements.
  • Interview candidates following company's rules and regulations.
  • Adhere to laws, rules, and regulations.
  • Adhere to personal data privacy regulations.
  • Responsible for end-to-end recruitment activities.
  • Responsible for hiring decisions Offer Negotiation / DOJ negotiations / Notice period Negotiations with candidates.
  • Rich experience in technical/Non-technical hiring across all levels (junior, middle & senior)

 

Read more
IQVIA
at IQVIA
6 recruiters
Nishigandha Wagh
Posted by Nishigandha Wagh
Pune
3 - 6 yrs
₹5L - ₹15L / yr
Data Warehouse (DWH)
Business Intelligence (BI)
skill iconAmazon Web Services (AWS)
SQL
MDM
+1 more
Consultants will have the opportunity to :
- Build a team with skills in ETL, reporting, MDM and ad-hoc analytics support
- Build technical solutions using latest open source and cloud based technologies
- Work closely with offshore senior consultant, onshore team and client's business and IT teams to gather project requirements
- Assist overall project execution from India - starting from project planning, team formation system design and development, testing, UAT and deployment
- Build demos and POCs in support of business development for new and existing clients
- Prepare project documents and PowerPoint presentations for client communication
- Conduct training sessions to train associates and help shape their growth
Read more
Z1 Media
at Z1 Media
2 recruiters
Shivali Kapoor`
Posted by Shivali Kapoor`
NCR (Delhi | Gurgaon | Noida)
8 - 10 yrs
₹18L - ₹20L / yr
skill iconPHP
MySQL
skill iconMongoDB
Technical Architecture
We are looking for a Tech Lead (8-10 years exp) to join our programming team! Someone who wants to do more than just code; we want a dreamer, a creator, a mentor, someone who likes to discuss ideas and enjoys working with others to design and create great features and products. Someone who understands that “what you build” is just as important as “how you build it”. Someone who would love the chance to get involved in all aspects of product development, discussing product design, architecture as well as testing and coding, whilst working on market-leading solutions in the programmatic video advertising space. You must be happy talking with non-technical people, discussing ideas, doing pair programming and be keen to develop your skills in all areas of product development.
Read more
DailyRounds/Marrow
at DailyRounds/Marrow
1 video
8 recruiters
Abhilasha Chatterjee
Posted by Abhilasha Chatterjee
Bengaluru (Bangalore)
1 - 3 yrs
₹7L - ₹11L / yr
skill iconAndroid Development
skill iconJava
skill iconKotlin
Data Structures
Algorithms
+1 more
Requirements - Bachelors or Master degree in Computer Science, Engineering or a related subject - 1+ Years of Native Android - Exceptional coding skills in any of Java, Kotlin - Excellent command over Data Structures and Algorithms - Strong problem solving and analytical skills - Solid understanding of SDLC process Good To Have - Have an app published in Playstore/App Store - Contribution to open source projects - Hands-on experience writing automation test scripts.
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
21,01,133
Matches delivered
37,12,187
Network size
15,000
Companies hiring
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos