Cutshort logo
Inncircles  logo
Sr. Big Data Engineer
Sr. Big Data Engineer
Inncircles 's logo

Sr. Big Data Engineer

Gangadhar M's profile picture
Posted by Gangadhar M
3 - 5 yrs
Best in industry
Hyderabad
Skills
PySpark
Spark
skill iconPython
ETL
Amazon EMR
Amazon S3
SQL
NOSQL Databases
athena
Apache Kafka
Apache Spark
skill iconMongoDB


We are looking for a highly skilled Sr. Big Data Engineer with 3-5 years of experience in

building large-scale data pipelines, real-time streaming solutions, and batch/stream

processing systems. The ideal candidate should be proficient in Spark, Kafka, Python, and

AWS Big Data services, with hands-on experience in implementing CDC (Change Data

Capture) pipelines and integrating multiple data sources and sinks.


Responsibilities

  • Design, develop, and optimize batch and streaming data pipelines using Apache Spark and Python.
  • Build and maintain real-time data ingestion pipelines leveraging Kafka and AWS Kinesis.
  • Implement CDC (Change Data Capture) pipelines using Kafka Connect, Debezium or similar frameworks.
  • Integrate data from multiple sources and sinks (databases, APIs, message queues, file systems, cloud storage).
  • Work with AWS Big Data ecosystem: Glue, EMR, Kinesis, Athena, S3, Lambda, Step Functions.
  • Ensure pipeline scalability, reliability, and performance tuning of Spark jobs and EMR clusters.
  • Develop data transformation and ETL workflows in AWS Glue and manage schema evolution.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable and high-quality data solutions.
  • Implement monitoring, logging, and alerting for critical data pipelines.
  • Follow best practices for data security, compliance, and cost optimization in cloud environments.


Required Skills & Experience

  • Programming: Strong proficiency in Python (PySpark, data frameworks, automation).
  • Big Data Processing: Hands-on experience with Apache Spark (batch & streaming).
  • Messaging & Streaming: Proficient in Kafka (brokers, topics, partitions, consumer groups) and AWS Kinesis.
  • CDC Pipelines: Experience with Debezium / Kafka Connect / custom CDC frameworks.
  • AWS Services: AWS Glue, EMR, S3, Athena, Lambda, IAM, CloudWatch.
  • ETL/ELT Workflows: Strong knowledge of data ingestion, transformation, partitioning, schema management.
  • Databases: Experience with relational databases (MySQL, Postgres, Oracle) and NoSQL (MongoDB, DynamoDB, Cassandra).
  • Data Formats: JSON, Parquet, Avro, ORC, Delta/Iceberg/Hudi.
  • Version Control & CI/CD: Git, GitHub/GitLab, Jenkins, or CodePipeline.
  • Monitoring/Logging: CloudWatch, Prometheus, ELK/Opensearch.
  • Containers & Orchestration (nice-to-have): Docker, Kubernetes, Airflow/Step
  • Functions for workflow orchestration.


Preferred Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
  • Experience in large-scale data lake / lake house architectures.
  • Knowledge of data warehousing concepts and query optimisation.
  • Familiarity with data governance, lineage, and cataloging tools (Glue Data Catalog, Apache Atlas).
  • Exposure to ML/AI data pipelines is a plus.


Tools & Technologies (must-have exposure)

  • Big Data & Processing: Apache Spark, PySpark, AWS EMR, AWS Glue
  • Streaming & Messaging: Apache Kafka, Kafka Connect, Debezium, AWS Kinesis
  • Cloud & Storage: AWS (S3, Athena, Lambda, IAM, CloudWatch)
  • Programming & Scripting: Python, SQL, Bash
  • Orchestration: Airflow / Step Functions
  • Version Control & CI/CD: Git, Jenkins/CodePipeline
  • Data Formats: Parquet, Avro, ORC, JSON, Delta, Iceberg, Hudi
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Inncircles

Founded :
2019
Type :
Product
Size :
100-500
Stage :
Profitable

About

Inncircles Arena – a Modern Construction Management Platform, is a composed one stop solution for construction management that is highly configurable, easily adaptable, data centric while leveraging advanced artificial intelligence and cloud technologies to ramp up construction company's digital transformation initiatives. It empowers construction teams to go beyond the tactical to strategic. Through the platform, residential, industrial and infrastructure construction companies gain real-time insights into their projects, a unified communication and collaboration space, powerful configuration capabilities to integrate with existing or legacy tools and technologies. Detailed products and solution packages allow for planning, scheduling, assigning, tracking, and analyzing construction work packages from pre-planning stage to handover. Solving Construction Business Challenges through Technology • Work Management: Elevate efficiency and transform project management by scheduling works, tracking progress while gaining real-time insights. • Quality Management: Ensure projects meet the required quality standards by performing regular inspections and issue tracking. • Safety Management: Record and monitor safety incidents in real-time, schedule inspections, and store pertinent safety documentation. • Advanced Work Package (AWP): Build package of works, schedule work packages and monitor progress. • Data and Analytics: Gain profound insights of projects through visually stunning dashboards that bring your data to life. • Mobile Application: Offer mobile accessibility to enable field teams to access information, submit updates, and communicate in real-time, improving overall efficiency. • Integrations: Foster connectivity in the construction eco system with Arenas potent integration capabilities. • Pre-Construction Planning: Drive project planning, budgeting, estimating, bidding, and collaboration during the pre-construction phase. • Procurement: Create procurement packages
Read more

Company social profiles

linkedin

Similar jobs

AerTrip India Limited
at AerTrip India Limited
1 recruiter
Aertrip HR
Posted by Aertrip HR
Remote only
3 - 12 yrs
₹3L - ₹15L / yr
skill iconPython
skill iconMongoDB
rasa
skill iconFlask
MySQL

Job Description:

As a developer in the chatbot team you’ll be responsible for expanding the scope of AERIN (Aertrip’s Chatbot) to handle all types of travel queries in a scalable and efficient manner. You will need to make AERIN as robust as possible and strive towards 100% accuracy. You should be able to design, develop, test and roll out new features quickly. You should be ready to tackle open-ended problems and build solutions for them.

 

Preferred qualifications & experience:

  1. Bachelor’s degree in Computer Science or equivalent experience.
  2. 1 or more years of work experience in the tech industry.
  3. Experience with NLP solutions.
  4. Experience with chatbot frameworks like RASA, DialogFlow is a bonus.
  5. Experience with spacy, nltk and other ML libraries.
  6. Familiarity with version control tools like Gitlab.
  7. Experience in object oriented design and programming.
  8. Sound programming skills to write algorithms for text analysis and context understanding.

Preferred Skills & Abilities:

  1. Should be able to write clean and manageable code.
  2. Passion for innovation.
  3. Out of the box thinking.
  4. Should be able to understand both the tech and business aspects of a system.
  5. Should be able to meet deadlines.
  6. Should be able to work in a collaborative environment.
  7. Should be able to work independently in small teams.
Read more
Monjura Parveen
Monjura Parveen
Posted by Monjura Parveen
Kolkata
4 - 8 yrs
₹5L - ₹12L / yr
FreeSWITCH
VoIP Protols
Audioconferencing
Videoconferencing
RTP proxy
+4 more

Job Opening: FreeSWITCH Developer

Location: Ecospace Business Park, Rajarhat, Newtown, Kolkata

Employment Type: Full-Time, Permanent

Shift: Day Shift

Experience: Minimum 4+ years in FreeSWITCH/VoIP Development

Responsibilities:

  • Design, develop, deploy, troubleshoot, and maintain tools and services supporting our cloud telephony network.
  • Customize FreeSWITCH for large-scale audio/video conferencing (1000–1500 concurrent calls).
  • Expertise in SIP, RTP, RTCP, TURN, STUN, NAT, TLS.
  • Hands-on experience with RTP Proxy and routed audio conferences.
  • Understanding of SDP Protocol offer/answer mechanism.
  • Work with load testing tools for FreeSWITCH audio conferences.
  • Deploy and manage multiple FreeSWITCH instances using load balancers.
  • Debug issues using packet captures (Wireshark/Ngrep).
  • Collaborate with mobile/API teams for integration and support.
  • Knowledge of codecs (PCMU, PCMA, G729, Opus) and open-source telephony technologies (FreeSWITCH, WebRTC).
  • Familiarity with SIP servers (SER/OpenSER), proxy servers, SBC, SIPX is a plus.

Qualifications:

  • Bachelor’s degree in Engineering (B.Tech/MCA).
  • 4+ years of hands-on experience in FreeSWITCH development or related VoIP technologies.
  • Strong knowledge of Linux environments and command-line tools.
  • Basic to intermediate SQL knowledge.
  • Proficiency in scripting (Bash, Python, Perl) for automation.
  • Strong understanding of VoIP protocols (SIP, RTP), networking principles (TCP/IP, DNS, DHCP, routing protocols).
  • Excellent troubleshooting skills with ability to resolve complex technical issues.
  • Strong problem-solving, communication, and collaboration skills.

Industry Preference:

  • Travel (Preferred)
  • Any Industry (Required)
Read more
Marktine
at Marktine
1 recruiter
Vishal Sharma
Posted by Vishal Sharma
Remote, Bengaluru (Bangalore)
3 - 7 yrs
₹10L - ₹24L / yr
skill iconData Science
skill iconR Programming
skill iconPython
SQL
skill iconMachine Learning (ML)
+1 more

Responsibilities:

  • Design and develop strong analytics system and predictive models
  • Managing a team of data scientists, machine learning engineers, and big data specialists
  • Identify valuable data sources and automate data collection processes
  • Undertake pre-processing of structured and unstructured data
  • Analyze large amounts of information to discover trends and patterns
  • Build predictive models and machine-learning algorithms
  • Combine models through ensemble modeling
  • Present information using data visualization techniques
  • Propose solutions and strategies to business challenges
  • Collaborate with engineering and product development teams

Requirements:

  • Proven experience as a seasoned Data Scientist
  • Good Experience in data mining processes
  • Understanding of machine learning and Knowledge of operations research is a value addition
  • Strong understanding and experience in R, SQL, and Python; Knowledge base with Scala, Java, or C++ is an asset
  • Experience using business intelligence tools (e. g. Tableau) and data frameworks (e. g. Hadoop)
  • Strong math skills (e. g. statistics, algebra)
  • Problem-solving aptitude
  • Excellent communication and presentation skills
  • Experience in Natural Language Processing (NLP)
  • Strong competitive coding skills
  • BSc/BA in Computer Science, Engineering or relevant field; graduate degree in Data Science or other quantitative field is preferred
Read more
appleute
at appleute
1 recruiter
Hanush Pillai
Posted by Hanush Pillai
Remote only
8 - 13 yrs
₹5L - ₹15L / yr
skill iconAngularJS (1.x)
skill iconAngular (2+)
skill iconReact.js
skill iconNodeJS (Node.js)
TypeScript


At Appleute, we're not just developing software; we're shaping the future of technology by partnering with startups and SMEs to bring groundbreaking ideas to life. From sculpting Minimum Viable Products (MVPs) to scaling full-fledged platforms for millions of users, we engage in projects that challenge the status quo, powered by a blend of innovative technology, visionary founders, and transformative business models. Our work is driven by a passion for creating market-leading products that redefine industries.


Why Join Us?


Global Remote Culture: Embrace the freedom to work from anywhere in the world. Our remote-first approach ensures you have the balance and flexibility to work where you thrive.


Results-Driven Environment: We value outcomes over hours. Focus on making a significant impact, with the flexibility to manage your schedule around what works best for you.


Flat Organizational Structure: Say goodbye to bureaucracy. Our flat hierarchies mean your ideas and contributions are valued and can make a direct impact from day one.


A Team of A-Players: Join an elite team of tech enthusiasts and innovators. Our selective hiring process ensures you're working among the best in the industry, fostering an environment of learning and growth.


Innovative Tech Stack: Dive into cutting-edge technologies and tools. From JavaScript and TypeScript to AWS, Docker, and React/Express, you'll have the opportunity to work on exciting projects using the latest in tech.


You're a perfect fit if you're dedicated to:


Excellence in Code Quality: You understand that great products are built on clean, maintainable code.


Collaborative Problem Solving: You thrive in open discussions and enjoy brainstorming solutions with your team.


Continuous Learning: Your curiosity drives you to stay on top of new technologies and coding practices.


Taking Ownership: You take pride in your work, from concept to deployment, ensuring excellence every step of the way.


Your Impact:


Build Scalable Solutions: Develop software that's ready to grow from thousands to millions of users, focusing on performance and scalability.


Drive Integrations and Deployment: Implement seamless software integrations and efficient deployment strategies to enhance product capabilities.


Enhance Mobile Experience: Craft mobile-optimized applications and features that deliver exceptional user experiences.


Ensure usable Documentation: Hone your skills in improving documentation to be clear and concise that supports your team and project needs.


Technologies You'll Work With:


Frontend and Backend: JavaScript, TypeScript, React/Express, React Native, Node.js


Database Management: MySQL, Postgres and Prisma


Deployment and Version Control: AWS, Docker, Git/Gitlab, NPM




Salary Range - ₹30L – ₹40L

Read more
TRUGlobal
at TRUGlobal
3 recruiters
Bibin Roy
Posted by Bibin Roy
Bangarmau, Pune, MEXICO CITY (Mexico)
10 - 18 yrs
₹1L - ₹15L / yr
Oracle EBS
Supply Chain Management (SCM)
P2P
SQL
Order management
+3 more
Hybrid Model Available (2 Days WFO & 3 Days WFH) 

WFO: 5 am - 11pm
WFH : 12am- 3am

Flexible Salary bracket
Read more
The Next Generation Technology Consulting & Services
The Next Generation Technology Consulting & Services
Agency job
via Jobdost by Riya Roy
Rajkot, Ahmedabad
2 - 8 yrs
₹6L - ₹15L / yr
Selenium
Appium
Test Automation (QA)
Software Testing (QA)
QA Automation - 5 Openings

● Experience level: +2 years
● Experience using technologies Cypress, Cucumber, Appium, Maven, Log4J2, TestNG,
Junit.
● Should have experience/knowledge of JAVA, Javascript, Gherkin.
● Experience developing and executing manual and automated test suites.
Familiarity with the tools, concepts, and methodologies of quality management.
● Good Communication Skills.
Read more
Bexhin Technology Pvt Ltd
Bexhin Technology Pvt Ltd
Agency job
via Aans Technologies by Aans Technologies
Remote only
5 - 8 yrs
₹5.5L - ₹7L / yr
Windows Azure
Azure
SQL Azure
Data Factory
Data Lake

Should Have Minimum 3 yrs of Exp in Data factory , Data Lake.

•             Should have exposure in Azure SQL Database, Azure Dataware house.

•             Exposure in  t-SQL skills with experience in Azure SQL DW

•             Experience handling Structured and unstructured datasets.

•             Excellent problem solving, Critical and Analytical thinking skills.

•             Understand and worked on Azure Concepts.

•               Good Communication

Read more
Chandrapura, Bengaluru (Bangalore)
1 - 3 yrs
₹2L - ₹3L / yr
Sales
Real Estate
Communication Skills
  • Responsible for client engagement & Manage the requirements
  • Communicate with clients to identify their requirements and choice of property
  • Coordinate the builder to arrange for the site visit and share the necessary information to clients
  • Visit the connected properties and identify the USPs to pitch better to clients
  • Manage the feedback from the client
  • Understand the real estate market & comparison of the product
  • Maintain good business relationship with client and builder
  • Participate in seminars, conferences, and events to enhance the market & expand the business network
  • Work with the team to share the responsibilities and work towards achieving team’s targets
Read more
Main Street
at Main Street
1 recruiter
Teja Dega
Posted by Teja Dega
Remote, Bengaluru (Bangalore)
5 - 8 yrs
₹15L - ₹25L / yr
skill iconRuby on Rails (ROR)
skill iconPostgreSQL
skill iconNodeJS (Node.js)
skill iconHTML/CSS

Main Street is hiring a Full Stack Engineer who will help design our technical architecture and build solutions that help business owners become entrepreneurs. Apply if you are excited about creating a platform for true entrepreneurship. 

 

As an early hire on the core team, you’ll work on shaping projects, big and small, over short three week sprints. We’re a small team, but we get a lot of features shipped. You would be responsible for driving full-stack projects from prototype to finished stage with a lot of autonomy and creativity. This is an exciting time to join the team. We've found solid early traction, have strong basic foundations and are pushing aggressively into some incredible new feature builds. We have a small team, so you can make an incredible impact on our product very quickly. 

 

Responsibilities:

  • Own the design of the overall architecture of the web application
  • Implement a robust set of services and effective APIs to power the web application
  • Build reusable and manageable code and libraries for future use
  • Optimize the application for maximum speed and scalability
  • Translate UI/UX wireframes to visual elements
  • Develop and handle well-functioning databases and applications
  • Test software to ensure responsiveness and efficiency 
  • Troubleshoot, debug and upgrade software

Experience:

  • Proven expertise in shipping features in Ruby on Rails. Strong background in Ruby on Rails, HTML, CSS and JavaScript
  • Strong experience in driving thought leadership and technical architecture
  • Experience with mentoring juniors in the team

 

You are a fit if :

  • You are a skilled coder. We need candidates with a strong history of shipping features in Rails with clean maintainable code. 
  • You are a good communicator and a really good writer. Since we are async and remote most of our work is written. You'll need to be able to proactively communicate in concise, clear, simple writing.
  • You are an expert project manager. You need to be able to break down a large, complex business deliverable into executable pieces that can be managed as individual user stories. You need to be able to keep a project on track, and cut scope when needed.
  • You don't need babysitting and are a Manager of One. We love people who have started companies or have freelanced in the past. You take ownership and take things across the finish line. Especially in our remote async approach, you'll need to be intrinsically motivated and keep your  projects on track.
  • You have strong business and product acumen. This job isn't just writing code, we solve real business problems for real people. That means you need to be able to look beyond the user story, have empathy for our end users and solve real problems for them.

About Main Street:

We're solving a problem that 62% of Americans have

62% of Americans last year thought about starting a business. The #1 reason they didn’t was because they didn’t know where to start. Currents options are:

  1. Going independent which is risky: no support or direction
  2. Paying a franchise which is expensive:  franchises charge $100K+ fees to get started, take 10% of topline revenue, don't provide marketing support or technology

The market we're going after is bigger than e-Commerce

We’re disrupting the $2.3T franchise industry by removing the franchise fee and offering a tech-enabled franchise like product.

We’re starting with home painting where our biggest competitor is https://certapro.com/">CertaPro a $550M revenue franchise (1% market share) before quickly taking on https://www.neighborlybrands.com/our-brands/">Neighborly Brands as we expand categories (pest control, lawn care, etc.).

As we expand categories, our customer does not change

Our customer is an aspiring business owner, like https://www.linkedin.com/in/nicklaforest">Nick or https://www.linkedin.com/in/hardin-h-hawes-9ba6a72/">Hardin who doesn't care what type of business they start, as long as it is profitable. Our customers are not actually the one painting houses or completing roofing jobs, rather they are the one that hires people to do the job. Our customer has years of experience in sales and general management, which makes them suitable for any type of business — which is why as we expand to 100 categories, our customer does not change.

Our solution is an end to end platform to help you start, grow and run a business

Main Street is a combination of well known technology platforms that provides an Uber like simple experience to starting a business

Take a look Main Street's Product walk through https://www.youtube.com/watch?v=teajumhL5DI&;feature=emb_title">here.

 

We're backed by top investors and built a world class team

We're a seed-stage startup and have raised $10M+ from top investors including Greylock and Khosla. Our team previously worked at places like DoorDash, OpenDoor, Amazon and OYO.

Read more
OctoSocial
at OctoSocial
1 recruiter
Pravin Kumar
Posted by Pravin Kumar
Remote only
3 - 10 yrs
₹3L - ₹6L / yr
Digital Marketing
Google Adwords
Web Analytics
skill iconGoogle Analytics
Facebook Marketing
+3 more
Responsibilities

* Plan and execute all digital marketing, including SEO/SEM, marketing database, email, social media, and display advertising campaigns
* Design, build and maintain our social media presence
* Measure and report performance of all digital marketing campaigns, and assess against goals (ROI and KPIs)
* Identify trends and insights, and optimize spend and performance based on the insights
* Brainstorm new and creative growth strategies
* Plan, execute, and measure experiments and conversion tests
* Collaborate with internal teams to create landing pages and optimize the user experience
* Utilize strong analytical ability to evaluate end-to-end customer experience across multiple channels and customer touchpoints
* Instrument conversion points and optimize user funnels
* Evaluate emerging technologies. Provide thought leadership and perspective for adoption where appropriate


Requirements
* BS/MS degree in marketing or a related field
* Experience in digital marketing agency would be an added advantage
* Demonstrable experience leading and managing SEO/SEM, marketing database, email, social media and/or display advertising campaigns
* Highly creative with experience in identifying target audiences and devising digital campaigns that engage, inform and motivate
* Experience in optimizing landing pages and user funnels
* Experience with A/B and multivariate experiments
* Solid knowledge of website analytics tools (e.g., Google Analytics, NetInsight, Omniture, WebTrends)
* Experience in setting up and optimizing Google Adwords campaigns
* Strong analytical skills and data-driven thinking
* Up-to-date with the latest trends and best practices in online marketing and measurement
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
21,01,133
Matches delivered
37,12,187
Network size
15,000
Companies hiring
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos