Cutshort logo
Global digital transformation solutions provider.  logo
Lead II - Data Engineering -Python - Databricks, PySpark, Python
Global digital transformation solutions provider.
Lead II - Data Engineering -Python - Databricks, PySpark, Python
Global digital transformation solutions provider. 's logo

Lead II - Data Engineering -Python - Databricks, PySpark, Python

at Global digital transformation solutions provider.

Agency job
7 - 9 yrs
₹15L - ₹28L / yr
Bengaluru (Bangalore)
Skills
databricks
skill iconPython
SQL
PySpark
skill iconAmazon Web Services (AWS)
ETL
Informatica
Performance tuning
AWS Lambda
Amazon Redshift
Data modeling
CI/CD
skill iconGit
EMR

Role Proficiency:

This role requires proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources. The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL. This position demands independence and proficiency across various data domains. Expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required.


Skill Examples:

  1. Proficiency in SQL Python or other programming languages used for data manipulation.
  2. Experience with ETL tools such as Apache Airflow Talend Informatica AWS Glue Dataproc and Azure ADF.
  3. Hands-on experience with cloud platforms like AWS Azure or Google Cloud particularly with data-related services (e.g. AWS Glue BigQuery).
  4. Conduct tests on data pipelines and evaluate results against data quality and performance specifications.
  5. Experience in performance tuning.
  6. Experience in data warehouse design and cost improvements.
  7. Apply and optimize data models for efficient storage retrieval and processing of large datasets.
  8. Communicate and explain design/development aspects to customers.
  9. Estimate time and resource requirements for developing/debugging features/components.
  10. Participate in RFP responses and solutioning.
  11. Mentor team members and guide them in relevant upskilling and certification.

 

Knowledge Examples:

  1. Knowledge of various ETL services used by cloud providers including Apache PySpark AWS Glue GCP DataProc/Dataflow Azure ADF and ADLF.
  2. Proficient in SQL for analytics and windowing functions.
  3. Understanding of data schemas and models.
  4. Familiarity with domain-related data.
  5. Knowledge of data warehouse optimization techniques.
  6. Understanding of data security concepts.
  7. Awareness of patterns frameworks and automation practices.


 

Additional Comments:

# of Resources: 22 Role(s): Technical Role Location(s): India Planned Start Date: 1/1/2026 Planned End Date: 6/30/2026

Project Overview:

Role Scope / Deliverables: We are seeking highly skilled Data Engineer with strong experience in Databricks, PySpark, Python, SQL, and AWS to join our data engineering team on or before 1st week of Dec, 2025.

The candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and analytics solutions that drive business insights and operational efficiency.

Design, build, and maintain scalable data pipelines using Databricks and PySpark.

Develop and optimize complex SQL queries for data extraction, transformation, and analysis.

Implement data integration solutions across multiple AWS services (S3, Glue, Lambda, Redshift, EMR, etc.).

Collaborate with analytics, data science, and business teams to deliver clean, reliable, and timely datasets.

Ensure data quality, performance, and reliability across data workflows.

Participate in code reviews, data architecture discussions, and performance optimization initiatives.

Support migration and modernization efforts for legacy data systems to modern cloud-based solutions.


Key Skills:

Hands-on experience with Databricks, PySpark & Python for building ETL/ELT pipelines.

Proficiency in SQL (performance tuning, complex joins, CTEs, window functions).

Strong understanding of AWS services (S3, Glue, Lambda, Redshift, CloudWatch, etc.).

Experience with data modeling, schema design, and performance optimization.

Familiarity with CI/CD pipelines, version control (Git), and workflow orchestration (Airflow preferred).

Excellent problem-solving, communication, and collaboration skills.

 

Skills: Databricks, Pyspark & Python, Sql, Aws Services

 

Must-Haves

Python/PySpark (5+ years), SQL (5+ years), Databricks (3+ years), AWS Services (3+ years), ETL tools (Informatica, Glue, DataProc) (3+ years)

Hands-on experience with Databricks, PySpark & Python for ETL/ELT pipelines.

Proficiency in SQL (performance tuning, complex joins, CTEs, window functions).

Strong understanding of AWS services (S3, Glue, Lambda, Redshift, CloudWatch, etc.).

Experience with data modeling, schema design, and performance optimization.

Familiarity with CI/CD pipelines, Git, and workflow orchestration (Airflow preferred).


******

Notice period - Immediate to 15 days

Location: Bangalore

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Remote, Bengaluru (Bangalore)
8 - 10 yrs
₹22L - ₹25L / yr
Security Information and Event Management (SIEM)
Google Chronicle
Sumologic
Splunk
Crowdstrike

Ø Develop and implement content for any SIEM platforms, including Google Chronicle, Sumologic, and Splunk.

Ø Configure and fine-tune use cases, correlation, grouping, and logical rules in SIEM tools.

Ø Integrate new log sources, assets with SIEM, and incremental threat intelligence feeds.

Ø Draft, test, and deploy YARA and Chronicle Backstory rules.

Ø Curate and update Incident Response Guides.

Ø Customize SIGMA rules and maintain familiarity with the MITRE ATT&CK Framework.

Ø Develop threat detection content for various datasets such as Proxy, VPN, Firewall, and DLP.

Ø Aid in process development/improvement for Security Operations.

Ø Recognize and propose new security controls to bridge existing gaps.

Ø Chronicle Backstory/ ELK Stack/ YARA / CrowdStrike rules experience is a plus.

 

Read more
Wissen Technology
at Wissen Technology
4 recruiters
VenkataRamanan S
Posted by VenkataRamanan S
Bengaluru (Bangalore)
6 - 11 yrs
Best in industry
skill iconPython
databricks
Snowflake

Required Skills:

· 6+ years of being a practitioner in data engineering or a related field.

· Proficiency in programming skills in Python

· Experience with data processing frameworks like Apache Spark or Hadoop.

· Experience working on Snowflake and Databricks.

· Familiarity with cloud platforms (AWS, Azure) and their data services.

· Experience with data warehousing concepts and technologies.

· Experience with message queues and streaming platforms (e.g., Kafka).

· Excellent communication and collaboration skills.

· Ability to work independently and as part of a geographically distributed team.

Read more
DevCare solutions pvt ltd
at DevCare solutions pvt ltd
2 candid answers
Guru Dharshini
Posted by Guru Dharshini
Chennai
1 - 4 yrs
₹3L - ₹5L / yr
US staffing

Responsible for handling Healthcare Recruiter from US based clients.

Responsible for full-cycle recruiting: interview, offer, negotiation and closed candidates for assigned requisitions.

Sourcing from Job Portal (Indeed), Making job postings on the Job Portal.

Must be having capability to work with a team or individually.

Maintain close contact with candidates/Vendors/ recruiters/offshore team.

Publish progress report to management every week.

Candidate should have worked on Healthcare requirements – Medical Assistant, CNA, RN, Nurse Practitioner, Physician.

Read more
Recro
at Recro
1 video
32 recruiters
Muskan Handa
Posted by Muskan Handa
Bengaluru (Bangalore), Gurugram, Noida
2 - 5 yrs
₹7L - ₹19L / yr
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
Problem solving
+1 more
  • 2+ years of experience in using Java/J2EE tech stacks
  • Good understanding of data structures and algorithms.
  • Excellent analytical and problem solving skills.
  • Ability to work in a fast paced internet start-up environment.
  • Experience in technical mentorship/coaching is highly desirable.
  • Understanding of AI/ML algorithms is a plus.


Read more
vakilsearch (vakilsearch.com)
Rajkumar Ganapathy
Posted by Rajkumar Ganapathy
Chennai
3 - 7 yrs
₹6L - ₹15L / yr
skill iconRuby on Rails (ROR)
skill iconRuby
skill iconJavascript
skill iconMongoDB
skill iconPostgreSQL

We are the largest online legal service providers in India. Our business is technology driven and our app is already servicing customers across India and abroad. We have offices in Chennai and Bangalore.


Our dev team comprises of seasoned and experienced developers along with Senior Resources for mentoring. We are looking forward to expand our team.


Job brief

Do you like writing mean and clean Ruby code? Come join our team and help us build amazing things.

We’re looking for someone with a passion for programming and for writing beautiful code.  You will join a team of exceptional developers working in a fast-paced environment to deliver world-class software.


Responsibilities

  • Write clean, maintainable and efficient code.
  • Design robust, scalable and secure features.
  • Drive continuous adoption and integration of relevant new technologies into design.
Read more
BooksApp
at BooksApp
2 recruiters
Veerendra Kumar
Posted by Veerendra Kumar
Bengaluru (Bangalore)
2 - 4 yrs
₹8L - ₹14L / yr
skill iconReact.js
skill iconJavascript
skill iconRedux/Flux
[ CTC is Flexible and at Par with Industry Standards]
We are looking for a react.js developer to join our front-end development team. In this role, you will be responsible for developing and implementing user interface components using React.js concepts and workflows such as Redux, Flux, and Webpack. You will also be responsible for profiling and improving front-end performance and documenting our front-end codebase.

  • Reviewing application requirements and interface designs.
  • Identifying web-based user interactions.
  • Developing and implementing highly responsive user interface components using react concepts.
  • Writing application interface codes using JavaScript following react.js workflows.
  • Troubleshooting interface software and debugging application codes.
  • Developing and implementing front-end architecture to support user interface concepts.
  • Monitoring and improving front-end performance.
  • Documenting application changes and developing updates.
Read more
A firm which works with US clients. Permanent WFH.
A firm which works with US clients. Permanent WFH.
Agency job
via Jobdost by Riya Roy
Remote only
4 - 10 yrs
₹8L - ₹18L / yr
skill iconPython
skill iconDjango
RESTful APIs
GraphQL
skill iconReact.js
+5 more

This person MUST have:

  • B.E Computer Science or equivalent
  • 5 years experience with the Django framework
  • Experience with building APIs (REST or GraphQL) 
  • Strong Troubleshooting and debugging skills
  • React.js knowledge would be an added bonus 
  • Understanding on how to use a database like Postgres (prefered choice), SQLite, MongoDB, MySQL.
  • Sound knowledge of object-oriented design and analysis.
  • A strong passion for writing simple, clean and efficient code.
  • Proficient understanding of code versioning tools Git.
  • Strong communication skills.

Experience:

  • Min 5 year experience
  • Startup experience is a must. 

Location:

  • Remote developer

Timings:

  • 40 hours a week but with 4 hours a day overlapping with client timezone.  Typically clients are in California PST Timezone.

Position:

  • Full time/Direct
  • We have great benefits such as PF, medical insurance, 12 annual company holidays, 12 PTO leaves per year, annual increments, Diwali bonus, spot bonuses and other incentives etc.
  • We dont believe in locking in people with large notice periods.  You will stay here because you love the company.  We have only a 15 days notice period.
Read more
smartdata enterprises inc.
Usha Shukla
Posted by Usha Shukla
Nagpur
3 - 8 yrs
₹4L - ₹7L / yr
skill iconNodeJS (Node.js)
skill iconMongoDB
skill iconAngularJS (1.x)
skill iconAngular (2+)
skill iconReact.js

Skills and Qualification: 

 

  1. Experience in working MEAN stack / Full Stack platform
  2. Front-end developer with knowledge of coding into NODE/EXPRESS, JavaScript, Jquery & its plugins.
  3. js/Redux/React-native.
  4. Knowledge of different NPM packages (Session, Async etc.).
  5. Knowledge of Mongo dB/Mongoose/Nosql.
  6. Knowledge of Any one framework (Mean.io/Ionic/Meteor).
  7. Experience in Backbone.js or Angular.js is a must.
  8. Knowledge of responsive design creation.
    CSS3 / HTML / HTML5.

                                                                 

Roles and responsibilities:

  • Proficient in SQL, NoSQL, relational database design and methods for efficiently retrieving data 
  • Experience building complex and non-interactive systems batch, distributed, etc.
  • Be responsible for the technical design and experience in guiding/mentoring others.
  • Strong OOPS knowledge, including experience with design patterns
  • Experience with relational databases like MySQL/Oracle
  • Experience with enterprise application servers like Tomcat/Weblogic/Websphere.
  • Exposure to Web Services SOAP and RESTFUL
  • Exposure to Agile/Scrum, TDD, and Continuous Integration tools like Jenkins, Bamboo etc.
  • Excellent analytical aptitude and problem solving skills.
  • Excellent communication and customer interfacing skill

 

Read more
Techmplish Consultancy Services
Komal Kadam
Posted by Komal Kadam
Mumbai
0 - 1 yrs
₹1L - ₹1L / yr
skill iconPHP
MySQL
skill iconCodeIgniter
skill iconHTML/CSS
skill iconjQuery
Must be able to develop a complete PHP based web application end to end single handedly, although freshers will get guidance from seniors and also 1 week of on job training. Candidates must possess strong knowledge in PHP and JQuery Candidates will get an exposure to real time projects of various domains and also will explore various ways in which Data Analytics can be used for clients.
Read more
Zomato Media Pvt Ltd
at Zomato Media Pvt Ltd
1 video
10 recruiters
Pooja Kanth
Posted by Pooja Kanth
NCR (Delhi | Gurgaon | Noida)
1 - 8 yrs
₹7L - ₹25L / yr
skill iconObjective C
skill iconSwift
skill iconiOS App Development
RESTful APIs
SDK
+20 more
Who are we?Zomato is an online platform that is responsible for powering amazing dining experiences for our users daily, both, in home and whilst dining out. We started with a vision to ensure that no one ever has a bad meal! Over 10 years Zomato has steadily built a search and discovery platform that constantly accelerates the growth of our transaction businesses. Today, Zomato has multiple products and services that ensure our users have a great experience whilst improving the food industry globally. The last 5 years is where most of Zomato's growth has happened; our mission has also evolved to something exponentially greater than being a restaurant reviews and food delivery platform. We are already taking baby steps towards being the biggest farm to fork food company in the world.Please see our blog for more: https://www.zomato.com/blog/10-yearsZomato's MomentumEvery month, Zomato connects more than 57m users with 1.3m restaurants globallyZomato annual revenues increased by 45% in FY'18Number of orders/day on Zomato is increasing considerably, currently at ~1 millionZomato Gold launched in November 2017 and already has 7500+ restaurant partners that serve our ever-increasing subscriber base of more than 700,000 peopleZomato food ordering business has been growing at a considerable rate since inception and currently grows by 2x every 3 months!To make sure we match the increasing number of orders with smooth delivery and customer satisfaction, we are supporting this function with a fleet of 150,000+ ridersPlease see our blog for more: https://www.zomato.com/blog/annual-report-fy18Become a part of our growth storyOur apps have constantly been featured on the App Store and we take immense pride in the care with which we build our products. Our tight-knit iOS team works on a wide range of cutting edge products from an array of high-quality consumer applications to building world class solutions for restaurant owners. If you know what it takes to build magical user experiences and consider performance to be the key feature of your app, this is definitely the right place for you. Your typical day as a Zoman will look like :- Own the entire development lifecycle as you prototype, create, optimise, test, launch, and refine iOS apps- Build features that will reach millions of users across 9 languages and 24 countries- Work with fellow designers and developers in an iterative environment with short and heavy impact development cycles- Develop cutting edge functional modules and kits that will be integrated across all our iOS apps- Build proprietary SDKs to enable partnerships with other industry leaders- Brainstorm, contribute new ideas, think of innovative strategies, and implement solutions to difficult problems- Work closely with our backend team to interface with RESTful API services and contribute to the backend code when needed- Work with the real-time feedback from our users to make the product better- Discover, design, develop, deploy, debug. Repeat!Here's what we're looking for (Technical Skills) :- Engineers with experience in shipping iOS apps with a large user base, ideally currently available on the App Store- Top-notch programming skills in Objective C/Swift and Apple's SDKs and frameworks (Foundation, UIKit, Core Data, etc) along with proficiency in using Xcode- Command of memory management, view hierarchy, battery optimisation, and in-depth experience with multithreaded and networked applications- Experience in working with with RESTful APIs, third party SDK Integrations and common technologies like HTTPS, JSON, OAuth, and SQL- Focus on product design and pixel-perfect implementation of design into code- Solid experience with version control systemsHere's what we're looking for (Soft Skills) :- Care for quality with an obsession for performance and willingness to spend time testing the team's work as well as yours- Ability to work towards seeking different creative approaches to a single problem- Ability to unlearn and adapt to the fast moving technology environment- Excellent communication skills - both verbal and written - Fits well in a collaborative and charged workspaceZomato will be providing you with :As a Zoman, you will be immersed in a culture of people who take pride in their company and their ability to change the lives of millions of people daily. We place Zomato first in our daily tasks with a high level of honesty, ownership and judgment on a foundation of constant and real time feedback. You will be challenged daily to be a better thinker and problem solve using a first principles approach. We promise you, in addition to the quest for 0 bad meals, you will also have an infinite amount of interesting days as a Zoman - if you have passion and see yourself in our vision go ahead and send in your application!
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos