Cutshort logo
MNC logo
Data Engineer (Pyspark Developer)
MNC
Data Engineer (Pyspark Developer)
MNC's logo

Data Engineer (Pyspark Developer)

at MNC

Agency job
4 - 12 yrs
₹15L - ₹30L / yr
Remote, Bengaluru (Bangalore)
Skills
PySpark
Pyspark Developer
skill iconScala
DevOps

EXP-Developer-4 to 12 years

 

 

         Must have low-level design and development skills.  Should able to design a solution for given use cases. 

  • Agile delivery-  Person must able to show design and code on a daily basis
  • Must be an experienced PySpark developer and  Scala coding.   Primary skill is PySpark
  • Must have experience in designing job orchestration, sequence, metadata design, Audit trail, dynamic parameter passing and error/exception handling
  • Good experience with unit, integration and UAT support
  • Able to design and code reusable components and functions
  • Should able to review design, code & provide review comments with justification
  • Zeal to learn new tool/technologies and adoption
  • Good to have experience with Devops and CICD
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

NeoGenCode Technologies Pvt Ltd
Akshay Patil
Posted by Akshay Patil
Bengaluru (Bangalore), Pune, Chennai
10 - 20 yrs
₹30L - ₹60L / yr
skill iconJava
skill iconSpring Boot
Microservices
Apache Kafka
skill iconAmazon Web Services (AWS)
+8 more

📍 Position : Java Architect

📅 Experience : 10 to 15 Years

🧑‍💼 Open Positions : 3+

📍 Work Location : Bangalore, Pune, Chennai

💼 Work Mode : Hybrid

📅 Notice Period : Immediate joiners preferred; up to 1 month maximum

🔧 Core Responsibilities :

  • Lead architecture design and development for scalable enterprise-level applications.
  • Own and manage all aspects of technical development and delivery.
  • Define and enforce best coding practices, architectural guidelines, and development standards.
  • Plan and estimate the end-to-end technical scope of projects.
  • Conduct code reviews, ensure CI/CD, and implement TDD/BDD methodologies.
  • Mentor and lead individual contributors and small development teams.
  • Collaborate with cross-functional teams, including DevOps, Product, and QA.
  • Engage in high-level and low-level design (HLD/LLD), solutioning, and cloud-native transformations.

🛠️ Required Technical Skills :

  • Strong hands-on expertise in Java, Spring Boot, Microservices architecture
  • Experience with Kafka or similar messaging/event streaming platforms
  • Proficiency in cloud platformsAWS and Azure (must-have)
  • Exposure to frontend technologies (nice-to-have)
  • Solid understanding of HLD, system architecture, and design patterns
  • Good grasp of DevOps concepts, Docker, Kubernetes, and Infrastructure as Code (IaC)
  • Agile/Lean development, Pair Programming, and Continuous Integration practices
  • Polyglot mindset is a plus (Scala, Golang, Python, etc.)

🚀 Ideal Candidate Profile :

  • Currently working in a product-based environment
  • Already functioning as an Architect or Principal Engineer
  • Proven track record as an Individual Contributor (IC)
  • Strong engineering fundamentals with a passion for scalable software systems
  • No compromise on code quality, craftsmanship, and best practices

🧪 Interview Process :

  1. Round 1: Technical pairing round
  2. Rounds 2 & 3: Technical rounds with panel (code pairing + architecture)
  3. Final Round: HR and offer discussion
Read more
Deqode
at Deqode
1 recruiter
Shraddha Katare
Posted by Shraddha Katare
Bengaluru (Bangalore), Pune, Chennai, Mumbai, Gurugram
5 - 7 yrs
₹5L - ₹19L / yr
skill iconAmazon Web Services (AWS)
skill iconPython
PySpark
SQL
redshift

Profile: AWS Data Engineer

Mode- Hybrid

Experience- 5+7 years

Locations - Bengaluru, Pune, Chennai, Mumbai, Gurugram


Roles and Responsibilities

  • Design and maintain ETL pipelines using AWS Glue and Python/PySpark
  • Optimize SQL queries for Redshift and Athena
  • Develop Lambda functions for serverless data processing
  • Configure AWS DMS for database migration and replication
  • Implement infrastructure as code with CloudFormation
  • Build optimized data models for performance
  • Manage RDS databases and AWS service integrations
  • Troubleshoot and improve data processing efficiency
  • Gather requirements from business stakeholders
  • Implement data quality checks and validation
  • Document data pipelines and architecture
  • Monitor workflows and implement alerting
  • Keep current with AWS services and best practices


Required Technical Expertise:

  • Python/PySpark for data processing
  • AWS Glue for ETL operations
  • Redshift and Athena for data querying
  • AWS Lambda and serverless architecture
  • AWS DMS and RDS management
  • CloudFormation for infrastructure
  • SQL optimization and performance tuning
Read more
Confidential
Confidential
Agency job
via Arnold Consultants by Sampreetha Pai
Bengaluru (Bangalore)
8 - 13 yrs
₹30L - ₹35L / yr
skill iconJava
skill iconMongoDB
skill iconC#
skill iconPython
skill iconNodeJS (Node.js)
+3 more

About this roleWe are seeking an experienced MongoDB Developer/DBA who will be

responsible for maintaining MongoDB databases while optimizing performance, security, and

the availability of MongoDB clusters. As a key member of our team, you’ll play a crucial role in

ensuring our data infrastructure runs smoothly.

You'll have the following responsibilities

 Maintain and Configure MongoDB Instances - Responsible for build, design, deploy,

maintain, and lead the MongoDB Atlas infrastructure. Keep clear documentation of the

database setup and architecture.

 Ownership of governance, defining and enforcing policies in MongoDB Atlas.Provide

consultancy in drawing the design and infrastructure (MongoDB Atlas) for use case.

 Service and Governance wrap will be in place to restrict over provisioning for server size,

number of clusters per project and scaling through MongoDB Atlas

 Gathering and documenting detailed business requirements applicable to the data

layer.Responsible for designing, configuring and managing MongoDB on Atlas.

 Design, develop, test, document, and deploy high-quality technical solutions on the

MongoDB Atlas platform based on industry best practices to solve business needs.

Resolves technical issues raised by the team and/or customer and manages escalations as

required.

 Migrate data from on-premise MongoDB and RDBMS to MongoDB AtlasCommunicate

and collaborate with other technical resources and customers in providing timely updates

on status of deliverables, shedding light on technical issues, and obtaining buy-in on

creative solutions.

 Write procedures for backup and disaster recovery.


You'll have the following skills & experience

 Excellent analytical, diagnostic skills, and problem-solving skills

 Should understand the Database concept and develop expertise in designing and

developing NoSQL databases such as MongoDB

 MongoDB query operation, import and export operation in database

 Experience in ETL methodology for performing Data Migration, Extraction,

Transformation, Data Profiling and Loading

 Migrating database by ETL, migrating database by manual process and designing,

development, implementation

 General networking skills, especially in the context of a public cloud (e.g. AWS – VPC,

subnets, routing tables, nat / internet gateways, dns, security groups)

 Experience using Terraform as an IaC tool for setting up infrastructure on AWS

CloudPerforming database backups and recovery

 Competence in at least one of the following languages (in no particular order): Java, C++,

C#, Python, Node.js (JavaScript), Ruby, Perl, Scala, Go

 Excellent communication skills, often being able to compromise but draw out risks and

constraints associated with solutions. Be able to work independently and collaborate with

other teams

 Proficiency in configuring schema and MongoDB data modeling.


 Strong understanding of SQL and NoSQL databases.

 Comfortable with MongoDB syntax.

 Experience with database security management.

 Performance Optimization - Ensure databases achieve maximum performance and

availability. Design effective indexing strategies.

Read more
Publicis Sapient
at Publicis Sapient
10 recruiters
Pooja Singh
Posted by Pooja Singh
Bengaluru (Bangalore), Mumbai, Gurugram, Noida, Hyderabad, Pune
4 - 19 yrs
₹1L - ₹15L / yr
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
Microservices
+7 more
  • Experience building large scale, large volume services & distributed apps., taking them through production and post-production life cycles
  • Experience in Programming Language: Java 8, Javascript
  • Experience in Microservice Development or Architecture
  • Experience with Web Application Frameworks: Spring or Springboot or Micronaut
  • Designing: High Level/Low-Level Design
  • Development Experience: Agile/ Scrum, TDD(Test Driven Development)or BDD (Behaviour Driven Development) Plus Unit Testing
  • Infrastructure Experience: DevOps, CI/CD Pipeline, Docker/ Kubernetes/Jenkins, and Cloud platforms like – AWS, AZURE, GCP, etc
  • Experience on one or more Database: RDBMS or NoSQL
  • Experience on one or more Messaging platforms: JMS/RabbitMQ/Kafka/Tibco/Camel
  • Security (Authentication, scalability, performance monitoring)
Read more
Product based company(Financial services)
Product based company(Financial services)
Agency job
via Qrata by Revathi Satish
Bengaluru (Bangalore)
4 - 8 yrs
₹30L - ₹35L / yr
skill iconPython
skill iconDjango
skill iconRuby
skill iconRuby on Rails (ROR)
skill iconScala
+2 more

Backend/Fullstack Developer

 

About the Company:

Company is India’s largest and best-established digital wealth management service that helps its customers create wealth for their long-and-short-term goals. Founded in 2012, Company is a pioneer in the digital financial services category and is recognised for creating simple and elegant user experiences in a complex domain. We do this by simplifying complex investing concepts and automating best practices, so our customers can grow their wealth without worry. We achieve this by combining cutting-edge technology, data-driven algorithms, awesome UX and friendly customer support. Our task is ambitious and we like to work hard as well as smart. We want to build a team that relishes challenges and contributes to a new way of thinking and investing in India. We are also invested in the growth of our colleagues and providing a supportive and thriving working environment for everyone. We have been recognised by Great Place To Work® as one of India’s best companies to work for.

Role specific info:

  • 4-12 years of experience building good quality production software
  • Excellent knowledge of at least one ecosystem based on Elixir/Phoenix, Ruby/Rails, Python/Django, Go/Scala/Clojure
  • Good OO skills, including strong design patterns knowledge
  • Familiar with datastores like MySQL, PostgreSQL, Redis, Redshift etc.
  • Familiarity with react.js/react-native, vue.js etc. • Knowledge of deploying software to AWS, GCP, Azure
  • Knowledge of software best practices, like Test-Driven Development (TDD) and Continuous Integration (CI)

 

What would you do here:

  • Writing quality code using language best practices.
  • Working in a highly collaborative team.
  • Building good software using the latest tools and techniques.
  • Participating in design reviews, coding modules, code reviews and unit testing.
  • Taking ownership of quality and usability of your code.
  • Mentoring co-workers.
  • Leading efforts in improving technology architecture.
Read more
Number Theory
at Number Theory
3 recruiters
Nidhi Mishra
Posted by Nidhi Mishra
Gurugram
2 - 4 yrs
₹10L - ₹15L / yr
Hadoop
Spark
HDFS
skill iconScala
skill iconJava
+2 more
Position Overview: Data Engineer (2+ yrs)
Our company is seeking to hire a skilled software developer to help with the development of our AI/ML platform.
Your duties will primarily revolve around building Platform by writing code in Scala, as well as modifying platform
to fix errors, work on distributed computing, adapt it to new cloud services, improve its performance, or upgrade
interfaces. To be successful in this role, you will need extensive knowledge of programming languages and the
software development life-cycle.

Responsibilities:
 Analyze, design develop, troubleshoot and debug Platform
 Writes code and guides other team membersfor best practices and performs testing and debugging of
applications.
 Specify, design and implementminor changes to existing software architecture. Build highly complex
enhancements and resolve complex bugs. Build and execute unit tests and unit plans.
 Duties and tasks are varied and complex, needing independent judgment. Fully competent in own area of
expertise

Experience:
The candidate should have about 2+ years of experience with design and development in Java/Scala. Experience in
algorithm, Distributed System, Data-structure, database and architectures of distributed System is mandatory.

Required Skills:
1. In-depth knowledge of Hadoop, Spark architecture and its componentssuch as HDFS, YARN and executor, cores and memory param
2. Knowledge of Scala/Java.
3. Extensive experience in developing spark job. Should possess good Oops knowledge and be aware of
enterprise application design patterns.
4. Good knowledge of Unix/Linux.
5. Experience working on large-scale software projects
6. Keep an eye out for technological trends, open-source projects that can be used.
7. Knows common programming languages Frameworks
Read more
senrysa technologies (p) ltd
Joyita Guha
Posted by Joyita Guha
Bengaluru (Bangalore)
4 - 10 yrs
₹6L - ₹15L / yr
skill iconRuby
skill iconRuby on Rails (ROR)
DevOps
skill iconAmazon Web Services (AWS)

Job Description

We are looking for a ROR developer. If you're a creative problem solver who is eager to develop amazing products and hungry for a new adventure, a word class workplace is waiting for you in the heart of Bengaluru / Kolkata or remote.

  • Production experience in Ruby.
  • A completed technical degree in Computer Science or any related fields.
  • 3+ years of professional product development experience.
  • Being comfortable with microservices architectures, API-based
  • You are a pragmatic programmer who understands what is needed to get things done.
  • Problem solving and collaborative mindset.
  • Experience working with DevOps (Docker, Kubernetes, Terraform).
  • Experience with AWS (RDS, DDB, Lambda, CW, EC2, SQS, SNS, Cognito, Kinesis).
  • Experience with performance improvements (Caching Techniques, SQL Query Optimization, Performance monitoring and profiling.
  • Deep understanding of service-oriented and microservices architectural patterns, troubleshooting methods and best practices.
  • Takes end to end ownership of the development and operation of complete features.
Read more
Jivox Software India Pvt Ltd
Kiranmai Banisetty
Posted by Kiranmai Banisetty
Bengaluru (Bangalore)
3 - 6 yrs
₹15L - ₹19L / yr
skill iconScala
skill iconJava
Data Structures
Algorithms
skill iconAmazon Web Services (AWS)

Your Opportunity

  • Own and drive business features into tech requirements
  • Design & develop large scale real time server side systems
  • Quickly create quality prototypes
  • Staying updated on emerging technologies
  • Ensuring that all deliverables adhere to our world class standards
  • Promote coding best practices
  • Mentor and develop junior developers in the team

Required Experience

  • 4+ years of relevant experience as described below
  • Excellent grasp of Core Java, Multi Threading and OO design patterns 
  • Experience with Scala, functional, reactive programming and Akka/Play is a plus
  • Excellent understanding of data structures and algorithms
  • Solid grasp of large scale distributed real time systems
  • Prior experience on building a scalable and resilient micro service
  • Solid understanding of relational databases, NoSQL databases and Caching systems
  • Good understanding of Big Data technologies such as Spark, Hadoop is a plus
  • Experience on one of AWS, Azure or GCP 

Who you are

  • You have excellent and effective communication and collaborative skills
  • You love problem solving
  • You stay up to date with the latest technologies and then apply them in real life
  • You love paying attention to detail
  • You thrive in meeting tight deadlines and prioritising workloads
  • Ability to collaborate across multiple functions

Education

Bachelor’s degree in Engineering or equivalent experience within the field

Read more
JaMocha Tech
at JaMocha Tech
1 video
1 recruiter
Sarika Kumari
Posted by Sarika Kumari
Bengaluru (Bangalore)
2 - 12 yrs
₹5L - ₹20L / yr
skill iconJava
Data Structures
Algorithms
skill iconScala
skill iconC++
+6 more

Job Position: JAVA Software Engineer
Job Location: Bangalore

Experience: 2 to 12 years

Job Responsibilities:
• Responsible for developing Web Applications using J2EE components.
• Responsible for developing standalone applications using Core Java and JDBC.

Desired Qualification: BE / B.Tech / MCA / MSc in Computer Science or IT

Added advantage: Sun Certified Java Programmer (SCJP), Sun Certified Web Component Developer (SCWCD) certification

Desired Skills:

  • Strong skills in JAVA programming languages and focus specially on the following areas Servlets, JSP, Web Services, Struts framework, JDBC/JPA/mybatis (ibatis)/Hibernate, Threads, Collections and Design Patterns.
  • Strong skills in web technologies HTML 4/5, JQuery, AJAX, Java Script and CSS.
  • Thorough understanding of Object Orientation.
  • Above average skills in SQL with familiarity in any of the standard RDBMS – MySQL, MSSQL or Oracle.
  • Must have good written and oral communication in English language.
  • Must be completely hands-on and an individual contributor and a self-starter; must be able to complete assignments (suitable to his/her experience) with minimal assistance.
  • Very good at Analytical and Logical skills.
  • Solid understanding of the web development lifecycle.
  • Ability to learn quickly and deliver software on time.

 

Read more
Scienaptic Systems
at Scienaptic Systems
1 recruiter
Zoheab Rehaman
Posted by Zoheab Rehaman
Bengaluru (Bangalore)
9 - 12 yrs
₹20L - ₹40L / yr
skill iconJava
Big Data
skill iconScala
Scienaptic (www.scienaptic.com) is a new age technology and analytics company based in NY and Bangalore. Our mission is to infuse robust decision science into organizations. Our mantra to achieve our mission is to - reduce friction- among technology, processes and humans. We believe that good design thinking needs to permeate all aspects of our activities so that our customers get the best possible aesthetic and least frictious experience of our software and services. As a Prinicipal Software development Engineer you will be responsible for the development and augmentation of the software components which will be used to solve the analytics problems of large enterprises. These components are highly scalable, connect with multiple data sources and implement some of the complex algorithms We are funded by very senior and eminent business leaders in India and US. Our lead investor is Pramod Bhasin, who is known as a pioneer of ITES revolution. We have the working environment of a new age, cool startup. We are firm believers that the best talent grounds will be non-hierarchical in structure and spirit. We expect you to enjoy, thrive and empower others by progressing that culture. Requirements : - Candidate should have all round experience in developing and delivering large-scale business applications in scale-up systems as well as scale-out distributed systems. - Identify the appropriate software technology / tools based on the requirements and design elements contained in a system specification - Should implement complex algorithms in a scalable fashion. - Work closely with product and Analytic managers, user interaction designers, and other software engineers to develop new product offerings and improve existing ones. Qualifications/Experience : - Bachelor's or Master's degree in computer science or related field - 10 to 12 years of experience in core Java programming: JDK 1.7/JDK 1.8 and Familiarity with Big data systems like Hadoop and Spark is an added bonus - Familiarity with dependency injection, Concurrency, Guice/Spring - Familiarity with JDBC API / Databases like MySQL, Oracle, Hadoop - Knowledge of graph databases and traversal - Knowlede of SOLR/ElasticSearch, Cloud based deployment would be preferred
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos