Cutshort logo
Thoughtworks logo
Lead Consultant - Data Engineer
Lead Consultant - Data Engineer
Thoughtworks's logo

Lead Consultant - Data Engineer

Sunidhi Thakur's profile picture
Posted by Sunidhi Thakur
10 - 13 yrs
Best in industry
Bengaluru (Bangalore)
Skills
Data modeling
PySpark
Data engineering
Big Data
Hadoop
Spark
Apache Spark
skill iconPython
skill iconScala
skill iconJava
skill iconAmazon Web Services (AWS)
Google Cloud Platform (GCP)
Apache Hive
Apache HBase
Cassandra

Lead Data Engineer

 

Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.

 

Job responsibilities

 

·      You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems

·      You will partner with teammates to create complex data processing pipelines in order to solve our clients' most ambitious challenges

·      You will collaborate with Data Scientists in order to design scalable implementations of their models

·      You will pair to write clean and iterative code based on TDD

·      Leverage various continuous delivery practices to deploy, support and operate data pipelines

·      Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available

·      Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions

·      Create data models and speak to the tradeoffs of different modeling approaches

·      On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product

·      Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process

·      Assure effective collaboration between Thoughtworks' and the client's teams, encouraging open communication and advocating for shared outcomes

 

Job qualifications Technical skills

·      You are equally happy coding and leading a team to implement a solution

·      You have a track record of innovation and expertise in Data Engineering

·      You're passionate about craftsmanship and have applied your expertise across a range of industries and organizations

·      You have a deep understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop

·      You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting

·      Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions

·      You are comfortable taking data-driven approaches and applying data security strategy to solve business problems

·      You're genuinely excited about data infrastructure and operations with a familiarity working in cloud environments

·      Working with data excites you: you have created Big data architecture, you can build and operate data pipelines, and maintain data storage, all within distributed systems

 

Professional skills


·      Advocate your data engineering expertise to the broader tech community outside of Thoughtworks, speaking at conferences and acting as a mentor for more junior-level data engineers

·      You're resilient and flexible in ambiguous situations and enjoy solving problems from technical and business perspectives

·      An interest in coaching others, sharing your experience and knowledge with teammates

·      You enjoy influencing others and always advocate for technical excellence while being open to change when needed

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Thoughtworks

Founded :
1993
Type :
Products & Services
Size :
5000+
Stage :
Profitable

About

Founded in 1993, we’ve grown from a small team in Chicago to a leading software consultancy of more than 8000 Thoughtworkers in 17 countries. Our cross-functional teams of strategists, developers, data engineers, and designers bring over two decades of global experience to every partnership.

 

Thoughtworks invented the concept of distributed agile and we know how to harness the power of global teams to deliver software excellence at scale. Today we help our clients to create their own path to digital fluency and to build organizational resilience to navigate the future.


Our job is to foster a vibrant community where people have the freedom to make an extraordinary impact on the world through technology.

 

As a Thoughtworker, you are free to seek out the most ambitious challenges. Free to change career paths. Free to use technology as a tool for social change. Free to be yourself.

Read more

Company video

Thoughtworks's video section
Thoughtworks's video section

Connect with the team

Profile picture
Yogita Singh
Profile picture
Apoorv Singhal
Profile picture
Santhosh Joshua
Profile picture
Vidyashree Kulkarni
Profile picture
Sana Singh
Profile picture
Nargise Manova
Profile picture
Ranjith Prabhu
Profile picture
M I Shifa Samrin
Profile picture
Valarmathi M
Profile picture
Anubhuti Pundir
Profile picture
Abhidev Gwari
Profile picture
Diksha Kalucha
Profile picture
Ramya S
Profile picture
Logeshwaran M
Profile picture
Sukriti Sharma
Profile picture
Megha Wadhwa
Profile picture
Jyoti Deswal
Profile picture
Anugrah Mathur
Profile picture
Aaliya Thaver
Profile picture
Aasif Syed
Profile picture
Maria Nancy
Profile picture
Neha Sharma
Profile picture
Charani GP
Profile picture
Ambica Bharadwaj
Profile picture
pavan Juluri
Profile picture
nadeem Shaikh
Juana Fernandes
icon
Connect

Company social profiles

bloginstagramlinkedintwitterfacebook

Similar jobs

PGAGI
Javeriya Shaik
Posted by Javeriya Shaik
Remote only
0 - 0.6 yrs
₹2L - ₹2L / yr
skill iconPython
Large Language Models (LLM)
Natural Language Processing (NLP)
skill iconDeep Learning
FastAPI
+1 more

We're at the forefront of creating advanced AI systems, from fully autonomous agents that provide intelligent customer interaction to data analysis tools that offer insightful business solutions. We are seeking enthusiastic interns who are passionate about AI and ready to tackle real-world problems using the latest technologies.


Duration: 6 months


Perks:

- Hands-on experience with real AI projects.

- Mentoring from industry experts.

- A collaborative, innovative and flexible work environment

After completion of the internship period, there is a chance to get a full-time opportunity as AI/ML engineer (Up to 12 LPA).


Compensation:

- Joining Bonus: A one-time bonus of INR 2,500 will be awarded upon joining.

- Stipend: Base is INR 8000/- & can increase up to 20000/- depending upon performance matrix.

Key Responsibilities

  • Experience working with python, LLM, Deep Learning, NLP, etc..
  • Utilize GitHub for version control, including pushing and pulling code updates.
  • Work with Hugging Face and OpenAI platforms for deploying models and exploring open-source AI models.
  • Engage in prompt engineering and the fine-tuning process of AI models.

Requirements

  • Proficiency in Python programming.
  • Experience with GitHub and version control workflows.
  • Familiarity with AI platforms such as Hugging Face and OpenAI.
  • Understanding of prompt engineering and model fine-tuning.
  • Excellent problem-solving abilities and a keen interest in AI technology.


To Apply Click below link and submit the Assignment

https://pgagi.in/jobs/28df1e98-f0c3-4d58-9509-d5b1a4ea9754

Read more
withRemote
at withRemote
2 candid answers
Sadhana MS
Posted by Sadhana MS
Remote only
5 - 12 yrs
₹15L - ₹25L / yr
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
skill iconKotlin
+3 more

Job Title: Backend Developer (Kotlin/Java)

Experience: 5-7 Years

Location: Remote (Candidates from Pune/Mumbai preferred for training-related travel)

Salary: ₹15-25 LPA CTC (1L up to 1.5L In hand Per Month)


Job Description:

We are seeking a highly skilled Backend Developer with expertise in Kotlin/Java (not Android) to join our dynamic team. The ideal candidate will have experience building scalable backend solutions, working with relational databases, and utilizing modern frameworks like Spring and Webflux.

This is a remote role, but candidates based in Pune/Mumbai who can travel for training purposes will be preferred.


  • Must-Have Skills:Kotlin/Java backend development (5+ years)
  • Spring Boot/Webflux (Reactor) (1+ year)
  • Relational Databases (PostgreSQL preferred) (2+ years)
  • Strong problem-solving skills and attention to detail
  • Excellent communication skills and professionalism


  • Key Responsibilities:Develop and maintain backend services using Kotlin/Java.
  • Design and implement scalable microservices with Spring Boot/Webflux (Reactor).
  • Work with relational databases (PostgreSQL preferred) and ensure optimal data handling.
  • Optimize backend performance and ensure system reliability.
  • Collaborate with frontend, DevOps, and product teams to deliver high-quality software solutions.
  • Ensure best practices in coding, security, and system design.


  • Nice to Have:Experience with NoSQL databases (Cassandra/MongoDB)
  • Knowledge of Kafka for event-driven architecture
  • Familiarity with AWS, Docker for cloud-based deployments
  • Exposure to full-stack development with React
  • Understanding of ClickHouse for analytics


Company:

The client is a technology company specializing in custom software development, mobile app creation, cloud services, data analytics, applied AI, cybersecurity, web design, and UX/UI design. They emphasize building robust, scalable, and world-class software products and applications. Their expertise spans various technology stacks, including MERN, MEAN, applied AI, Oracle, AWS, GCP, Microsoft Azure, and PAAS. They have developed products such as Digirestro, Task Manager, and Attendance Management. The company operates from multiple locations, including Pune and Nagpur in India, and Kendall Park, New Jersey, USA.


  • Why Join Us?Competitive salary (₹15-25 LPA)
  • Remote work flexibility (with occasional travel to Pune/Mumbai for training)
  • Opportunity to work on modern tech stacks
  • Collaborative and professional work environment


Read more
Care Infotech
Aman Shaikh
Posted by Aman Shaikh
Noida
4 - 6 yrs
₹12L - ₹13L / yr
Computer Networking
Network
Cisco Certified Network Associate (CCNA)
Neural networks
LAN
+1 more

L2 Network Engineer (NOC - Operation & Monitoring)

We are looking for experienced L2 Network Engineers to join our NOC (Network Operations Center) team. The ideal candidate should have 3+ years of experience in IT networking with expertise in troubleshooting, configuration, and monitoring of network devices.

Job Details:

  • Location: Sec-24, Noida, UP
  • Working Hours: 9:30 AM – 6:30 PM
  • Support Days: Monday to Saturday
  • Experience Required: 3+ years in IT Networking
  • Education: B.Tech / MCA (Passing year 2021, No distance learning)
  • Certifications: CCNA / CCNP
  • Salary: ₹3.0 - ₹3.6 LPA

Key Responsibilities:

  • Monitor and manage network performance to ensure smooth operations.
  • Troubleshoot and configure routers, switches, and other network devices.
  • Work with ITSM tools for incident and problem management.
  • Analyze and resolve network issues related to IP addressing, subnetting, VLANs, and routing protocols (OSPF, BGP).
  • Implement security policies, access controls, and network configurations as per company standards.
  • Support and optimize LAN/WAN infrastructure to ensure minimal downtime.
  • Configure and manage MPLS, SD-WAN, and VLAN Trunking Protocol (VTP).
  • Handle STP (Spanning Tree Protocol), Inter-VLAN Routing, and SNMP monitoring.

Technical Skills Required:

  • Networking Protocols: OSPF, BGP, MPLS, ICMP, SNMP
  • Network Components: Routers, Switches, Firewalls, Load Balancers
  • IP Addressing & Subnetting: IPv4, IPv6, CIDR
  • Network Monitoring & ITSM Tools: Experience with at least one tool
  • LAN & WAN Technologies: VLAN, Trunking, Inter-VLAN Routing
  • Security & Management: Access Control, SNMP Monitoring, Community Strings

Who Should Apply?

  • Candidates with strong troubleshooting and network configuration skills.
  • Those looking to grow in NOC operations, network monitoring, and L2 network support.
  • Individuals with a CCNA/CCNP certification and experience in handling enterprise-level networks.


Read more
Programmingcom
Harshita Saxena
Posted by Harshita Saxena
Bengaluru (Bangalore)
6 - 12 yrs
₹9L - ₹15L / yr
Software Testing (QA)
Test Automation (QA)
Appium
Selenium
Team Management

Position- Automation Lead 

Location Bangalore (Wfo/Hybrid)

Experience :5+ years


Qualifications Bachelor of Engineering/ Masters in Computer Science (OR) any equivalent 


Job Overview 


  • Good experience & strong knowledge in Java programming with Object Oriented Programming methodologies
  • Good experience in web/mobile application UI automation & related tools/packages (like selenium/appium)
  • Good exposure to functional testing, regression testing, api testing
  • Very good in analytical & logical thinking on different perspectives. Ability to handle challenges & resolve blockers independently
  • Self-driven, Accountability and ownership of problems to achieve team goals & ability to contribute independently
  • Good team player & proactive in giving new ideas/suggestions/solutions & constructive analysis of team member’s ideas
  • Hands on experience with different tools - Testing tool, Version Control systems (Git), Test Management Software, Bug Tracking software and CI tools (Jenkins etc.)


Read more
DuckTale IT Services Pvt Ltd
Shivani Gautam
Posted by Shivani Gautam
Remote, Mohali
1 - 5 yrs
₹2L - ₹8L / yr
skill iconAngularJS (1.x)
skill iconAngular (2+)

Designation: Angular Developer

Experience: 1 to 5 years

Skill Set: Angular 2+ Versions, HTML, CSS

Requirements:

  • Must have experience on Angular 2+ versions
  • Design, develop, test, deploy, maintain and improve software in the area of Web Development.
  • Ensure the technical feasibility of UIUX designs.
  • Manage individual project priorities, deadlines and deliverable.
  • Making complex technical and design decisions for Angular projects.
  • Adapting interface for modern internet applications using the latest front-end technologies.
Read more
Bengaluru (Bangalore)
0 - 4 yrs
₹1L - ₹3L / yr
MS-Word
MS-PowerPoint
MS-Excel
Communication Skills
Customer Relationship Management (CRM)
The desired candidate preferably a female should be an MBA/Mcom/ should possess excellent written and oral communications with at least  1-2 years of experience as an admin/support in financial services. Should communicate and coordinate with internal departments to handle & resolve client’s complaints, direct requests and unresolved issues to the designated resource. Should be able to manage the day-to-day operations of the office. Organizie and maintain files and records etc. should be ·         Proficient in financial terminologies. ·         Good mathematical skills ·         MS Office Prior experience in a financial institution/wealth management firm would be added advantage
Read more
SportsActive Online Services Pvt Ltd
Thomas Abraham
Posted by Thomas Abraham
Remote, Pune
2 - 4 yrs
₹2L - ₹5L / yr
skill icon.NET
skill iconC#
skill iconHTML/CSS
Entity Framework
SQL
+5 more
Full Stack Dot Net Developer , BE/BTECH/ME/Mtech/MCA having following skillsets, .NET, ASP.NET, ADO.NET, C#, HTML, CSS, JQuery, Entity Framework, SQL, Microsoft SQL Servers Framework, 4.0, 4.1, 4.2, 4.3, 4.4, 4.5 and above Bootstrap, HTML, able to do the following,
1) understand the requirement gathering process completely 
2) Understand Change Request Proess and other Software Development Process
3) Prepare Tecnical Documenation .
4) Problem Solving ad thinking laterallyas part of a team or individually to meet needs of the project 
5) Manual Testing , unit testing of the software developed 
6) Experience in Deployment and Build Management 
7) Consulting Client /colleagues concerning the maintennce and performance of software appllications and obraining information clarify details and implement information 
Read more
It's a USA based IT company with the Staregnth of 5000 peopl
It's a USA based IT company with the Staregnth of 5000 peopl
Agency job
via Absolute Talent by Rahul Sharma
Noida, NCR (Delhi | Gurgaon | Noida)
10 - 20 yrs
₹26L - ₹60L / yr
Recruitment/Talent Acquisition
Recruitment
Talent management
Human Resources (HR)
Us staffing

Job description

Job Description:

You will lead a team of specialists in the areas of talent acquisition, manpower planning and budgeting. You will manage manpower planning in accordance to expansion requirements and provide strategic advice on talent acquisition and workforce management.
You should have a degree in the relevant business discipline and 10 years of experience in an MNC or local organization with regional footprint. You will come with a proven track record in managing a team and strong stakeholder management skills.
You must be able to successfully manage a team, prioritize and achieve hiring goal against a timeline and have experience setting benchmarks, metrics, and understand how to prioritize to hit all customer SLAs. You will need to thrive in an innovative, fast-paced environment, can roll up their sleeves, work hard, have fun, and get the job done.

Job Responsibilities:

1.   Should have experience in Delivery, Client Interaction and Team Handling.

2.   Preferably should have experience working on Direct Clients and Implementation Partners.

3.   Good to have: knowledge of US Tax terms (1099, C2C, W2)/Also should have experience working on Job Portals (Naukri, Dice, Monster,…etc)

4.   Should have excellent communication skills.

5.   Should be proactive and ensure to revert / submit profiles with quick turnaround time for timely submittals to meet sharp deadlines of submittals for each requisition. (Plan & execute the client deliverables).

6.   Making use of effective resourcing strategies, such as head hunting, Internet sourcing, networking, employee referrals.

7.   Responsible for achieving a good conversion ratio of submittals into interview and placement.

8.   Should be well versed with various visas.

9.   Should be an expert in full recruitment life cycle.


Objectives/ Deliverable:

1.   Lead an experienced team of Recruiters to support the hiring demand of our rapidly growing businesses and deliver annual hiring plan and target

2.   Manage team performance and hire and develop the best

3.   Collaborate with customers and peers to drive hiring initiatives and experiments

4.   Provide recruiting leadership and consultancy in participating business/HR projects

5.   Optimize operational process flows and identify ways to deliver a consistent, scalable, customer obsessed recruitment experience to all stakeholders

6.   Drive simplification and contribute to increase productivity across the team

7.   Drive the best candidate and referee experience, delivering on key internal SLA’s

8.   Drive candidate feedback surveys and participate in other recruiting/HR initiatives to provide the best recruiting support

Read more
Artificial Learning Systems
at Artificial Learning Systems
1 video
3 recruiters
Bindu Varma
Posted by Bindu Varma
Bengaluru (Bangalore)
3 - 7 yrs
₹1L - ₹2L / yr
skill iconHTML/CSS
skill iconAngular (2+)
skill iconBootstrap
skill iconJavascript
The UI Developer is entrusted with the following responsibilities:  Design and build UIs on any server platform in a team environment  Integrate front-end application with the application business layer  Ensure design consistency with clients development standards and guidelines  Conduct usability testing to resolve interface problems  Stick to established coding standards and group procedures individually and in teams  Guide and maintain developer teams and best practices
Read more
Fintech company
Fintech company
Agency job
via Ixora Global Services by Tilak Perala
Bengaluru (Bangalore)
3 - 5 yrs
₹7L - ₹10L / yr
Spring
skill iconJava
Struts
Hibernate (Java)
skill iconJavascript
+4 more
Greyamp Consulting is in the Digital Transformation space working with clients to enable Business agility while also helping them build sustainable talent within their organisations As part of our team, Greyamp is looking to hire java developers who would be interested in building a product for Greyamp to help our enterprise clients solve real problems, build lasting solutions along with build capability in their teams. Must Have: - Bachelors in Computer Science; Masters Preferred - 3 - 5 yrs of working experience - Strong working knowledge of Java, Spring Frameworks, JavaScript, Python, PHP, Ruby, MySql&NoSql. - Hands on experience in databased like MySQL, PostgreSQL, NoSQL, etc. Familiarity with Elastic, Cassandra etc is a bonus - Strong practical knowledge of OO including design patters knowledge - Exposure to and Docker containers & Kubernetes etc - Good written verbal communication skills - Willingness to travel - Exposure to Agile methodologies Good to Have: - Xtreme programming practices - Exposure to Client facing roles projects is a added advantage What you can expect: - An opportunity to experience multiple contexts, situations, and cultures - Work on path breaking products in the area of digital transformation - A chance to work with large enterprises across all levels to bridge technology, business & people - Work with the best minds in the industry - A learning curve to be jealous of - Opportunity to get your hands dirty with initiatives across disciplines in the organisation and continuously push limits
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
21,01,133
Matches delivered
37,12,187
Network size
15,000
Companies hiring
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos