Cutshort logo
For Employers
Thoughtworks logo
Lead Consultant: Data Engineer
Lead Consultant: Data Engineer
Thoughtworks's logo

Lead Consultant: Data Engineer

Vidyashree Kulkarni's profile picture
Posted by Vidyashree Kulkarni
9 - 15 yrs
Best in industry
Remote only
Skills
PySpark
Data engineering
Big Data
Hadoop
Spark
skill iconScala
Data Structures
Data modeling
Data security
Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.

Job responsibilities
  • You will partner with teammates to create complex data processing pipelines in order to solve our clients' most complex challenges
  • You will collaborate with Data Scientists in order to design scalable implementations of their models
  • You will pair to write clean and iterative code based on TDD
  • Leverage various continuous delivery practices to deploy, support and operate data pipelines
  • Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
  • Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
  • Create data models and speak to the tradeoffs of different modeling approaches
  • Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process
  • Assure effective collaboration between Thoughtworks' and the client's teams, encouraging open communication and advocating for shared outcomes
Job qualifications

Technical skills

  • You have a good understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
  • You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
  • Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions
  • You are comfortable taking data-driven approaches and applying data security strategy to solve business problems
  • Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
  • You're genuinely excited about data infrastructure and operations with a familiarity working in cloud environments
  • Professional skills
  • You're resilient and flexible in ambiguous situations and enjoy solving problems from technical and business perspectives
  • An interest in coaching, sharing your experience and knowledge with teammates
  • You enjoy influencing others and always advocate for technical excellence while being open to change when needed
  • Presence in the external tech community: you willingly share your expertise with others via speaking engagements, contributions to open source, blogs and more
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Thoughtworks

Founded :
1993
Type :
Products & Services
Size :
5000+
Stage :
Profitable

About

Founded in 1993, we’ve grown from a small team in Chicago to a leading software consultancy of more than 8000 Thoughtworkers in 17 countries. Our cross-functional teams of strategists, developers, data engineers, and designers bring over two decades of global experience to every partnership.

 

Thoughtworks invented the concept of distributed agile and we know how to harness the power of global teams to deliver software excellence at scale. Today we help our clients to create their own path to digital fluency and to build organizational resilience to navigate the future.


Our job is to foster a vibrant community where people have the freedom to make an extraordinary impact on the world through technology.

 

As a Thoughtworker, you are free to seek out the most ambitious challenges. Free to change career paths. Free to use technology as a tool for social change. Free to be yourself.

Read more

Company video

Thoughtworks's video section
Thoughtworks's video section

Connect with the team

Profile picture
Yogita Singh
Profile picture
Apoorv Singhal
Profile picture
Santhosh Joshua
Profile picture
Vidyashree Kulkarni
Profile picture
Sana Singh
Profile picture
Nargise Manova
Profile picture
Ranjith Prabhu
Profile picture
M I Shifa Samrin
Profile picture
Valarmathi M
Profile picture
Anubhuti Pundir
Profile picture
Abhidev Gwari
Profile picture
Diksha Kalucha
Profile picture
Ramya S
Profile picture
Logeshwaran M
Profile picture
Sukriti Sharma
Profile picture
Megha Wadhwa
Profile picture
Jyoti Deswal
Profile picture
Anugrah Mathur
Profile picture
Aaliya Thaver
Profile picture
Aasif Syed
Profile picture
Maria Nancy
Profile picture
Neha Sharma
Profile picture
Charani GP
Profile picture
Ambica Bharadwaj
Profile picture
pavan Juluri
Profile picture
nadeem Shaikh
Juana Fernandes
icon
Connect

Company social profiles

bloginstagramlinkedintwitterfacebook

Similar jobs

Remote only
5 - 7 yrs
₹7L - ₹12L / yr
Playwright
TypeScript
skill iconJavascript
  • Web Application Testing: Design, document, and execute comprehensive functional test plans and test cases for complex, highly interactive web applications, ensuring they meet specified requirements and provide an excellent user experience.
  • Backend API Testing: Possess deep expertise in validating backend RESTful and/or SOAP APIs. This includes testing request/response payloads, status codes, data integrity, security, and robust error handling mechanisms.
  • Data Validation with SQL: Write and execute complex SQL queries (joins, aggregations, conditional logic) to perform backend data checks, verify application states, and ensure data integrity across integration points.
  • I Automation (Playwright & TypeScript):
  • Design, develop, and maintain robust, scalable, and reusable UI automation scripts using Playwright and TypeScript.
  • Integrate automation suites into Continuous Integration/Continuous Deployment (CI/CD) pipelines.
  • Implement advanced automation patterns and frameworks (e.g., Page Object Model) to enhance maintainability.
  • Prompt-Based Automation: Demonstrate familiarity or hands-on experience with emerging AI-driven or prompt-based automation approaches and tools to accelerate test case generation and execution.
  • API Automation: Develop and maintain automated test suites for APIs to ensure reliability and performance.
  • JMeter Proficiency: Utilize Apache JMeter to design, script, and execute robust API load testing and stress testing scenarios.
  • Analyze performance metrics, identify bottlenecks (e.g., response time, throughput), and provide actionable reports to development teams.
  • Experience: 4+ years of professional experience in Quality Assurance and Software Testing, with a strong focus on automation.
  • Automation Stack: Expert-level proficiency in developing and maintaining automation scripts using Playwright and TypeScript.
  • Testing Tools: Proven experience with API testing tools (e.g., Postman, Swagger) and strong functional testing methodologies.
  • Database Skills: Highly proficient in writing and executing complex SQL queries for data validation and backend verification.
  • Performance: Hands-on experience with Apache JMeter for API performance and load testing.
  • Communication: Excellent communication and collaboration skills to work effectively with cross-functional teams (Developers, Product Managers).
  • Problem-Solving: Strong analytical and debugging skills to efficiently isolate and report defects.


Read more
A B2B content marketplace
A B2B content marketplace
Agency job
via Qrata by Monish Kulkarni
Mumbai
5 - 10 yrs
₹40L - ₹80L / yr
Product Management
Product Strategy
Product development
  • As Director of Product, you are at the helm of the Product function of the organization. Your position overlooks the entire product organization right from cultivating a high growth team, overseeing the day to day management of ongoing projects to strategizing with CTO/founders, and planning the execution of upcoming projects/initiatives. 
  • You are responsible for recruiting and mentoring product teams, setting up proper team structures, and keeping everyone aligned with the Vision at all times. 
  • You are also required to maintain and oversee the design and data functions within the product org and build scalable structures for efficient output.
  • You're expected to be involved in decisions (with Founders/CTO) that are to be taken in driving New Product initiatives and plan their execution and keep the management updated on the progress.
·  In the end, you're expected to be great at execution as well as management.
Read more
Agiletech Info Solutions pvt ltd
Chennai
4 - 8 yrs
₹4L - ₹15L / yr
ETL
Informatica
Data Warehouse (DWH)
Spark
SQL
+1 more
We are looking for a Data Engineer to join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoy optimizing data systems and building them from the ground up.

The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.
Responsibilities for Data Engineer
• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes,
optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data
from a wide variety of data sources using SQL and AWS big data technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer
acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with
data-related technical issues and support their data infrastructure needs.
• Create data tools for analytics and data scientist team members that assist them in building and
optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
Qualifications for Data Engineer
• Experience building and optimizing big data ETL pipelines, architectures and data sets.
• Advanced working SQL knowledge and experience working with relational databases, query
authoring (SQL) as well as working familiarity with a variety of databases.
• Experience performing root cause analysis on internal and external data and processes to
answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and
workload management.
• A successful history of manipulating, processing and extracting value from large disconnected
datasets.
Read more
Getinz
at Getinz
11 recruiters
kousalya k
Posted by kousalya k
Bengaluru (Bangalore), Pune
3 - 7 yrs
₹6L - ₹15L / yr
skill iconJava
skill iconSpring Boot
Data Structures
Algorithms
  • Candidates with good experience with SDLC
  • Minimum 3+ years of hard core coding experience is mandatory
  • Excellent hands on coding in Java, J2EE, Spring, Struts
Read more
DigiChefs
at DigiChefs
2 recruiters
Marmik Ajmera
Posted by Marmik Ajmera
Mumbai
3 - 5 yrs
₹4L - ₹5L / yr
Human Resources (HR)
Recruitment
Recruitment/Talent Acquisition
Screening
HR management

Job Responsibilities.

  • Develop and implement HR strategies and initiatives aligned with the overall business strategy.
  • Bridge management and employee relations by addressing demands, grievances, or other issues.
  • Manage the recruitment and selection process.
  • Support current and future business needs through the development, engagement, motivation, and preservation of human capital.
  • Develop and monitor overall HR strategies, systems, tactics, and procedures across the organization.
  • Nurture a positive working environment.
  • Oversee and manage a performance appraisal system that drives high performance.
  • Maintain pay plan and benefits program.
  • Assess training needs to apply and monitor training programs.
  • Report to management and provide decision support through HR metrics.
  • Ensure legal compliance throughout human resource management.
Read more
Technology service company
Technology service company
Agency job
via Jobdost by Riya Roy
Remote only
5 - 10 yrs
₹12L - ₹20L / yr
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
  • Preferred Education & Experience:

  • Bachelor’s or master’s degree in Computer Engineering, Computer Science, Computer Applications, Mathematics, Statistics or related technical field or equivalent practical experience. Relevant experience of at least 3 years in lieu of above if from a different stream of education.

  • Well-versed in and 5+ years of hands-on demonstrable experience with:
    ▪ Object Oriented Modeling, Design, & Programming
    ▪ Microservices Architecture, API Design, & Implementation
    ▪ Relational, Document, & Graph Data Modeling, Design, & Implementation

  • Well-versed in and hands-on demonstrable experience with:
    ▪ Stream & Batch Big Data Pipeline Processing
    ▪ Distributed Cloud Native Computing
    ▪ Serverless Computing & Cloud Functions

  • 5+ years of hands-on development experience in Java programming.

  • 3+ years of hands-on development experience in one or more libraries & frameworks such as Spring Boot, Apache Camel, Akka, etc.;
    extra points if you can demonstrate your knowledge with working examples.
    2+ years of hands-on development experience in one or more Relational and NoSQL datastores such as Amazon S3, Amazon DocumentDB, Amazon Elasticsearch Service, Amazon Aurora, AWS DynamoDB, Amazon Athena, etc.

  • 2+ years of hands-on development experience in one or more technologies such as Amazon Simple Queue Service, Amazon Kinesis, Apache Kafka, AWS Lambda, AWS Batch, AWS Glue, AWS Step Functions, Amazon API Gateway, etc.

  • 2+ years of hands-on development experience in one or more technologies such as AWS Developer Tools, AWS Management & Governance, AWS Networking and Content Delivery, AWS Security, Identity, and Compliance, etc.

  • Well-versed in Virtualization & Containerization; must demonstrate experience in technologies such as Kubernetes, Istio, Docker, OpenShift, Anthos, Oracle VirtualBox, Vagrant, etc.

  • Well-versed with demonstrable working experience with API Management, API Gateway, Service Mesh, Identity & Access Management, Data Protection & Encryption.

  • Hands-on experience with demonstrable working experience with DevOps tools and platforms viz., Jira, GIT, Jenkins, Code Quality & Security Plugins, Maven, Artifactory, Terraform, Ansible/Chef/Puppet, Spinnaker, etc.

  • Well-versed with Storage, Networks and Storage Networking basics which will enable you to work in a Cloud environment.

  •  Experience : 5+Years

  • Job Location : Remote/Pune

Read more
Cloud infrastructure solutions and support company. (SE)
Cloud infrastructure solutions and support company. (SE)
Agency job
via Multi Recruit by Ranjini A R
Bengaluru (Bangalore)
4 - 8 yrs
₹15L - ₹24L / yr
skill icon.NET
skill iconC#
Web API
skill iconAngular (2+)
Azure
+1 more
  • Develop detailed technical analysis and design specifications based on user stories.
  • Collaborate heavily with fellow developers, designers, and analysts to solve problems.
  • End-to-End Design and Development of cloud applications utilizing Service Fabric / Web API
  • Develop complete front-end stack of projects using HTML/CSS and TypeScript with Angular.
  • Integrate front-end UI with back-end services.
  • Debug cross-browser compatibility issues.
  • Ensure responsive implementation.

Critical Competencies / Leadership Anchors

  • Builds strong commitment and promotes open dialogue.
  • Defines success in terms of the entire team.
  • Is action-oriented and has a strong sense of urgency.
  • Uses logic and rigorous problem-solving skills to resolve difficult problems and drive effective solutions.
  • Relentlessly customer-focused and dedicated to exceeding expectations of internal and external customers.
  • Drives innovation to bring creative ideas to market.

Requirements

  • Bachelor’s degree in computer science, engineering, or equivalent.
  • 4+ years of experience developing dynamic, highly interactive user interfaces for the web.
  • 4+ years of proven development experience using .NET/C# with a focus on Web API cloud applications
  • 4 years experience with RESTful Web API and JSON
  • Strong hands-on development with Single Page Applications with at least one year of experience focused on Angular Development
  • Disciplined approach to testing and quality assurance.
  • Experience with Git, Redux, Typescript, and Angular Material.
  • Demonstrated ability to produce technical design documentation for assigned features and user stories.
  • Strong drive to learn and produce beautiful code

 

Read more
Ivan Infotech  IT Outsourcing Company in India
Kolkata
1 - 3 yrs
₹2L - ₹3.5L / yr
skill iconReact Native
Apache Cordova
Ionic
skill iconiOS App Development
skill iconAndroid Development

Ivan Infotech is looking for talented Hybrid developer.

Job Responsibility :

  • Develop hybrid mobile application using ionic or react native.
  • Manage the complete app development life cycle, which includes Screen Layout development, API integration, Testing and Submission to app stores .
  • Experience of building Apps using CLI for Android and iOS. Experience of interacting with Apple Store and Play Store.
  • Capability to integrate with third party APIs .
  • Should have experience in Front end design and back end coding.

Required Skilled:

  • Minimum 1.5 year of year experience hybrid mobile application development.
  • Experience with HTML5, CSS3, Angular js, jQuery and Responsive Web Design frameworks .
  • Should have knowledge of developing Hybrid Mobile applications using Ionic or React native framework.
  • Should have good knowledge in cordova.

Benefit:

  • Monday to Friday working.
  • Flexible working hour.
  • Hassle free working environment.

 

Read more
Infinitum Global
at Infinitum Global
7 recruiters
Sashi panditha
Posted by Sashi panditha
Bengaluru (Bangalore), Pune, Kochi (Cochin)
4 - 8 yrs
₹8L - ₹16L / yr
skill iconNodeJS (Node.js)
skill iconReact.js
skill iconExpress
skill iconMongoDB
skill iconAngular (2+)
+1 more
Greetings from Infintium Global.. We have openings for MERN Stack Developer. Please find the company profile and JD and Kindly revert back to me for your interest. Job Description: We are looking at some on who has minimum 4.0+ yrs. of experience on MERN Stack Developer, Bangalore. Skills • Should have work experience in a MERN Stack Developer • Experience on NodeJS, ExpressJS or Restify, ReactJS. • Experience creating front end applications using HTML, CSS. • Hands on experience with JavaScript Development on both client and server-side • Experience with modern frameworks and design patterns, minimum one-year experience with MERN Full stack paradigm • Knowledge of the following will be considered as an advantage: o Consumer Web Development Experience for High-Traffic, Public Facing web applications o Experience with cloud technologies also a plus o Creating secure RESTful-based web services in XML and JSON, JavaScript, JQuery o Continuous integration (Jenkins/Hudson) and version control (SVN, Git) o Also have Onsite opportunity. Note: You will be on permanent with “MethodHub” and work in client Location at TCS, Bangalore. About TCS: Tata Consultancy Services is an IT services, consulting and business solutions organization that has been partnering with many of the world’s largest businesses in their transformation journeys for the last fifty years. TCS offers a consulting-led, Cognitive powered, integrated portfolio of IT, Business & Technology Services, and engineering. This is delivered through its unique Location Independent Agile delivery model, recognized as a benchmark of excellence in software development. A part of the Tata group, India's largest multinational business group, TCS has over 400,000 of the world’s best-trained consultants in 46 countries. The company generated consolidated revenues of US $19.09 billion for year ended March 31, 2018 and is listed on the BSE (formerly Bombay Stock Exchange) and the NSE (National Stock Exchange) in India. TCS' proactive stance on climate change and award winning work with communities across the world have earned it a place in leading sustainability indices such as the Dow Jones Sustainability Index (DJSI), MSCI Global Sustainability Index and the FTSE4Good Emerging Index. For more information, visit us at www.tcs.com. Website http://www.tcs.com About MethodHub: MethodHub is a Life Sciences-focused Product Engineering and software development company based out of USA with domestic presence in Bangalore, Hyderabad and Chennai. An IdeaScaler initiative, MethodHub is built around identifying and bringing appropriate talent for client needs! Especial to note is our unique ability to curate and deliver Expertise, solving demanding problems through extreme specialists! We also orchestrate Solutions and Work Products! Part of deep competency strategy is our ongoing quest for deep specialists. Recent brilliance: • React Native • Drupal CMS • Ruby on Rails • Apigee solution architecture • RPA solutions using the "Big 3" New areas of growth in the technology space includes Process Automation ( RPA & IOT ) and Blockchain MethodHub has innovative retention policies including deferred bonuses, timed milestone-based growth in roles and global opportunities to ensure we retain our best talent. MethodHub constantly explores opportunities for consultants to upskill and cross-skill. MethodHub has a compensation structure which incentivizes behaviour-based on the right parameters for direct, support and advisory roles. Part of the IdeaScaler Group- Other investments include Ostium, Pharmazon/Trade and Famidoc. This group works with the top 20 Life Sciences organizations across India For more information, visit: http://methodhub.in/; https://www.method-hub.com/ Company size 501-1,000 employees Our Clients are as follows, Harman connected services, Infosys, TCS, Mphasis, Capgemini, Madiba Technologies, Moonraft Innovation, Wipro, Moolya Testing and many other Tier 2 companies.
Read more
Fission Labs
at Fission Labs
5 recruiters
Sudhir Suguru
Posted by Sudhir Suguru
Hyderabad
10 - 13 yrs
₹18L - ₹22L / yr
skill iconJava
skill iconAmazon Web Services (AWS)
Mobile App Development
• 10+ Years of experience in software development • Track record of successfully delivering large scale solutions • Hands-on on any language preferably Java, Javascript, NodeJS, Python. • Product development experience • Ability to narrow down on technology and design choices • Experience with Agile, Test Driven Development, and Continuous Integration environments • Experience with various architectures preferably Service Oriented Architectures • An active interest in the open-source technologies, frameworks, tools and community
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos