Cutshort logo
Thoughtworks logo
Lead Consultant: Data Engineer
Lead Consultant: Data Engineer
Thoughtworks's logo

Lead Consultant: Data Engineer

Vidyashree Kulkarni's profile picture
Posted by Vidyashree Kulkarni
9 - 15 yrs
Best in industry
Remote only
Skills
PySpark
Data engineering
Big Data
Hadoop
Spark
skill iconScala
Data Structures
Data modeling
Data security
Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.

Job responsibilities
  • You will partner with teammates to create complex data processing pipelines in order to solve our clients' most complex challenges
  • You will collaborate with Data Scientists in order to design scalable implementations of their models
  • You will pair to write clean and iterative code based on TDD
  • Leverage various continuous delivery practices to deploy, support and operate data pipelines
  • Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
  • Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
  • Create data models and speak to the tradeoffs of different modeling approaches
  • Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process
  • Assure effective collaboration between Thoughtworks' and the client's teams, encouraging open communication and advocating for shared outcomes
Job qualifications

Technical skills

  • You have a good understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
  • You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
  • Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions
  • You are comfortable taking data-driven approaches and applying data security strategy to solve business problems
  • Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
  • You're genuinely excited about data infrastructure and operations with a familiarity working in cloud environments
  • Professional skills
  • You're resilient and flexible in ambiguous situations and enjoy solving problems from technical and business perspectives
  • An interest in coaching, sharing your experience and knowledge with teammates
  • You enjoy influencing others and always advocate for technical excellence while being open to change when needed
  • Presence in the external tech community: you willingly share your expertise with others via speaking engagements, contributions to open source, blogs and more
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Thoughtworks

Founded :
1993
Type :
Products & Services
Size :
5000+
Stage :
Profitable

About

Founded in 1993, we’ve grown from a small team in Chicago to a leading software consultancy of more than 8000 Thoughtworkers in 17 countries. Our cross-functional teams of strategists, developers, data engineers, and designers bring over two decades of global experience to every partnership.

 

Thoughtworks invented the concept of distributed agile and we know how to harness the power of global teams to deliver software excellence at scale. Today we help our clients to create their own path to digital fluency and to build organizational resilience to navigate the future.


Our job is to foster a vibrant community where people have the freedom to make an extraordinary impact on the world through technology.

 

As a Thoughtworker, you are free to seek out the most ambitious challenges. Free to change career paths. Free to use technology as a tool for social change. Free to be yourself.

Read more

Company video

Thoughtworks's video section
Thoughtworks's video section

Connect with the team

Profile picture
Yogita Singh
Profile picture
Apoorv Singhal
Profile picture
Santhosh Joshua
Profile picture
Vidyashree Kulkarni
Profile picture
Sana Singh
Profile picture
Nargise Manova
Profile picture
Ranjith Prabhu
Profile picture
M I Shifa Samrin
Profile picture
Valarmathi M
Profile picture
Anubhuti Pundir
Profile picture
Abhidev Gwari
Profile picture
Diksha Kalucha
Profile picture
Ramya S
Profile picture
Logeshwaran M
Profile picture
Sukriti Sharma
Profile picture
Megha Wadhwa
Profile picture
Jyoti Deswal
Profile picture
Anugrah Mathur
Profile picture
Aaliya Thaver
Profile picture
Aasif Syed
Profile picture
Maria Nancy
Profile picture
Neha Sharma
Profile picture
Charani GP
Profile picture
Ambica Bharadwaj
Profile picture
pavan Juluri
Profile picture
nadeem Shaikh
Juana Fernandes
icon
Connect

Company social profiles

bloginstagramlinkedintwitterfacebook

Similar jobs

Pattem Digital
Vidyasagar Pattem
Posted by Vidyasagar Pattem
Bengaluru (Bangalore)
9 - 20 yrs
₹20L - ₹30L / yr
Adobe Experience Manager (AEM)
OSGi
Apache Sling
Akamai
skill iconJava

[Qualifications]

Must have 10 - 12 years of development and project handling experience using Adobe AEM (AMS & Cloud) and proficient in Java/J2EE.

 

[Job Overview]:

1.     Looking for AEM Senior Technical Specialist with strong experience in AEM stack, such as AEM modules, Granite UI, OSGi, and JCR.

2.     Strong AEM product & architectural exposure to lead teams, assign task and guide technically in BAU/Enhancements.

3.     Prior experience and strong articulation skills to Lead teams in technical discussions with Business and Engineering teams.

 

[Primary Skills]:

MUST HAVE

4.     10 – 12 years of development experience on Adobe AEM/CQ experience implementing full cycle Adobe AEM projects.

5.     Extensive experience in AEM architecture & technology suite, Sling web framework & Apache Felix OSGi framework, DAM, Workflow (custom, OOTB), multi-site manager (MSM), Dispatcher filter & mapping rules.

6.     Solid experience with AEM building blocks including templates, components, dialogs, widgets, pages, experience fragments & bundles.

                            

7.     Experience in integrating AEM with other platforms, technologies, third-party systems, APIs, different internal servers, databases using both frontend (ajax, cookies) & backend components (servlet, OSGI, Sling)

8.     Troubleshoot complex issues by analysing Heap & Thread dumps.

9.     Provide architectural guidance & suggestion around server environment configurations, data management, caching, CDN usage, security, etc. 

10. Excellent written and verbal communication skills.

 

NICE TO HAVE

  1. AEM Sites Developer Certified.
  2. Familiarity in AMS Cloud Manager.
  3. Experience integrating Adobe Analytics, Target, Campaign.
  4. Experience with data migration in AEM.
  5. Experience with Junit and other testing frameworks in AEM.
  6. Familiarity with front-end technologies, HTML, CSS, JavaScript
  7. Experience with JIRA / ROD ticket tracking tools.

 

Read more
CredenceSoft
Pratikhya Routray
Posted by Pratikhya Routray
Bhubaneswar, Navi Mumbai
3 - 8 yrs
₹19L - ₹25L / yr
GCP
Azure Data

Skills Required:


✔️ Strong expertise in data, AI, and cloud infrastructure.


✔️ Proficiency in Python, SQL, and basic scripting (Shell/Bash).


✔️ Experience working with Azure Databricks, GCP, or AWS.


✔️ Hands-on experience with Apache Spark, Hive, Hadoop, and Big Data ETL tools.


✔️ Knowledge of Jenkins, JIRA, Bitbucket, Bamboo, Azure DevOps, and Git.


✔️ Strong experience in SQL databases (Oracle, SQL Server, DB2) and NoSQL databases.




Key Responsibilities:


🔹 Develop and maintain scalable data pipelines from multiple data sources.


🔹 Collaborate with cross-functional teams to source, process, and integrate data efficiently.


🔹 Support in designing AI-driven data solutions for business intelligence and automation.


🔹 Partner with stakeholders to manage project risks and track delivery progress.


🔹 Ensure timely execution of project milestones, deadlines, and budgets.


🔹 Troubleshoot, fine-tune, and optimize large-scale data processing jobs for performance.

Read more
Neoxam
Neoxam
Agency job
via Hunarstreet Technologies Pvt Ltd by Priyanka Londhe
Remote, Noida
12 - 14 yrs
₹35L - ₹55L / yr
skill iconJava
Spring
Spring MVC
hibernate
RDBMS
+12 more

TECHNICAL MANAGER

Department: Product Engineering Location: Noida/Chennai

Experience: 12+ years with 2+ years in a similar role

Job Summary:

We are looking for an inspiring leader to lead a dynamic R&D team with a strong “Product & Customer” spirit. As an Engineering Manager, you will be responsible for the entire process, from design and specification to code quality, process integration and delivery performance

Key Responsibilities:

Collaborate closely with Product Management teams to design and develop business modules.

As a manager, coordinate a diverse team and ensure collaboration between different departments. Empathetic and fair yet demanding management with particular attention to operational excellence.

Actively participate in resolving technical issues and challenges that the team encounters during development and escalated client implementation and production issues

Anticipate technical challenges and work to address them proactively to minimize disruptions to the development process. Guides the team in making architectural choices

Promote and advocate for best practices in software development, including coding standards, testing practices, and documentation.

Make informed decisions on technical trade-offs and communicate those decisions to the team and stakeholders.

Be on top of critical client/ implementation issues and keep stakeholders informed.

PROFILE

Good proficiency overlaps with technologies like: Java17, Spring, SpringMVC, RESTful web services, Hibernate, RDBMS, Spring Security, Ansible, Docker, Kubernetes, JMeter, Angular.

Strong experience in development tools, CI/CD pipelines. Extensive experience with Agile.

Deep understanding of cloud technologies on at least one of the cloud platforms AWS, Azure or Google Cloud

Strong communicator with ability to collaborate cross-functionally, build relationships, and achieve broader organizational goals.

Proven leadership skills

Product development experience preferred. Fintech or lending domain experience is a plus.

Engineering degree or equivalent.

Read more
Fintech Startup
Fintech Startup
Agency job
via Qrata by Blessy Fernandes
Remote only
7 - 14 yrs
₹18L - ₹26L / yr
MEAN stack
MERN Stack
LAMP Stack
Artificial Intelligence (AI)
skill iconMachine Learning (ML)
+5 more
Key Responsibilities:
1. Experience as a Technical / Enterprise Architect with strong knowledge of technologies /
paradigms such as AI/ML, Micro-services, REST APIs, using MEAN /MERN & MongoDB
2. Experience in defining and implementing and managing Cloud infrastructure  
3. Experience in setting up Architecture and technology vision and adoption of technology
advancements into product delivery
4. Should have worked with key technology partners in architecting various solutions
5. Developing, Training, and mentoring in-house & outsourced Engineering teams
6. Setting best practices across coding process
7. Experience in establishing Enterprise technology standards and architecture review
board for governance of Technology & Architectural changes and roadmap
8. Define the complete technical infrastructure environment
9. Maintain current knowledge of all relevant technology developments and seek
opportunities to deploy new technologies that benefit the client
10. Create architecture scope, definition, construction, and execution
11. Provide technical leadership to project resources and the client to meet project deadlines
and ensure that project objectives are met
12. Provide technical leadership to project resources and the client to meet project deadlines
and ensure that project objectives are met
13. Ability to work with cross functional teams and articulate the technology vision and drive
adoption across the Unit

Requirements:
1. Experience in Fintech and B2C SaaS applications
2. Proven experience as a Technical Architect or Project Manager
3. Hands-on experience with software development and system administration
4. Understanding of strategic IT solutions
5. Experience in project management and service-oriented architecture (SOA)
6. Knowledge of selected coding languages (PHP, Python)
7. Familiarity with various operating systems (eg Windows, UNIX) and databases (eg
MongoDB / MySQL)
8. Ability to articulate technical solutions & ROI associated to multiple stakeholders such as
Product engineering teams, business leaders, CXOs etc
9. Organizational and leadership skills
10. BSc/BA in Computer Science, Engineering or a related field
11. Experience with Product-centric Organizations
Read more
client of peoplefirst consultants
client of peoplefirst consultants
Agency job
via People First Consultants by Aishwarya KA
Chennai
6 - 10 yrs
Best in industry
skill iconBootstrap
skill iconHTML/CSS
skill iconAngular (2+)
skill iconJavascript
User Interface (UI) Development
+3 more

SKILLS: UI Development,Angular,Javascript,HTML,CSS,Monitering Management,etc.,

Role

We are looking for a Lead UI Engineer to design and build multiple channels for user interaction.

Responsibilities

  • Design/Architect and develop core features of the UI/frontend.
  • Design/Architect and develop policy framework to provide rich UI for various features of the solution.

 

Requirements:

  • Must have 6+ years of experience in working in frontend technologies
  • Must have at least 2-3 years. of experience with Angular2 or above versions.
  • Must have at least 5+ years of experience with HTML/CSS/JavaScript/TypeScript.
  • Experience with PrimeNG and Vega is a plus
  • Strong background in developing UI for Monitoring and Management systems, dealing with topology, and different telemetry such as metrics, traces and logs
  • Familiar with containerization solutions like Docker/Kubernetes etc.
  • Familiar with serverless technologies like AWS Lambda.
  • B.E/B.Tech/MS degree in Computer Science, or equivalent

 

Read more
codersbrain
at codersbrain
1 recruiter
Tanuj Uppal
Posted by Tanuj Uppal
Delhi
4 - 8 yrs
₹2L - ₹15L / yr
Spark
Hadoop
Big Data
Data engineering
PySpark
+5 more
  • Mandatory - Hands on experience in Python and PySpark.

 

  • Build pySpark applications using Spark Dataframes in Python using Jupyter notebook and PyCharm(IDE).

 

  • Worked on optimizing spark jobs that processes huge volumes of data.

 

  • Hands on experience in version control tools like Git.

 

  • Worked on Amazon’s Analytics services like Amazon EMR, Lambda function etc

 

  • Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS.

 

  • Experience/knowledge of bash/shell scripting will be a plus.

 

  • Experience in working with fixed width, delimited , multi record file formats etc.

 

  • Hands on experience in tools like Jenkins to build, test and deploy the applications

 

  • Awareness of Devops concepts and be able to work in an automated release pipeline environment.

 

  • Excellent debugging skills.
Read more
Leading Consulting firm in the World
Leading Consulting firm in the World
Agency job
Remote, Bengaluru (Bangalore)
10 - 20 yrs
₹40L - ₹55L / yr
skill icon.NET
skill iconJava
skill iconGo Programming (Golang)
skill iconScala
Architecture
+11 more
  • Bachelor's or Master’s degree in Computer Science or equivalent area
  • 10 to 20 years of experience in software development
  • Hands-on experience designing and building B2B or B2C products
  • 3+ years architecting SaaS/Web based customer facing products, leading engineering teams as software/technical architect
  • Experiences of engineering practices such as code refactoring, microservices, design and enterprise integration patterns, test and design-driven development, continuous integration, building highly scalable applications, application and infrastructure security
  • Strong cloud infrastructure experience with AWS and/or Azure
  • Experience building event driven systems and working with message queues/topics
  • Broad working experience across multiple programming languages and frameworks with in-depth experience in one or more of the following: .Net, Java, Scala or Go-lang
  • Hands-on experience with relational databases like SQL Server, PostgreSQL and document stores like Elasticsearch or MongoDB
  • Hands-on experience with Big Data processing technologies like Hadoop/Spark is a plus
  • Hands-on experience with container technologies like Docker, Kubernetes
  • Knowledge of Agile software development process
Read more
Recro
at Recro
1 video
32 recruiters
Mohit Arora
Posted by Mohit Arora
Bengaluru (Bangalore)
3 - 7 yrs
Best in industry
skill iconReact.js
skill iconJavascript
skill iconRedux/Flux

Job Description:


We are looking for a good JavaScript developer who is proficient with React.js. Your primary focus will be on developing user interface components and implementing them following well-known React.js workflows (such as Flux or Redux). You will ensure that these components and the overall application are robust and easy to maintain. You will coordinate with the rest of the team working on different layers of the infrastructure. Therefore, a commitment to collaborative problem solving, sophisticated design, and quality product is important.


Responsibilities

  • Developing new user-facing features using React.js
  • Building reusable components and front-end libraries for future use
  • Translating designs and wireframes into high-quality code
  • Optimizing components for maximum performance across a vast array of web-capable devices and browsers

Skills:

  • Strong proficiency in JavaScript, including DOM manipulation and the JavaScript object model
  • Thorough understanding of React.js and its core principles
  • Experience with popular React.js workflows (Redux)
  • Familiarity with newer specifications of ECMAScript
  • Experience with data structure libraries (e.g., Immutable.js)
  • Knowledge of isomorphic React is a plus
  • Familiarity with RESTful APIs
  • Knowledge of modern authorization mechanisms, such as JSON Web Token
  • Familiarity with modern front-end build pipelines and tools
  • Ability to understand business requirements and translate them into technical requirements
  • A knack for benchmarking and optimization
  • Familiarity with code versioning tools (such as Git, SVN, and Mercurial)


Read more
Accuracy
at Accuracy
1 recruiter
PratapChandran Nair
Posted by PratapChandran Nair
Remote only
3 - 5 yrs
₹5L - ₹10L / yr
skill iconPython
skill iconDjango
skill iconFlask
We are looking for senior Python developer to join our Automation development team
and help us create dynamic software applications for our clients. In this role, you will be
responsible for writing and testing scalable code, developing back-end components,
and integrating user-facing elements in collaboration with front-end developers. You
should possess in-depth knowledge of object-relational mapping, experience with
server-side logic, and above-average knowledge of Python programming.
Responsibilities:
· Coordinating with development teams to determine application requirements.
· Writing error-free, scalable code using Python programming language.
· Testing and debugging applications.
· Coordinating with front-end developers.
· Reprogramming existing packages to improve functionality.
· Priorities bug-fixes, features. Plan and coordinate QA/Prod release
· Bringing and establishing industry standard practices
Qualification and Experience
o master’s degree in computer science, Computer Engineering, or related field.
o 5+ years of experience as IT Developer
o 3+ years of Python development experience
o 1+ years’ experience of automation experience in any python packages
o Hands-on experience on any python automation libraries.
o Ability to collaborate on projects and work independently when required
o Ability to up-skills team members and delegate the works

Budget : Constraint( From current max 30% hike)
Notice Period: Immediate to 15 Days
Read more
Avestan Technologies
at Avestan Technologies
1 recruiter
Chetan Chitriv
Posted by Chetan Chitriv
Nagpur
1 - 2 yrs
₹1.2L - ₹3L / yr
Angular
skill iconAngularJS (1.x)
skill iconAngular (2+)
  • Should have exp. 01 to 02 years
  • Should have expert level knowledge of HTML5, CSS3, JavaScript, Angular and writing cross browser compatible code
  • Good understanding of AJAX and JavaScript Dom manipulation, Responsive Design
  • Proficiency with building Single Page Apps using Javascript component libraries.
  • Should have experience in Agile Software Development projects
  • Should have good hands on experience with Object oriented JavaScript and Design Patterns
  • Should have knowledge on experience on Java, Web services using REST /SOAP over JSON is a plus

Job Types: Full-time, Walk-In

Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos