Data Engineer (Consultant/Senior consultant)

at Thoughtworks

DP
Posted by sabarinath konath
icon
Pune, Bengaluru (Bangalore), Coimbatore, Hyderabad, Gurugram
icon
3 - 10 yrs
icon
₹18L - ₹40L / yr
icon
Full time
Skills
Apache Kafka
Spark
Hadoop
Apache Hive
Big Data
Scala
Python
Cloud Computing
Apache Oozie
NOSQL Databases

Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.



You’ll spend time on the following:

  • You will partner with teammates to create complex data processing pipelines in order to solve our clients’ most ambitious challenges
  • You will collaborate with Data Scientists in order to design scalable implementations of their models
  • You will pair to write clean and iterative code based on TDD
  • Leverage various continuous delivery practices to deploy data pipelines
  • Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
  • Develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
  • Create data models and speak to the tradeoffs of different modeling approaches

Here’s what we’re looking for:

 

  • You have a good understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
  • You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
  • Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions
  • You are comfortable taking data-driven approaches and applying data security strategy to solve business problems 
  • Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
  • Strong communication and client-facing skills with the ability to work in a consulting environment
Read more

About Thoughtworks

Founded in 1993, we’ve grown from a small team in Chicago to a leading software consultancy of more than 8000 Thoughtworkers in 17 countries. Our cross-functional teams of strategists, developers, data engineers, and designers bring over two decades of global experience to every partnership.

 

Thoughtworks invented the concept of distributed agile and we know how to harness the power of global teams to deliver software excellence at scale. Today we help our clients to create their own path to digital fluency and to build organizational resilience to navigate the future.

Our job is to foster a vibrant community where people have the freedom to make an extraordinary impact on the world through technology.

 

As a Thoughtworker, you are free to seek out the most ambitious challenges. Free to change career paths. Free to use technology as a tool for social change. Free to be yourself.

Read more
Founded
1993
Type
Products & Services
Size
5000+ employees
Stage
Profitable
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Data Science - Manager

at Vahak

Founded 2016  •  Product  •  20-100 employees  •  Raised funding
Data Science
Data Analytics
R Programming
Python
SQL
PowerBI
Tableau
icon
Bengaluru (Bangalore)
icon
5 - 12 yrs
icon
₹20L - ₹40L / yr

Who Are We?

 

Vahak (https://www.vahak.in) is India’s largest & most trusted online transport marketplace & directory for road transport businesses and individual commercial vehicle (Trucks, Trailers, Containers, Hyva, LCVs) owners for online truck and load booking, transport business branding and transport business network expansion. Lorry owners can find intercity and intracity loads from all over India and connect with other businesses to find trusted transporters and best deals in the Indian logistics services market. With the Vahak app, users can book loads and lorries from a live transport marketplace with over 7 Lakh + Transporters and Lorry owners in over 10,000+ locations for daily transport requirements.

Vahak has raised a capital of $5+ Million in a Pre-Series A round from RTP Global along with participation from Luxor Capital and Leo Capital. The other marquee angel investors include Kunal Shah, Founder and CEO, CRED; Jitendra Gupta, Founder and CEO, Jupiter; Vidit Aatrey and Sanjeev Barnwal, Co-founders, Meesho; Mohd Farid, Co-founder, Sharechat; Amrish Rau, CEO, Pine Labs; Harsimarbir Singh, Co-founder, Pristyn Care; Rohit and Kunal Bahl, Co-founders, Snapdeal; and Ravish Naresh, Co-founder and CEO, Khatabook.


 

Manager Data Science:

We at Vahak, are looking for an enthusiastic and passionate Manager of Data Science, to join our young & diverse team.You will play a key role in the data science group, working with different teams, identifying the use cases that could be solved by application of data science techniques.

Our goal as a group is to drive powerful, big data analytics products with scalable results.We love people who are humble and collaborative with hunger for excellence.


Responsibilities:

  • Mine and Analyze end to end business data and generate actionable insights. Work will involve analyzing Customer transaction data, Marketing Campaign performance analysis, identifying process bottlenecks, business performance analysis etc.
  • Identify data driven opportunities to drive optimization and improvement of product development, marketing techniques and business strategies.
  • Collaborate with Product and growth teams to test and learn at unprecedented pace and help the team achieve substantial upside in key metrics
  • Actively participate in the OKR process and help team democratize the key KPIs and metrics that drive various objectives
  • Comfortable with digital marketing campaign concepts, use of marketing campaign platforms such as Google Adwords and Facebook Ads
  • Responsible for design of algorithms that require different advanced analytics techniques  and heuristics to work together
  • Create dashboard and visualization from scratch and present data in logical manner to all the stakeholders
  • Collaborates with internal teams to create actionable items based off analysis; works with the datasets to conduct complex quantitative analysis and helps drive the innovation for our customers

Requirements:

  • Bachelor’s or Masters degree in  Engineering, Science, Maths,  Economics or other quantitative fields. MBA is a plus but not required
  • 5+ years of proven experience working in Data Science field preferably in ecommerce/web based  or consumer technology companies
  • Thorough understanding of implementation and analysis of product and marketing metrics at scale
  • Strong problem solving skills with an emphasis on product development.
  • Fluency in statistical computer languages like SQL, Python, R as well as a deep understanding of statistical analysis, experiments designs and common pitfalls of data analysis
  • Should have worked in a relational database like Oracle or Mysql, experience in Big Data systems like Bigquery or  Redshift a definite plus
  • Experience using business intelligence tools e.g. Tableau, Power BI would be an added advantage (not mandatory)

 

Read more
Job posted by
Vahak Talent

Data Engineer - AWS

at A global business process management company

Agency job
via Jobdost
Data engineering
Data modeling
data pipeline
Data integration
Data Warehouse (DWH)
Data engineer
AWS RDS
Glue
AWS CloudFormation
Amazon Web Services (AWS)
DevOps
AWS Lambda
Python
Django
Data Pipeline
Step functions
RDS
icon
Gurugram, Pune, Mumbai, Bengaluru (Bangalore), Chennai, Nashik
icon
4 - 12 yrs
icon
₹12L - ₹15L / yr

 

 

Designation – Deputy Manager - TS


Job Description

  1. Total of  8/9 years of development experience Data Engineering . B1/BII role
  2. Minimum of 4/5 years in AWS Data Integrations and should be very good on Data modelling skills.
  3. Should be very proficient in end to end AWS Data solution design, that not only includes strong data ingestion, integrations (both Data @ rest and Data in Motion) skills but also complete DevOps knowledge.
  4. Should have experience in delivering at least 4 Data Warehouse or Data Lake Solutions on AWS.
  5. Should be very strong experience on Glue, Lambda, Data Pipeline, Step functions, RDS, CloudFormation etc.
  6. Strong Python skill .
  7. Should be an expert in Cloud design principles, Performance tuning and cost modelling. AWS certifications will have an added advantage
  8. Should be a team player with Excellent communication and should be able to manage his work independently with minimal or no supervision.
  9. Life Science & Healthcare domain background will be a plus

Qualifications

BE/Btect/ME/MTech

 

Read more
Job posted by
Saida Jabbar

ETL-Database Developer/Lead

at Wissen Technology

Founded 2000  •  Products & Services  •  1000-5000 employees  •  Profitable
ETL
Informatica
Data Warehouse (DWH)
Data modeling
Spark
Databases
Shell Scripting
Perl
Python
KDB
icon
Bengaluru (Bangalore)
icon
5 - 12 yrs
icon
₹15L - ₹35L / yr

Job Description

The applicant must have a minimum of 5 years of hands-on IT experience, working on a full software lifecycle in Agile mode.

Good to have experience in data modeling and/or systems architecture.
Responsibilities will include technical analysis, design, development and perform enhancements.

You will participate in all/most of the following activities:
- Working with business analysts and other project leads to understand requirements.
- Modeling and implementing database schemas in DB2 UDB or other relational databases.
- Designing, developing, maintaining and Data processing using Python, DB2, Greenplum, Autosys and other technologies

 

Skills /Expertise Required :

Work experience in developing large volume database (DB2/Greenplum/Oracle/Sybase).

Good experience in writing stored procedures, integration of database processing, tuning and optimizing database queries.

Strong knowledge of table partitions, high-performance loading and data processing.
Good to have hands-on experience working with Perl or Python.
Hands on development using Spark / KDB / Greenplum platform will be a strong plus.
Designing, developing, maintaining and supporting Data Extract, Transform and Load (ETL) software using Informatica, Shell Scripts, DB2 UDB and Autosys.
Coming up with system architecture/re-design proposals for greater efficiency and ease of maintenance and developing software to turn proposals into implementations.

Need to work with business analysts and other project leads to understand requirements.
Strong collaboration and communication skills

Read more
Job posted by
Lokesh Manikappa

Tableau Engineer

at Aideo Technologies

Founded 2009  •  Product  •  100-500 employees  •  Bootstrapped
Tableau
Natural Language Processing (NLP)
Computer Vision
Python
RESTful APIs
Microservices
Flask
SQL
icon
Mumbai, Navi Mumbai
icon
3 - 8 yrs
icon
₹4L - ₹22L / yr

We are establishing infrastructure for internal and external reporting using Tableau and are looking for someone with experience building visualizations and dashboards in Tableau and using Tableau Server to deliver them to internal and external users. 

 

Required Experience 

  • Implementation of interactive visualizations using Tableau Desktop  
  • Integration with Tableau Server and support of production dashboards and embedded reports with it 
  • Writing and optimization of SQL queries  
  • Proficient in Python including the use of Pandas and numpy libraries to perform data exploration and analysis 
  • 3  years of experience working as a Software Engineer / Senior Software Engineer 
  • Bachelors in Engineering – can be Electronic and comm , Computer , IT  
  • Well versed with Basic Data Structures Algorithms and system design 
  • Should be capable of working well in a team – and should possess very good communication skills 
  • Self-motivated and fun to work with and organized 
  • Productive and efficient working remotely 
  • Test driven mindset with a knack for finding issues and problems at earlier stages of development 
  • Interest in learning and picking up a wide range of cutting edge technologies 
  • Should be curious and interested in learning some Data science related concepts and domain knowledge 
  • Work alongside other engineers on the team to elevate technology and consistently apply best practices 

 

Highly Desirable 

  • Data Analytics 
  • Experience in AWS cloud or any cloud technologies 
  • Experience in BigData technologies and streaming like – pyspark, kafka is a big plus 
  • Shell scripting  
  • Preferred tech stack – Python, Rest API, Microservices, Flask/Fast API, pandas, numpy, linux, shell scripting, Airflow, pyspark 
  • Has a strong backend experience – and worked with Microservices and Rest API’s - Flask, FastAPI, Databases Relational and Non-relational 
Read more
Job posted by
Akshata Alekar

Senior Data Consultant (Talend DI)

at Pinghala

Founded 2018  •  Products & Services  •  20-100 employees  •  Profitable
PowerBI
Data Visualization
Data architecture
Informatica PowerCenter
SQL
Business Intelligence (BI)
Cloud Computing
Data Analytics
Talend DI
talend
icon
Pune
icon
3 - 5 yrs
icon
₹6L - ₹10L / yr

Pingahla is recruiting business intelligence Consultants/Senior consultants who can help us with Information Management projects (domestic, onshore and offshore) as developers and team leads. The candidates are expected to have 3-6 years of experience with Informatica Power Center/Talend DI/Informatica Cloud and must be very proficient with Business Intelligence in general. The job is based out of our Pune office.

Responsibilities:

  • Manage the customer relationship by serving as the single point of contact before, during and after engagements.
  • Architect data management solutions.
  • Provide technical leadership to other consultants and/or customer/partner resources.
  • Design, develop, test and deploy data integration solutions in accordance with customer’s schedule.
  • Supervise and mentor all intermediate and junior level team members.
  • Provide regular reports to communicate status both internally and externally.
  • Qualifications:
  • A typical profile that would suit this position would be if the following background:
  • A graduate from a reputed engineering college 
  • An excellent I.Q and analytical skills and should be able to grasp new concepts and learn new technologies.
  • A willingness to work with a small team in a fast-growing environment.
  • A good knowledge of Business Intelligence concepts

 

Mandatory Requirements:

  • Knowledge of Business Intelligence
  • Good knowledge of at least one of the following data integration tools - Informatica Powercenter, Talend DI, Informatica Cloud
  • Knowledge of SQL
  • Excellent English and communication skills
  • Intelligent, quick to learn new technologies
  • Track record of accomplishment and effectiveness with handling customers and managing complex data management needs
     

 

Read more
Job posted by
Ashwini Dhaipule

Senior Artificial intelligence/ Machine Learning Developer

at A firm which woks with US clients. Permanent WFH.

Agency job
via Jobdost
Artificial Intelligence (AI)
Machine Learning (ML)
Python
Data Structures
Data modeling
Software architecture
Algorithms
Git
icon
Remote only
icon
1 - 8 yrs
icon
₹8L - ₹18L / yr

This person MUST have:

  • B.E Computer Science or equivalent
  • 5 years experience with the Django framework
  • Experience with building APIs (REST or GraphQL) 
  • Strong Troubleshooting and debugging skills
  • React.js knowledge would be an added bonus 
  • Understanding on how to use a database like Postgres (prefered choice), SQLite, MongoDB, MySQL.
  • Sound knowledge of object-oriented design and analysis.
  • A strong passion for writing simple, clean and efficient code.
  • Proficient understanding of code versioning tools Git.
  • Strong communication skills.

Experience:

  • Min 5 year experience
  • Startup experience is a must. 

Location:

  • Remote developer

Timings:

  • 40 hours a week but with 4 hours a day overlapping with client timezone.  Typically clients are in California PST Timezone.

Position:

  • Full time/Direct
  • We have great benefits such as PF, medical insurance, 12 annual company holidays, 12 PTO leaves per year, annual increments, Diwali bonus, spot bonuses and other incentives etc.
  • We dont believe in locking in people with large notice periods.  You will stay here because you love the company.  We have only a 15 days notice period.
Read more
Job posted by
Riya Roy

Data Governance Engineer

at European Bank headquartered at Copenhagen, Denmark.

Agency job
via Apical Mind
Data governance
DevOps
Data integration
Data engineering
Python
Java
Apache Kafka
Git
GitHub
gitlab
Elastic Search
InfluxDB
Kibana
kafka integration
Apache Airflow
Airflow
Telegraf
Prometheus
Grafana
icon
NCR (Delhi | Gurgaon | Noida)
icon
2 - 12 yrs
icon
₹25L - ₹40L / yr
Data Platforms (Data Integration) is responsible for envisioning, building and operating the Bank’s data integration platforms. The successful candidate will work out of Gurgaon as a part of a high performing team who is distributed across our two development centers – Copenhagen and Gurugram. The individual must be driven, passionate about technology and display a level of customer service that is second to none.

Roles & Responsibilities

  • Designing and delivering a best-in-class, highly scalable data governance platform
  • Improving processes and applying best practices
  • Contribute in all scrum ceremonies; assuming the role of ‘scum master’ on a rotational basis
  •  Development, management and operation of our infrastructure to ensure it is easy to deploy, scalable, secure and fault-tolerant
  • Flexible on working hours as per business needs
Read more
Job posted by
Rajeev T

Product Analyst - Ad Tech

at MX Player

Founded 2011  •  Product  •  500-1000 employees  •  Profitable
Python
SQL
Tableau
icon
Mumbai, NCR (Delhi | Gurgaon | Noida)
icon
2 - 5 yrs
icon
Best in industry

About MX Player (https://play.google.com/store/apps/details?id=com.mxtech.videoplayer.ad&;hl=en_IN">Playstore Link)


MX Player is the world’s #1 entertainment superapp offering 100,000+ hours of premium OTT (over the top) content spanning acclaimed MX Originals, Web Shows, TV (Live & OnDemand), movies, music videos and hyper-casual games, music streaming, short form video and more. With more than 1 billion installs worldwide – MX Player is present on 1 out of every 2 smartphones, making it the largest entertainment app/platform in the world.

 

Position : Product Analyst / Business Analyst - Ad Tech


Key Responsibilities:

 

  • Driving the collection of new data that would help build the next generation of algorithms (E.g. audience segmentation, contextual targeting)
  • Understanding user behavior and performing root-cause analysis of changes in data trends to identify corrections or propose desirable enhancements in product & across different verticals
  • Excellent problem solving skills and the ability to make sound judgments based on trade-offs for different solutions to complex problem constraints
  • Defining and monitoring KPIs for product/content/business performance and identifying ways to improve them
  • Should be a strong advocate of data driven approach and drive analytics decisions by doing user testing, data analysis, and A/B testing
  • Help in defining the analytics roadmap for the product
  • Prior knowledge and experience in ad tech industry or other advertising platforms will be preferred

Tools/ Skillset:

  • Knowledge of Google DFP (prefered)
  • SQL
  • R/Python (preferred) 
  • Any BI Tool such as tableau, sisense (preferred)
  • Go getter attitude
  • Ability to thrive in a fast paced dynamic environment
  • Self - Starter
Read more
Job posted by
Payal Thakker

Data Scientist

at mPaani Solutions Pvt Ltd

Founded 2013  •  Product  •  20-100 employees  •  Raised funding
Machine Learning (ML)
Python
Data Science
Big Data
R Programming
Haskell
Hadoop
icon
Mumbai
icon
3 - 7 yrs
icon
₹5L - ₹15L / yr
Data Scientist - We are looking for a candidate to build great recommendation engines and power an intelligent m.Paani user journey Responsibilities : - Data Mining using methods like associations, correlations, inferences, clustering, graph analysis etc. - Scale machine learning algorithm that powers our platform to support our growing customer base and increasing data volume - Design and implement machine learning, information extraction, probabilistic matching algorithms and models - Care about designing the full machine learning pipeline. - Extending company's data with 3rd party sources. - Enhancing data collection procedures. - Processing, cleaning and verifying data collected. - Ad hoc analysis of the data and present clear results. - Creating advanced analytics products that provide actionable insights. The Individual : - We are looking for a candidate with the following skills, experience and attributes: Required : - Someone with 2+ years of work experience in machine learning. - Educational qualification relevant to the role. Degree in Statistics, certificate courses in Big Data, Machine Learning etc. - Knowledge of Machine Learning techniques and algorithms. - Knowledge in languages and toolkits like Python, R, Numpy. - Knowledge of data visualization tools like D3,js, ggplot2. - Knowledge of query languages like SQL, Hive, Pig . - Familiar with Big Data architecture and tools like Hadoop, Spark, Map Reduce. - Familiar with NoSQL databases like MongoDB, Cassandra, HBase. - Good applied statistics skills like distributions, statistical testing, regression etc. Compensation & Logistics : This is a full-time opportunity. Compensation will be in line with startup, and will be based on qualifications and experience. The position is based in Mumbai, India, and the candidate must live in Mumbai or be willing to relocate.
Read more
Job posted by
Julie K

Backend Developer

at Poker Yoga

Founded  •  Product  •  0-20 employees  •  Bootstrapped
Elastic Search
MongoDB
NOSQL Databases
Redis
Relational Database (RDBMS)
icon
Bengaluru (Bangalore)
icon
2 - 4 yrs
icon
₹13L - ₹18L / yr
At Poker Yoga we aim to make poker a tool towards self transformation. By providing the necessary tools to improve his skill, necessary learning frame work to bring skill to the core of his game approach and experiences to enhance his perception. We are looking at passionate coders who love building products that speak for themselves. It's an invitation to join a family, not a company. Looking forward to work with you!
Read more
Job posted by
Anuj Kumar Kodam
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Thoughtworks?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort