Data Engineer

at Talent500

Agency job
icon
Bengaluru (Bangalore)
icon
1 - 10 yrs
icon
₹5L - ₹30L / yr
icon
Full time
Skills
Python
ETL
SQL
SQL Server Reporting Services (SSRS)
Data Warehouse (DWH)
SSIS
Google Cloud Platform (GCP)
GitHub
SQL server
MuleSoft
Azure Synapse

A proficient, independent contributor that assists in technical design, development, implementation, and support of data pipelines; beginning to invest in less-experienced engineers.

Responsibilities:

- Design, Create and maintain on premise and cloud based data integration pipelines. 
- Assemble large, complex data sets that meet functional/non functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
- Build analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics.
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Create data pipelines to enable BI, Analytics and Data Science teams that assist them in building and optimizing their systems
- Assists in the onboarding, training and development of team members.
- Reviews code changes and pull requests for standardization and best practices
- Evolve existing development to be automated, scalable, resilient, self-serve platforms
- Assist the team in the design and requirements gathering for technical and non technical work to drive the direction of projects

 

Technical & Business Expertise:

-Hands on integration experience in SSIS/Mulesoft
- Hands on experience Azure Synapse
- Proven advanced level of writing database experience in SQL Server
- Proven advanced level of understanding about Data Lake
- Proven intermediate level of writing Python or similar programming language
- Intermediate understanding of Cloud Platforms (GCP) 
- Intermediate understanding of Data Warehousing
- Advanced Understanding of Source Control (Github)

Read more
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Startup Focused on simplifying Buying Intent
Agency job
via Qrata by Blessy Fernandes
Bengaluru (Bangalore)
4 - 9 yrs
₹28L - ₹56L / yr
Big Data
Apache Spark
Spark
Hadoop
ETL
+7 more
5+ years of experience in a Data Engineer role.
 Proficiency in Linux.
 Must have SQL knowledge and experience working with relational databases,
query authoring (SQL) as well as familiarity with databases including Mysql,
Mongo, Cassandra, and Athena.
 Must have experience with Python/Scala.
Must have experience with Big Data technologies like Apache Spark.
 Must have experience with Apache Airflow.
 Experience with data pipeline and ETL tools like AWS Glue.
 Experience working with AWS cloud services: EC2, S3, RDS, Redshift.
Read more
at Persistent Systems
1 video
1 recruiter
Agency job
Bengaluru (Bangalore), Pune, Mumbai, Nagpur, Goa, Indore, Hyderabad
5 - 10 yrs
₹20L - ₹30L / yr
PySpark
Data engineering
Big Data
Hadoop
Spark
+2 more

Responsibilities

 

  • Develop process workflows for data preparations, modeling, and mining Manage configurations to build reliable datasets for analysis Troubleshooting services, system bottlenecks, and application integration.
  • Designing, integrating, and documenting technical components, and dependencies of big data platform Ensuring best practices that can be adopted in the Big Data stack and shared across teams.
  • Design and Development of Data pipeline on AWS Cloud
  • Data Pipeline development using Pyspark, AWS, and Python.
  • Developing Pyspark streaming applications

Eligibility

 

  • Hands-on experience in Spark, Python, and Cloud
  • Highly analytical and data-oriented
  • Good to have - Databricks
Read more
This is with one of the top conglomerate
Agency job
via Seven N Half by Gurpreet Desai
Remote only
2 - 5 yrs
₹10L - ₹20L / yr
Natural Language Processing (NLP)
Data Science
Machine Learning (ML)
Computer Vision
Python
+2 more
● Good understanding of data structures, algorithms, and the first principles of mathematics.
● Proficient in Python and using packages like NLTK, Numpy, Pandas
● Should have worked on deep learning frameworks (like Tensorflow, Keras, PyTorch, etc)
● Hands-on experience in Natural Language Processing, Sequence, and RNN Based models
● Mathematical intuition of ML and DL algorithms
● Should be able to perform thorough model evaluation by creating hypotheses on the basis of statistical
analyses
● Should be comfortable in going through open-source code and reading research papers.
● Should be curious or thoughtful enough to answer the “WHYs” pertaining to the most cherished
observations, thumb rules, and ideas across the data science community.
Read more
at Synapsica Healthcare
6 candid answers
1 video
DP
Posted by Human Resources
Bengaluru (Bangalore)
4 - 8 yrs
₹20L - ₹45L / yr
SVM
OpenCV
Machine Learning (ML)
Deep Learning
Artificial Intelligence (AI)
+6 more

Introduction

Synapsica is a series-A funded HealthTech startup founded by alumni from IIT Kharagpur, AIIMS New Delhi, and IIM Ahmedabad. We believe healthcare needs to be transparent and objective while being affordable. Every patient has the right to know exactly what is happening in their bodies and they don't have to rely on cryptic 2 liners given to them as a diagnosis. 

Towards this aim, we are building an artificial intelligence enabled cloud based platform to analyse medical images and create v2.0 of advanced radiology reporting.  We are backed by IvyCap, Endia Partners, YCombinator and other investors from India, US, and Japan. We are proud to have GE and The Spinal Kinetics as our partners. Here’s a small sample of what we’re building: https://www.youtube.com/watch?v=FR6a94Tqqls

 

Your Roles and Responsibilities

Synapsica is looking for a Principal AI Researcher to lead and drive AI based research and development efforts. Ideal candidate should have extensive experience in Computer Vision and AI Research, either through studies or industrial R&D projects and should be excited to work on advanced exploratory research and development projects in computer vision and machine learning to create the next generation of advanced radiology solutions.

The role involves computer vision tasks including development customization and training of Convolutional Neural Networks (CNNs); application of ML techniques (SVM, regression, clustering etc.), and traditional Image Processing (OpenCV, etc.). The role is research-focused and would involve going through and implementing existing research papers, deep dive of problem analysis, frequent review of results, generating new ideas, building new models from scratch, publishing papers, automating and optimizing key processes. The role will span from real-world data handling to the most advanced methods such as transfer learning, generative models, reinforcement learning, etc., with a focus on understanding quickly and experimenting even faster. Suitable candidate will collaborate closely both with the medical research team, software developers and AI research scientists. The candidate must be creative, ask questions, and be comfortable challenging the status quo. The position is based in our Bangalore office.

 

 

Primary Responsibilities

  • Interface between product managers and engineers to design, build, and deliver AI models and capabilities for our spine products.
  • Formulate and design AI capabilities of our stack with special focus on computer vision.
  • Strategize end-to-end model training flow including data annotation, model experiments, model optimizations, model deployment and relevant automations
  • Lead teams, engineers, and scientists to envision and build new research capabilities and ensure delivery of our product roadmap.
  • Organize regular reviews and discussions.
  • Keep the team up-to-date with latest industrial and research updates.
  • Publish research and clinical validation papers

 

Requirements

  • 6+ years of relevant experience in solving complex real-world problems at scale using computer vision-based deep learning.
  • Prior experience in leading and managing a team.
  • Strong problem-solving ability
  • Prior experience with Python, cuDNN, Tensorflow, PyTorch, Keras, Caffe (or similar Deep Learning frameworks).
  • Extensive understanding of computer vision/image processing applications like object classification, segmentation, object detection etc
  • Ability to write custom Convolutional Neural Network Architecture in Pytorch (or similar)
  • Background in publishing research papers and/or patents 
  • Computer Vision and AI Research background in medical domain will be a plus
  • Experience of GPU/DSP/other Multi-core architecture programming
  • Effective communication with other project members and project stakeholders
  • Detail-oriented, eager to learn, acquire new skills
  • Prior Project Management and Team Leadership experience
  • Ability to plan work and meet the deadline
Read more
at Marktine
1 recruiter
DP
Posted by Vishal Sharma
Remote, Bengaluru (Bangalore)
3 - 10 yrs
₹5L - ₹15L / yr
Big Data
ETL
PySpark
SSIS
Microsoft Windows Azure
+4 more

Must Have Skills:

- Solid Knowledge on DWH, ETL and Big Data Concepts

- Excellent SQL Skills (With knowledge of SQL Analytics Functions)

- Working Experience on any ETL tool i.e. SSIS / Informatica

- Working Experience on any Azure or AWS Big Data Tools.

- Experience on Implementing Data Jobs (Batch / Real time Streaming)

- Excellent written and verbal communication skills in English, Self-motivated with strong sense of ownership and Ready to learn new tools and technologies

Preferred Skills:

- Experience on Py-Spark / Spark SQL

- AWS Data Tools (AWS Glue, AWS Athena)

- Azure Data Tools (Azure Databricks, Azure Data Factory)

Other Skills:

- Knowledge about Azure Blob, Azure File Storage, AWS S3, Elastic Search / Redis Search

- Knowledge on domain/function (across pricing, promotions and assortment).

- Implementation Experience on Schema and Data Validator framework (Python / Java / SQL),

- Knowledge on DQS and MDM.

Key Responsibilities:

- Independently work on ETL / DWH / Big data Projects

- Gather and process raw data at scale.

- Design and develop data applications using selected tools and frameworks as required and requested.

- Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.

- Perform tasks such as writing scripts, web scraping, calling APIs, write SQL queries, etc.

- Work closely with the engineering team to integrate your work into our production systems.

- Process unstructured data into a form suitable for analysis.

- Analyse processed data.

- Support business decisions with ad hoc analysis as needed.

- Monitoring data performance and modifying infrastructure as needed.

Responsibility: Smart Resource, having excellent communication skills

 

 
Read more
Remote only
0 - 2 yrs
₹2.4L - ₹10L / yr
Quantitative analyst
Algorithmic trading
R Programming
Matlab
SPSS
+3 more

Work Experience : 0-2 years

Responsibilities :

- Design and implement mathematical models for fundamental valuation of securities. The person will need to understand latest research in quantitative finance and implement the same.

- Design, back-testing and implementation of high-frequency trading strategies on international exchanges. Work as part of the market-making team to determine the signals and trading strategies to go live with.

- Conduct performance attribution of live portfolios.

Required Skills :

- Strong candidates should have 0-2 years of work experience and successful track record in quantitative analysis preferably in the capital markets domain.

- Post-Graduate degree in statistics, finance, mathematics, engineering (Computer Science preferred) or other quantitative or computational disciplines

- Experience in using some or all of the following packages: R, MATLAB, SPSS, CART, C# .Net, Python

- Good written and oral communication skills.

- Strong experience working both independently and in a team-oriented collaborative environment.

- Entrepreneurial, self-motivated individual - high energy, high activity levels - passion for working with an innovative, small but rapidly growing company.

Read more
at Kaleidofin
3 recruiters
DP
Posted by Poornima B
Chennai, Bengaluru (Bangalore)
2 - 4 yrs
Best in industry
PowerBI
Business Intelligence (BI)
Python
Tableau
SQL
+1 more
We are looking for a developer to design and deliver strategic data-centric insights leveraging the next generation analytics and BI technologies. We want someone who is data-centric and insight-centric, less report centric. We are looking for someone wishing to make an impact by enabling innovation and growth; someone with passion for what they do and a vision for the future.

Responsibilities:
  • Be the analytical expert in Kaleidofin, managing ambiguous problems by using data to execute sophisticated quantitative modeling and deliver actionable insights.
  • Develop comprehensive skills including project management, business judgment, analytical problem solving and technical depth.
  • Become an expert on data and trends, both internal and external to Kaleidofin.
  • Communicate key state of the business metrics and develop dashboards to enable teams to understand business metrics independently.
  • Collaborate with stakeholders across teams to drive data analysis for key business questions, communicate insights and drive the planning process with company executives.
  • Automate scheduling and distribution of reports and support auditing and value realization.
  • Partner with enterprise architects to define and ensure proposed.
  • Business Intelligence solutions adhere to an enterprise reference architecture.
  • Design robust data-centric solutions and architecture that incorporates technology and strong BI solutions to scale up and eliminate repetitive tasks.
 Requirements:
  • Experience leading development efforts through all phases of SDLC.
  • 2+ years "hands-on" experience designing Analytics and Business Intelligence solutions.
  • Experience with Quicksight, PowerBI, Tableau and Qlik is a plus.
  • Hands on experience in SQL, data management, and scripting (preferably Python).
  • Strong data visualisation design skills, data modeling and inference skills.
  • Hands-on and experience in managing small teams.
  • Financial services experience preferred, but not mandatory.
  • Strong knowledge of architectural principles, tools, frameworks, and best practices.
  • Excellent communication and presentation skills to communicate and collaborate with all levels of the organisation.
  • Preferred candidates with less than 30 days notice period.
Read more
A Global IT Service company
Agency job
via Multi Recruit by Manjunath Multirecruit
Bengaluru (Bangalore)
5 - 8 yrs
₹20L - ₹30L / yr
Data engineering
Data Bricks
data engineer
PySpark
ETL
+3 more
  • Insurance P&C and Specialty domain experience a plus
  • Experience in a cloud-based architecture preferred, such as Databricks, Azure Data Lake, Azure Data Factory, etc.
  • Strong understanding of ETL fundamentals and solutions. Should be proficient in writing advanced / complex SQL, expertise in performance tuning and optimization of SQL queries required.
  • Strong experience in Python/PySpark and Spark SQL
  • Experience in troubleshooting data issues, analyzing end to end data pipelines, and working with various teams in resolving issues and solving complex problems.
  • Strong experience developing Spark applications using PySpark and SQL for data extraction, transformation, and aggregation from multiple formats for analyzing & transforming the data to uncover insights and actionable intelligence for internal and external use
Read more
at Alien Brains
5 recruiters
DP
Posted by Praveen Baheti
Kolkata
2 - 7 yrs
₹5L - ₹8L / yr
Java
Python
Javascript
Machine Learning (ML)
Deep Learning
+2 more
A research lab with roots laid in innovation, we are looking for someone who can take reins of our AI based development think tank. Given the work ethics and results, the salary can be re-negotiated in 5 months.
Read more
AI-driven platform designed for education,e-commerce sectors
Agency job
via Anzy by Kushala U
Remote, Bengaluru (Bangalore)
8 - 11 yrs
₹10L - ₹22L / yr
Data Science
Machine Learning (ML)
Python
Image Processing

The Job
The Architect, Machine Learning and Artificial Intelligence including Computer Vision will grow and lead a team of talented Machine Learning (ML), Computer Vision (CV) and Artificial Intelligence (AI) researchers and engineers to develop innovative machine learning algorithms, scalable ML system, and AI applications for Racetrack. This role will be focused on developing and deploying personalization and recommender system, search, experimentation, audience, and content AI solutions to drive user experience and growth.


The Daily

  • Develop innovative data science solutions that utilize machine learning and deep learning algorithms, statistical and quantitative modelling approaches to support product, engineering, content, and marketing initiatives.
  • Build and lead a world-class team of ML and AI scientists and engineers.
  • Be a hands-on leader to mentor the team in latest machine learning and deep learning approaches, and to introduce new technologies and processes. Single headedly manage the MVP and PoCs
  • Work with ML engineers to design solution architecture and develop scalable machine learning system to accelerate learning cycle.
  • Identify data science opportunities that deliver business value.
  • Develop ML/AI/CV roadmap and educate both internal and external stakeholders at all levels to drive implementation and measurement.
  • Hands on experience in Image processing for auto industry
  • BFSI domain knowledge is a plus
  • Provide thought leadership to enable ML/AI applications.
  • Manage products priorities and ensure timely delivery.
  • Develop and evangelize best practices for scoping, building, validating, deploying, and monitoring ML/AI products.
  • Prepare and present ML modelling results and analytical insights that help drive the business to senior leadership.


The Essentials

  • 8 + years of work experience in Machine Learning, AI and Data Science with a proven track record to drive innovation and business impacts
  • 4 + years of managing a team of data scientists, ML and AI researchers and engineers
  • Strong machine learning, deep learning, and statistical modelling expertise, such as causal inference modelling, ensembles, neural networks, reinforcement learning, NLP, and computer vision
  • Advanced knowledge of SQL and experience with big data platform (AWS, Snowflake, Spark, Google Cloud etc.)
  • Proficiency in machine learning and deep learning languages and platforms (Python, R, TensorFlow, Keras, PyTorch, MXNet etc.)
  • Experience in deploying machine learning algorithms and advanced modelling solutions
  • Experience in developing advanced analytics and ML infrastructure and system
  • Self-starter and self-motivated with the proven ability to deliver results in a fast-paced, high-energy environment
  • Strong communication skills and the ability to explain complex analysis and algorithms to non-technical audience
  • Works effectively cross functional teams to build trusted partnership
  • Working experience in digital media and entertainment industry preferred
  • Experience with Agile methodologies preferred
Read more
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Talent500?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort