Pyspark Lead/Pyspark Dev

at Virtusa

Agency job
icon
Chennai, Bengaluru (Bangalore), Pune, Mumbai, Hyderabad
icon
3 - 10 yrs
icon
₹10L - ₹24L / yr (ESOP available)
icon
Full time
Skills
PySpark
Python
Amazon Web Services (AWS)
Apache Spark
Glue semantics
Apache Kafka
Amazon Redshift
AWS Lambda
  • Minimum 1 years of relevant experience, in PySpark (mandatory)
  • Hands on experience in development, test, deploy, maintain and improving data integration pipeline in AWS cloud environment is added plus 
  • Ability to play lead role and independently manage 3-5 member of Pyspark development team 
  • EMR ,Python and PYspark mandate.
  • Knowledge and awareness working with AWS Cloud technologies like Apache Spark, , Glue, Kafka, Kinesis, and Lambda in S3, Redshift, RDS
Read more

About Virtusa

Virtusa help clients change, disrupt, and unlock new value that surpasses their wildest expectations not just to reach our best, but to redefine yours.
Read more
Founded
1996
Type
Services
Size
100-1000 employees
Stage
Profitable
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Sr Software Engineer - Python

at Energy Exemplar

Founded 1999  •  Product  •  100-500 employees  •  Profitable
Spark
Hadoop
Big Data
Data engineering
PySpark
Apache Spark
Web Scraping
icon
Pune
icon
6 - 8 yrs
icon
₹15L - ₹22L / yr
Greetings!!

The Energy Exemplar (EE) data team is looking for an experienced Python Developer (Data Engineer) to join our Pune office. As a dedicated Data Engineer on our Research team, you will apply data engineering expertise, work very closely with the core data team to identify different data sources for specific energy markets and create an automated data pipeline. The pipeline will then incrementally pull the data from its sources and maintain a dataset, which in turn provides tremendous value to hundreds of EE customers.

 

At EE, you’ll have access to vast amounts of energy-related data from our sources. Our data pipelines are curated and supported by engineering teams. We also offer many company-sponsored classes and conferences that focus on data engineering, data platform. There’s a great growth opportunity for data engineering at EE..

Responsibilities

  •  Develop, test and maintain architectures, such as databases and large-scale processing systems using high-performance data pipelines.
  •  Recommend and implement ways to improve data reliability, efficiency, and quality.
  •  Identify performant features and make them universally accessible to our teams across EE.
  •  Work together with data analysts and data scientists to wrangle the data and provide quality datasets and insights to business-critical decisions
  • Take end-to-end responsibility for the development, quality, testing, and production readiness of the services you build.
  • Define and evangelize Data Engineering best standards and practices to ensure engineering excellence at every stage of a development cycle.
  • Act as a resident expert for data engineering, feature engineering, exploratory data analysis.
  • Agile methodologies, acting as Scrum Master would be an added plus.

Qualifications

  • 6+ years of professional experience in developing data pipelines for large-scale, complex datasets from varieties of data sources.
  • Data Engineering expertise with strong experience working with Python, Beautiful Soup, Selenium, Regular Expression, Web Scraping.
  • Best practices with Python Development, Doc String, Type Hints, Unit Testing, etc.
  • Experience working with Cloud-based data technologies such as Azure Data lake, Azure Data Factory, Azure Data Bricks is optionally desirable.
  • Moderate coding skills. SQL or similar required. C# or other languages strongly preferred.
  • Outstanding communication and collaboration skills. You can learn from and teach others.
  • Strong drive for results. You have a proven record of shepherding experiments to create successful shipping products/services
  • A Bachelor or Masters degree in Computer Science or Engineering with coursework in Python, Big Data, Data Engineering is highly desirable.
Read more
Job posted by
Pratibha Shukla

Data Scientist

at Top startup of India - News App

Agency job
via Jobdost
Data Science
Machine Learning (ML)
Natural Language Processing (NLP)
Computer Vision
TensorFlow
Deep Learning
Python
PySpark
MongoDB
Hadoop
Spark
icon
Noida
icon
6 - 10 yrs
icon
₹35L - ₹65L / yr
This will be an individual contributor role and people from Tier 1/2 and Product based company can only apply.

Requirements-

● B.Tech/Masters in Mathematics, Statistics, Computer Science or another quantitative field
● 2-3+ years of work experience in ML domain ( 2-5 years experience )
● Hands-on coding experience in Python
● Experience in machine learning techniques such as Regression, Classification,Predictive modeling, Clustering, Deep Learning stack, NLP.
● Working knowledge of Tensorflow/PyTorch
Optional Add-ons-
● Experience with distributed computing frameworks: Map/Reduce, Hadoop, Spark etc.
● Experience with databases: MongoDB
Read more
Job posted by
Sathish Kumar

Quantitative Developer - Strategy & Algorithm

at Trisha Digital Services

Founded 2022  •  Products & Services  •  0-20 employees  •  Bootstrapped
Quantitative analyst
Algorithmic trading
R Programming
Matlab
SPSS
C#
Python
.NET
icon
Remote only
icon
0 - 2 yrs
icon
₹2.4L - ₹10L / yr

Work Experience : 0-2 years

Responsibilities :

- Design and implement mathematical models for fundamental valuation of securities. The person will need to understand latest research in quantitative finance and implement the same.

- Design, back-testing and implementation of high-frequency trading strategies on international exchanges. Work as part of the market-making team to determine the signals and trading strategies to go live with.

- Conduct performance attribution of live portfolios.

Required Skills :

- Strong candidates should have 0-2 years of work experience and successful track record in quantitative analysis preferably in the capital markets domain.

- Post-Graduate degree in statistics, finance, mathematics, engineering (Computer Science preferred) or other quantitative or computational disciplines

- Experience in using some or all of the following packages: R, MATLAB, SPSS, CART, C# .Net, Python

- Good written and oral communication skills.

- Strong experience working both independently and in a team-oriented collaborative environment.

- Entrepreneurial, self-motivated individual - high energy, high activity levels - passion for working with an innovative, small but rapidly growing company.

Read more
Job posted by
Vikas Gaud

Data Scientist

at It's a deep-tech and research company.

Agency job
via wrackle
Data Science
Python
Natural Language Processing (NLP)
Deep Learning
Long short-term memory (LSTM)
TensorFlow
Keras
NumPy
PyTorch
OpenCV
Machine Learning (ML)
Artificial Intelligence (AI)
Artificial Neural Network (ANN)
icon
Bengaluru (Bangalore)
icon
3 - 8 yrs
icon
₹10L - ₹25L / yr
Job Description: 
 
We are seeking passionate engineers experienced in software development using Machine Learning (ML) and Natural Language Processing (NLP) techniques to join our development team in Bangalore, India. We're a fast-growing startup working on an enterprise product - An intelligent data extraction Platform for various types of documents. 
 
Your responsibilities: 
 
• Build, improve and extend NLP capabilities 
• Research and evaluate different approaches to NLP problems 
• Must be able to write code that is well designed, produce deliverable results 
• Write code that scales and can be deployed to production 
 
You must have: 
 
• Fundamentals of statistical methods is a must 
• Experience in named entity recognition, POS Tagging, Lemmatization, vector representations of textual data and neural networks - RNN, LSTM 
• A solid foundation in Python, data structures, algorithms, and general software development skills. 
• Ability to apply machine learning to problems that deal with language 
• Engineering ability to build robustly scalable pipelines
 • Ability to work in a multi-disciplinary team with a strong product focus
Read more
Job posted by
Naveen Taalanki
Data Science
R Programming
Python
Mathematical modeling
Machine Learning (ML)
Deep Learning
SQL
Microsoft Windows Azure
Spark
icon
Vadodara
icon
4 - 10 yrs
icon
₹15L - ₹20L / yr
Must-Have Skills:
  • Extract and present valuable information from data
  • Understand business requirements and generate insights
  • Build mathematical models, validate and work with them
  • Explain complex topics tailored to the audience
  • Validate and follow up on results
  • Work with large and complex data sets
  • Establish priorities with clear goals and responsibilities to achieve a high level of performance.
  • Work in an agile and iterative manner on solving problems
  • Evaluate different options proactively and the ability to solve problems in an innovative way. Develop new solutions or combine existing methods to create new approaches.
  • Good understanding of Digital & analytics
  • Strong communication skills, orally and in writing

Job Overview:

As a Data Scientist, you will work in collaboration with our business and engineering people, on creating value from data. Often the work requires solving complex problems by turning vast amounts of data into business insights through advanced analytics, modeling, and machine learning. You have a strong foundation in analytics, mathematical modeling, computer science, and math - coupled with a strong business sense. You proactively fetch information from various sources and analyze it for better understanding of how the business performs. Furthermore, you model and build AI tools that automate certain processes within the company. The solutions produced will be implemented to impact business results.
The Data Scientist believes in a non-hierarchical culture of collaboration, transparency, safety, and trust. Working with a focus on value creation, growth, and serving customers with full ownership and accountability. Delivering exceptional customer and business results
Industry: Any (prefer – Manufacturing, Logistics); willingness to learn manufacturing systems (OT systems and data stores)

Primary Responsibilities:

  • Develop an understanding of business obstacles, create solutions based on advanced analytics and draw implications for model development
    • Combine, explore, and draw insights from data. Often large and complex data assets from different parts of the business.
    • Design and build explorative, predictive- or prescriptive models, utilizing optimization, simulation, and machine learning techniques
    • Prototype and pilot new solutions and be a part of the aim of ‘productizing’ those valuable solutions that can have an impact at a global scale
    • Guides and coaches other chapter colleagues to help solve data/technical problems at an operational level, and in methodologies to help improve development processes
    • Identifies and interprets trends and patterns in complex data sets to enable the business to make data-driven decisions




Read more
Job posted by
Priyanka U

Associate Director - Data Science

at Tiger Analytics

Founded 2012  •  Services  •  100-1000 employees  •  Profitable
Data Science
Machine Learning (ML)
Python
R Programming
icon
Remote, Chennai, Remote, Bengaluru (Bangalore), Hyderabad
icon
8 - 14 yrs
icon
₹20L - ₹40L / yr
Associate Director – Data Science

Tiger Analytics is a global AI & analytics consulting firm. With data and technology at the core of our solutions, we are solving some of the toughest problems out there. Our culture is modeled around expertise and mutual respect with a team first mindset. Working at Tiger, you’ll be at the heart of this AI revolution. You’ll work with teams that push the boundaries of what-is-possible and build solutions that energize and inspire.
We are headquartered in the Silicon Valley and have our delivery centres across the globe. The below role is for our Chennai or Bangalore office, or you can choose to work remotely.

About the Role:

As an Associate Director - Data Science at Tiger Analytics, you will lead data science aspects of endto-end client AI & analytics programs. Your role will be a combination of hands-on contribution, technical team management, and client interaction.
• Work closely with internal teams and client stakeholders to design analytical approaches to
solve business problems
• Develop and enhance a broad range of cutting-edge data analytics and machine learning
problems across a variety of industries.
• Work on various aspects of the ML ecosystem – model building, ML pipelines, logging &
versioning, documentation, scaling, deployment, monitoring and maintenance etc.
• Lead a team of data scientists and engineers to embed AI and analytics into the client
business decision processes.

Desired Skills:

• High level of proficiency in a structured programming language, e.g. Python, R.
• Experience designing data science solutions to business problems
• Deep understanding of ML algorithms for common use cases in both structured and
unstructured data ecosystems.
• Comfortable with large scale data processing and distributed computing
• Excellent written and verbal communication skills
• 10+ years exp of which 8 years of relevant data science experience including hands-on
programming.

Designation will be commensurate with expertise/experience. Compensation packages among the best in the industry.
Read more
Job posted by
Muthu Thiagarajan

Senior Data Engineer

at Data Team

Agency job
via Oceanworld
Big Data
Data engineering
Hadoop
data engineer
Apache Hive
Apache Kafka
icon
Remote only
icon
8 - 12 yrs
icon
₹10L - ₹20L / yr
Senior Data Engineer (SDE)

(Hadoop, HDFS, Kafka, Spark, Hive)

Overall Experience - 8 to 12 years

Relevant exp on Big data - 3+ years in above

Salary: Max up-to 20LPA 

Job location - Chennai / Bangalore / 

Notice Period - Immediate joiner / 15-to-20-day Max 

The Responsibilities of The Senior Data Engineer Are:

- Requirements gathering and assessment

- Breakdown complexity and translate requirements to specification artifacts and story boards to build towards, using a test-driven approach

- Engineer scalable data pipelines using big data technologies including but not limited to Hadoop, HDFS, Kafka, HBase, Elastic

- Implement the pipelines using execution frameworks including but not limited to MapReduce, Spark, Hive, using Java/Scala/Python for application design.

- Mentoring juniors in a dynamic team setting

- Manage stakeholders with proactive communication upholding TheDataTeam's brand and values

A Candidate Must Have the Following Skills:

- Strong problem-solving ability

- Excellent software design and implementation ability

- Exposure and commitment to agile methodologies

- Detail oriented with willingness to proactively own software tasks as well as management tasks, and see them to completion with minimal guidance

- Minimum 8 years of experience

- Should have experience in full life-cycle of one big data application

- Strong understanding of various storage formats (ORC/Parquet/Avro)

- Should have hands on experience in one of the Hadoop distributions (Hortoworks/Cloudera/MapR)

- Experience in at least one cloud environment (GCP/AWS/Azure)

- Should be well versed with at least one database (MySQL/Oracle/MongoDB/Postgres)

- Bachelor's in Computer Science, and preferably, a Masters as well - Should have good code review and debugging skills

Additional skills (Good to have):

- Experience in Containerization (docker/Heroku)

- Exposure to microservices

- Exposure to DevOps practices - Experience in Performance tuning of big data applications
Read more
Job posted by
Chandan J
Scala
Big Data
Hadoop
Spark
JVM
Apache Kafka
Akka
icon
Remote, Bengaluru (Bangalore)
icon
15 - 20 yrs
icon
₹50L - ₹120L / yr
About the Company, Conviva:
Conviva is the leader in streaming media intelligence, powered by its real-time platform. More than 250 industry leaders and brands – including CBS, CCTV, Cirque Du Soleil, DAZN, Disney+, HBO, Hulu, Sky, Sling TV, TED, Univision, and Warner Media – rely on Conviva to maximize their consumer engagement, deliver the quality experiences viewers expect and drive revenue growth. With a global footprint of more than 500 million unique viewers watching 150 billion streams per year across 3 billion applications streaming on devices, Conviva offers streaming providers unmatched scale for continuous video measurement, intelligence and benchmarking across every stream, every screen, every second. Conviva is privately held and headquartered in Silicon Valley, California, with offices around the world. For more information, please visit us at www.conviva.com.

What you get to do:

 Be a thought leader. As one of the senior most technical minds in the India centre, influence our technical evolution journey by pushing the boundaries of possibilities by testing forwarding looking ideas and demonstrating its value.
 Be a technical leader: Demonstrate pragmatic skills of translating requirements into technical design.
 Be an influencer. Understand challenges and collaborate across executives and stakeholders in a geographically distributed environment to influence them.
 Be a technical mentor. Build respect within team. Mentor senior engineers technically and
contribute to the growth of talent in the India centre.
 Be a customer advocate. Be empathetic to customer and domain by resolving ambiguity efficiently with the customer in mind.
 Be a transformation agent. Passionately champion engineering best practices and sharing across teams.
 Be hands-on. Participate regularly in code and design reviews, drive technical prototypes and actively contribute to resolving difficult production issues.

What you bring to the role:
 Thrive in a start-up environment and has a platform mindset.
 Excellent communicator. Demonstrated ability to succinctly communicate and describe complexvtechnical designs and technology choices both to executives and developers.
 Expert in Scala coding. JVM based stack is a bonus.
 Expert in big data technologies like Druid, Spark, Hadoop, Flink (or Akka) & Kafka.
 Passionate about one or more engineering best practices that influence design, quality of code or developer efficiency.
 Familiar with building distributed applications using webservices and RESTful APIs.
 Familiarity in building SaaS platforms on either in-house data centres or public cloud providers.
Read more
Job posted by
Bevin Baby

Data Scientist

at A Fintech startup in Dubai

Agency job
via Jobbie
Data Science
Python
R Programming
icon
Remote, Dubai, Bengaluru (Bangalore), Mumbai
icon
2 - 18 yrs
icon
₹14L - ₹38L / yr
RESPONSIBILITIES AND QUALIFICATIONS The mission of the Marcus Surveillance Analytics team is to deliver a platform which detects security incidents which have a tangible business impact and actionable response. You will work alongside industry leading technologists from who have recently joined Goldman from across consumer security, technology, fintech, finance and quant firms. The role has a broad scope which will involve interacting with senior leaders of Goldman and the Consumer business on a regular basis. The position is hands-on and requires a driven and “take ownership” oriented individual who is intently focused on execution. You will work directly with developers, business leaders, vendors and partners in order to deliver security assets to the consumer business. Develop a team, vision and platform which identifies/prioritizes actionable security & fraud risks which have tangible businesses impact across Goldman's consumer and commercial banking businesses. Develop response and recovery technology and programs to ensure resilience from fraud and abuse events. Manage, develop and operationalize analytics which discover security & fraud events and identifies risks for all of Goldman's consumer businesses. Partner with fraud / abuse operations and leadership to ensure consumer fraud rates are within industry norms and own outcomes related to fraud improvements. Skills And Experience We Are Looking For BA/BS degree in Computer Science, Cybersecurity, or other relevant Computer/Data/Engineering degrees 2+ years of experience as a security professional or data analyst/scientist/engineer Python, PySpark, R, Bash, SQL, Splunk (search, ES, UBA) Experience with cloud infrastructure/big data tool sets Visualization tools such as Tableau or D3 Research and development to create innovative predictive detections for security and fraud Build a 24/7 real-time monitoring system with long term vision for scaling to new lines of consumer businesses Strong focus on customer experience and product usability Ability to work closely with the business, fraud, and security incident response teams on creating actionable detections
Read more
Job posted by
Sourav Nandi

Data Scientist

at Woodcutter Film Technologies Pvt. Ltd.

Founded 2018  •  Products & Services  •  0-20 employees  •  Bootstrapped
Data Science
R Programming
Python
icon
Hyderabad
icon
1 - 5 yrs
icon
₹3L - ₹6L / yr
We're an early stage film-tech startup with a mission to empower filmmakers and independent content creators with data-driven decision-making tools. We're looking for a data person to join the core team. Please get in touch if you would be excited to join us on this super exciting journey of disrupting the film production and distribution business. We are currently collaborating with Rana Daggubatt's Suresh Productions, and work out of their studio in Hyderabad - so exposure and opportunities to work on real issues faced by the media industry will be in plenty.
Read more
Job posted by
Athul Krishnan
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Virtusa?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort