20+ Hadoop Jobs in Mumbai | Hadoop Job openings in Mumbai

Experience: 12-15 Years

Key Responsibilities: 

• Client Engagement & Requirements Gathering: Independently engage with client stakeholders to
• understand data landscapes and requirements, translating them into functional and technical specifications.
• Data Architecture & Solution Design: Architect and implement Hadoop-based Cloudera CDP solutions,
• including data integration, data warehousing, and data lakes.
• Data Processes & Governance: Develop data ingestion and ETL/ELT frameworks, ensuring robust data governance and quality practices.
• Performance Optimization: Provide SQL expertise and optimize Hadoop ecosystems (HDFS, Ozone, Kudu, Spark Streaming, etc.) for maximum performance.
• Coding & Development: Hands-on coding in relevant technologies and frameworks, ensuring project deliverables meet stringent quality and performance standards.
• API & Database Management: Integrate APIs and manage databases (e.g., PostgreSQL, Oracle) to support seamless data flows.
• Leadership & Mentoring: Guide and mentor a team of data engineers and analysts, fostering collaboration and technical excellence.

Skills Required:

• a. Technical Proficiency:
• • Extensive experience with Hadoop ecosystem tools and services (HDFS, YARN, Cloudera
• Manager, Impala, Kudu, Hive, Spark Streaming, etc.).
• • Proficiency in programming languages like Spark, Python, Scala and a strong grasp of SQL
• performance tuning.
• • ETL tool expertise (e.g., Informatica, Talend, Apache Nifi) and data modelling knowledge.
• • API integration skills for effective data flow management.
• b. Project Management & Communication:
• • Proven ability to lead large-scale data projects and manage project timelines.
• • Excellent communication, presentation, and critical thinking skills.
• c. Client & Team Leadership:
• • Engage effectively with clients and partners, leading onsite and offshore teams.

About the company

DCB Bank is a new generation private sector bank with 442 branches across India.It is a scheduled commercial bank regulated by the Reserve Bank of India. DCB Bank’s business segments are Retail banking, Micro SME, SME, mid-Corporate, Agriculture, Government, Public Sector, Indian Banks, Co-operative Banks and Non-Banking Finance Companies.

Job Description

Department: Risk Analytics

CTC: Max 18 Lacs

Grade: Sr Manager/AVP

Experience: Min 4 years of relevant experience

We are looking for a Data Scientist to join our growing team of Data Science experts and manage the processes and people responsible for accurate data collection, processing, modelling, analysis, implementation, and maintenance.

Responsibilities

• Understand, monitor and maintain existing financial scorecards (ML Based) and make changes to the model when required.
• Perform Statistical analysis in R and assist IT team with deployment of ML model and analytical frameworks in Python.
• Should be able to handle multiple tasks and must know how to prioritize the work.
• Lead cross-functional projects using advanced data modelling and analysis techniques to discover insights that will guide strategic decisions and uncover optimization opportunities.
• Develop clear, concise and actionable solutions and recommendations for client’s business needs and actively explore client’s business and formulate solutions/ideas which can help client in terms of efficient cost cutting or in achieving growth/revenue/profitability targets faster.
• Build, develop and maintain data models, reporting systems, data automation systems, dashboards and performance metrics support that support key business decisions.
• Design and build technical processes to address business issues.
• Oversee the design and delivery of reports and insights that analyse business functions and key operations and performance metrics.
• Manage and optimize processes for data intake, validation, mining, and engineering as well as modelling, visualization, and communication deliverables.
• Communicate results and business impacts of insight initiatives to the Management of the company.

Requirements

• Industry knowledge
• 4 years or more of experience in financial services industry particularly retail credit industry is a must.
• Candidate should have either worked in banking sector (banks/ HFC/ NBFC) or consulting organizations serving these clients.
• Experience in credit risk model building such as application scorecards, behaviour scorecards, and/ or collection scorecards.
• Experience in portfolio monitoring, model monitoring, model calibration
• Knowledge of ECL/ Basel preferred.
• Educational qualification: Advanced degree in finance, mathematics, econometrics, or engineering.
• Technical knowledge: Strong data handling skills in databases such as SQL and Hadoop. Knowledge with data visualization tools, such as SAS VI/Tableau/PowerBI is preferred.
• Expertise in either R or Python; SAS knowledge will be plus.

Soft skills:

• Ability to quickly adapt to the analytical tools and development approaches used within DCB Bank
• Ability to multi-task good communication and team working skills.
• Ability to manage day-to-day written and verbal communication with relevant stakeholders.
• Ability to think strategically and make changes to data when required.

Key Responsibilities:

• Install, configure, and maintain Hadoop clusters.

• Monitor cluster performance and ensure high availability.

• Manage Hadoop ecosystem components (HDFS, YARN, Ozone, Spark, Kudu, Hive).

• Perform routine cluster maintenance and troubleshooting.

• Implement and manage security and data governance.

• Monitor systems health and optimize performance.

• Collaborate with cross-functional teams to support big data applications.

• Perform Linux administration tasks and manage system configurations.

• Ensure data integrity and backup procedures.

• Architectural Leadership:
• Design and architect robust, scalable, and high-performance Hadoop solutions.
• Define and implement data architecture strategies, standards, and processes.
• Collaborate with senior leadership to align data strategies with business goals.
• Technical Expertise:
• Develop and maintain complex data processing systems using Hadoop and its ecosystem (HDFS, YARN, MapReduce, Hive, HBase, Pig, etc.).
• Ensure optimal performance and scalability of Hadoop clusters.
• Oversee the integration of Hadoop solutions with existing data systems and third-party applications.
• Strategic Planning:
• Develop long-term plans for data architecture, considering emerging technologies and future trends.
• Evaluate and recommend new technologies and tools to enhance the Hadoop ecosystem.
• Lead the adoption of big data best practices and methodologies.
• Team Leadership and Collaboration:
• Mentor and guide data engineers and developers, fostering a culture of continuous improvement.
• Work closely with data scientists, analysts, and other stakeholders to understand requirements and deliver high-quality solutions.
• Ensure effective communication and collaboration across all teams involved in data projects.
• Project Management:
• Lead large-scale data projects from inception to completion, ensuring timely delivery and high quality.
• Manage project resources, budgets, and timelines effectively.
• Monitor project progress and address any issues or risks promptly.
• Data Governance and Security:
• Implement robust data governance policies and procedures to ensure data quality and compliance.
• Ensure data security and privacy by implementing appropriate measures and controls.
• Conduct regular audits and reviews of data systems to ensure compliance with industry standards and regulations.

Role: Principal Software Engineer

We looking for a passionate Principle Engineer - Analytics to build data products that extract valuable business insights for efficiency and customer experience. This role will require managing, processing and analyzing large amounts of raw information and in scalable databases. This will also involve developing unique data structures and writing algorithms for the entirely new set of products. The candidate will be required to have critical thinking and problem-solving skills. The candidates must be experienced with software development with advanced algorithms and must be able to handle large volume of data. Exposure with statistics and machine learning algorithms is a big plus. The candidate should have some exposure to cloud environment, continuous integration and agile scrum processes.

Responsibilities:

• Lead projects both as a principal investigator and project manager, responsible for meeting project requirements on schedule

• Software Development that creates data driven intelligence in the products which deals with Big Data backends

• Exploratory analysis of the data to be able to come up with efficient data structures and algorithms for given requirements

• The system may or may not involve machine learning models and pipelines but will require advanced algorithm development

• Managing, data in large scale data stores (such as NoSQL DBs, time series DBs, Geospatial DBs etc.)

• Creating metrics and evaluation of algorithm for better accuracy and recall

• Ensuring efficient access and usage of data through the means of indexing, clustering etc.

• Collaborate with engineering and product development teams.

Requirements:

• Master’s or Bachelor’s degree in Engineering in one of these domains - Computer Science, Information Technology, Information Systems, or related field from top-tier school

• OR Master’s degree or higher in Statistics, Mathematics, with hands on background in software development.

• Experience of 8 to 10 year with product development, having done algorithmic work

• 5+ years of experience working with large data sets or do large scale quantitative analysis

• Understanding of SaaS based products and services.

• Strong algorithmic problem-solving skills

• Able to mentor and manage team and take responsibilities of team deadline.

Skill set required:

• In depth Knowledge Python programming languages

• Understanding of software architecture and software design

• Must have fully managed a project with a team

• Having worked with Agile project management practices

• Experience with data processing analytics and visualization tools in Python (such as pandas, matplotlib, Scipy, etc.)

• Strong understanding of SQL and querying to NoSQL database (eg. Mongo, Casandra, Redis

Role / Purpose - Lead Developer - API and Microservices

Must have a strong hands-on development track record building integration utilizing a variety of integration products, tools, protocols, technologies, and patterns.

• Must have an in-depth understanding of SOA/EAI/ESB concepts, SOA Governance, Event-Driven Architecture, message-based architectures, file sharing, and exchange platforms, data virtualization and caching strategies, J2EE design patterns, frameworks
• Should possess experience with at least one of middleware technologies (Application Servers, BPMS, BRMS, ESB & Message Brokers), Programming languages (e.g. Java/J2EE, JavaScript, COBOL, C), Operating Systems (e.g. Windows, Linux, MVS), and Databases (DB2, MySQL, No SQL Databases like MongoDB, Cassandra, Hadoop, etc.)
• Must have experience implementing API Service architectures (SOAP, REST) using any of the market-leading API Management tools such as Apigee and frameworks such as Spring Boot for Microservices
• Should have Advanced skills in implementing API Service architectures (SOAP, REST) using any of the market-leading API Management tools such as Apigee or similar frameworks such as Spring Boot for Microservices 
• Appetite to manage large-scale projects and multiple tracks
•  Experience and knowhow of the e-commerce domain and retail experience are preferred
•  Good communication & people managerial skills

Job Description for :

Role: Data/Integration Architect

Experience – 8-10 Years

Notice Period: Under 30 days

Key Responsibilities: Designing, Developing frameworks for batch and real time jobs on Talend. Leading migration of these jobs from Mulesoft to Talend, maintaining best practices for the team, conducting code reviews and demos.

Core Skillsets:

Talend Data Fabric - Application, API Integration, Data Integration. Knowledge on Talend Management Cloud, deployment and scheduling of jobs using TMC or Autosys.

Programming Languages - Python/Java
Databases: SQL Server, Other Databases, Hadoop

Should have worked on Agile

Sound communication skills

Should be open to learning new technologies based on business needs on the job

Additional Skills:

Awareness of other data/integration platforms like Mulesoft, Camel

Awareness Hadoop, Snowflake, S3

• upGrad was awarded the Best Tech for Education by IAMAI for 2018-19,
• upGrad was also ranked as one of the LinkedIn Top Startups 2018: The 25 most sought-after startups in India.
• upGrad was earlier selected as one of the top ten most innovative companies in India by FastCompany.
• We were also covered by the Financial Times along with other disruptors in Ed-Tech.
• upGrad is the official education partner for Government of India - Startup India program.
• Our program with IIIT B has been ranked #1 program in the country in the domain of Artificial Intelligence and Machine Learning.

• Work with Product Managers and Business to understand business/product requirements and vision.
• Provide a clear architectural vision in line with business and product vision.
• Lead a team of architects, developers, and data engineers to provide platform services to other engineering teams.
• Provide architectural oversight to engineering teams across the organization.
• Hands on design and development of platform services and features owned by self - this is a hands-on coding role.
• Define guidelines for best practices covering design, unit testing, secure coding etc.
• Ensure quality by reviewing design, code, test plans, load test plans etc. as appropriate.
• Work closely with the QA and Support teams to track quality and proactively identify improvement opportunities.
• Work closely with DevOps and IT to ensure highly secure and cost optimized operations in the cloud.
• Grow technical skills in the team - identify skill gaps with plans to address them, participate in hiring, mentor other architects and engineers.
• Support other engineers in resolving complex technical issues as a go-to person.

• 12+ years of experience in design and development of ecommerce scale systems and highly scalable SaaS or enterprise products.
• Extensive experience in developing extensible and scalable web applications with
  • Java, Spring Boot, Go
  • Web Services - REST, OAuth, OData
  • Database/Caching - MySQL, Cassandra, MongoDB, Memcached/Redis
  • Queue/Broker services - RabbitMQ/Kafka
  • Microservices architecture via Docker on AWS or Azure.
  • Experience with web front end technologies - HTML5, CSS3, JavaScript libraries and frameworks such as jQuery, AngularJS, React, Vue.js, Bootstrap etc.

• Extensive experience with cloud based architectures and how to optimize design for cost.
• Expert level understanding of secure application design practices and a working understanding of cloud infrastructure security.
• Experience with CI/CD processes and design for testability.
• Experience working with big data technologies such as Spark/Storm/Hadoop/Data Lake Architectures is a big plus.
• Action and result-oriented problem-solver who works well both independently and as part of a team; able to foster and develop others' ideas as well as his/her own.
• Ability to organize, prioritize and schedule a high workload and multiple parallel projects efficiently.
• Excellent verbal and written communication with stakeholders in a matrixed environment.
• Long term experience with at least one product from inception to completion and evolution of the product over multiple years.

At Karza technologies, we take pride in building one of the most comprehensive digital onboarding & due-diligence platforms by profiling millions of entities and trillions of associations amongst them using data collated from more than 700 publicly available government sources. Primarily in the B2B Fintech Enterprise space, we are headquartered in Mumbai in Lower Parel with 100+ strong workforce. We are truly furthering the cause of Digital India by providing the entire BFSI ecosystem with tech products and services that aid onboarding customers, automating processes and mitigating risks seamlessly, in real-time and at fraction of the current cost.

A few recognitions:

• Recognized as Top25 startups in India to work with 2019 by LinkedIn
• Winner of HDFC Bank's Digital Innovation Summit 2020
• Super Winners (Won every category) at Tecnoviti 2020 by Banking Frontiers
• Winner of Amazon AI Award 2019 for Fintech
• Winner of FinTech Spot Pitches at Fintegrate Zone 2018 held at BSE
• Winner of FinShare 2018 challenge held by ShareKhan
• Only startup in Yes Bank Global Fintech Accelerator to win the account during the Cohort
• 2nd place Citi India FinTech Challenge 2018 by Citibank
• Top 3 in Viacom18's Startup Engagement Programme VStEP

What your average day would look like:

• Deploy and maintain mission-critical information extraction, analysis, and management systems
• Manage low cost, scalable streaming data pipelines
• Provide direct and responsive support for urgent production issues
• Contribute ideas towards secure and reliable Cloud architecture
• Use open source technologies and tools to accomplish specific use cases encountered within the project
• Use coding languages or scripting methodologies to solve automation problems
• Collaborate with others on the project to brainstorm about the best way to tackle a complex infrastructure, security, or deployment problem
• Identify processes and practices to streamline development & deployment to minimize downtime and maximize turnaround time

What you need to work with us:

• Proficiency in at least one of the general-purpose programming languages like Python, Java, etc.
• Experience in managing the IAAS and PAAS components on popular public Cloud Service Providers like AWS, Azure, GCP etc.
• Proficiency in Unix Operating systems and comfortable with Networking concepts
• Experience with developing/deploying a scalable system
• Experience with the Distributed Database & Message Queues (like Cassandra, ElasticSearch, MongoDB, Kafka, etc.)
• Experience in managing Hadoop clusters
• Understanding of containers and have managed them in production using container orchestration services.
• Solid understanding of data structures and algorithms.
• Applied exposure to continuous delivery pipelines (CI/CD).
• Keen interest and proven track record in automation and cost optimization.

Experience:

• 1-4 years of relevant experience
• BE in Computer Science / Information Technology