19+ Hadoop Jobs in Delhi, NCR and Gurgaon | Hadoop Job openings in Delhi, NCR and Gurgaon

Job Title: Data Engineer

Job Summary: As a Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure and tools necessary for data collection, storage, processing, and analysis. You will work closely with data scientists and analysts to ensure that data is available, accessible, and in a format that can be easily consumed for business insights.

Responsibilities:

• Design, build, and maintain data pipelines to collect, store, and process data from various sources.
• Create and manage data warehousing and data lake solutions.
• Develop and maintain data processing and data integration tools.
• Collaborate with data scientists and analysts to design and implement data models and algorithms for data analysis.
• Optimize and scale existing data infrastructure to ensure it meets the needs of the business.
• Ensure data quality and integrity across all data sources.
• Develop and implement best practices for data governance, security, and privacy.
• Monitor data pipeline performance / Errors and troubleshoot issues as needed.
• Stay up-to-date with emerging data technologies and best practices.

Requirements:

Bachelor's degree in Computer Science, Information Systems, or a related field.

Experience with ETL tools like Matillion,SSIS,Informatica

Experience with SQL and relational databases such as SQL server, MySQL, PostgreSQL, or Oracle.

Experience in writing complex SQL queries

Strong programming skills in languages such as Python, Java, or Scala.

Experience with data modeling, data warehousing, and data integration.

Strong problem-solving skills and ability to work independently.

Excellent communication and collaboration skills.

Familiarity with big data technologies such as Hadoop, Spark, or Kafka.

Familiarity with data warehouse/Data lake technologies like Snowflake or Databricks

Familiarity with cloud computing platforms such as AWS, Azure, or GCP.

Familiarity with Reporting tools

Teamwork/ growth contribution

• Helping the team in taking the Interviews and identifying right candidates
• Adhering to timelines
• Intime status communication and upfront communication of any risks
• Tech, train, share knowledge with peers.
• Good Communication skills
• Proven abilities to take initiative and be innovative
• Analytical mind with a problem-solving aptitude

Good to have :

Master's degree in Computer Science, Information Systems, or a related field.

Experience with NoSQL databases such as MongoDB or Cassandra.

Familiarity with data visualization and business intelligence tools such as Tableau or Power BI.

Knowledge of machine learning and statistical modeling techniques.

If you are passionate about data and want to work with a dynamic team of data scientists and analysts, we encourage you to apply for this position.

• KSQL
• Data Engineering spectrum (Java/Spark)
• Spark Scala / Kafka Streaming
• Confluent Kafka components
• Basic understanding of Hadoop

Responsibilities:

• Designing and implementing fine-tuned production ready data/ML pipelines in Hadoop platform.
• Driving optimization, testing and tooling to improve quality.
• Reviewing and approving high level & amp; detailed design to ensure that the solution delivers to the business needs and aligns to the data & analytics architecture principles and roadmap.
• Understanding business requirements and solution design to develop and implement solutions that adhere to big data architectural guidelines and address business requirements.
• Following proper SDLC (Code review, sprint process).
• Identifying, designing, and implementing internal process improvements: automating manual processes, optimizing data delivery, etc.
• Building robust and scalable data infrastructure (both batch processing and real-time) to support needs from internal and external users.
• Understanding various data security standards and using secure data security tools to apply and adhere to the required data controls for user access in the Hadoop platform.
• Supporting and contributing to development guidelines and standards for data ingestion.
• Working with a data scientist and business analytics team to assist in data ingestion and data related technical issues.
• Designing and documenting the development & deployment flow.

Requirements:

• Experience in developing rest API services using one of the Scala frameworks.
• Ability to troubleshoot and optimize complex queries on the Spark platform
• Expert in building and optimizing ‘big data’ data/ML pipelines, architectures and data sets.
• Knowledge in modelling unstructured to structured data design.
• Experience in Big Data access and storage techniques.
• Experience in doing cost estimation based on the design and development.
• Excellent debugging skills for the technical stack mentioned above which even includes analyzing server logs and application logs.
• Highly organized, self-motivated, proactive, and ability to propose best design solutions.
• Good time management and multitasking skills to work to deadlines by working independently and as a part of a team.

Job Description:

The data science team is responsible for solving business problems with complex data. Data complexity could be characterized in terms of volume, dimensionality and multiple touchpoints/sources. We understand the data, ask fundamental-first-principle questions, apply our analytical and machine learning skills to solve the problem in the best way possible. 

Our ideal candidate

The role would be a client facing one, hence good communication skills are a must. 

The candidate should have the ability to communicate complex models and analysis in a clear and precise manner. 

The candidate would be responsible for:

• Comprehending business problems properly - what to predict, how to build DV, what value addition he/she is bringing to the client, etc.
• Understanding and analyzing large, complex, multi-dimensional datasets and build features relevant for business
• Understanding the math behind algorithms and choosing one over another
• Understanding approaches like stacking, ensemble and applying them correctly to increase accuracy

Desired technical requirements

• Proficiency with Python and the ability to write production-ready codes. 
• Experience in pyspark, machine learning and deep learning
• Big data experience, e.g. familiarity with Spark, Hadoop, is highly preferred
• Familiarity with SQL or other databases.

Key Responsibilities : ( Data Developer Python, Spark)

Exp : 2 to 9 Yrs 

Development of data platforms, integration frameworks, processes, and code.

Develop and deliver APIs in Python or Scala for Business Intelligence applications build using a range of web languages

Develop comprehensive automated tests for features via end-to-end integration tests, performance tests, acceptance tests and unit tests.

Elaborate stories in a collaborative agile environment (SCRUM or Kanban)

Familiarity with cloud platforms like GCP, AWS or Azure.

Experience with large data volumes.

Familiarity with writing rest-based services.

Experience with distributed processing and systems

Experience with Hadoop / Spark toolsets

Experience with relational database management systems (RDBMS)

Experience with Data Flow development

Knowledge of Agile and associated development techniques including:

n 

• We are looking for : Data engineer
• Sprak
• Scala
• Hadoop

Experience – 3 – 12 yrs

Budget - Open

Location - PAN India (Noida/Bangaluru/Hyderabad/Chennai)

Presto Developer (4)

Understanding of distributed SQL query engine running on Hadoop 

Design and develop core components for Presto 

Contribute to the ongoing Presto development by implementing new features, bug fixes, and other improvements 

Develop new and extend existing Presto connectors to various data sources 

Lead complex and technically challenging projects from concept to completion 

Write tests and contribute to ongoing automation infrastructure development 

Run and analyze software performance metrics 

Collaborate with teams globally across multiple time zones and operate in an Agile development environment 

Hands-on experience and interest with Hadoop 

Company Description

At Bungee Tech, we help retailers and brands meet customers everywhere and, on every occasion, they are in. We believe that accurate, high-quality data matched with compelling market insights empowers retailers and brands to keep their customers at the center of all innovation and value they are delivering. 

We provide a clear and complete omnichannel picture of their competitive landscape to retailers and brands. We collect billions of data points every day and multiple times in a day from publicly available sources. Using high-quality extraction, we uncover detailed information on products or services, which we automatically match, and then proactively track for price, promotion, and availability. Plus, anything we do not match helps to identify a new assortment opportunity.

Empowered with this unrivalled intelligence, we unlock compelling analytics and insights that once blended with verified partner data from trusted sources such as Nielsen, paints a complete, consolidated picture of the competitive landscape.

We are looking for a Big Data Engineer who will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them.

You will also be responsible for integrating them with the architecture used in the company.

We're working on the future. If you are seeking an environment where you can drive innovation, If you want to apply state-of-the-art software technologies to solve real world problems, If you want the satisfaction of providing visible benefit to end-users in an iterative fast paced environment, this is your opportunity.

Responsibilities

As an experienced member of the team, in this role, you will:

• Contribute to evolving the technical direction of analytical Systems and play a critical role their design and development

• You will research, design and code, troubleshoot and support. What you create is also what you own.

• Develop the next generation of automation tools for monitoring and measuring data quality, with associated user interfaces.

• Be able to broaden your technical skills and work in an environment that thrives on creativity, efficient execution, and product innovation.

BASIC QUALIFICATIONS

• Bachelor’s degree or higher in an analytical area such as Computer Science, Physics, Mathematics, Statistics, Engineering or similar.
• 5+ years relevant professional experience in Data Engineering and Business Intelligence
• 5+ years in with Advanced SQL (analytical functions), ETL, Data Warehousing.
• Strong knowledge of data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures, data modeling and performance tuning.
• Ability to effectively communicate with both business and technical teams.
• Excellent coding skills in Java, Python, C++, or equivalent object-oriented programming language
• Understanding of relational and non-relational databases and basic SQL
• Proficiency with at least one of these scripting languages: Perl / Python / Ruby / shell script

PREFERRED QUALIFICATIONS

• Experience with building data pipelines from application databases.
• Experience with AWS services - S3, Redshift, Spectrum, EMR, Glue, Athena, ELK etc.
• Experience working with Data Lakes.
• Experience providing technical leadership and mentor other engineers for the best practices on the data engineering space
• Sharp problem solving skills and ability to resolve ambiguous requirements
• Experience on working with Big Data
• Knowledge and experience on working with Hive and the Hadoop ecosystem
• Knowledge of Spark
• Experience working with Data Science teams

Position: Big Data Engineer

What You'll Do

Punchh is seeking to hire Big Data Engineer at either a senior or tech lead level. Reporting to the Director of Big Data, he/she will play a critical role in leading Punchh’s big data innovations. By leveraging prior industrial experience in big data, he/she will help create cutting-edge data and analytics products for Punchh’s business partners.

This role requires close collaborations with data, engineering, and product organizations. His/her job functions include

• Work with large data sets and implement sophisticated data pipelines with both structured and structured data.
• Collaborate with stakeholders to design scalable solutions.
• Manage and optimize our internal data pipeline that supports marketing, customer success and data science to name a few.
• A technical leader of Punchh’s big data platform that supports AI and BI products.
• Work with infra and operations team to monitor and optimize existing infrastructure 
• Occasional business travels are required.

What You'll Need

• 5+ years of experience as a Big Data engineering professional, developing scalable big data solutions.
• Advanced degree in computer science, engineering or other related fields.
• Demonstrated strength in data modeling, data warehousing and SQL.
• Extensive knowledge with cloud technologies, e.g. AWS and Azure.
• Excellent software engineering background. High familiarity with software development life cycle. Familiarity with GitHub/Airflow.
• Advanced knowledge of big data technologies, such as programming language (Python, Java), relational (Postgres, mysql), NoSQL (Mongodb), Hadoop (EMR) and streaming (Kafka, Spark).
• Strong problem solving skills with demonstrated rigor in building and maintaining a complex data pipeline.
• Exceptional communication skills and ability to articulate a complex concept with thoughtful, actionable recommendations.

Dear Candidate,,        

Greetings of the day!

As discussed, Please find the below job description.

Job Title           :            Hadoop developer

Experience      :            3+ years

Job Location   :            New Delhi

Job type           :            Permanent

Knowledge and Skills Required:

Brief Skills:

Hadoop, Spark, Scala and Spark SQL

Main Skills:

1. Strong experience in Hadoop development
2. Experience in Spark
3. Experience in Scala
4. Experience in Spark SQL

Why OTSi!

Working with OTSi gives you the assurance of a successful, fast-paced career.

Exposure to infinite opportunities to learn and grow, familiarization with cutting-edge technologies, cross-domain experience and a harmonious environment are some of the prime attractions for a career-driven workforce.

Join us today, as we assure you 2000+ friends and a great career; Happiness begins at a great workplace..!

Feel free to refer this opportunity to your friends and associates.

About OTSI: (CMMI Level 3): Founded in 1999 and headquartered in Overland Park, Kansas, OTSI offers global reach and local delivery to companies of all sizes, from start-ups to Fortune 500s. Through offices across the US and around the world, we provide universal access to exceptional talent and innovative solutions in a variety of delivery models to reduce overall risk while optimizing outcomes & enabling our customers to thrive in a global economy.http://otsi-usa.com/?page_id=2806">
OTSI's global presence, scalable and sustainable world-class infrastructure, business continuity processes, ISO 9001:2000, CMMI 3 certifications makes us a preferred service provider for our clients. OTSI has the expertise in different technologies enhanced by our http://otsi-usa.com/?page_id=2933">partnerships and alliances with industry giants like HP, Microsoft, IBM, Oracle, and SAP and others. Highly repetitive local company with a proven success of serving the UAE Government IT needs is seeking to attract, employ and develop people with exceptional skills who want to make a difference in a challenging environment.Object Technology Solutions India Pvt Ltd is a leading Global Information Technology (IT) Services and Solutions company offering a wide array of Solutions for a range of key Verticals. The company is headquartered in Overland Park, Kansas, and has a strong presence in US, Europe and Asia-Pacific with a Global Delivery Center based in India. OTSI offers a broad range of IT application solutions and services including; e-Business solutions, Enterprise Resource Planning (ERP) implementation and Post Implementation Support, Application development, Application Maintenance, Software customizations services.

OTSI Partners & Practices

• SAP Partner
• Microsoft Silver Partner
• Oracle Gold Partner
• Microsoft CoE
• DevOps Consulting
• Cloud
• Mobile & IoT
• Digital Transformation
• Big data & Analytics
• Testing Solutions

OTSI Honor’s & Awards:

• #91 in Inc.5000 .
• Fastest growing IT Companies in Inc.5000…

JD:

Required Skills:

• Intermediate to Expert level hands-on programming using one of programming language- Java or Python or Pyspark or Scala.
• Strong practical knowledge of SQL.
  Hands on experience on Spark/SparkSQL
• Data Structure and Algorithms
• Hands-on experience as an individual contributor in Design, Development, Testing and Deployment of Big Data technologies based applications
• Experience in Big Data application tools, such as Hadoop, MapReduce, Spark, etc
• Experience on NoSQL Databases like HBase, etc
• Experience with Linux OS environment (Shell script, AWK, SED)
• Intermediate RDBMS skill, able to write SQL query with complex relation on top of big RDMS (100+ table)