Data Engineer

10+ years of experience in web application design and development

• 10+ years experience with Ruby on Rails

• 5+ years of experience in Jenkins and must have an understanding of CI/CD pipelines

• Analyze, design, develop, and maintain scalable solutions.

• Participate in code review.

• Collaborate with design team to create customer friendly solutions.

• Work with the support team to fix any bugs that arise in code.

• Perform debugging and troubleshooting of existing code base. • Good knowledge of Dockers containers. • Experience with Git/GitHub/AWS DevOps. • Work within a wide range of new and legacy code and technologies in a mature codebase o Experience with React-o Experience with AWS Lambda

Nice to have: Experience with building integrations with 3rd party services/tools such as:o Experience with Sinatra Experience with WordPress, Drupal, Sitecore Experience with MongoDB

• Build campaign generation services which can send app notifications at a speed of 10 million a minute
• Dashboards to show Real time key performance indicators to clients
• Develop complex user segmentation engines which creates segments on Terabytes of data within few seconds
• Building highly available & horizontally scalable platform services for ever growing data
• Use cloud based services like AWS Lambda for blazing fast throughput & auto scalability
• Work on complex analytics on terabytes of data like building Cohorts, Funnels, User path analysis, Recency Frequency & Monetary analysis at blazing speed
• You will build backend services and APIs to create scalable engineering systems.
• As an individual contributor, you will tackle some of our broadest technical challenges that requires deep technical knowledge, hands-on software development and seamless collaboration with all functions.
• You will envision and develop features that are highly reliable and fault tolerant to deliver a superior customer experience.
• Collaborating various highly-functional teams in the company to meet deliverables throughout the software development lifecycle.
• Identify and improvise areas of improvement through data insights and research.

• 2-5 years of experience in backend development and must have worked on Java/shell/Perl/python scripting.
• Solid understanding of engineering best practices, continuous integration, and incremental delivery.
• Strong analytical skills, debugging and troubleshooting skills, product line analysis.
• Follower of agile methodology (Sprint planning, working on JIRA, retrospective etc).
• Proficiency in usage of tools like Docker, Maven, Jenkins and knowledge on frameworks in Java like spring, spring boot, hibernate, JPA.
• Ability to design application modules using various concepts like object oriented, multi-threading, synchronization, caching, fault tolerance, sockets, various IPCs, database interfaces etc.
• Hands on experience on Redis, MySQL and streaming technologies like Kafka producer consumers and NoSQL databases like mongo dB/Cassandra.
• Knowledge about versioning like Git and deployment processes like CICD.

Experience:

The candidate should have about 2+ years of experience with design and development in Java/Scala. Experience in algorithm, data-structure, database and distributed System is mandatory.

Required Skills:

Mandatory: -

1. Core Java or Scala
2. Experience in Big Data, Spark
3. Extensive experience in developing spark job. Should possess good Oops knowledge and be aware of enterprise application design patterns.
4. Should have the ability to analyze, design, develop and test complexity of spark job.
5. Working knowledge of Unix/Linux.
6. Hands on experience in Spark, creating RDD, applying operation - transformation-action

Good To have: -

1. Python
2. Spark streaming
3. Py Spark
4. Azure/AWS Cloud Knowledge of Data Storage and Compute side

Cloudera Data Warehouse Hive team looking for a passionate senior developer to join our growing engineering team. This group is targeting the biggest enterprises wanting to utilize Cloudera’s services in a private and public cloud environment. Our product is built on open source technologies like Hive, Impala, Hadoop, Kudu, Spark and so many more providing unlimited learning opportunities. 

A Day in the Life

Over the past 10+ years, Cloudera has experienced tremendous growth making us the leading contributor to Big Data platforms and ecosystems and a leading provider for enterprise solutions based on Apache Hadoop. You will work with some of the best engineers in the industry who are tackling challenges that will continue to shape the Big Data revolution.  We foster an engaging, supportive, and productive work environment where you can do your best work. The team culture values engineering excellence, technical depth, grassroots innovation, teamwork, and collaboration.

You will manage product development for our CDP components, develop engineering tools and scalable services to enable efficient development, testing, and release operations.  You will be immersed in many exciting, cutting-edge technologies and projects, including collaboration with developers, testers, product, field engineers, and our external partners, both software and hardware vendors.

Opportunity:

Cloudera is a leader in the fast-growing big data platforms market. This is a rare chance to make a name for yourself in the industry and in the Open Source world. The candidate will responsible for Apache Hive and CDW projects. We are looking for a candidate who would like to work on these projects upstream and downstream. If you are curious about the project and code quality you can check the project and the code at the following link. You can start the development before you join. This is one of the beauties of the OSS world. 

https://hive.apache.org/" target="_blank">Apache Hive

Responsibilities:

• Build robust and scalable data infrastructure software

• Design and create services and system architecture for your projects 

• Improve code quality through writing unit tests, automation, and code reviews

• The candidate would write Java code and/or build several services in the Cloudera Data Warehouse. 

• Worked with a team of engineers who reviewed each other's code/designs and held each other to an extremely high bar for the quality of code/designs

• The candidate has to understand the basics of Kubernetes.

• Build out the production and test infrastructure. 

• Develop automation frameworks to reproduce issues and prevent regressions.

• Work closely with other developers providing services to our system.

• Help to analyze and to understand how customers use the product and improve it where necessary.  

Qualifications:

• Deep familiarity with Java programming language.

• Hands-on experience with distributed systems. 

• Knowledge of database concepts, RDBMS internals.

• Knowledge of the Hadoop stack, containers, or Kubernetes is a strong plus.  

• Has experience working in a distributed team. 

• Has 3+ years of experience in software development.