Big Data

Level of skills and experience:

5 years of hands-on experience in using Python, Spark,Sql.

Experienced in AWS Cloud usage and management.

Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).

Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.

Experience with orchestrators such as Airflow and Kubeflow.

Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).

Fundamental understanding of Parquet, Delta Lake and other data file formats.

Proficiency on an IaC tool such as Terraform, CDK or CloudFormation.

Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

• Augmenting, improving, redesigning, and/or re-implementing Dolat's low-latency/high-throughput production trading environment, which collects data from and disseminates orders to exchanges around the world
• Optimizing this platform by using network and systems programming, as well as other advanced techniques
• Developing systems that provide easy access to historical market data and trading simulations
• Building risk-management and performance-tracking tools
• Shaping the future of Dolat through regular interviewing and infrequent campus recruiting trips
• Implementing domain-optimized data structures
• Learn and internalize the theories behind current trading system
• Participate in the design, architecture and implementation of automated trading systems
• Take ownership of system from design through implementation

Cloudera Data Warehouse Hive team looking for a passionate senior developer to join our growing engineering team. This group is targeting the biggest enterprises wanting to utilize Cloudera’s services in a private and public cloud environment. Our product is built on open source technologies like Hive, Impala, Hadoop, Kudu, Spark and so many more providing unlimited learning opportunities. 

A Day in the Life

Over the past 10+ years, Cloudera has experienced tremendous growth making us the leading contributor to Big Data platforms and ecosystems and a leading provider for enterprise solutions based on Apache Hadoop. You will work with some of the best engineers in the industry who are tackling challenges that will continue to shape the Big Data revolution.  We foster an engaging, supportive, and productive work environment where you can do your best work. The team culture values engineering excellence, technical depth, grassroots innovation, teamwork, and collaboration.

You will manage product development for our CDP components, develop engineering tools and scalable services to enable efficient development, testing, and release operations.  You will be immersed in many exciting, cutting-edge technologies and projects, including collaboration with developers, testers, product, field engineers, and our external partners, both software and hardware vendors.

Opportunity:

Cloudera is a leader in the fast-growing big data platforms market. This is a rare chance to make a name for yourself in the industry and in the Open Source world. The candidate will responsible for Apache Hive and CDW projects. We are looking for a candidate who would like to work on these projects upstream and downstream. If you are curious about the project and code quality you can check the project and the code at the following link. You can start the development before you join. This is one of the beauties of the OSS world. 

https://hive.apache.org/" target="_blank">Apache Hive

Responsibilities:

• Build robust and scalable data infrastructure software

• Design and create services and system architecture for your projects 

• Improve code quality through writing unit tests, automation, and code reviews

• The candidate would write Java code and/or build several services in the Cloudera Data Warehouse. 

• Worked with a team of engineers who reviewed each other's code/designs and held each other to an extremely high bar for the quality of code/designs

• The candidate has to understand the basics of Kubernetes.

• Build out the production and test infrastructure. 

• Develop automation frameworks to reproduce issues and prevent regressions.

• Work closely with other developers providing services to our system.

• Help to analyze and to understand how customers use the product and improve it where necessary.  

Qualifications:

• Deep familiarity with Java programming language.

• Hands-on experience with distributed systems. 

• Knowledge of database concepts, RDBMS internals.

• Knowledge of the Hadoop stack, containers, or Kubernetes is a strong plus.  

• Has experience working in a distributed team. 

• Has 3+ years of experience in software development.

JD:

Your role will include:

• Writing and testing your code, innovating and contributing towards increasing the value delivered by your team.
• Setting a high bar through your design, development, analysis and deployment activities
• Understanding and participating in evolving the architecture of our products.
• Keeping up-to-date with new technologies, best practices, and work on optimizing the tooling and automation.
• Understanding the latest development and engineering paradigms like Scrum/Agile/TDD/BDD/DDD etc.

You have experience with the following:

• Strong experience of leading and being part of technical teams preferably following agile methodology.
• Strong technical background with ability to provide technical guidance to other team members.
• Knowledge of microservices and must have experience of implementing a few microservices by the least.
• Knowledge of API driven platform development & Software Integration.
• You have hands-on experience in building secure, high-performing and scalable systems in Java.
• Exposure to JVM based languages like Java, Scala, Clojure.

Data Platform engineering at Uber is looking for a strong Technical Lead (Level 5a Engineer) who has built high quality platforms and services that can operate at scale. 5a Engineer at Uber exhibits following qualities: 

• Demonstrate tech expertise › Demonstrate technical skills to go very deep or broad in solving classes of problems or creating broadly leverageable solutions. 
• Execute large scale projects › Define, plan and execute complex and impactful projects. You communicate the vision to peers and stakeholders.
• Collaborate across teams › Domain resource to engineers outside your team and help them leverage the right solutions. Facilitate technical discussions and drive to a consensus.
• Coach engineers › Coach and mentor less experienced engineers and deeply invest in their learning and success. You give and solicit feedback, both positive and negative, to others you work with to help improve the entire team.
• Tech leadership › Lead the effort to define the best practices in your immediate team, and help the broader organization establish better technical or business processes.

What You’ll Do

• Build a scalable, reliable, operable and performant data analytics platform for Uber’s engineers, data scientists, products and operations teams.
• Work alongside the pioneers of big data systems such as Hive, Yarn, Spark, Presto, Kafka, Flink to build out a highly reliable, performant, easy to use software system for Uber’s planet scale of data. 
• Become proficient of multi-tenancy, resource isolation, abuse prevention, self-serve debuggability aspects of a high performant, large scale, service while building these capabilities for Uber's engineers and operation folks.

What You’ll Need

• 7+ years experience in building large scale products, distributed systems in a high caliber environment.
• Architecture: Identify and solve major architectural problems by going deep in your field or broad across different teams. Extend, improve, or, when needed, build solutions to address architectural gaps or technical debt.
• Software Engineering/Programming: Create frameworks and abstractions that are reliable and reusable. advanced knowledge of at least one programming language, and are happy to learn more. Our core languages are Java, Python, Go, and Scala.
• Platform Engineering: Solid understanding of distributed systems and operating systems fundamentals such as concurrency, multithreading, file systems, locking etc.
• Execution & Results: You tackle large technical projects/problems that are not clearly defined. You anticipate roadblocks and have strategies to de-risk timelines. You orchestrate work that spans multiple teams and keep your stakeholders informed.
• A team player: You believe that you can achieve more on a team that the whole is greater than the sum of its parts. You rely on others’ candid feedback for continuous improvement.
• Business acumen: You understand requirements beyond the written word. Whether you’re working on an API used by other developers, an internal tool consumed by our operation teams, or a feature used by millions of customers, your attention to details leads to a delightful user experience.

Job Description:

Perform high quality software analysis, development, testing and maintenance for the client’s core technology platform  

* Work with Product owners, Architects and X-Vertical teams to analyse requirements.

* Implement the features based on the design.

* Follow good development practices.

* Product deployment and support.

* Mentor junior developers.

Job Requirements

* Excellent analytical skills and problem solving skills.

* Expert in JEE/ Java programming.

* Competent in delivering a high quality product.

* Competent in software engineering tools and best practices

* Conversant in web front-end technologies - HTML5/CSS/JavaScript
* Object-oriented Design and Analysis (OOA and OOD)

* Estimate engineering effort; plan implementation, and rollout system changes.

* Exhibit creativity and resourcefulness at problem solving while collaborating and working effectively   with best in class designers, engineers of different technical backgrounds, architects and product managers.

* Expert in data structures and algorithms.

* Aware of Agile methodology of Software Development Life Cycle.

* Should be a self-starter and a good team player.

* Great communication skills both verbal and written to ensure effective communication across product teams, functions and locations.

* Can make decisions without always asking others what has to be done.

* Ability to mentor junior developers.

* Experience working with remote teams.

* Flexible to work on any technology as per project demand.

Requirements:

• Academic degree (BE / MCA) with 3-10 years of experience in back-end Development.
• Strong knowledge of OOPS concepts, Analyzing, Designing, Development and Unit testing
• Scala technologies, AKKA, REST Webservices, SOAP, Jackson JSON API, JUnit, Mockito, Maven
• Hands-on experience with Play framework
• Familiarity with Microservice Architecture
• Experience working with Apache Tomcat server or TomEE
• Experience working with SQL databases (MySQL, PostgreSQL, Cassandra), writing custom queries, procedures, and designing schemas.
• Good to have front end experience (JavaScript, Angular JS/ React JS)