Big Data

Requirements

• 3+ years work experience with production-grade python. Contribution to open source repos is preferred
• Experience writing concurrent and distributed programs, AWS lambda, Kubernetes, Docker, Spark is preferred.
• Experience with one relational & 1 non-relational DB is preferred
• Prior work in the ML domain will be a big boost

What You’ll Do

• Help realize the product vision: Production-ready machine learning models with monitoring within moments, not months.
• Help companies deploy their machine learning models at scale across a wide range of use-cases and sectors.
• Build integrations with other platforms to make it easy for our customers to use our product without changing their workflow.
• Write maintainable, scalable performant python code
• Building gRPC, rest API servers
• Working with Thrift, Protobufs, etc.

The Sr. Analytics Engineer would provide technical expertise in needs identification, data modeling, data movement, and transformation mapping (source to target), automation and testing strategies, translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective.

Understands and leverages best-fit technologies (e.g., traditional star schema structures, cloud, Hadoop, NoSQL, etc.) and approaches to address business and environmental challenges.

Provides data understanding and coordinates data-related activities with other data management groups such as master data management, data governance, and metadata management.

Actively participates with other consultants in problem-solving and approach development.

Responsibilities :

Provide a consultative approach with business users, asking questions to understand the business need and deriving the data flow, conceptual, logical, and physical data models based on those needs.

Perform data analysis to validate data models and to confirm the ability to meet business needs.

Assist with and support setting the data architecture direction, ensuring data architecture deliverables are developed, ensuring compliance to standards and guidelines, implementing the data architecture, and supporting technical developers at a project or business unit level.

Coordinate and consult with the Data Architect, project manager, client business staff, client technical staff and project developers in data architecture best practices and anything else that is data related at the project or business unit levels.

Work closely with Business Analysts and Solution Architects to design the data model satisfying the business needs and adhering to Enterprise Architecture.

Coordinate with Data Architects, Program Managers and participate in recurring meetings.

Help and mentor team members to understand the data model and subject areas.

Ensure that the team adheres to best practices and guidelines.

Requirements :

- Strong working knowledge of at least 3 years of Spark, Java/Scala/Pyspark, Kafka, Git, Unix / Linux, and ETL pipeline designing.

- Experience with Spark optimization/tuning/resource allocations

- Excellent understanding of IN memory distributed computing frameworks like Spark and its parameter tuning, writing optimized workflow sequences.

- Experience of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., Redshift, Bigquery, Cassandra, etc).

- Familiarity with Docker, Kubernetes, Azure Data Lake/Blob storage, AWS S3, Google Cloud storage, etc.

- Have a deep understanding of the various stacks and components of the Big Data ecosystem.

- Hands-on experience with Python is a huge plus

Role- Software Development Engineer-2

As a Software Development Engineer at Amazon, you have industry-leading technical abilities and demonstrate breadth and depth of knowledge. You build software to deliver business impact, making smart technology choices. You work in a team and drive things forward.

Top Skills

You write high quality, maintainable, and robust code, often in Java or C++ or C#

You recognize and adopt best practices in software engineering: design, testing, version control, documentation, build, deployment, and operations.

You have experience building scalable software systems that are high-performance, highly-available, highly transactional, low latency and massively distributed.

Roles & Responsibilities

You solve problems at their root, stepping back to understand the broader context.

You develop pragmatic solutions and build flexible systems that balance engineering complexity and timely delivery, creating business impact.

You understand a broad range of data structures and algorithms and apply them to deliver high-performing applications.

You recognize and use design patterns to solve business problems.

You understand how operating systems work, perform and scale.

You continually align your work with Amazon’s business objectives and seek to deliver business value.

You collaborate to ensure that decisions are based on the merit of the proposal, not the proposer.

You proactively support knowledge-sharing and build good working relationships within the team and with others in Amazon.

You communicate clearly with your team and with other groups and listen effectively.

Skills & Experience

Bachelors or Masters in Computer Science or relevant technical field.

Experience in software development and full product life-cycle.

Excellent programming skills in any object-oriented programming languages - preferably Java, C/C++/C#, Perl, Python, or Ruby.

Strong knowledge of data structures, algorithms, and designing for performance, scalability, and availability.

Proficiency in SQL and data modeling.

1. Should be very strong Scala development(Coding)
2. With Any combination of Java/Python/Spark/Bigdata
3. 3+ years experience in Core Java/Scala with good understanding of multithreading
4. The candidate must be good with Computer Science fundamentals
5. Exposure to python/perl and Unix / K-Shell scripting
6. Code management tools such as Git/Perforce.
7. Experience with large batch-oriented systems 
8. DB2/Sybase or any RDBMS
9. Prior experience with financial products, particularly OTC Derivatives
10. Exposure to counterparty risk, margining, collateral or confirmation systems

Data Platform engineering at Uber is looking for a strong Technical Lead (Level 5a Engineer) who has built high quality platforms and services that can operate at scale. 5a Engineer at Uber exhibits following qualities: 

• Demonstrate tech expertise › Demonstrate technical skills to go very deep or broad in solving classes of problems or creating broadly leverageable solutions. 
• Execute large scale projects › Define, plan and execute complex and impactful projects. You communicate the vision to peers and stakeholders.
• Collaborate across teams › Domain resource to engineers outside your team and help them leverage the right solutions. Facilitate technical discussions and drive to a consensus.
• Coach engineers › Coach and mentor less experienced engineers and deeply invest in their learning and success. You give and solicit feedback, both positive and negative, to others you work with to help improve the entire team.
• Tech leadership › Lead the effort to define the best practices in your immediate team, and help the broader organization establish better technical or business processes.

What You’ll Do

• Build a scalable, reliable, operable and performant data analytics platform for Uber’s engineers, data scientists, products and operations teams.
• Work alongside the pioneers of big data systems such as Hive, Yarn, Spark, Presto, Kafka, Flink to build out a highly reliable, performant, easy to use software system for Uber’s planet scale of data. 
• Become proficient of multi-tenancy, resource isolation, abuse prevention, self-serve debuggability aspects of a high performant, large scale, service while building these capabilities for Uber's engineers and operation folks.

What You’ll Need

• 7+ years experience in building large scale products, distributed systems in a high caliber environment.
• Architecture: Identify and solve major architectural problems by going deep in your field or broad across different teams. Extend, improve, or, when needed, build solutions to address architectural gaps or technical debt.
• Software Engineering/Programming: Create frameworks and abstractions that are reliable and reusable. advanced knowledge of at least one programming language, and are happy to learn more. Our core languages are Java, Python, Go, and Scala.
• Platform Engineering: Solid understanding of distributed systems and operating systems fundamentals such as concurrency, multithreading, file systems, locking etc.
• Execution & Results: You tackle large technical projects/problems that are not clearly defined. You anticipate roadblocks and have strategies to de-risk timelines. You orchestrate work that spans multiple teams and keep your stakeholders informed.
• A team player: You believe that you can achieve more on a team that the whole is greater than the sum of its parts. You rely on others’ candid feedback for continuous improvement.
• Business acumen: You understand requirements beyond the written word. Whether you’re working on an API used by other developers, an internal tool consumed by our operation teams, or a feature used by millions of customers, your attention to details leads to a delightful user experience.