Data Engineer

Company Overview:

Rakuten, Inc. (TSE's first section: 4755) is the largest ecommerce company in Japan, and third largest eCommerce marketplace company worldwide. Rakuten provides a variety of consumer and business-focused services including e-commerce, e-reading, travel, banking, securities, credit card, e-money, portal and media, online marketing and professional sports. The company is expanding globally and currently has operations throughout Asia, Western Europe, and the Americas. Founded in 1997, Rakuten is headquartered in Tokyo, with over 17,000 employees and partner staff worldwide. Rakuten's 2018 revenues were 1101.48 billions yen. 　 -In Japanese, Rakuten stands for ‘optimism.’ -It means we believe in the future. -It’s an understanding that, with the right mind-set, -we can make the future better by what we do today. Today, our 70+ businesses span e-commerce, digital content, communications and FinTech, bringing the joy of discovery to more than 1.2 billion members across the world.

Website : https://www.rakuten.com/

Crunchbase : Rakuten has raised a total of $42.4M in funding over 2 rounds

Companysize : 10,001 + Employees

Founded : 1997

Headquarters : Tokyo, Japan

Work location : Bangalore (M.G.Road)

Please find below Job Description.

Role Description – Data Engineer for AN group (Location - India)

 

Key responsibilities include:

 

We are looking for engineering candidate in our Autonomous Networking Team. The ideal candidate must have following abilities –

 

• Hands- on experience in big data computation technologies (at least one and potentially several of the following: Spark and Spark Streaming, Hadoop, Storm, Kafka Streaming, Flink, etc)
• Familiar with other related big data technologies, such as big data storage technologies (e.g., Phoenix/HBase, Redshift, Presto/Athena, Hive, Spark SQL, BigTable, BigQuery, Clickhouse, etc), messaging layer (Kafka, Kinesis, etc), Cloud and container- based deployments (Docker, Kubernetes etc), Scala, Akka, SocketIO, ElasticSearch, RabbitMQ, Redis, Couchbase, JAVA, Go lang.
• Partner with product management and delivery teams to align and prioritize current and future new product development initiatives in support of our business objectives
• Work with cross functional engineering teams including QA, Platform Delivery and DevOps
• Evaluate current state solutions to identify areas to improve standards, simplify, and enhance functionality and/or transition to effective solutions to improve supportability and time to market
• Not afraid of refactoring existing system and guiding the team about same.
• Experience with Event driven Architecture, Complex Event Processing
• Extensive experience building and owning large- scale distributed backend systems.

Responsibilities Description:

Responsible for the development and implementation of machine learning algorithms and techniques to solve business problems and optimize member experiences. Primary duties may include are but not limited to: Design machine learning projects to address specific business problems determined by consultation with business partners. Work with data-sets of varying degrees of size and complexity including both structured and unstructured data. Piping and processing massive data-streams in distributed computing environments such as Hadoop to facilitate analysis. Implements batch and real-time model scoring to drive actions. Develops machine learning algorithms to build customized solutions that go beyond standard industry tools and lead to innovative solutions. Develop sophisticated visualization of analysis output for business users.

 

Experience Requirements:

BS/MA/MS/PhD in Statistics, Computer Science, Mathematics, Machine Learning, Econometrics, Physics, Biostatistics or related Quantitative disciplines. 2-4 years of experience in predictive analytics and advanced expertise with software such as Python, or any combination of education and experience which would provide an equivalent background. Experience in the healthcare sector. Experience in Deep Learning strongly preferred.

 

Required Technical Skill Set:

• Full cycle of building machine learning solutions,

o   Understanding of wide range of algorithms and their corresponding problems to solve

o   Data preparation and analysis

o   Model training and validation

o   Model application to the problem

• Experience using the full open source programming tools and utilities
• Experience in working in end-to-end data science project implementation.
• 2+ years of experience with development and deployment of Machine Learning applications
• 2+ years of experience with NLP approaches in a production setting
• Experience in building models using bagging and boosting algorithms
• Exposure/experience in building Deep Learning models for NLP/Computer Vision use cases preferred
• Ability to write efficient code with good understanding of core Data Structures/algorithms is critical
• Strong python skills following software engineering best practices
• Experience in using code versioning tools like GIT, bit bucket
• Experience in working in Agile projects
• Comfort & familiarity with SQL and Hadoop ecosystem of tools including spark
• Experience managing big data with efficient query program good to have
• Good to have experience in training ML models in tools like Sage Maker, Kubeflow etc.
• Good to have experience in frameworks to depict interpretability of models using libraries like Lime, Shap etc.
• Experience with Health care sector is preferred
• MS/M.Tech or PhD is a plus

Roles and Responsibilities:

• Responsible for developing and maintaining applications with PySpark 

In this role you'll get.

• Being part of core team member for data platform, setup platform foundation while adhering all required quality standards and design patterns
• Write efficient and quality code that can scale
• Adopt Bookr quality standards, recommend process standards and best practices
• Research, learn & adapt new technologies to solve problems & improve existing solutions
• Contribute to engineering excellence backlog
• Identify performance issues
• Effective code and design reviews
• Improve reliability of overall production system by proactively identifying patterns of failure
• Leading and mentoring junior engineers by example
• End-to-end ownership of stories (including design, serviceability, performance, failure handling)
• Strive hard to provide the best experience to anyone using our products
• Conceptualise innovative and elegant solutions to solve challenging big data problems
• Engage with Product Management and Business to drive the agenda, set your priorities and deliver awesome products
• Adhere to company policies, procedures, mission, values, and standards of ethics and integrity

 

On day one we'll expect you to.

• B. E/B. Tech from a reputed institution
• Minimum 5 years of software development experience and at least a year experience in leading/guiding people
• Expert coding skills in Python/PySpark or Java/Scala
• Deep understanding in Big Data Ecosystem - Hadoop and Spark
• Must have project experience with Spark
• Ability to independently troubleshoot Spark jobs
• Good understanding of distributed systems
• Fast learner and quickly adapt to new technologies
• Prefer individuals with high ownership and commitment
• Expert hands on experience with RDBMS
• Fast learner and quickly adapt to new technologies
• Prefer individuals with high ownership and commitment
• Ability to work independently as well as working collaboratively in a team

 

Added bonuses you have.

• Hands on experience with EMR/Glue/Data bricks
• Hand on experience with Airflow
• Hands on experience with AWS Big Data ecosystem

 

We are looking for passionate Engineers who are always hungry for challenging problems. We believe in creating opportunistic, yet balanced, work environment for savvy, entrepreneurial tech individuals. We are thriving on remote work with team working across multiple timezones.

 

 

• Flexible hours & Remote work - We are a results focused bunch, so we encourage you to work whenever and wherever you feel most creative and focused.
• Unlimited PTOWe want you to feel free to recharge your batteries when you need it!
• Stock Options - Opportunity to participate in Company stock plan
• Flat hierarchy - Team leaders at your fingertips
• BFC(Stands for bureaucracy-free company). We're action oriented and don't bother with dragged-out meetings or pointless admin exercises - we'd rather get our hands dirty!
• Working along side Leaders - You being part of core team, will give you opportunity to directly work with founding and management team

 

Good knowledge of SQL , Microsoft Excel One Programming language in SAA/Python or R

Understanding business objectives and developing models that help to achieve them,
along with metrics to track their progress
Managing available resources such as hardware, data, and personnel so that deadlines
are met
Analysing the ML algorithms that could be used to solve a given problem and ranking
them by their success probability
Exploring and visualizing data to gain an understanding of it, then identifying
differences in data distribution that could affect performance when deploying the model
in the real world
Verifying data quality, and/or ensuring it via data cleaning
Supervising the data acquisition process if more data is needed
Defining validation strategies
Defining the pre-processing or feature engineering to be done on a given dataset
Defining data augmentation pipelines
Training models and tuning their hyper parameters
Analysing the errors of the model and designing strategies to overcome them
Deploying models to production

What's the role?

Your role as a Principal Engineer will involve working with various team. As a principal engineer, will need full knowledge of the software development lifecycle and Agile methodologies. You will demonstrate multi-tasking skills under tight deadlines and constraints. You will regularly contribute to the development of work products (including analyzing, designing, programming, debugging, and documenting software) and may work with customers to resolve challenges and respond to suggestions for improvements and enhancements. You will setup the standard and principal for the product he/she drives.

• Setup coding practice, guidelines & quality of the software delivered.
• Determines operational feasibility by evaluating analysis, problem definition, requirements, solution development, and proposed solutions.
• Documents and demonstrates solutions by developing documentation, flowcharts, layouts, diagrams, charts, code comments and clear code.
• Prepares and installs solutions by determining and designing system specifications, standards, and programming.
• Improves operations by conducting systems analysis; recommending changes in policies and procedures.
• Updates job knowledge by studying state-of-the-art development tools, programming techniques, and computing equipment; participating in educational opportunities; reading professional publications; maintaining personal networks; participating in professional organizations.
• Protects operations by keeping information confidential.
• Develops software solutions by studying information needs; conferring with users; studying systems flow, data usage, and work processes; investigating problem areas; following the software development lifecycle. Who are you? You are a go-getter, with an eye for detail, strong problem-solving and debugging skills, and having a degree in BE/MCA/M.E./ M Tech degree or equivalent degree from reputed college/university.

 

Essential Skills / Experience:

• 10+ years of engineering experience
• Experience in designing and developing high volume web-services using API protocols and data formats
• Proficient in API modelling languages and annotation
• Proficient in Java programming
• Experience with Scala programming
• Experience with ETL systems
• Experience with Agile methodologies
• Experience with Cloud service & storage
• Proficient in Unix/Linux operating systems
• Excellent oral and written communication skills Preferred:
• Functional programming languages (Scala, etc)
• Scripting languages (bash, Perl, Python, etc)
• Amazon Web Services (Redshift, ECS etc)

WyngCommerce is building state of the art AI software for the Global Consumer Brands & Retailers to enable best-in-class customer experiences. Our vision is to democratize machine learning algorithms for our customers and help them realize dramatic improvements in speed, cost and flexibility. Backed by a clutch of prominent angel investors & having some of the category leaders in the retail industry as clients, we are looking to hire for our data science team. The data science team at WyngCommerce is on a mission to challenge the norms and re-imagine how retail business should be run across the world. As a Junior Data Scientist in the team, you will be driving and owning the thought leadership and impact on one of our core data science problems. You will work collaboratively with the founders, clients and engineering team to formulate complex problems, run Exploratory Data Analysis and test hypotheses, implement ML-based solutions and fine tune them with more data. This is a high impact role with goals that directly impact our business. Your Role & Responsibilities: - Implement data-driven solutions based on advanced ML and optimization algorithms to address business problems - Research, experiment, and innovate ML/statistical approaches in various application areas of interest and contribute to IP - Partner with engineering teams to build scalable, efficient, automated ML-based pipelines (training/evaluation/monitoring) - Deploy, maintain, and debug ML/decision models in production environment - Analyze and assess data to ensure high data quality and correctness of downstream processes - Communicate results to stakeholders and present data/insights to participate in and drive decision making Desired Skills & Experiences: - Bachelors or Masters in a quantitative field from a top tier college - 1-2 years experience in a data science / analytics role in a technology / analytics company - Solid mathematical background (especially in linear algebra & probability theory) - Familiarity with theoretical aspects of common ML techniques (generalized linear models, ensembles, SVMs, clustering algos, graphical models, etc.), statistical tests/metrics, experiment design, and evaluation methodologies - Demonstrable track record of dealing with ambiguity, prioritizing needs, bias for iterative learning, and delivering results in a dynamic environment with minimal guidance - Hands-on experience in at least one of the following: (a) Anomaly Detection, (b) Time Series Analysis, (c) Product Clustering, (d) Demand Forecasting, (e) Intertemporal Optimization - Good programming skills (fluent in Java/Python/SQL) with experience of using common ML toolkits (e.g., sklearn, tensor flow, keras, nltk) to build models for real world problems - Computational thinking and familiarity with practical application requirements (e.g., latency, memory, processing time) - Excellent written and verbal communication skills for both technical and non-technical audiences - (Plus Point) Experience of applying ML / other techniques in the domain of supply chain - and particularly in retail - for inventory optimization, demand forecasting, assortment planning, and other such problems - (Nice to have) Research experience and publications in top ML/Data science conferences

Telesourcia is a company who want to be a leader in the Big data , all around The Data (machine learning,data Analytics ,Data science etc.. at Madagascar , and we invite you to be part of this adventure