Senior Database Architect

Role: Data Engineer

Total Experience: 5 to 8 Years

Job Location: Gurgaon

Budget -26 28 LPA

Must have - Technical & Soft Skills:

• Python: Data Structures, List, Libraries, Data engineering basics
• SQL: Joins, Groups, Aggregations, Windowing functions, analytic functions etc.
• Worked in AWS services S3, EC2, Glue, Data Pipeline, Athena and Redshift
• Solid hands-on working experience in Big-Data Technologies
• Strong hands-on experience of programming languages like Python, Scala with Spark.
• Good command and working experience on Hadoop/Map Reduce, HDFS, Hive, HBase, and No-SQL Databases
• Hands on working experience on any of the data engineering/analytics platform AWS preferred
• Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Ozzie
• Hands on working experience of data processing at scale with event driven systems, message queues (Kafka/ Flink/Spark Streaming)
• Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API
• Gateway, Lake Foundation
• Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.)
• Feature Engineering/Data Processing to be used for Model development
• Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)
• Hands-on working experience in analyzing source system data and data flows, working with structured and unstructured data

DATA ENGINEER

Overview

They started with a singular belief - what is beautiful cannot and should not be defined in marketing meetings. It's defined by the regular people like us, our sisters, our next-door neighbours, and the friends we make on the playground and in lecture halls. That's why we stand for people-proving everything we do. From the inception of a product idea to testing the final formulations before launch, our consumers are a part of each and every process. They guide and inspire us by sharing their stories with us. They tell us not only about the product they need and the skincare issues they face but also the tales of their struggles, dreams and triumphs. Skincare goes deeper than skin. It's a form of self-care for many. Wherever someone is on this journey, we want to cheer them on through the products we make, the content we create and the conversations we have. What we wish to build is more than a brand. We want to build a community that grows and glows together - cheering each other on, sharing knowledge, and ensuring people always have access to skincare that really works.

Job Description:

We are seeking a skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining the data infrastructure and systems that enable efficient data collection, storage, processing, and analysis. You will collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to implement data pipelines and ensure the availability, reliability, and scalability of our data platform.

Responsibilities:

Design and implement scalable and robust data pipelines to collect, process, and store data from various sources.

Develop and maintain data warehouse and ETL (Extract, Transform, Load) processes for data integration and transformation.

Optimize and tune the performance of data systems to ensure efficient data processing and analysis.

Collaborate with data scientists and analysts to understand data requirements and implement solutions for data modeling and analysis.

Identify and resolve data quality issues, ensuring data accuracy, consistency, and completeness.

Implement and maintain data governance and security measures to protect sensitive data.

Monitor and troubleshoot data infrastructure, perform root cause analysis, and implement necessary fixes.

Stay up-to-date with emerging technologies and industry trends in data engineering and recommend their adoption when appropriate.

Qualifications:

Bachelor’s or higher degree in Computer Science, Information Systems, or a related field.

Proven experience as a Data Engineer or similar role, working with large-scale data processing and storage systems.

Strong programming skills in languages such as Python, Java, or Scala.

Experience with big data technologies and frameworks like Hadoop, Spark, or Kafka.

Proficiency in SQL and database management systems (e.g., MySQL, PostgreSQL, or Oracle).

Familiarity with cloud platforms like AWS, Azure, or GCP, and their data services (e.g., S3, Redshift, BigQuery).

Solid understanding of data modeling, data warehousing, and ETL principles.

Knowledge of data integration techniques and tools (e.g., Apache Nifi, Talend, or Informatica).

Strong problem-solving and analytical skills, with the ability to handle complex data challenges.

Excellent communication and collaboration skills to work effectively in a team environment.

Preferred Qualifications:

Advanced knowledge of distributed computing and parallel processing.

Experience with real-time data processing and streaming technologies (e.g., Apache Kafka, Apache Flink).

Familiarity with machine learning concepts and frameworks (e.g., TensorFlow, PyTorch).

Knowledge of containerization and orchestration technologies (e.g., Docker, Kubernetes).

Experience with data visualization and reporting tools (e.g., Tableau, Power BI).

Certification in relevant technologies or data engineering disciplines.

Job Description

Job Responsibilities

• Design and implement robust database solutions including

  • Security, backup and recovery

  • Performance, scalability, monitoring and tuning,

  • Data management and capacity planning,

  • Planning, and implementing failover between database instances.

• Create data architecture strategies for each subject area of the enterprise data model.

• Communicate plans, status and issues to higher management levels.

• Collaborate with the business, architects and other IT organizations to plan a data strategy, sharing important information related to database concerns and constrains

• Produce all project data architecture deliverables..

• Create and maintain a corporate repository of all data architecture artifacts.

Skills Required:

• Understanding of data analysis, business principles, and operations

• Software architecture and design Network design and implementation

• Data visualization, data migration and data modelling

• Relational database management systems

• DBMS software, including SQL Server  

• Database and cloud computing design, architectures and data lakes

• Information management and data processing on multiple platforms 

• Agile methodologies and enterprise resource planning implementation

• Demonstrate database technical functionality, such as performance tuning, backup and recovery, monitoring.

• Excellent skills with advanced features such as database encryption, replication, partitioning, etc.

• Strong problem solving, organizational and communication skill.

Primary Duties and Responsibilities 

• Experience with Informatica Multidomain MDM 10.4 tool suite preferred
• Partnering with data architects and engineers to ensure an optimal data model design and implementation for each MDM domain in accordance with industry and MDM best practices
• Works with data governance and business steward(s) to design, develop, and configure business rules for data validation, standardize, match, and merge
• Implementation of Data Quality policies, procedures and standards along with Data Governance Team for maintenance of customer, location, product, and other data domains; Experience with Informatica IDQ tool suite preferred.
• Performs data analysis and source-to-target mapping for ingest and egress of data.
• Maintain compliance with change control, SDLC, and development standards.
• Champion the creation and contribution to technical documentation and diagrams.
• Establishes a technical vision and strategy with the team and works with the team to turn it into reality.
• Emphasis on coaching and training to cultivate skill development of team members within the department.
• Responsible for keeping up with industry best practices and trends.
• Monitor, troubleshoot, maintain, and continuously improve the MDM ecosystem.

Secondary Duties and Responsibilities

• May participate in off-hours on-call rotation.
• Attends and is prepared to participate in team, department and company meetings.
• Performs other job related duties and special projects as assigned.

Supervisory Responsibilities

This is a non-management role

Education and Experience

• Bachelor's degree in MIS, Computer Sciences, Business Administration, or related field; or High School Degree/General Education Diploma and 4 years of relevant experience in lieu of Bachelor's degree.
• 5+ years of experience in implementing MDM solutions using Informatica MDM.
• 2+ years of experience in data stewardship, data governance, and data management concepts.
• Professional working knowledge of Customer 360 solution
• Professional working knowledge in multi domain MDM data modeling.
• Strong understanding of company master data sets and their application in complex business processes and support data profiling, extraction, and cleansing activities using Informatica Data Quality (IDQ).
• Strong knowledge in the installation and configuration of the Informatica MDM Hub.
• Familiarity with real-time, near real-time and batch data integration.
• Strong experience and understanding of Informatica toolsets including Informatica MDM Hub, Informatica Data Quality (IDQ), Informatica Customer 360, Informatica EDC, Hierarchy Manager (HM), Business Entity Service Model, Address Doctor, Customizations & Composite Services
• Experience with event-driven architectures (e.g. Kafka, Google Pub/Sub, Azure Event Hub, etc.).
• Professional working knowledge of CI/CD technologies such as Concourse, TeamCity, Octopus, Jenkins, and CircleCI.
• Team player that exhibits high energy, strategic thinking, collaboration, direct communication and results orientation.

Physical Requirements

• Visual requirements include: ability to see detail at near range with or without correction. Must be physically able to perform sedentary work: occasionally lifting or carrying objects of no more than 10 pounds, and occasionally standing or walking, reaching, handling, grasping, feeling, talking, hearing and repetitive motions.

Working Conditions

• The duties of this position are performed through a combination of an open office setting and remote work options. Full remote work options available for employees that reside outside of the Des Moines Metro Area. There is frequent pressure to meet deadlines and handle multiple projects in a day.

Equipment Used to Perform Job

• Windows, or Mac computer and various software solutions.

Financial Responsibility

• Responsible for company assets including maintenance of software solutions.

Contacts

• Has frequent contact with office personnel in other departments related to the position, as well as occasional contact with users and customers. Engages stakeholders from other area in the business.

Confidentiality

• Has access to confidential information including trade secrets, intellectual property, various financials, and customer data.

You will be responsible for building and maintaining highly scalable data infrastructure for our cloud-hosted SAAS product. You will work closely with the Product Managers and Technical team to define and implement data pipelines for customer-facing and internal reports.

• Design and develop resilient data pipelines.
• Write efficient queries to fetch data from the report database.
• Work closely with application backend engineers on data requirements for their stories.
• Designing and developing report APIs for the front end to consume.
• Focus on building highly available, fault-tolerant report systems.
• Constantly improve the architecture of the application by clearing the technical backlog. 
• Adopt a culture of learning and development to constantly keep pace with and adopt new technolgies.

An enthusiastic individual with the following skills. Please do not hesitate to apply if you do not match all of it. We are open to promising candidates who are passionate about their work and are team players.

• Education - BE/MCA or equivalent
• Overall 8+ years of experience
• Expert level understanding of database concepts and BI.
• Well verse in databases such as MySQL, MongoDB and hands on experience in creating data models. 
• Must have designed and implemented low latency data warehouse systems.
• Must have strong understanding of Kafka and related systems.
• Experience in clickhouse database preferred.
• Must have good knowledge of APIs and should be able to build interfaces for frontend engineers.
• Should be innovative and communicative in approach
• Will be responsible for functional/technical track of a project

You will work with a top-notch tech team, working closely with the CTO and product team.  

A wholesome opportunity in a fast-paced environment that will enable you to juggle between concepts, yet maintain the quality of content, interact, and share your ideas and have loads of learning while at work. Work with a team of highly talented young professionals and enjoy the benefits of being at Xoxoday.

Xoxoday is a rapidly growing fintech SaaS firm that propels business growth while focusing on human motivation. Backed by Giift and Apis Partners Growth Fund II, Xoxoday offers a suite of three products - Plum, Empuls, and Compass. Xoxoday works with more than 2000 clients across 10+ countries and over 2.5 million users. Headquartered in Bengaluru, Xoxoday is a 300+ strong team with four global offices in San Francisco, Dublin, Singapore, New Delhi.

We look forward to connecting with you. As you may take time to review this opportunity, we will wait for a reasonable time of around 3-5 days before we screen the collected applications and start lining up job discussions with the hiring manager. We however assure you that we will attempt to maintain a reasonable time window for successfully closing this requirement. The candidates will be kept informed and updated on the feedback and application status.

‍

Company Overview:

Rakuten, Inc. (TSE's first section: 4755) is the largest ecommerce company in Japan, and third largest eCommerce marketplace company worldwide. Rakuten provides a variety of consumer and business-focused services including e-commerce, e-reading, travel, banking, securities, credit card, e-money, portal and media, online marketing and professional sports. The company is expanding globally and currently has operations throughout Asia, Western Europe, and the Americas. Founded in 1997, Rakuten is headquartered in Tokyo, with over 17,000 employees and partner staff worldwide. Rakuten's 2018 revenues were 1101.48 billions yen. 　 -In Japanese, Rakuten stands for ‘optimism.’ -It means we believe in the future. -It’s an understanding that, with the right mind-set, -we can make the future better by what we do today. Today, our 70+ businesses span e-commerce, digital content, communications and FinTech, bringing the joy of discovery to more than 1.2 billion members across the world.

Website : https://www.rakuten.com/">https://www.rakuten.com/

Crunchbase : https://www.crunchbase.com/organization/rakuten">Rakuten has raised a total of https://www.crunchbase.com/search/funding_rounds/field/organizations/funding_total/rakuten">$42.4M in funding over https://www.crunchbase.com/search/funding_rounds/field/organizations/num_funding_rounds/rakuten">2 rounds

Companysize : 10,001 + Employees

Founded : 1997

Headquarters : Tokyo, Japan

Work location : Bangalore (M.G.Road)

Please find below Job Description.

Role Description – Data Engineer for AN group (Location - India)

Key responsibilities include:

We are looking for engineering candidate in our Autonomous Networking Team. The ideal candidate must have following abilities –

• Hands- on experience in big data computation technologies (at least one and potentially several of the following: Spark and Spark Streaming, Hadoop, Storm, Kafka Streaming, Flink, etc)
• Familiar with other related big data technologies, such as big data storage technologies (e.g., Phoenix/HBase, Redshift, Presto/Athena, Hive, Spark SQL, BigTable, BigQuery, Clickhouse, etc), messaging layer (Kafka, Kinesis, etc), Cloud and container- based deployments (Docker, Kubernetes etc), Scala, Akka, SocketIO, ElasticSearch, RabbitMQ, Redis, Couchbase, JAVA, Go lang.
• Partner with product management and delivery teams to align and prioritize current and future new product development initiatives in support of our business objectives
• Work with cross functional engineering teams including QA, Platform Delivery and DevOps
• Evaluate current state solutions to identify areas to improve standards, simplify, and enhance functionality and/or transition to effective solutions to improve supportability and time to market
• Not afraid of refactoring existing system and guiding the team about same.
• Experience with Event driven Architecture, Complex Event Processing
• Extensive experience building and owning large- scale distributed backend systems.

• Key responsibility is to design and develop a data pipeline including the architecture, prototyping, and development of data extraction, transformation/processing, cleansing/standardizing, and loading in Data Warehouse at real-time/near the real-time frequency. Source data can be structured, semi-structured, and/or unstructured format.
• Provide technical expertise to design efficient data ingestion solutions to consolidate data from RDBMS, APIs, Messaging queues, weblogs, images, audios, documents, etc of  Enterprise Applications, SAAS applications, external 3rd party sites or APIs, etc through ETL/ELT, API integrations, Change Data Capture, Robotic Process Automation, Custom Python/Java Coding, etc
• Development of complex data transformation using Talend (BigData edition), Python/Java transformation in Talend, SQL/Python/Java UDXs, AWS S3, etc to load in OLAP Data Warehouse in Structured/Semi-structured form
• Development of data model and creating transformation logic to populate models for faster data consumption with simple SQL.
• Implementing automated Audit & Quality assurance checks in Data Pipeline
• Document & maintain data lineage to enable data governance
• Coordination with BIU, IT, and other stakeholders to provide best-in-class data pipeline solutions, exposing data via APIs, loading in down streams, No-SQL Databases, etc

Requirements

• Programming experience using Python / Java, to create functions / UDX
• Extensive technical experience with SQL on RDBMS (Oracle/MySQL/Postgresql etc) including code optimization techniques
• Strong ETL/ELT skillset using Talend BigData Edition. Experience in Talend CDC & MDM functionality will be an advantage.
• Experience & expertise in implementing complex data pipelines, including semi-structured & unstructured data processing
• Expertise to design efficient data ingestion solutions to consolidate data from RDBMS, APIs, Messaging queues, weblogs, images, audios, documents, etc of  Enterprise Applications, SAAS applications, external 3rd party sites or APIs, etc through ETL/ELT, API integrations, Change Data Capture, Robotic Process Automation, Custom Python/Java Coding, etc
• Good understanding & working experience in OLAP Data Warehousing solutions (Redshift, Synapse, Snowflake, Teradata, Vertica, etc) and cloud-native Data Lake (S3, ADLS, BigQuery, etc) solutions
• Familiarity with AWS tool stack for Storage & Processing. Able to recommend the right tools/solutions available to address a technical problem
• Good knowledge of database performance and tuning, troubleshooting, query optimization, and tuning
• Good analytical skills with the ability to synthesize data to design and deliver meaningful information
• Good knowledge of Design, Development & Performance tuning of 3NF/Flat/Hybrid Data Model
• Know-how on any No-SQL DB (DynamoDB, MongoDB, CosmosDB, etc) will be an advantage.
• Ability to understand business functionality, processes, and flows
• Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently

Functional knowledge

• Data Governance & Quality Assurance
• Distributed computing
• Linux
• Data structures and algorithm
• Unstructured Data Processing