AWS Data Migration Consultant-Delhi

 Job DescriptionPosition: Sr Data Engineer – Databricks & AWS

Experience: 4 - 5 Years

Company Profile:

Exponentia.ai is an AI tech organization with a presence across India, Singapore, the Middle East, and the UK. We are an innovative and disruptive organization, working on cutting-edge technology to help our clients transform into the enterprises of the future. We provide artificial intelligence-based products/platforms capable of automated cognitive decision-making to improve productivity, quality, and economics of the underlying business processes. Currently, we are transforming ourselves and rapidly expanding our business.

Exponentia.ai has developed long-term relationships with world-class clients such as PayPal, PayU, SBI Group, HDFC Life, Kotak Securities, Wockhardt and Adani Group amongst others.

One of the top partners of Cloudera (leading analytics player) and Qlik (leader in BI technologies), Exponentia.ai has recently been awarded the ‘Innovation Partner Award’ by Qlik in 2017.

Get to know more about us on our website: http://www.exponentia.ai/ and Life @Exponentia.

​Role Overview: 

·         A Data Engineer understands the client requirements and develops and delivers the data engineering solutions as per the scope.

·         The role requires good skills in the development of solutions using various services required for data architecture on Databricks Delta Lake, streaming, AWS, ETL Development, and data modeling.

Job Responsibilities

•         Design of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions to support the analytics needs of the organization.

•         Apply best practices during design in data modeling (logical, physical) and ETL pipelines (streaming and batch) using cloud-based services.

•         Design, develop and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (documentation, exploration) of the data.

•         Interact with stakeholders regarding data landscape understanding, conducting discovery exercises, developing proof of concepts and demonstrating it to stakeholders.

Technical Skills 

•         Has more than 2 Years of experience in developing data lakes, and datamarts on the Databricks platform.

•         Proven skill sets in AWS Data Lake services such as - AWS Glue, S3, Lambda, SNS, IAM, and skills in Spark, Python, and SQL.

•         Experience in Pentaho

•         Good understanding of developing a data warehouse, data marts etc.

•         Has a good understanding of system architectures, and design patterns and should be able to design and develop applications using these principles.

Personality Traits

•         Good collaboration and communication skills

•         Excellent problem-solving skills to be able to structure the right analytical solutions.

•         Strong sense of teamwork, ownership, and accountability

•         Analytical and conceptual thinking 

•         Ability to work in a fast-paced environment with tight schedules.

•         Good presentation skills with the ability to convey complex ideas to peers and management.

Education:

BE / ME / MS/MCA.

Data Engineer- Senior

Cubera is a data company revolutionizing big data analytics and Adtech through data share value principles wherein the users entrust their data to us. We refine the art of understanding, processing, extracting, and evaluating the data that is entrusted to us. We are a gateway for brands to increase their lead efficiency as the world moves towards web3.

What are you going to do?

Design & Develop high performance and scalable solutions that meet the needs of our customers.

Closely work with the Product Management, Architects and cross functional teams.

Build and deploy large-scale systems in Java/Python.

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Create data tools for analytics and data scientist team members that assist them in building and optimizing their algorithms.

Follow best practices that can be adopted in Bigdata stack.

Use your engineering experience and technical skills to drive the features and mentor the engineers.

What are we looking for ( Competencies) :

Bachelor’s degree in computer science, computer engineering, or related technical discipline.

Overall 5 to 8 years of programming experience in Java, Python including object-oriented design.

Data handling frameworks: Should have a working knowledge of one or more data handling frameworks like- Hive, Spark, Storm, Flink, Beam, Airflow, Nifi etc.

Data Infrastructure: Should have experience in building, deploying and maintaining applications on popular cloud infrastructure like AWS, GCP etc.

Data Store: Must have expertise in one of general-purpose No-SQL data stores like Elasticsearch, MongoDB, Redis, RedShift, etc.

Strong sense of ownership, focus on quality, responsiveness, efficiency, and innovation.

Ability to work with distributed teams in a collaborative and productive manner.

Benefits:

Competitive Salary Packages and benefits.

Collaborative, lively and an upbeat work environment with young professionals.

Job Category: Development

Job Type: Full Time

Job Location: Bangalore

Responsibilities :

• Involve in planning, design, development and maintenance of large-scale data repositories, pipelines, analytical solutions and knowledge management strategy
• Build and maintain optimal data pipeline architecture to ensure scalability, connect operational systems data for analytics and business intelligence (BI) systems
• Build data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
• Reporting and obtaining insights from large data chunks on import/export and communicating relevant pointers for helping in decision-making
• Preparation, analysis, and presentation of reports to the management for further developmental activities
• Anticipate, identify and solve issues concerning data management to improve data quality

Requirements :

• Ability to build and maintain ETL pipelines 
• Technical Business Analysis experience and hands-on experience developing functional spec
• Good understanding of Data Engineering principles including data modeling methodologies
• Sound understanding of PostgreSQL
• Strong analytical and interpersonal skills as well as reporting capabilities

This profile will include the following responsibilities:

- Develop Parsers for XML and JSON Data sources/feeds

- Write Automation Scripts for product development

- Build API Integrations for 3^rd Party product integration

- Perform Data Analysis

- Research on Machine learning algorithms

- Understand AWS cloud architecture and work with 3 party vendors for deployments

Key deliverables for the Data Science Engineer would be to help us discover the information hidden in vast amounts of data, and help us make smarter decisions to deliver even better products. Your primary focus will be on applying data mining techniques, doing statistical analysis, and building high-quality prediction systems integrated with our products.

What will you do?

• You will be building and deploying ML models to solve specific business problems related to NLP, computer vision, and fraud detection.
• You will be constantly assessing and improving the model using techniques like Transfer learning
• You will identify valuable data sources and automate collection processes along with undertaking pre-processing of structured and unstructured data
• You will own the complete ML pipeline - data gathering/labeling, cleaning, storage, modeling, training/testing, and deployment.
• Assessing the effectiveness and accuracy of new data sources and data gathering techniques.
• Building predictive models and machine-learning algorithms to apply to data sets.
• Coordinate with different functional teams to implement models and monitor outcomes.
•  
• Presenting information using data visualization techniques and proposing solutions and strategies to business challenges

We would love to hear from you if :

• You have 2+ years of experience as a software engineer at a SaaS or technology company
• Demonstrable hands-on programming experience with Python/R Data Science Stack
• Ability to design and implement workflows of Linear and Logistic Regression, Ensemble Models (Random Forest, Boosting) using R/Python
• Familiarity with Big Data Platforms (Databricks, Hadoop, Hive), AWS Services (AWS, Sagemaker, IAM, S3, Lambda Functions, Redshift, Elasticsearch)
• Experience in Probability and Statistics, ability to use ideas of Data Distributions, Hypothesis Testing and other Statistical Tests.
• Demonstrable competency in Data Visualisation using the Python/R Data Science Stack.
• Preferable Experience Experienced in web crawling and data scraping
• Strong experience in NLP. Worked on libraries such as NLTK, Spacy, Pattern, Gensim etc.
• Experience with text mining, pattern matching and fuzzy matching

• Brand new Macbook
• Stock Options
• Health Insurance
• Unlimited Sick Leaves
• Passion Fund (Invest in yourself or your passion project)
• Wind Down

About Company:

Working with a multitude of clients populating the FTSE and Fortune 500s, Audit Partnership is a people focused organization with a strong belief in our employees. We hire the best people to provide the best services to our clients.

APL offers profit recovery services to organizations of all sizes across a number of sectors. APL was borne out of a desire to offer an alternative from the stagnant service provision on offer in the profit recovery industry.

Every year we cover million of pounds for our clients and also work closely with them, sharing our audit findings to minimize future losses. Our dedicated and highly experienced audit teams utilize progressive & dynamic financial service solutions & industry leading technology to achieve maximum success.

We provide dynamic work environments focused on delivering data-driven solutions at a rapidly increased pace over traditional development. Be a part of our passionate and motivated team who are excited to use the latest in software technologies within financial services.

Headquartered in the UK, we have expanded from a small team in 2002 to a market leading organization serving clients across the globe while keeping our clients at the heart of all decisions we make.

Job description:

We are looking for a high-potential, enthusiastic SQL Data Engineer with a strong desire to build a career in data analysis, database design and application solutions. Reporting directly to our UK based Technology team, you will provide support to our global operation in the delivery of data analysis, conversion, and application development to our core audit functions.

Duties will include assisting with data loads, using T-SQL to analyse data, front-end code changes, data housekeeping, data administration, and supporting the Data Services team as a whole.  Your contribution will grow in line with your experience and skills, becoming increasingly involved in the core service functions and client delivery.  A self-starter with a deep commitment to the highest standards of quality and customer service. We are offering a fantastic career opportunity, working for a leading international financial services organisation, serving the world’s largest organisations.

What we are looking for:

• 1-2 years of previous experience in a similar role
• Data analysis and conversion skills using Microsoft SQL Server is essential
• An understanding of relational database design and build
• Schema design, normalising data, indexing, query performance analysis
• Ability to analyse complex data to identify patterns and detect anomalies
• Assisting with ETL design and implementation projects
• Knowledge or experience in one or more of the key technologies below would be preferable:
  • Microsoft SQL Server (SQL Server Management Studio, Stored Procedure writing etc)
  • T-SQL
  • Programming languages (C#, VB, Python etc)
  • Use of Python to manipulate and import data
  •  
  • Experience of ETL/automation advantageous but not essential (SSIS/Prefect/Azure)
• A self-starter who can drive projects with minimal guidance
• Meeting stakeholders to agree system requirements
• Someone who is enthusiastic and eager to learn
• Very good command of English and excellent communication skills

Perks & Benefits:

• A fantastic work life balance
• Competitive compensation and benefits
• Exposure of working with Fortune 500 organization
• Expert guidance and nurture from global leaders
• Opportunities for career and personal advancement with our continued global growth strategy
• Industry leading training programs
• A working environment that is exciting, fun and engaging

• Experience implementing large-scale ETL processes using Informatica PowerCenter.
• Design high-level ETL process and data flow from the source system to target databases.
• Strong experience with Oracle databases and strong SQL.
• Develop & unit test Informatica ETL processes for optimal performance utilizing best practices.
• Performance tune Informatica ETL mappings and report queries.
• Develop database objects like Stored Procedures, Functions, Packages, and Triggers using SQL and PL/SQL.
• Hands-on Experience in Unix.
• Experience in Informatica Cloud (IICS).
• Work with appropriate leads and review high-level ETL design, source to target data mapping document, and be the point of contact for any ETL-related questions.
• Good understanding of project life cycle, especially tasks within the ETL phase.
• Ability to work independently and multi-task to meet critical deadlines in a rapidly changing environment.
• Excellent communication and presentation skills.
• Effectively worked on the Onsite and Offshore work model.

Responsibilities Description:

Responsible for the development and implementation of machine learning algorithms and techniques to solve business problems and optimize member experiences. Primary duties may include are but not limited to: Design machine learning projects to address specific business problems determined by consultation with business partners. Work with data-sets of varying degrees of size and complexity including both structured and unstructured data. Piping and processing massive data-streams in distributed computing environments such as Hadoop to facilitate analysis. Implements batch and real-time model scoring to drive actions. Develops machine learning algorithms to build customized solutions that go beyond standard industry tools and lead to innovative solutions. Develop sophisticated visualization of analysis output for business users.

Experience Requirements:

BS/MA/MS/PhD in Statistics, Computer Science, Mathematics, Machine Learning, Econometrics, Physics, Biostatistics or related Quantitative disciplines. 2-4 years of experience in predictive analytics and advanced expertise with software such as Python, or any combination of education and experience which would provide an equivalent background. Experience in the healthcare sector. Experience in Deep Learning strongly preferred.

Required Technical Skill Set:

• Full cycle of building machine learning solutions,

o   Understanding of wide range of algorithms and their corresponding problems to solve

o   Data preparation and analysis

o   Model training and validation

o   Model application to the problem

• Experience using the full open source programming tools and utilities
• Experience in working in end-to-end data science project implementation.
• 2+ years of experience with development and deployment of Machine Learning applications
• 2+ years of experience with NLP approaches in a production setting
• Experience in building models using bagging and boosting algorithms
• Exposure/experience in building Deep Learning models for NLP/Computer Vision use cases preferred
• Ability to write efficient code with good understanding of core Data Structures/algorithms is critical
• Strong python skills following software engineering best practices
• Experience in using code versioning tools like GIT, bit bucket
• Experience in working in Agile projects
• Comfort & familiarity with SQL and Hadoop ecosystem of tools including spark
• Experience managing big data with efficient query program good to have
• Good to have experience in training ML models in tools like Sage Maker, Kubeflow etc.
• Good to have experience in frameworks to depict interpretability of models using libraries like Lime, Shap etc.
• Experience with Health care sector is preferred
• MS/M.Tech or PhD is a plus