Big Data Developer

Why LiftOff? 

We at LiftOff specialize in product creation, for our main forte lies in helping Entrepreneurs realize their dream. We have helped businesses and entrepreneurs launch more than 70 plus products.

Many on the team are serial entrepreneurs with a history of successful exits.

As a Data Engineer, you will work directly with our founders and alongside our engineers on a variety of software projects covering various languages, frameworks, and application architectures.

About the Role

If you’re driven by the passion to build something great from scratch, a desire to innovate, and a commitment to achieve excellence in your craft, LiftOff is a great place for you.

• Architecture/design / configure the data ingestion pipeline for data received from 3rd party vendors
• Data loading should be configured with ease/flexibility for adding new data sources & also refresh of the previously loaded data
• Design & implement a consumer graph, that provides an efficient means to query the data via email, phone, and address information (using any one of the fields or combination)
• Expose the consumer graph/search capability for consumption by our middleware APIs, which would be shown in the portal
• Design / review the current client-specific data storage, which is kept as a copy of the consumer master data for easier retrieval/query for subsequent usage

Please Note that this is for a Consultant Role

Candidates who are okay with freelancing/Part-time can apply

Responsibilities:

• Must be able to write quality code and build secure, highly available systems.
• Assemble large, complex datasets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing datadelivery, re-designing infrastructure for greater scalability, etc with the guidance.
• Create datatools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Monitoring performance and advising any necessary infrastructure changes.
• Defining dataretention policies.
• Implementing the ETL process and optimal data pipeline architecture
• Build analytics tools that utilize the datapipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Create design documents that describe the functionality, capacity, architecture, and process.
• Develop, test, and implement datasolutions based on finalized design documents.
• Work with dataand analytics experts to strive for greater functionality in our data
• Proactively identify potential production issues and recommend and implement solutions

Skillsets:

• Good understanding of optimal extraction, transformation, and loading of datafrom a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Proficient understanding of distributed computing principles
• Experience in working with batch processing/ real-time systems using various open-source technologies like NoSQL, Spark, Pig, Hive, Apache Airflow.
• Implemented complex projects dealing with the considerable datasize (PB).
• Optimization techniques (performance, scalability, monitoring, etc.)
• Experience with integration of datafrom multiple data sources
• Experience with NoSQL databases, such as HBase, Cassandra, MongoDB, etc.,
• Knowledge of various ETL techniques and frameworks, such as Flume
• Experience with various messaging systems, such as Kafka or RabbitMQ
• Good understanding of Lambda Architecture, along with its advantages and drawbacks
• Creation of DAGs for dataengineering
• Expert at Python /Scala programming, especially for dataengineering/ ETL purposes

HIRING SQL Developers with max 20 days Of NOTICE PERIOD

Job ID: TNS2023DB01

Who Should apply?

• Only for Serious job seekers who are ready to work in night shift
• Technically Strong Candidates who are willing to take up challenging roles and want to raise their Career graph
• No DBAs & BI Developers, please

• Exposure to the latest technology
• Opportunity to work on different platforms
• Rapid Career Growth
• Friendly Knowledge-Sharing Environment

• BE/MTech/MCA/MSc
• 2yrs Hands Experience in MS SQL / NOSQL
• Immediate joiners preferred/ Maximum notice period between 15 to 20 days
• Candidates will be selected based on logical/technical and scenario-based testing
• Work time - 10:00 pm to 6:00 am

Note: Candidates who have attended the interview process with TnS in the last 6 months will not be eligible.

1. Technical Skills Desired:
   1. Experience in MS SQL Server, and one of these Relational DB’s, PostgreSQL / AWS Aurora DB / MySQL / any of NoSQL DBs (MongoDB / DynamoDB / DocumentDB) in an application development environment and eagerness to switch
   2. Design database tables, views, indexes
   3. Write functions and procedures for Middle Tier Development Team
   4. Work with any front-end developers in completing the database modules end to end (hands-on experience in the parsing of JSON & XML in Stored Procedures would be an added advantage).
   5. Query Optimization for performance improvement
   6. Design & develop SSIS Packages or any other Transformation tools for ETL

1. Functional Skills Desired:
   1. The banking / Insurance / Retail domain would be a
   2. Interaction with a client a

1. Knowledge in a Cloud Platform (AWS / Azure)
2. Knowledge on version control system (SVN / Git)
3. Exposure to Quality and Process Management
4. Knowledge in Agile Methodology

1. Soft skills: (additional)
   1. Team building (attitude to train, work along, and mentor juniors)
   2. Communication skills (all kinds)
   3. Quality consciousness
   4. Analytical acumen to all business requirements
   5. Think out-of-box for business solution

We are looking for an outstanding ML Architect (Deployments) with expertise in deploying Machine Learning solutions/models into production and scaling them to serve millions of customers. A candidate with an adaptable and productive working style which fits in a fast-moving environment.

Skills:

- 5+ years deploying Machine Learning pipelines in large enterprise production systems.

- Experience developing end to end ML solutions from business hypothesis to deployment / understanding the entirety of the ML development life cycle.
- Expert in modern software development practices; solid experience using source control management (CI/CD).
- Proficient in designing relevant architecture / microservices to fulfil application integration, model monitoring, training / re-training, model management, model deployment, model experimentation/development, alert mechanisms.
- Experience with public cloud platforms (Azure, AWS, GCP).
- Serverless services like lambda, azure functions, and/or cloud functions.
- Orchestration services like data factory, data pipeline, and/or data flow.
- Data science workbench/managed services like azure machine learning, sagemaker, and/or AI platform.
- Data warehouse services like snowflake, redshift, bigquery, azure sql dw, AWS Redshift.
- Distributed computing services like Pyspark, EMR, Databricks.
- Data storage services like cloud storage, S3, blob, S3 Glacier.
- Data visualization tools like Power BI, Tableau, Quicksight, and/or Qlik.
- Proven experience serving up predictive algorithms and analytics through batch and real-time APIs.
- Solid working experience with software engineers, data scientists, product owners, business analysts, project managers, and business stakeholders to design the holistic solution.
- Strong technical acumen around automated testing.
- Extensive background in statistical analysis and modeling (distributions, hypothesis testing, probability theory, etc.)
- Strong hands-on experience with statistical packages and ML libraries (e.g., Python scikit learn, Spark MLlib, etc.)
- Experience in effective data exploration and visualization (e.g., Excel, Power BI, Tableau, Qlik, etc.)
- Experience in developing and debugging in one or more of the languages Java, Python.
- Ability to work in cross functional teams.
- Apply Machine Learning techniques in production including, but not limited to, neuralnets, regression, decision trees, random forests, ensembles, SVM, Bayesian models, K-Means, etc.

Roles and Responsibilities:

Deploying ML models into production, and scaling them to serve millions of customers.

Technical solutioning skills with deep understanding of technical API integrations, AI / Data Science, BigData and public cloud architectures / deployments in a SaaS environment.

Strong stakeholder relationship management skills - able to influence and manage the expectations of senior executives.
Strong networking skills with the ability to build and maintain strong relationships with both business, operations and technology teams internally and externally.

Provide software design and programming support to projects.

 Qualifications & Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Machine Learning Architect (Deployments) or a similar role for 5-7 years.

CANDIDATE WILL BE DEPLOYED IN A FINANCIAL CAPTIVE ORGANIZATION @ PUNE (KHARADI)

Below are the job Details :-

Experience 10 to 18 years

Mandatory skills –

• data migration,
• data flow

The ideal candidate for this role will have the below experience and qualifications:  

• Experience of building a range of Services in a Cloud Service provider (ideally GCP)  
• Hands-on design and development of Google Cloud Platform (GCP), across a wide range of GCP services including hands on experience of GCP storage & database technologies. 
• Hands-on experience in architecting, designing or implementing solutions on GCP, K8s, and other Google technologies. Security and Compliance, e.g. IAM and cloud compliance/auditing/monitoring tools 
• Desired Skills within the GCP stack - Cloud Run, GKE, Serverless, Cloud Functions, Vision API, DLP, Data Flow, Data Fusion 
• Prior experience of migrating on-prem applications to cloud environments. Knowledge and hands on experience on Stackdriver, pub-sub, VPC, Subnets, route tables, Load balancers, firewalls both for on premise and the GCP.  
• Integrate, configure, deploy and manage centrally provided common cloud services (e.g. IAM, networking, logging, Operating systems, Containers.)  
• Manage SDN in GCP Knowledge and experience of DevOps technologies around Continuous Integration & Delivery in GCP using Jenkins.  
• Hands on experience of Terraform, Kubernetes, Docker, Stackdriver, Terraform  
• Programming experience in one or more of the following languages: Python, Ruby, Java, JavaScript, Go, Groovy, Scala  
• Knowledge or experience in DevOps tooling such as Jenkins, Git, Ansible, Splunk, Jira or Confluence, AppD, Docker, Kubernetes  
• Act as a consultant and subject matter expert for internal teams to resolve technical deployment obstacles, improve product's vision. Ensure compliance with centrally defined Security 
• Financial experience is preferred 
• Ability to learn new technologies and rapidly prototype newer concepts 
• Top-down thinker, excellent communicator, and great problem solver

Exp:- 10  to 18 years

Location:- Pune

Candidate must have experience in below.

• GCP Data Platform
• Data Processing:- Data Flow, Data Prep, Data Fusion
• Data Storage:- Big Query, Cloud Sql,
• Pub Sub, GCS Bucket

REQUIREMENT:

•  Previous experience of working in large scale data engineering
•  4+ years of experience working in data engineering and/or backend technologies with cloud experience (any) is mandatory.
•  Previous experience of architecting and designing backend for large scale data processing.
•  Familiarity and experience of working in different technologies related to data engineering – different database technologies, Hadoop, spark, storm, hive etc.
•  Hands-on and have the ability to contribute a key portion of data engineering backend.
•  Self-inspired and motivated to drive for exceptional results.
•  Familiarity and experience working with different stages of data engineering – data acquisition, data refining, large scale data processing, efficient data storage for business analysis.
•  Familiarity and experience working with different DB technologies and how to scale them.

RESPONSIBILITY:

•  End to end responsibility to come up with data engineering architecture, design, development and then implementation of it.
•  Build data engineering workflow for large scale data processing.
•  Discover opportunities in data acquisition.
•  Bring industry best practices for data engineering workflow.
•  Develop data set processes for data modelling, mining and production.
•  Take additional tech responsibilities for driving an initiative to completion
•  Recommend ways to improve data reliability, efficiency and quality
•  Goes out of their way to reduce complexity.
•  Humble and outgoing - engineering cheerleaders.