Data Engineer

Why LiftOff? 

We at LiftOff specialize in product creation, for our main forte lies in helping Entrepreneurs realize their dream. We have helped businesses and entrepreneurs launch more than 70 plus products.

Many on the team are serial entrepreneurs with a history of successful exits.

As a Data Engineer, you will work directly with our founders and alongside our engineers on a variety of software projects covering various languages, frameworks, and application architectures.

About the Role

If you’re driven by the passion to build something great from scratch, a desire to innovate, and a commitment to achieve excellence in your craft, LiftOff is a great place for you.

• Architecture/design / configure the data ingestion pipeline for data received from 3rd party vendors
• Data loading should be configured with ease/flexibility for adding new data sources & also refresh of the previously loaded data
• Design & implement a consumer graph, that provides an efficient means to query the data via email, phone, and address information (using any one of the fields or combination)
• Expose the consumer graph/search capability for consumption by our middleware APIs, which would be shown in the portal
• Design / review the current client-specific data storage, which is kept as a copy of the consumer master data for easier retrieval/query for subsequent usage

Please Note that this is for a Consultant Role

Candidates who are okay with freelancing/Part-time can apply

Technical/Core skills

1. Minimum 3 yrs of exp in Informatica Big data Developer(BDM) in Hadoop environment.
2. Have knowledge of informatica Power exchange (PWX).
3. Minimum 3 yrs of exp in big data querying tool like Hive and Impala.
4. Ability to designing/development of complex mappings using informatica Big data Developer.
5. Create and manage Informatica power exchange and CDC real time implementation
6. Strong Unix knowledge skills for writing shell scripts and troubleshoot of existing scripts.
7. Good knowledge of big data platforms and its framework.
8. Good to have an experience in cloudera data platform (CDP)
9. Experience with building stream processing systems using Kafka and spark
10. Excellent SQL knowledge

Soft skills :

1. Ability to work independently 
2. Strong analytical and problem solving skills
3. Attitude of learning new technology
4. Regular interaction with vendors, partners and stakeholders

Pingahla is recruiting business intelligence Consultants/Senior consultants who can help us with Information Management projects (domestic, onshore and offshore) as developers and team leads. The candidates are expected to have 3-6 years of experience with Informatica Power Center/Talend DI/Informatica Cloud and must be very proficient with Business Intelligence in general. The job is based out of our Pune office.

Responsibilities:

• Manage the customer relationship by serving as the single point of contact before, during and after engagements.
• Architect data management solutions.
• Provide technical leadership to other consultants and/or customer/partner resources.
• Design, develop, test and deploy data integration solutions in accordance with customer’s schedule.
• Supervise and mentor all intermediate and junior level team members.
• Provide regular reports to communicate status both internally and externally.
• Qualifications:
• A typical profile that would suit this position would be if the following background:
• A graduate from a reputed engineering college 
• An excellent I.Q and analytical skills and should be able to grasp new concepts and learn new technologies.
• A willingness to work with a small team in a fast-growing environment.
• A good knowledge of Business Intelligence concepts

Mandatory Requirements:

• Knowledge of Business Intelligence
• Good knowledge of at least one of the following data integration tools - Informatica Powercenter, Talend DI, Informatica Cloud
• Knowledge of SQL
• Excellent English and communication skills
• Intelligent, quick to learn new technologies
• Track record of accomplishment and effectiveness with handling customers and managing complex data management needs
   

• Expert software implementation and automated testing
• Promoting development standards, code reviews, mentoring, knowledge sharing
• Improving our Agile methodology maturity
• Product and feature design, scrum story writing
• Build, release, and deployment automation
• Product support & troubleshooting

Who we have in mind: 

• Demonstrated experience as a Java
• Should have a deep understanding of Enterprise/Distributed Architecture patterns and should be able to demonstrate the relevant usage of the same
• Turn high-level project requirements into application-level architecture and collaborate with the team members to implement the solution
• Strong experience and knowledge in Spring boot framework and microservice architecture
• Experience in working with Apache Spark
• Solid demonstrated object-oriented software development experience with Java, SQL, Maven, relational/NoSQL databases and testing frameworks 
• Strong working experience with developing RESTful services
• Should have experience working on Application frameworks such as Spring, Spring Boot, AOP
• Exposure to tools – Jira, Bamboo, Git, Confluence would be an added advantage
• Excellent grasp of the current technology landscape, trends and emerging technologies

• Total Experience of 7-10 years and should be interested in teaching and research
• 3+ years’ experience in data engineering which includes data ingestion, preparation, provisioning, automated testing, and quality checks.
• 3+ Hands-on experience in Big Data cloud platforms like AWS and GCP, Data Lakes and Data Warehouses
• 3+ years of Big Data and Analytics Technologies. Experience in SQL, writing code in spark engine using python, scala or java Language. Experience in Spark, Scala
• Experience in designing, building, and maintaining ETL systems
• Experience in data pipeline and workflow management tools like Airflow
• Application Development background along with knowledge of Analytics libraries, opensource Natural Language Processing, statistical and big data computing libraries
• Familiarity with Visualization and Reporting Tools like Tableau, Kibana.
• Should be good at storytelling in Technology

Roles and responsibilities:

1. Responsible for development and maintenance of applications with technologies involving Enterprise Java and Distributed  technologies.
2. Experience in Hadoop, Kafka, Spark, Elastic Search, SQL, Kibana, Python, experience w/ machine learning and Analytics     etc.
3. Collaborate with developers, product manager, business analysts and business users in conceptualizing, estimating and developing new software applications and enhancements..
4. Collaborate with QA team to define test cases, metrics, and resolve questions about test results.
5. Assist in the design and implementation process for new products, research and create POC for possible solutions.
6. Develop components based on business and/or application requirements
7. Create unit tests in accordance with team policies & procedures
8. Advise, and mentor team members in specialized technical areas as well as fulfill administrative duties as defined by support process
9. Work with cross-functional teams during crisis to address and resolve complex incidents and problems in addition to assessment, analysis, and resolution of cross-functional issues.