Senior Data engineer

Senior Data engineer at Clairvoyant India Private Limited · Remote only · 4 - 9 years · ₹10L - ₹15L / yr · Profitable · Remote only · Posted 6 Jun 2022

Senior Data engineer

at Clairvoyant India Private Limited

Posted by Taruna Roy

4 - 9 yrs

₹10L - ₹15L / yr

Remote only

Skills

Java

Apache Spark

Spark

SQL

HiveQL

Apache Hive

Must-Have:

5+ years of experience in software development.
At least 2 years of relevant work experience on large scale Data applications
Good attitude, strong problem-solving abilities, analytical skills, ability to take ownership as appropriate
Should be able to do coding, debugging, performance tuning, and deploying the apps to Prod.
Should have good working experience Hadoop ecosystem (HDFS, Hive, Yarn, File formats like Avro/Parquet)
Kafka
J2EE Frameworks (Spring/Hibernate/REST)
Spark Streaming or any other streaming technology.
Java programming language is mandatory.
Good to have experience with Java
Ability to work on the sprint stories to completion along with Unit test case coverage.
Experience working in Agile Methodology
Excellent communication and coordination skills
Knowledgeable (and preferred hands-on) - UNIX environments, different continuous integration tools.
Must be able to integrate quickly into the team and work independently towards team goals

Role & Responsibilities:

Take the complete responsibility of the sprint stories’ execution
Be accountable for the delivery of the tasks in the defined timelines with good quality
Follow the processes for project execution and delivery.
Follow agile methodology
Work with the team lead closely and contribute to the smooth delivery of the project.
Understand/define the architecture and discuss the pros-cons of the same with the team
Involve in the brainstorming sessions and suggest improvements in the architecture/design.
Work with other team leads to get the architecture/design reviewed.
Work with the clients and counterparts (in US) of the project.
Keep all the stakeholders updated about the project/task status/risks/issues if there are any.

Must-Have:

5+ years of experience in software development.
At least 2 years of relevant work experience on large scale Data applications
Good attitude, strong problem-solving abilities, analytical skills, ability to take ownership as appropriate
Should be able to do coding, debugging, performance tuning, and deploying the apps to Prod.
Should have good working experience Hadoop ecosystem (HDFS, Hive, Yarn, File formats like Avro/Parquet)
Kafka
J2EE Frameworks (Spring/Hibernate/REST)
Spark Streaming or any other streaming technology.
Java programming language is mandatory.
Good to have experience with Java
Ability to work on the sprint stories to completion along with Unit test case coverage.
Experience working in Agile Methodology
Excellent communication and coordination skills
Knowledgeable (and preferred hands-on) - UNIX environments, different continuous integration tools.
Must be able to integrate quickly into the team and work independently towards team goals

Role & Responsibilities:

Take the complete responsibility of the sprint stories’ execution
Be accountable for the delivery of the tasks in the defined timelines with good quality
Follow the processes for project execution and delivery.
Follow agile methodology
Work with the team lead closely and contribute to the smooth delivery of the project.
Understand/define the architecture and discuss the pros-cons of the same with the team
Involve in the brainstorming sessions and suggest improvements in the architecture/design.
Work with other team leads to get the architecture/design reviewed.
Work with the clients and counterparts (in US) of the project.
Keep all the stakeholders updated about the project/task status/risks/issues if there are any.

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Shubham Vishwakarma

Full Stack Developer - Averlon

I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.

Companies hiring on Cutshort

About Clairvoyant India Private Limited

Founded :

2014

Type :

Services

Size :

100-1000

Stage :

Profitable

About

Clairvoyant is a global technology consulting and services company. We help organizations build innovative products and solutions using big data, analytics, and the cloud. We provide the best-in-class solutions and services that leverage big data and continually exceed client expectations. Our deep vertical knowledge combined with expertise on multiple, enterprise-grade big data platforms helps support purpose-built solutions to meet our client’s business needs. Our global team consists of experienced professionals, with backgrounds in design, software engineering, analytics, and data science. Each member of our team is highly energetic and committed to helping our clients achieve their goals.

Afreen Shaikh

Sandeep Bharate

Unnati Yadav

Taruna Roy

Chakravarthi Peram

Company social profiles

Similar jobs (2)

Java Spark Data Engineer

at Wissen Technology

4 recruiters

Posted by Meghana Shinde

Pune

2.5 - 5 yrs

Best in industry

Java

Spark

Hadoop

Apache HBase

SQL

+1 more

Company Name – Wissen Technology

Group of companies in India – Wissen Technology & Wissen Infotech

Work Location – Whitefield, Bangalore

Website and Company profile:

www.wissen.com

LinkedIn Page:

https://www.linkedin.com/company/wissen-technology/

While you may already know about Wissen and the company history, here is a quick rundown for you.

About Wissen Technology:

· The Wissen Group was founded in the year 2000. Wissen Technology, a part of Wissen Group, was established in the year 2015.

· Wissen Technology is a specialized technology company that delivers high-end consulting for organizations in the Banking & Finance, Telecom, and Healthcare domains. We help clients build world class products.

· Our workforce has highly skilled professionals, with leadership and senior management executives who have graduated from Ivy League Universities like Wharton, MIT, IITs, IIMs, and NITs and with rich work experience in some of the biggest companies in the world.

· Wissen Technology has grown its revenues by 400% in these five years without any external funding or investments.

· Globally present with offices US, India, UK, Australia, Mexico, and Canada.

· We offer an array of services including Application Development, Artificial Intelligence & Machine Learning, Big Data & Analytics, Visualization & Business Intelligence, Robotic Process Automation, Cloud, Mobility, Agile & DevOps, Quality Assurance & Test Automation.

· Wissen Technology has been certified as a Great Place to Work®.

· Wissen Technology has been voted as the Top 20 AI/ML vendor by CIO Insider in 2020.

· Over the years, Wissen Group has successfully delivered $650 million worth of projects for more than 20 of the Fortune 500 companies.

· We have served client across sectors like Banking, Telecom, Healthcare, Manufacturing, and Energy. They include likes of Morgan Stanley, Goldman Sachs, MSCI, StateStreet, Flipkart, Swiggy, Trafigura, GE to name a few.

About Role :

Key Responsibilities

Build and maintain data transformation pipelines using java Spark
Develop and optimize large-scale/CPU intensive data processing using Apache Spark
Orchestrate workflows using Airflow
Implement data quality checks, testing, and monitoring for pipeline. Good to have exposer into managing metadata, cataloguing, and lineage
Support schema evolution, backfills, and incremental processing
Ensure pipelines meet SLAs for freshness, reliability, and performance
Expertise/working knowledge in Spark and HBase(semantic layer, virtual datasets, Reflections)

Required Skills & Qualifications

Strong hands-on experience with
HBase
Apache Spark
Experience with HBase or similar lakehouse query engines
Airflow
Understanding of data catalogs and lineage (e.g., OpenLineage, DataHub, Apache Polaris , openlineage)
Proficiency in Java
Experience with Git-based development and CI/CD

Nice-to-Have Skills

OpenTable format/Iceberg ,Apache Arrow
CDC-based analytics pipelines
Cloud platforms (AWS)
Kubernetes-based data platforms

Company Name – Wissen Technology

Group of companies in India – Wissen Technology & Wissen Infotech

Work Location – Whitefield, Bangalore

Website and Company profile:

www.wissen.com

LinkedIn Page:

https://www.linkedin.com/company/wissen-technology/

While you may already know about Wissen and the company history, here is a quick rundown for you.

About Wissen Technology:

· The Wissen Group was founded in the year 2000. Wissen Technology, a part of Wissen Group, was established in the year 2015.

· Wissen Technology has grown its revenues by 400% in these five years without any external funding or investments.

· Globally present with offices US, India, UK, Australia, Mexico, and Canada.

· Wissen Technology has been certified as a Great Place to Work®.

· Wissen Technology has been voted as the Top 20 AI/ML vendor by CIO Insider in 2020.

· Over the years, Wissen Group has successfully delivered $650 million worth of projects for more than 20 of the Fortune 500 companies.

About Role :

Key Responsibilities

Build and maintain data transformation pipelines using java Spark
Develop and optimize large-scale/CPU intensive data processing using Apache Spark
Orchestrate workflows using Airflow
Implement data quality checks, testing, and monitoring for pipeline. Good to have exposer into managing metadata, cataloguing, and lineage
Support schema evolution, backfills, and incremental processing
Ensure pipelines meet SLAs for freshness, reliability, and performance
Expertise/working knowledge in Spark and HBase(semantic layer, virtual datasets, Reflections)

Required Skills & Qualifications

Strong hands-on experience with
HBase
Apache Spark
Experience with HBase or similar lakehouse query engines
Airflow
Understanding of data catalogs and lineage (e.g., OpenLineage, DataHub, Apache Polaris , openlineage)
Proficiency in Java
Experience with Git-based development and CI/CD

Nice-to-Have Skills

OpenTable format/Iceberg ,Apache Arrow
CDC-based analytics pipelines
Cloud platforms (AWS)
Kubernetes-based data platforms

Sr. Software Engineer I (Viz)

at Planview

3 candid answers

3 recruiters

Posted by Srikanth Bajgur

Bengaluru (Bangalore)

6 - 12 yrs

Best in industry

Java

Spark

Spring Boot

RESTful APIs

React.js

Join our product development team at Planview as a Senior Software Engineer I and become a pivotal force on the Viz Core Team. This role offers the unique opportunity to shape and lead the development of data-processing pipelines and APIs that are at the heart of our software solutions. These solutions are designed to streamline and enhance the efficiency of software delivery, resonating deeply with software engineers who strive to build better and faster.

At Planview Viz, which is powered by the innovative “Flow Framework,” you will tackle complex data challenges and develop scalable solutions within an AWS cloud environment. Your efforts will be crucial in revolutionizing how businesses harness and interpret vast amounts of workflow data, transforming it into actionable insights that propel organizational efficiency and effectiveness.

Responsibilities (What you'll do)

Create and refine powerful data-processing architectures that integrate seamlessly with a diverse array of external tools, enhancing the way software is delivered across industries.
Drive operational excellence by critically analyzing problems, defining requirements, and devising robust solutions that push the boundaries of technology.
Lead and inspire a team of talented engineers, promoting a culture of ownership, meticulous attention to quality, and proactive problem-solving.
Stay at the cutting edge of technology by updating and expanding your team’s knowledge of cloud architectures, data processing, and advanced analytics, ensuring that you and your team remain leaders in technological innovation.
Make a direct impact on the efficiency and effectiveness of software delivery worldwide through innovation and leadership.

Qualifications (What you'll bring)

Who We’re Looking For

The ideal candidate is a seasoned professional in cloud software development with a strong foundation in data processing and scalable software systems. You thrive in collaborative environments and are passionate about advancing cloud technology and data architecture to new heights. You are a leader who enjoys mentoring, guiding, and inspiring others.

Preferred Qualifications

Skills, Knowledge, and Expertise

A degree in Computer Science, Engineering, or a related field.
6+ years of experience with a modern programming language, with a focus on back-end systems and cloud-based technologies.
Strong capability in architecting and designing robust, scalable software systems.
Proficiency in AWS or another popular cloud platform.

Additional Qualifications

Experience with big-data technologies such as Spark and Apache Kafka.
Experience with Java (version 17), Scala, or other JVM programming languages.
Experience with tools such as Amazon Redshift and MongoDB.
Experience with continuous integration and continuous deployment tools such as GitHub, Jenkins, Travis CI, or similar platforms.
Proficiency in test-driven development.
Experience with containers and orchestration tools such as Kubernetes.
Experience working on remote or distributed teams and projects.

Responsibilities (What you'll do)

Create and refine powerful data-processing architectures that integrate seamlessly with a diverse array of external tools, enhancing the way software is delivered across industries.
Drive operational excellence by critically analyzing problems, defining requirements, and devising robust solutions that push the boundaries of technology.
Lead and inspire a team of talented engineers, promoting a culture of ownership, meticulous attention to quality, and proactive problem-solving.
Stay at the cutting edge of technology by updating and expanding your team’s knowledge of cloud architectures, data processing, and advanced analytics, ensuring that you and your team remain leaders in technological innovation.
Make a direct impact on the efficiency and effectiveness of software delivery worldwide through innovation and leadership.

Qualifications (What you'll bring)

Who We’re Looking For

Preferred Qualifications

Skills, Knowledge, and Expertise

A degree in Computer Science, Engineering, or a related field.
6+ years of experience with a modern programming language, with a focus on back-end systems and cloud-based technologies.
Strong capability in architecting and designing robust, scalable software systems.
Proficiency in AWS or another popular cloud platform.

Additional Qualifications

Experience with big-data technologies such as Spark and Apache Kafka.
Experience with Java (version 17), Scala, or other JVM programming languages.
Experience with tools such as Amazon Redshift and MongoDB.
Experience with continuous integration and continuous deployment tools such as GitHub, Jenkins, Travis CI, or similar platforms.
Proficiency in test-driven development.
Experience with containers and orchestration tools such as Kubernetes.
Experience working on remote or distributed teams and projects.

Why apply to jobs via Cutshort