GCP ARCHITECT / LEAD ENGINEER

MNC Pune based IT company

GCP ARCHITECT / LEAD ENGINEER

Bhs Staffing Solutions Pvt Ltd

Company

Home

GCP ARCHITECT / LEAD ENGINEER

at MNC Pune based IT company

Agency job

via Bhs Staffing Solutions Pvt Ltd

10 - 18 yrs

₹35L - ₹40L / yr

Pune

Skills

Google Cloud Platform (GCP)

Dataflow architecture

Data migration

Data processing

Big Data

cloud sql

big query

pobsub

gcs bucket

CANDIDATE WILL BE DEPLOYED IN A FINANCIAL CAPTIVE ORGANIZATION @ PUNE (KHARADI)

Below are the job Details :-

Experience 10 to 18 years

Mandatory skills –

data migration,
data flow

The ideal candidate for this role will have the below experience and qualifications:

Experience of building a range of Services in a Cloud Service provider (ideally GCP)
Hands-on design and development of Google Cloud Platform (GCP), across a wide range of GCP services including hands on experience of GCP storage & database technologies.
Hands-on experience in architecting, designing or implementing solutions on GCP, K8s, and other Google technologies. Security and Compliance, e.g. IAM and cloud compliance/auditing/monitoring tools
Desired Skills within the GCP stack - Cloud Run, GKE, Serverless, Cloud Functions, Vision API, DLP, Data Flow, Data Fusion
Prior experience of migrating on-prem applications to cloud environments. Knowledge and hands on experience on Stackdriver, pub-sub, VPC, Subnets, route tables, Load balancers, firewalls both for on premise and the GCP.
Integrate, configure, deploy and manage centrally provided common cloud services (e.g. IAM, networking, logging, Operating systems, Containers.)
Manage SDN in GCP Knowledge and experience of DevOps technologies around Continuous Integration & Delivery in GCP using Jenkins.
Hands on experience of Terraform, Kubernetes, Docker, Stackdriver, Terraform
Programming experience in one or more of the following languages: Python, Ruby, Java, JavaScript, Go, Groovy, Scala
Knowledge or experience in DevOps tooling such as Jenkins, Git, Ansible, Splunk, Jira or Confluence, AppD, Docker, Kubernetes
Act as a consultant and subject matter expert for internal teams to resolve technical deployment obstacles, improve product's vision. Ensure compliance with centrally defined Security
Financial experience is preferred
Ability to learn new technologies and rapidly prototype newer concepts
Top-down thinker, excellent communicator, and great problem solver

Exp:- 10 to 18 years

Location:- Pune

Candidate must have experience in below.

GCP Data Platform
Data Processing:- Data Flow, Data Prep, Data Fusion
Data Storage:- Big Query, Cloud Sql,
Pub Sub, GCS Bucket

CANDIDATE WILL BE DEPLOYED IN A FINANCIAL CAPTIVE ORGANIZATION @ PUNE (KHARADI)

Below are the job Details :-

Experience 10 to 18 years

Mandatory skills –

data migration,
data flow

The ideal candidate for this role will have the below experience and qualifications:

Experience of building a range of Services in a Cloud Service provider (ideally GCP)
Hands-on design and development of Google Cloud Platform (GCP), across a wide range of GCP services including hands on experience of GCP storage & database technologies.
Hands-on experience in architecting, designing or implementing solutions on GCP, K8s, and other Google technologies. Security and Compliance, e.g. IAM and cloud compliance/auditing/monitoring tools
Desired Skills within the GCP stack - Cloud Run, GKE, Serverless, Cloud Functions, Vision API, DLP, Data Flow, Data Fusion
Prior experience of migrating on-prem applications to cloud environments. Knowledge and hands on experience on Stackdriver, pub-sub, VPC, Subnets, route tables, Load balancers, firewalls both for on premise and the GCP.
Integrate, configure, deploy and manage centrally provided common cloud services (e.g. IAM, networking, logging, Operating systems, Containers.)
Manage SDN in GCP Knowledge and experience of DevOps technologies around Continuous Integration & Delivery in GCP using Jenkins.
Hands on experience of Terraform, Kubernetes, Docker, Stackdriver, Terraform
Programming experience in one or more of the following languages: Python, Ruby, Java, JavaScript, Go, Groovy, Scala
Knowledge or experience in DevOps tooling such as Jenkins, Git, Ansible, Splunk, Jira or Confluence, AppD, Docker, Kubernetes
Act as a consultant and subject matter expert for internal teams to resolve technical deployment obstacles, improve product's vision. Ensure compliance with centrally defined Security
Financial experience is preferred
Ability to learn new technologies and rapidly prototype newer concepts
Top-down thinker, excellent communicator, and great problem solver

Exp:- 10 to 18 years

Location:- Pune

Candidate must have experience in below.

GCP Data Platform
Data Processing:- Data Flow, Data Prep, Data Fusion
Data Storage:- Big Query, Cloud Sql,
Pub Sub, GCS Bucket

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Subodh Popalwar

Software Engineer, Memorres

For 2 years, I had trouble finding a company with good work culture and a role that will help me grow in my career. Soon after I started using Cutshort, I had access to information about the work culture, compensation and what each company was clearly offering.

Companies hiring on Cutshort

Similar jobs

Senior Software Engineer - Data Support

at DeepIntent

2 candid answers

17 recruiters

Posted by Indrajeet Deshmukh

Pune

4 - 8 yrs

Best in industry

Data Warehouse (DWH)

Informatica

ETL

SQL

Google Cloud Platform (GCP)

+3 more

Who We Are:

DeepIntent is leading the healthcare advertising industry with data-driven solutions built for the future. From day one, our mission has been to improve patient outcomes through the artful use of advertising, data science, and real-world clinical data.

What You’ll Do:

We are looking for a Senior Software Engineer based in Pune, India who can master both DeepIntent’s data architectures and pharma research and analytics methodologies to make significant contributions to how health media is analyzed by our clients. This role requires an Engineer who not only understands DBA functions but also how they impact research objectives and can work with researchers and data scientists to achieve impactful results.

This role will be in the Analytics Organization and will require integration and partnership with the Engineering Organization. The ideal candidate is a self-starter who is inquisitive who is not afraid to take on and learn from challenges and will constantly seek to improve the facets of the business they manage. The ideal candidate will also need to demonstrate the ability to collaborate and partner with others.

Serve as the Engineering interface between Analytics and Engineering teams
Develop and standardized all interface points for analysts to retrieve and analyze data with a focus on research methodologies and data based decisioning
Optimize queries and data access efficiencies, serve as expert in how to most efficiently attain desired data points
Build “mastered” versions of the data for Analytics specific querying use cases
Help with data ETL, table performance optimization
Establish formal data practice for the Analytics practice in conjunction with rest of DeepIntent
Build & operate scalable and robust data architectures
Interpret analytics methodology requirements and apply to data architecture to create standardized queries and operations for use by analytics teams
Implement DataOps practices
Master existing and new Data Pipelines and develop appropriate queries to meet analytics specific objectives
Collaborate with various business stakeholders, software engineers, machine learning engineers, analysts
Operate between Engineers and Analysts to unify both practices for analytics insight creation

Who You Are:

Adept in market research methodologies and using data to deliver representative insights
Inquisitive, curious, understands how to query complicated data sets, move and combine data between databases
Deep SQL experience is a must
Exceptional communication skills with ability to collaborate and translate with between technical and non technical needs
English Language Fluency and proven success working with teams in the U.S.
Experience in designing, developing and operating configurable Data pipelines serving high volume and velocity data
Experience working with public clouds like GCP/AWS
Good understanding of software engineering, DataOps, and data architecture, Agile and DevOps methodologies
Experience building Data architectures that optimize performance and cost, whether the components are prepackaged or homegrown
Proficient with SQL,Python or JVM based language, Bash
Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc.and big data databases like BigQuery, Clickhouse, etc
Ability to think big, take bets and innovate, dive deep, hire and develop the best talent, learn and be curious
Comfortable to work in EST Time Zone

Who We Are:

What You’ll Do:

Serve as the Engineering interface between Analytics and Engineering teams
Develop and standardized all interface points for analysts to retrieve and analyze data with a focus on research methodologies and data based decisioning
Optimize queries and data access efficiencies, serve as expert in how to most efficiently attain desired data points
Build “mastered” versions of the data for Analytics specific querying use cases
Help with data ETL, table performance optimization
Establish formal data practice for the Analytics practice in conjunction with rest of DeepIntent
Build & operate scalable and robust data architectures
Interpret analytics methodology requirements and apply to data architecture to create standardized queries and operations for use by analytics teams
Implement DataOps practices
Master existing and new Data Pipelines and develop appropriate queries to meet analytics specific objectives
Collaborate with various business stakeholders, software engineers, machine learning engineers, analysts
Operate between Engineers and Analysts to unify both practices for analytics insight creation

Who You Are:

Adept in market research methodologies and using data to deliver representative insights
Inquisitive, curious, understands how to query complicated data sets, move and combine data between databases
Deep SQL experience is a must
Exceptional communication skills with ability to collaborate and translate with between technical and non technical needs
English Language Fluency and proven success working with teams in the U.S.
Experience in designing, developing and operating configurable Data pipelines serving high volume and velocity data
Experience working with public clouds like GCP/AWS
Good understanding of software engineering, DataOps, and data architecture, Agile and DevOps methodologies
Experience building Data architectures that optimize performance and cost, whether the components are prepackaged or homegrown
Proficient with SQL,Python or JVM based language, Bash
Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc.and big data databases like BigQuery, Clickhouse, etc
Ability to think big, take bets and innovate, dive deep, hire and develop the best talent, learn and be curious
Comfortable to work in EST Time Zone

Senior Software Engineer

at DeepIntent

2 candid answers

17 recruiters

Posted by Indrajeet Deshmukh

Pune

3 - 5 yrs

Best in industry

PySpark

Data engineering

Big Data

Hadoop

Spark

+5 more

About DeepIntent:

DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioural, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences and message them on a one-to-one basis in a privacy-compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

What You’ll Do:

Establish formal data practice for the organisation.
Build & operate scalable and robust data architectures.
Create pipelines for the self-service introduction and usage of new data
Implement DataOps practices
Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
Engineers.
Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
to deploy and manage.
Collaborate with various business stakeholders, software engineers, machine learning
engineers, and analysts.

Who You Are:

Experience in designing, developing and operating configurable Data pipelines serving high
volume and velocity data.
Experience working with public clouds like GCP/AWS.
Good understanding of software engineering, DataOps, data architecture, Agile and
DevOps methodologies.
Experience building Data architectures that optimize performance and cost, whether the
components are prepackaged or homegrown
Proficient with SQL, Java, Spring boot, Python or JVM-based language, Bash
Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
etc. and big data databases like BigQuery, Clickhouse, etc
Good communication skills with the ability to collaborate with both technical and non-technical
people.
Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

About DeepIntent:

What You’ll Do:

Establish formal data practice for the organisation.
Build & operate scalable and robust data architectures.
Create pipelines for the self-service introduction and usage of new data
Implement DataOps practices
Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
Engineers.
Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
to deploy and manage.
Collaborate with various business stakeholders, software engineers, machine learning
engineers, and analysts.

Who You Are:

Experience in designing, developing and operating configurable Data pipelines serving high
volume and velocity data.
Experience working with public clouds like GCP/AWS.
Good understanding of software engineering, DataOps, data architecture, Agile and
DevOps methodologies.
Experience building Data architectures that optimize performance and cost, whether the
components are prepackaged or homegrown
Proficient with SQL, Java, Spring boot, Python or JVM-based language, Bash
Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
etc. and big data databases like BigQuery, Clickhouse, etc
Good communication skills with the ability to collaborate with both technical and non-technical
people.
Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

Data Engineer

Startup Focused on simplifying Buying Intent

Agency job

via Qrata by Blessy Fernandes

Bengaluru (Bangalore)

4 - 9 yrs

₹28L - ₹56L / yr

Big Data

Apache Spark

Spark

Hadoop

ETL

+7 more

5+ years of experience in a Data Engineer role.
 Proficiency in Linux.
 Must have SQL knowledge and experience working with relational databases,
query authoring (SQL) as well as familiarity with databases including Mysql,
Mongo, Cassandra, and Athena.
 Must have experience with Python/Scala.
Must have experience with Big Data technologies like Apache Spark.
 Must have experience with Apache Airflow.
 Experience with data pipeline and ETL tools like AWS Glue.
 Experience working with AWS cloud services: EC2, S3, RDS, Redshift.

Hadoop Developer

at Pion Global Solutions LTD

2 recruiters

Posted by Sheela P

Mumbai

3 - 100 yrs

₹4L - ₹15L / yr

Spark

Big Data

Hadoop

HDFS

Apache Sqoop

+2 more

Looking for Big data Developers in Mumbai Location

Data Engineer

Picture the future

Agency job

via Jobdost by Sathish Kumar

Hyderabad

4 - 7 yrs

₹5L - ₹15L / yr

PySpark

Data engineering

Big Data

Hadoop

Spark

+7 more

CORE RESPONSIBILITIES

Create and manage cloud resources in AWS
Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies
Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform
Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations
Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
Define process improvement opportunities to optimize data collection, insights and displays.
Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible
Identify and interpret trends and patterns from complex data sets
Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders.
Key participant in regular Scrum ceremonies with the agile teams
Proficient at developing queries, writing reports and presenting findings
Mentor junior members and bring best industry practices

QUALIFICATIONS

5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)
Strong background in math, statistics, computer science, data science or related discipline
Advanced knowledge one of language: Java, Scala, Python, C#
Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
Proficient with
Data mining/programming tools (e.g. SAS, SQL, R, Python)
Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
Data visualization (e.g. Tableau, Looker, MicroStrategy)
Comfortable learning about and deploying new technologies and tools.
Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.
Good written and oral communication skills and ability to present results to non-technical audiences
Knowledge of business intelligence and analytical tools, technologies and techniques.

Mandatory Requirements

Experience in AWS Glue
Experience in Apache Parquet
Proficient in AWS S3 and data lake
Knowledge of Snowflake
Understanding of file-based ingestion best practices.
Scripting language - Python & pyspark

CORE RESPONSIBILITIES

Create and manage cloud resources in AWS
Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies
Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform
Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations
Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
Define process improvement opportunities to optimize data collection, insights and displays.
Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible
Identify and interpret trends and patterns from complex data sets
Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders.
Key participant in regular Scrum ceremonies with the agile teams
Proficient at developing queries, writing reports and presenting findings
Mentor junior members and bring best industry practices

QUALIFICATIONS

5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)
Strong background in math, statistics, computer science, data science or related discipline
Advanced knowledge one of language: Java, Scala, Python, C#
Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
Proficient with
Data mining/programming tools (e.g. SAS, SQL, R, Python)
Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
Data visualization (e.g. Tableau, Looker, MicroStrategy)
Comfortable learning about and deploying new technologies and tools.
Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.
Good written and oral communication skills and ability to present results to non-technical audiences
Knowledge of business intelligence and analytical tools, technologies and techniques.

Mandatory Requirements

Experience in AWS Glue
Experience in Apache Parquet
Proficient in AWS S3 and data lake
Knowledge of Snowflake
Understanding of file-based ingestion best practices.
Scripting language - Python & pyspark

Big Data Engineer

Multinational Company providing Automation digital Solutions

Agency job

via Jobdost by Sathish Kumar

Hyderabad

4 - 7 yrs

₹12L - ₹28L / yr

Python

Spark

Big Data

Hadoop

Apache Hive

Must have :

At least 4 to 7 years of relevant experience as Big Data Engineer
Hands-on experience in Scala or Python
Hands-on experience on major components in Hadoop Ecosystem like HDFS, Map Reduce, Hive, Impala.
Strong programming experience in building applications/platform using Scala or Python.
Experienced in implementing Spark RDD Transformations, actions to implement business analysis

We are specialized in productizing solutions of new technology.
Our vision is to build engineers with entrepreneurial and leadership mindsets who can create highly impactful products and solutions using technology to deliver immense value to our clients.
We strive to develop innovation and passion into everything we do, whether it is services or products, or solutions.

Must have :

At least 4 to 7 years of relevant experience as Big Data Engineer
Hands-on experience in Scala or Python
Hands-on experience on major components in Hadoop Ecosystem like HDFS, Map Reduce, Hive, Impala.
Strong programming experience in building applications/platform using Scala or Python.
Experienced in implementing Spark RDD Transformations, actions to implement business analysis

Senior Systems Engineer – Big Data

at Couture.ai

4 recruiters

Posted by Deleted User

Bengaluru (Bangalore)

2 - 5 yrs

₹5L - ₹10L / yr

Big Data

Hadoop

DevOps

Apache Spark

Spark

+5 more

Skills Requirements
 Knowledge of Hadoop ecosystem installation, initial-configuration and performance tuning.
 Expert with Apache Ambari, Spark, Unix Shell scripting, Kubernetes and Docker
 Knowledge on python would be desirable.
 Experience with HDP Manager/clients and various dashboards.
 Understanding on Hadoop Security (Kerberos, Ranger and Knox) and encryption and Data masking.
 Experience with automation/configuration management using Chef, Ansible or an equivalent.
 Strong experience with any Linux distribution.
 Basic understanding of network technologies, CPU, memory and storage.
 Database administration a plus.
Qualifications and Education Requirements
 2 to 4 years of experience with and detailed knowledge of Core Hadoop Components solutions and
dashboards running on Big Data technologies such as Hadoop/Spark.
 Bachelor degree or equivalent in Computer Science or Information Technology or related fields.

Data Engineer

Service based company

Agency job

via Tech - Soul Technologies by Rohini Shinde

Pune

6 - 12 yrs

₹6L - ₹28L / yr

Big Data

Apache Kafka

Data engineering

Cassandra

Java

+1 more

Primary responsibilities:

Architect, Design and Build high performance Search systems for personalization, optimization, and targeting
Designing systems with Solr, Akka, Cassandra, Kafka
Algorithmic development with primary focus Machine Learning
Working with rapid and innovative development methodologies like: Kanban, Continuous Integration and Daily deployments
Participation in design and code reviews and recommend improvements
Unit testing with JUnit, Performance testing and tuning
Coordination with internal and external teams
Mentoring junior engineers
Participate in Product roadmap and Prioritization discussions and decisions
Evangelize the solution with Professional services and Customer Success teams

Primary responsibilities:

Architect, Design and Build high performance Search systems for personalization, optimization, and targeting
Designing systems with Solr, Akka, Cassandra, Kafka
Algorithmic development with primary focus Machine Learning
Working with rapid and innovative development methodologies like: Kanban, Continuous Integration and Daily deployments
Participation in design and code reviews and recommend improvements
Unit testing with JUnit, Performance testing and tuning
Coordination with internal and external teams
Mentoring junior engineers
Participate in Product roadmap and Prioritization discussions and decisions
Evangelize the solution with Professional services and Customer Success teams

Data Engineer

at Fint Solutions

1 recruiter

Posted by Thallada Naveen

Hyderabad

7 - 10 yrs

₹10L - ₹20L / yr

Data engineering

Data engineer

SSIS

Azure data factory

Data migration

+2 more

Job Locations: Hyderabad, Telangana, India
Required Experience: 5 - 7 Years
Skills : ADF, Azure, SSIS, python
Job Description
Azure Data Engineer with hands on SSIS migrations and ADF expertise.

Roles & Responsibilities
•Overall, 6+ years’ experience in Cloud Data Engineering, with hands on experience in ADF (Azure Data Factory) is required.
Hands on experience with SSIS to ADF migration is preferred.
SQL Server Integration Services (SSIS) workloads to SSIS in ADF. ( Must have done at least one migration)
Hands on experience implementing Azure Data Factory frameworks, scheduling, and performance tuning.
Hands on experience in migrating SSIS solutions to ADF
Hands on experience in ADF coding side.
Hands on experience with MPP Database architecture
Hands on experience in python

Hadoop Developer

at Object Technology Solutions Inc. (OTSI)

1 recruiter

Posted by Uma Nageswara Rao Guggilam

NCR (Delhi | Gurgaon | Noida)

3 - 5 yrs

₹2L - ₹8L / yr

Scala

Big Data

Spark

Hadoop

Dear Candidate,,

Greetings of the day!

As discussed, Please find the below job description.

Job Title : Hadoop developer

Experience : 3+ years

Job Location : New Delhi

Job type : Permanent

Knowledge and Skills Required:

Brief Skills:

Hadoop, Spark, Scala and Spark SQL

Main Skills:

Strong experience in Hadoop development
Experience in Spark
Experience in Scala
Experience in Spark SQL

Why OTSi!

Working with OTSi gives you the assurance of a successful, fast-paced career.

Exposure to infinite opportunities to learn and grow, familiarization with cutting-edge technologies, cross-domain experience and a harmonious environment are some of the prime attractions for a career-driven workforce.