Cutshort logo
For Employers
DataMetica logo
Bigdata Lead Architecture
Bigdata Lead Architecture

Bigdata Lead Architecture at DataMetica · Pune, Hyderabad · 7 - 12 years · ₹12L - ₹33L / yr · Profitable · Posted 25 Aug 2021

DataMetica's logo

Bigdata Lead Architecture

Nikita Aher's profile picture
Posted by Nikita Aher
7 - 12 yrs
₹12L - ₹33L / yr
Pune, Hyderabad
Skills
Big Data
Hadoop
Spark
Apache Spark
Apache Hive
PySpark
skill iconPython
Data engineering

Job description

Role : Lead Architecture (Spark, Scala, Big Data/Hadoop, Java)

Primary Location : India-Pune, Hyderabad

Experience : 7 - 12 Years

Management Level: 7

Joining Time: Immediate Joiners are preferred


  • Attend requirements gathering workshops, estimation discussions, design meetings and status review meetings
  • Experience of Solution Design and Solution Architecture for the data engineer model to build and implement Big Data Projects on-premises and on cloud.
  • Align architecture with business requirements and stabilizing the developed solution
  • Ability to build prototypes to demonstrate the technical feasibility of your vision
  • Professional experience facilitating and leading solution design, architecture and delivery planning activities for data intensive and high throughput platforms and applications
  • To be able to benchmark systems, analyses system bottlenecks and propose solutions to eliminate them
  • Able to help programmers and project managers in the design, planning and governance of implementing projects of any kind.
  • Develop, construct, test and maintain architectures and run Sprints for development and rollout of functionalities
  • Data Analysis, Code development experience, ideally in Big Data Spark, Hive, Hadoop, Java, Python, PySpark,
  • Execute projects of various types i.e. Design, development, Implementation and migration of functional analytics Models/Business logic across architecture approaches
  • Work closely with Business Analysts to understand the core business problems and deliver efficient IT solutions of the product
  • Deployment sophisticated analytics program of code using any of cloud application.


Perks and Benefits we Provide!


  • Working with Highly Technical and Passionate, mission-driven people
  • Subsidized Meals & Snacks
  • Flexible Schedule
  • Approachable leadership
  • Access to various learning tools and programs
  • Pet Friendly
  • Certification Reimbursement Policy
  • Check out more about us on our website below!

www.datametica.com

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About DataMetica

Founded :
2013
Type :
Services
Size :
100-1000
Stage :
Profitable

About

As a global leader in Data Warehouse Migration, Data Modernization, and Data Analytics, we empower businesses through automation and help you attain excellence. Our Belief is to Empowering companies to master their businesses and helping them achieve their full potential, we nurture clients with our innovative frameworks. Our embedded values help us strengthen the bond with our clients, ensuring growth for all. Datametica is a preferred partner with leading cloud vendors. We offer solutions related to migration from current Enterprise Data Warehouses to the Cloud determining which of these is best suited to your needs. We are giving Data Wings.
Read more

Company video

DataMetica's video section
DataMetica's video section

Photos

Company featured pictures
Company featured pictures
Company featured pictures
Company featured pictures
Company featured pictures
Company featured pictures
Company featured pictures
Company featured pictures

Connect with the team

Profile picture
Sumangali Desai
Profile picture
Shivani Mahale
Profile picture
Nitish Saxena
Profile picture
Nikita Aher
Profile picture
Pooja Gaikwad
Profile picture
Sayali Kachi
Profile picture
syed raza

Company social profiles

bloglinkedintwitterfacebook

Similar jobs (10)

Top MNC
Actively reviewing applications
Top MNC
Agency job
via VY SYSTEMS PRIVATE LIMITED by sandhiya lakshmi
Remote only
9 - 15 yrs
₹22L - ₹28L / yr
Data Engineer
Genie
Azure
skill iconAmazon Web Services (AWS)
SQL
+1 more

Job Summary

We are looking for an experienced Databricks Lead to lead and manage data engineering initiatives. The ideal candidate should have strong hands-on experience with Databricks, data engineering, and cloud-based data platforms, along with the ability to guide a team, drive technical decisions, and ensure successful project delivery.

Key Responsibilities

  • Lead and manage Databricks-based data engineering projects.
  • Design, develop, and implement scalable data pipelines using Databricks.
  • Provide technical guidance and mentorship to team members.
  • Work on data ingestion, transformation, processing, and integration.
  • Develop and optimize ETL/ELT workflows using PySpark and Spark SQL.
  • Handle performance tuning and optimization of Databricks workloads.
  • Collaborate with business stakeholders, architects, and cross-functional teams.
  • Identify and resolve technical issues and ensure smooth project delivery.
  • Follow best practices for data quality, security, and governance.
  • Participate in technical discussions, code reviews, and solution design.

Mandatory Skills

  • Strong hands-on experience with Databricks.
  • Good knowledge of Apache Spark, PySpark, and Spark SQL.
  • Experience in data engineering and ETL/ELT pipelines.
  • Strong programming knowledge in Python and SQL.
  • Experience with cloud platforms such as AWS, Azure, or GCP.
  • Understanding of data lake and lakehouse architecture.
  • Experience in team leadership and technical coordination.
  • Strong problem-solving and communication skills.


Read more
VY SYSTEMS PRIVATE LIMITED
Actively reviewing applications
Revathi R
Posted by Revathi R
Mumbai, Pune, Bengaluru (Bangalore)
6 - 12 yrs
₹5L - ₹25L / yr
skill iconJava
Apache Spark
Snowflake
SQL
Snow flake schema
+2 more

🚀 WE ARE HIRING | JAVA + APACHE SPARK DEVELOPER 🚀


📌 Job Details:

  • Position: Java + Apache Spark Developer
  • Experience: 6+ Years
  • Locations: Mumbai / Bangalore
  • Work Mode: Hybrid (3 days onsite per week)


💻 Required Technical Skills:

  • Java
  • Apache Spark
  • Snowflake
  • Spark SQL
  • Databricks
  • SQL
  • ETL


Interested candidates can share their updated resumes:

Email: revathi.r@ vysystems.com

WhatsApp: 6 3 8 5 0 7 6 1 4 2

Read more
NeoGenCode Technologies Pvt Ltd
Actively reviewing applications
Akshay Patil
Posted by Akshay Patil
Bengaluru (Bangalore)
14 - 25 yrs
₹50L - ₹70L / yr
Data engineering
databricks
Apache Spark
PySpark
skill iconPython
+19 more

Job Title : Senior Data Engineer – Databricks

Experience : 14 to 20 Years

Location : HSR Layout, Bangalore

Work Mode : Hybrid – 3 Days WFO

Shift : 11:30 AM – 07:30 PM IST

Positions : 2

Notice Period : Immediate Joiners Only

Interview : 1 Technical Round + 2 Client Rounds


Role Overview :

We are looking for a Senior Data Engineer to build and lead enterprise-scale data platforms for a Switzerland-based commodity client.

The role requires a strong hands-on Data Engineering professional with expertise in Databricks, PySpark, Python, SQL, and AWS, along with technical leadership and stakeholder management experience.


Must-Have Skills :

  • 14 to 20 years of Data Engineering experience
  • Databricks & Apache Spark / PySpark
  • Python & SQL
  • AWS Cloud
  • Lakehouse Architecture
  • ETL / ELT & Distributed Data Processing
  • Batch & Streaming Pipelines
  • Data Pipeline Optimization & Data Modeling
  • CDC & Incremental Processing
  • Git, CI/CD & Testing
  • Data Quality, Monitoring & Observability
  • Technical Leadership & Stakeholder Management


Key Responsibilities :

  • Design and build scalable data pipelines using Databricks, PySpark, Python, SQL, and AWS.
  • Own data products from design through production.
  • Develop batch / streaming pipelines and reusable ETL / ELT frameworks.
  • Optimize pipelines for performance, scalability, reliability, and cost.
  • Design scalable data architectures and data models.
  • Implement data quality, monitoring, lineage, and CI/CD practices.
  • Lead technical discussions and mentor engineering teams.
  • Collaborate with business stakeholders, architects, product owners, and engineering teams.
  • Remain hands-on while providing technical leadership.


Ideal Candidate :

A 14 to 20 years experienced, hands-on Data Engineering leader with strong Databricks + PySpark + AWS expertise, excellent communication, stakeholder management, and experience delivering enterprise-scale data platforms.

🔴 Super Urgent : Only Bangalore-based immediate joiners.

Read more
IT services and consulting organization,
IT services and consulting organization,
Agency job
via Unique Occupational by Mantasha Naaz
Bengaluru (Bangalore)
5 - 8 yrs
₹16L - ₹18L / yr
Microsoft Fabric, Data Engineer, Azure Databricks

Job Description:

We are seeking a skilled Senior Data Engineer with expertise in Databricks to join our dynamic data team. The ideal candidate will design, build, and maintain scalable data pipelines and architectures to support our organization's data-driven initiatives and leverage Databricks to process large-scale datasets, optimize data workflows, enable advanced analytics and machine learning, and integrate Power BI for data visualization and reporting.

Key Responsibilities

·        5-8 years of professional work experience in a relevant field

·        Proficient in Microsoft Fabric platform, Azure Databricks, ADF, Delta Lake, SQL Data Warehouse, Unity Catalog.

·        Good Experience on Microsoft Dynamics 365

·        Experience/ prior knowledge on semi structure data and Structured Streaming, Azure synapse, data lake, data warehouse.

·        Proficient in creating Azure Data Factory pipelines for ETL/ELT processing; copy activity, custom Azure development etc.

·        Good knowledge of SQL and Python for data manipulation, transformation, and analysis

·        Understand business requirements to set functional specifications for reporting applications

  • Data Pipeline Development: Design, develop, and maintain robust, scalable data pipelines using Databricks, Apache Spark, and other cloud-based technologies.
  • Data Integration: Ingest, transform, and integrate data from diverse sources, including APIs, databases, streaming platforms, and third-party systems, into Databricks for analytics and reporting.
  • Power BI Integration: Develop and optimize data models and datasets in Databricks for use in Power BI, ensuring efficient data connections and high-quality visualizations.
  • Performance Optimization: Optimize data workflows, API calls, and queries on Databricks and Power BI for performance, cost-efficiency, and scalability.
  • Data Modeling: Build and maintain data models to support business requirements, ensuring data quality, consistency, and accessibility for analytics and reporting.
  • Cloud Integration: Implement data solutions on cloud platforms (e.g., AWS, Azure, GCP) integrated with Databricks, Power BI, and API ecosystems.
  • Security & Compliance: Implement data governance, security, and compliance best practices within Databricks, Power BI, and API environments.
  • Technical Skills:
  • Proficiency in Databricks, including Delta Lake, Spark SQL.
  • Strong programming skills in Python, Scala, or Java.
  • Experience with Apache Spark for big data processing.
  • Knowledge of SQL for querying and transforming data.
  • Proficiency in Power BI for creating data models, DAX queries, and interactive dashboards.

Preferred Qualifications

  • Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).

Experience with real-time data processing and streaming

Read more
VY SYSTEMS PRIVATE LIMITED
Actively reviewing applications
Revathi R
Posted by Revathi R
Mumbai
5 - 10 yrs
₹5L - ₹25L / yr
Apache Spark
skill iconJava
SQL
databricks
ETL

🚀 WE ARE HIRING | JAVA + APACHE SPARK DEVELOPER

📍 Location: Mumbai

💼 Experience: 5+ Years

🏢 Work Mode: Hybrid (3 Days Work From Office)


Are you an experienced Java + Apache Spark Developer looking for your next career opportunity? We are looking for talented professionals with strong expertise in Java, Big Data technologies, and data engineering.

🔥 Required Technical Skills:

  • Strong experience in Java Development
  • Hands-on experience with Apache Spark
  • Good knowledge of Spark SQL
  • Experience with Databricks
  • Strong SQL skills
  • Experience in ETL processes and data pipelines
  • Understanding of distributed data processing
  • Good problem-solving and analytical skills


📩 Interested candidates can share their updated resumes:

📧 Email: revathi.r @vysystems.com

📱 WhatsApp: 6 3 8 5 0 7 6 1 4 2


Read more
A leading data & analytics intelligence technology solutions
Actively reviewing applications
A leading data & analytics intelligence technology solutions
Agency job
via HyrHub by Neha Koshy
Bengaluru (Bangalore), Mangalore
10 - 15 yrs
₹25L - ₹35L / yr
Data engineering
Artificial Intelligence (AI)
Generative AI
skill iconPython
Systems design

Key Skills:

Python & System Design: Strong expertise in Python and system design, with proven experience architecting scalable, production-grade systems.

LLM & Agentic AI: Proven experience building LLM-based and agentic systems from the ground up and evaluating them based on reproducibility, calibration, and cost.

Cloud & Data Architecture: Deep understanding of data and cloud architecture across AWS and Azure, including multi-store data design, distributed systems, streaming, security, and cost optimization.

Azure Databricks: Hands-on production experience with Azure Databricks architecture, including Unity Catalog, Delta Lake, cluster policy design, job orchestration, and Terraform provisioning.

Supply Chain & Inventory Management: Strong domain knowledge of supply chain and inventory operations, including demand planning, MRP logic, inventory parameters, and translating business planning decisions into data requirements.

System Architecture & Technical Decision-Making: Ability to make sound architectural decisions based on first principles, evaluate technical trade-offs, and avoid unnecessary complexity and abstraction.

LLM & Agentic AI: Proven experience building LLM-based and agentic systems from the ground up and evaluating them based on reproducibility, calibration, and cost.

Technical Leadership: Ability to influence teams without direct authority, establish technical standards, prepare technical specifications and documentation, and mentor team members.

Experience in pharmaceutical or regulated industries is highly valued.

Read more
MNC
Actively reviewing applications
MNC
Agency job
via VY SYSTEMS PRIVATE LIMITED by Akilandeswari Panneerselvam
Bengaluru (Bangalore)
4 - 12 yrs
₹6L - ₹20L / yr
skill iconJava
SCA
skill iconPython
Data engineering
databricks
+2 more


We are hiring for a Data Engineer & Java scala for a full-time opportunity in Bangalore.


Experience: 4–10 Years


Must-Have Skills:


Java / Scala / Python, Data Engineering, Databricks / Snowflake, and Cloud (AWS / Azure / GCP).


We are looking for strong software engineers with hands-on experience in data platform development, scalable backend programming, data integration, and cloud technologies.


Experience in Banking, Capital Markets, Market Risk, or Regulatory Reporting will be an added advantage.

Read more
Wissen Technology
Actively reviewing applications
at Wissen Technology
4 recruiters
Vishakha Walunj
Posted by Vishakha Walunj
Pune, Mumbai, Bengaluru (Bangalore)
4 - 9 yrs
Best in industry
PySpark
SQL
databricks
skill iconPython
Apache Airflow
+2 more

Technical Stack

- PySpark

- Databricks

- Microsoft Azure

- SQL

- Kubernetes


Key Responsibilities & Expectations

- Must be self-driven, proactive, and capable of independently delivering high-quality solutions.

- Demonstrate strong ownership and accountability for assigned tasks and deliverables.

- Ability to work with minimal supervision while consistently meeting project objectives and quality standards.

Read more
VY SYSTEMS PRIVATE LIMITED
Bengaluru (Bangalore), Hyderabad
5 - 10 yrs
₹18L - ₹28L / yr
skill iconPython
PySpark
SQL

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

9+yrs

Bangalore/Hyderabad

immediate to 15days.

Read more
Global analytics and digital solutions company
Actively reviewing applications
Global analytics and digital solutions company
Agency job
via Bean HR Consulting by Sandeep Kothari
Gurugram, Pune, Noida, Bengaluru (Bangalore)
11 - 15 yrs
₹40L - ₹45L / yr
data Architect
Microsoft Windows Azure
databricks
Snow flake schema
SQL
+2 more

Role: Technical Architect – Data & Cloud

 

We are looking for an experienced Technical Architect to lead the design and implementation of modern cloud-based data platforms. The ideal candidate should have strong expertise in Azure and/or AWS, Databricks, Snowflake, modern data architecture, and large-scale data engineering.

Key Responsibilities:

  • Design enterprise-scale Data Lakehouse and Data Warehouse architectures.
  • Define data ingestion, transformation, and serving architecture.
  • Lead architecture discussions and technical governance.
  • Design scalable ETL/ELT frameworks using Databricks and Snowflake.
  • Define best practices for security, performance optimization, CI/CD, and DevOps.
  • Provide technical guidance to engineering teams during implementation.
  • Collaborate with business, product, architecture, and data governance teams.

 

Must-Have Skills:

  • Azure and/or AWS
  • Databricks
  • Snowflake
  • Data Lakehouse Architecture
  • PySpark
  • SQL
  • Data Modelling
  • ETL/ELT Architecture
  • Performance Optimization
  • CI/CD
  • Terraform / Infrastructure as Code, preferred

Preferred Skills:

  • Insurance domain experience
  • dbt
  • Unity Catalog
  • Delta Lake
  • Iceberg
  • Azure Data Factory / AWS Glue
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos