Big Data Engineer at Virtusa · Chennai · 11 - 15 years · ₹15L - ₹33L / yr · Profitable · Posted 14 Feb 2023

We are looking for a Big Data Engineer with java for Chennai Location
Location : Chennai
Exp : 11 to 15 Years
Job description
Required Skill:
1. Candidate should have minimum 7 years of experience as total
2. Candidate should have minimum 4 years of experience in Big Data design and development
3. Candidate should have experience in Java, Spark, Hive & Hadoop, Python
4. Candidate should have experience in any RDBMS.
Roles & Responsibility:
1. To create work plans, monitor and track the work schedule for on time delivery as per the defined quality standards.
2. To develop and guide the team members in enhancing their technical capabilities and increasing productivity.
3. To ensure process improvement and compliance in the assigned module, and participate in technical discussions or review.
4. To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalation
Regards,
Priyanka S
7P8R9I9Y4A0N8K8A7S7

Similar jobs (10)
Job Summary
We are looking for an experienced Databricks Lead to lead and manage data engineering initiatives. The ideal candidate should have strong hands-on experience with Databricks, data engineering, and cloud-based data platforms, along with the ability to guide a team, drive technical decisions, and ensure successful project delivery.
Key Responsibilities
- Lead and manage Databricks-based data engineering projects.
- Design, develop, and implement scalable data pipelines using Databricks.
- Provide technical guidance and mentorship to team members.
- Work on data ingestion, transformation, processing, and integration.
- Develop and optimize ETL/ELT workflows using PySpark and Spark SQL.
- Handle performance tuning and optimization of Databricks workloads.
- Collaborate with business stakeholders, architects, and cross-functional teams.
- Identify and resolve technical issues and ensure smooth project delivery.
- Follow best practices for data quality, security, and governance.
- Participate in technical discussions, code reviews, and solution design.
Mandatory Skills
- Strong hands-on experience with Databricks.
- Good knowledge of Apache Spark, PySpark, and Spark SQL.
- Experience in data engineering and ETL/ELT pipelines.
- Strong programming knowledge in Python and SQL.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Understanding of data lake and lakehouse architecture.
- Experience in team leadership and technical coordination.
- Strong problem-solving and communication skills.
We are hiring experienced Data Engineers for a permanent opportunity in Bangalore.
Job Description
Role: Data Engineer
Location: Bangalore
Experience: 6+ Years
Key Responsibilities:
- Design, develop, and maintain scalable data processing pipelines using Amazon EMR.
- Develop data engineering solutions using Python and Scala.
- Process and transform large volumes of structured and unstructured data.
- Build and optimize distributed data processing workflows for performance and scalability.
- Work with big data technologies and cloud-based data processing environments.
- Perform data integration, validation, transformation, and quality checks.
- Troubleshoot data pipeline issues and optimize processing performance.
- Collaborate with data architects, analysts, and development teams to deliver reliable data solutions.
Required Skills:
- Strong hands-on experience with Amazon EMR.
- Proficiency in Python and Scala.
- Experience in big data processing and data pipeline development.
- Good understanding of distributed computing and data transformation techniques.
- Experience in performance tuning and troubleshooting data processing jobs.
- Strong analytical, problem-solving, and communication skills.

Job Title : Senior Data Engineer – Databricks
Experience : 14 to 20 Years
Location : HSR Layout, Bangalore
Work Mode : Hybrid – 3 Days WFO
Shift : 11:30 AM – 07:30 PM IST
Positions : 2
Notice Period : Immediate Joiners Only
Interview : 1 Technical Round + 2 Client Rounds
Role Overview :
We are looking for a Senior Data Engineer to build and lead enterprise-scale data platforms for a Switzerland-based commodity client.
The role requires a strong hands-on Data Engineering professional with expertise in Databricks, PySpark, Python, SQL, and AWS, along with technical leadership and stakeholder management experience.
Must-Have Skills :
- 14 to 20 years of Data Engineering experience
- Databricks & Apache Spark / PySpark
- Python & SQL
- AWS Cloud
- Lakehouse Architecture
- ETL / ELT & Distributed Data Processing
- Batch & Streaming Pipelines
- Data Pipeline Optimization & Data Modeling
- CDC & Incremental Processing
- Git, CI/CD & Testing
- Data Quality, Monitoring & Observability
- Technical Leadership & Stakeholder Management
Key Responsibilities :
- Design and build scalable data pipelines using Databricks, PySpark, Python, SQL, and AWS.
- Own data products from design through production.
- Develop batch / streaming pipelines and reusable ETL / ELT frameworks.
- Optimize pipelines for performance, scalability, reliability, and cost.
- Design scalable data architectures and data models.
- Implement data quality, monitoring, lineage, and CI/CD practices.
- Lead technical discussions and mentor engineering teams.
- Collaborate with business stakeholders, architects, product owners, and engineering teams.
- Remain hands-on while providing technical leadership.
Ideal Candidate :
A 14 to 20 years experienced, hands-on Data Engineering leader with strong Databricks + PySpark + AWS expertise, excellent communication, stakeholder management, and experience delivering enterprise-scale data platforms.
🔴 Super Urgent : Only Bangalore-based immediate joiners.

🚀 WE ARE HIRING | JAVA + APACHE SPARK DEVELOPER
📍 Location: Mumbai
💼 Experience: 5+ Years
🏢 Work Mode: Hybrid (3 Days Work From Office)
Are you an experienced Java + Apache Spark Developer looking for your next career opportunity? We are looking for talented professionals with strong expertise in Java, Big Data technologies, and data engineering.
🔥 Required Technical Skills:
- Strong experience in Java Development
- Hands-on experience with Apache Spark
- Good knowledge of Spark SQL
- Experience with Databricks
- Strong SQL skills
- Experience in ETL processes and data pipelines
- Understanding of distributed data processing
- Good problem-solving and analytical skills
📩 Interested candidates can share their updated resumes:
📧 Email: revathi.r @vysystems.com
📱 WhatsApp: 6 3 8 5 0 7 6 1 4 2
Dear Candidate,
Thanks for showing interest in the opportunity!!!
As discussed, we have an opening for ETL Developer – Hadoop & PySpark with Mphasis for the Bangalore location – Permanent.
Job Description:
We are looking for an experienced ETL Developer with strong expertise in ETL development, Hadoop, and PySpark. The candidate should have hands-on experience in data processing, building and maintaining ETL pipelines, data transformation, and handling large datasets using Big Data technologies.
Primary & Mandatory Skills:
- ETL Development
- Hadoop / HDFS
- PySpark / Apache Spark
- Data Extraction, Transformation & Loading
- SQL and Data Processing
Good to Have:
- Hive / Spark SQL
- Data Pipeline Development
- Big Data Processing
- Python Programming
Company: Mphasis
Role: ETL Developer – Hadoop & PySpark
Experience: As per requirement
Location: Bangalore
Employment Type: Permanent
Technical Stack
- PySpark
- Databricks
- Microsoft Azure
- SQL
- Kubernetes
Key Responsibilities & Expectations
- Must be self-driven, proactive, and capable of independently delivering high-quality solutions.
- Demonstrate strong ownership and accountability for assigned tasks and deliverables.
- Ability to work with minimal supervision while consistently meeting project objectives and quality standards.
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
9+yrs
Bangalore/Hyderabad
immediate to 15days.
Job Description
We are looking for an experienced ETL Developer with strong expertise in ETL development, Hadoop, PySpark, Python, and SQL. The candidate should have hands-on experience in developing data pipelines, processing and transforming large datasets, and working with data warehousing and big data technologies.
Key Responsibilities
- Design, develop, and maintain ETL processes and data pipelines.
- Work with Hadoop and PySpark to process and transform large volumes of data.
- Develop and optimize Python scripts for data processing and automation.
- Write complex SQL queries for data extraction, transformation, and validation.
- Perform data cleansing, integration, transformation, and validation.
- Work with data warehousing concepts and large-scale data processing systems.
- Optimize ETL workflows for performance, scalability, and reliability.
- Troubleshoot data pipeline failures and resolve data quality issues.
- Collaborate with data engineers, analysts, and other technical teams.
- Follow data governance, quality, and development best practices.
Required Skills
- Strong experience in ETL Development – Mandatory
- Hands-on experience with Hadoop – Mandatory
- Strong experience in PySpark – Mandatory
- Good programming skills in Python – Mandatory
- Strong knowledge of SQL – Mandatory
- Experience in Data Processing and Transformation – Mandatory
- Knowledge of Data Warehousing – Mandatory
- Experience with Big Data Technologies – Mandatory
Opportunity Details
Role: ETL Developer – Hadoop & PySpark
Experience: 6–10 Years
Location: Bangalore
Mandatory Skills
ETL Development | Hadoop | PySpark | Python | SQL | Data Processing & Transformation | Data Warehousing | Big Data Technologies
Join our product development team at Planview as a Senior Software Engineer I and become a pivotal force on the Viz Core Team. This role offers the unique opportunity to shape and lead the development of data-processing pipelines and APIs that are at the heart of our software solutions. These solutions are designed to streamline and enhance the efficiency of software delivery, resonating deeply with software engineers who strive to build better and faster.
At Planview Viz, which is powered by the innovative “Flow Framework,” you will tackle complex data challenges and develop scalable solutions within an AWS cloud environment. Your efforts will be crucial in revolutionizing how businesses harness and interpret vast amounts of workflow data, transforming it into actionable insights that propel organizational efficiency and effectiveness.
Responsibilities (What you'll do)
- Create and refine powerful data-processing architectures that integrate seamlessly with a diverse array of external tools, enhancing the way software is delivered across industries.
- Drive operational excellence by critically analyzing problems, defining requirements, and devising robust solutions that push the boundaries of technology.
- Lead and inspire a team of talented engineers, promoting a culture of ownership, meticulous attention to quality, and proactive problem-solving.
- Stay at the cutting edge of technology by updating and expanding your team’s knowledge of cloud architectures, data processing, and advanced analytics, ensuring that you and your team remain leaders in technological innovation.
- Make a direct impact on the efficiency and effectiveness of software delivery worldwide through innovation and leadership.
Qualifications (What you'll bring)
Who We’re Looking For
The ideal candidate is a seasoned professional in cloud software development with a strong foundation in data processing and scalable software systems. You thrive in collaborative environments and are passionate about advancing cloud technology and data architecture to new heights. You are a leader who enjoys mentoring, guiding, and inspiring others.
Preferred Qualifications
Skills, Knowledge, and Expertise
- A degree in Computer Science, Engineering, or a related field.
- 6+ years of experience with a modern programming language, with a focus on back-end systems and cloud-based technologies.
- Strong capability in architecting and designing robust, scalable software systems.
- Proficiency in AWS or another popular cloud platform.
Additional Qualifications
- Experience with big-data technologies such as Spark and Apache Kafka.
- Experience with Java (version 17), Scala, or other JVM programming languages.
- Experience with tools such as Amazon Redshift and MongoDB.
- Experience with continuous integration and continuous deployment tools such as GitHub, Jenkins, Travis CI, or similar platforms.
- Proficiency in test-driven development.
- Experience with containers and orchestration tools such as Kubernetes.
- Experience working on remote or distributed teams and projects.

Job Title : Tech Lead – Data Lake Platform
Number of Positions : 2
Experience : 7+ Years
Role Type : Technical Lead / Data Platform Lead
Domain : Data Engineering / Data Platform / AWS
About the Role :
We are looking for an experienced Tech Lead – Data Lake Platform to lead the design, development, and operations of an enterprise-scale AWS-based Data Lake Platform.
The platform will ingest data from multiple business systems, process it through structured data layers, and serve data for analytics, reporting, APIs, and operational applications.
As a Tech Lead, you will be responsible for setting the technical direction, leading data engineering teams, driving platform reliability and performance, and owning the platform's delivery, governance, and production support end to end.
Core Tech Stack :
AWS | S3 | EMR | Glue | Athena | Redshift | DMS | Lambda | RDS | IAM | Airflow | Spark/PySpark | SQL | Data Modeling | Hasura | GraphQL | DBT | PostgreSQL/Aurora | Kafka
Key Responsibilities :
- Own the overall Data Lake Platform architecture, covering data ingestion, staging, curated layers, consumption, analytics, and API/data serving.
- Lead the design and development of production-grade data pipelines using AWS Glue, EMR/Spark, Airflow, DBT, Athena, and Redshift.
- Design scalable batch and analytics pipelines with a focus on reliability, performance, data quality, and maintainability.
- Own the API/data-serving architecture from consumption data → RDS/PostgreSQL → Hasura GraphQL → Lambda/API Gateway.
- Drive improvements in platform stability, including orchestration failures, cluster sizing, pipeline SLAs, query performance, and production reliability.
- Design and implement appropriate AWS security, access control, IAM, PII handling, and data governance practices.
- Lead technical discussions, architecture decisions, code reviews, and engineering best practices.
- Mentor and guide data engineers while ensuring high-quality and scalable engineering delivery.
- Own production support, incident management, troubleshooting, and root-cause analysis (RCA) for critical data platform issues.
- Develop and maintain runbooks, operational procedures, monitoring, and incident response practices.
- Collaborate with business, product, application, and analytics teams to onboard new datasets and support reporting, API, and data consumption requirements.
- Ensure the platform meets defined availability, performance, security, data quality, and compliance requirements.
Must-Have Skills :
- 7+ years of experience in Data Engineering, Data Platform Engineering, or related areas, including experience in technical leadership or architecture.
- Strong hands-on experience with AWS Data Services, including:
- Amazon S3
- AWS EMR
- AWS Glue
- Amazon Athena
- Amazon Redshift
- AWS DMS
- AWS Lambda
- Amazon RDS
- AWS IAM
- Strong production experience with Apache Airflow.
- Strong hands-on experience with Apache Spark / PySpark.
- Strong SQL skills and experience with data modeling, including layered data architecture, data marts, and enterprise data models.
- Experience with Hasura or a similar GraphQL layer for PostgreSQL-based data/API serving.
- Strong understanding of data lake architecture and enterprise data platforms.
- Proven experience leading engineers and driving technical decisions.
- Hands-on experience with production support, troubleshooting, incident management, and RCA.
- Strong understanding of data platform performance, scalability, reliability, and SLA management.
Good-to-Have Skills :
- Experience with DBT and modern data transformation practices.
- Experience with lakehouse table formats on Amazon S3, such as Apache Iceberg or similar technologies.
- Strong knowledge of Amazon Redshift workload optimization, including :
- Distribution keys
- Sort keys
- Spectrum
- External tables
- Experience with Kafka or other streaming/data ingestion technologies.
- Experience with PostgreSQL / Amazon Aurora as a data-serving layer.
- Experience with Lambda and API Gateway for API-based data serving.
- Experience with enterprise data governance, data quality, security, and compliance.
- Experience in BFSI / Banking / Financial Services / Insurance domain.
Ideal Candidate :
The ideal candidate is a hands-on Data Platform / Data Engineering Lead who can operate at both the architecture and implementation level. You should be comfortable designing an AWS Data Lake from end to end, leading engineers, troubleshooting production issues, and working closely with business and application teams to deliver reliable data products.







