Customer Success / Analytics Team

Job Title: Hadoop/Big Data Developer (PySpark/Scala)

Location: Chennai, Tamil Nadu, India

Company: Tata Consultancy Services (TCS)

Experience: 2-5 Years

Job Description:

Tata Consultancy Services (TCS) is seeking a talented and passionate Hadoop/Big Data Developer

with expertise in PySpark and Scala to join our growing team in Chennai. In this role, you will be

responsible for developing, implementing, and maintaining robust and scalable big data solutions for

our clients. You will leverage your skills in Hadoop, Spark, PySpark, and Scala to process and analyze

large datasets, contributing to data-driven insights.

Responsibilities:

 Design, develop, and maintain efficient and scalable big data pipelines using Hadoop, Spark,

PySpark, and Scala.

 Develop and implement data ingestion, processing, and transformation processes for large

datasets.

 Write optimized and performant code for data processing and analysis.

 Develop and maintain data models and schemas for big data platforms.

 Implement data quality checks and monitoring to ensure data accuracy and consistency.

 Collaborate with data scientists and analysts to understand data requirements and deliver

solutions.  

 Troubleshoot and resolve performance and data-related issues.

 Develop and maintain documentation for data pipelines and solutions.

 Stay up-to-date with the latest big data technologies and trends.

 Participate in code reviews and contribute to best practices.

 Work with cloud-based big data platforms (e.g., AWS EMR, Azure HDInsight, Google Cloud

Dataproc) is a plus.

Required Skills and Experience:

 Bachelor’s degree in Computer Science, Information Technology, or a related field.

 2-5 years of hands-on experience in big data development.

 Strong understanding of Hadoop ecosystem (HDFS, MapReduce, YARN).

 Proficiency in Apache Spark and experience with PySpark and Scala.

 Experience with data processing and transformation using Spark.

 Strong understanding of data warehousing and ETL concepts.

 Proficiency in SQL and experience with NoSQL databases.

 Experience with version control systems (e.g., Git).

 Strong problem-solving and analytical skills.

 Excellent communication and teamwork skills.  

Preferred Skills:

 Experience with data streaming technologies (e.g., Kafka, Flume).  

 Knowledge of cloud-based big data platforms (AWS, Azure, GCP).

 Experience with data visualization tools (e.g., Tableau, Power BI).

 Knowledge of machine learning concepts and algorithms.

 Experience with containerization technologies (e.g., Docker, Kubernetes).

 Experience with CI/CD pipelines.

About TCS:

Tata Consultancy Services is a global leader in IT services, consulting, and business solutions with a

strong presence in Chennai. We are committed to providing our employees with a challenging and

rewarding work environment, and opportunities for professional growth.

Experience:

Should have a minimum of 10-12 years of Experience.

Should have experience on Product Development/Maintenance/Production Support experience in a support organization

Should have a good understanding of services business for fortune 1000 from the operations point of view

Ability to read, understand and communicate complex technical information

Ability to express ideas in an organized, articulate and concise manner

Ability to face stressful situation with positive attitude

Any certification in regards to support services will be an added advantage

Education: BE, B- Tech (CS), MCA

Location: India

Primary Skills:

Hands on experience with OpenStack framework. Ability to set up private cloud using OpenStack environment. Awareness to various OpenStack services and modules

Strong experience with OpenStack services like Neutron, Cinder, Keystone, etc.

Proficiency in programming languages such as Python, Ruby, or Go.

Strong knowledge of Linux systems administration and networking.

Familiarity with virtualization technologies like KVM or VMware.

Experience with configuration management and IaC tools like Ansible, Terraform.

Subject matter expertise in OpenStack security

Solid experience with Linux and shell scripting

Sound knowledge of cloud computing concepts & technologies, such as docker, Kubernetes, AWS, GCP, Azure etc.

Ability to configure OpenStack environment for optimum resources

Good knowledge of security, operations in open stack environment

Strong knowledge of Linux internals, networking, storage, security

Strong knowledge of VMware Enterprise products (ESX, vCenter)

Hands on experience with HEAT orchestration

Experience with CI/CD, monitoring, operational aspects

Strong experience working with Rest API's, JSON

Exposure to Big data technologies ( Messaging queues, Hadoop/MPP, NoSQL databases)

Hands on experience with open source monitoring tools like Grafana/Prometheus/Nagios/Ganglia/Zabbix etc.

Strong verbal and written communication skills are mandatory

Excellent analytical and problem solving skills are mandatory

Role & Responsibilities

Advise customers and colleagues on cloud and virtualization topics

Work with the architecture team on cloud design projects using openstack

Collaborate with product, customer success, and presales on customer projects

Participate in onsite assessments and workshops when requested 

Provide subject matter expertise and mentor colleagues

Set up open stack environments for projects

Design, deploy, and maintain OpenStack infrastructure.

Collaborate with cross-functional chapters to integrate OpenStack with other services (k8s, DBaaS)

Develop automation scripts and tools to streamline OpenStack operations.

Troubleshoot and resolve issues related to OpenStack services.

Monitor and optimize the performance and scalability of OpenStack components.

Stay updated with the latest OpenStack releases and contribute to the OpenStack community.

Work closely with Architects and Product Management to understand requirement

should be capable of working independently & responsible for end-to-end implementation

Should work with complete ownership and handle all issues without missing SLA's

Work closely with engineering team and support team

Should be able to debug the issues and report appropriately in the ticketing system

Contribute to improve the efficiency of the assignment by quality improvements & innovative suggestions

Should be able to debug/create scripts for automation

Should be able to configure monitoring utilities & set up alerts

Should be hands on in setting up OS, applications, databases and have passion to learn new technologies

Should be able to scan logs, errors, exception and get to the root cause of the issue

Contribute in developing a knowledge base on collaboration with other team members

Maintain customer loyalty through Integrity and accountability

Groom and mentor team members on project technologies and work

Responsibilities

• Manage and drive a team of Data Analysts and Sr. Data Analysts to provide logistics and supply chain solutions.
• Conduct meetings with Clients to gather the requirements and understand the scope.
• Conduct meetings with internal stake holders to walk them through the solution and handover the analysis.
• Define business problems, identify solutions, provide analysis and insights from the client's data.
• 5 Conduct scheduled progress reviews on all projects and interact with onsite team daily.
• Ensure solutions are delivered error free and submitted on time.
• Implement ETL processes using Pentaho Data Integration (Pentaho ETL)Design and implement data models in Hadoop.
• Provide end-user training and technical assistance to maximize utilization of tools.
• Deliver technical guidance to team, including hands-on development as necessary; oversee standards, change controls and documentation library for training and reuse.

Requirements

• Bachelor's degree in Engineering.
• 16+ years of experience in Supply Chain and logistics or related industry and Analytics experience.
• 3 years of experience in team handling(8+People) and interacting with the executive leadership teams.
• Strong project and time management skills with ability to multitask and prioritize workload.
• Solid expertise with MS Excel, SQL, any visualization tools like Tableau/Power BI, any ETL tools.
• Proficiency in Hadoop / Hive.
• Experience Pentaho ETL, Pentaho Visualization API, Tableau.
• Hands on experience of working with Big data sets (Data sets with millions of records).
• Strong technical and Management experience.

Desired Skills and Experience

• NET,ASP.NET