11+ TAC Jobs in Hyderabad | TAC Job openings in Hyderabad

ETL Developer – Talend

Job Duties:

• ETL Developer is responsible for Design and Development of ETL Jobs which follow standards,

best practices and are maintainable, modular and reusable.

• Proficiency with Talend or Pentaho Data Integration / Kettle.
• ETL Developer will analyze and review complex object and data models and the metadata

repository in order to structure the processes and data for better management and efficient

access.

• Working on multiple projects, and delegating work to Junior Analysts to deliver projects on time.
• Training and mentoring Junior Analysts and building their proficiency in the ETL process.
• Preparing mapping document to extract, transform, and load data ensuring compatibility with

all tables and requirement specifications.

• Experience in ETL system design and development with Talend / Pentaho PDI is essential.
• Create quality rules in Talend.
• Tune Talend / Pentaho jobs for performance optimization.
• Write relational(sql) and multidimensional(mdx) database queries.
• Functional Knowledge of Talend Administration Center/ Pentaho data integrator, Job Servers &

Load balancing setup, and all its administrative functions.

• Develop, maintain, and enhance unit test suites to verify the accuracy of ETL processes,

dimensional data, OLAP cubes and various forms of BI content including reports, dashboards,

and analytical models.

• Exposure in Map Reduce components of Talend / Pentaho PDI.
• Comprehensive understanding and working knowledge in Data Warehouse loading, tuning, and

maintenance.

• Working knowledge of relational database theory and dimensional database models.
• Creating and deploying Talend / Pentaho custom components is an add-on advantage.
• Nice to have java knowledge.

Skills and Qualification:

• BE, B.Tech / MS Degree in Computer Science, Engineering or a related subject.
• Having an experience of 3+ years.
• Proficiency with Talend or Pentaho Data Integration / Kettle.
• Ability to work independently.
• Ability to handle a team.
• Good written and oral communication skills.

Technical Skills:

• Ability to understand and translate business requirements into design.
• Proficient in AWS infrastructure components such as S3, IAM, VPC, EC2, and Redshift.
• Experience in creating ETL jobs using Python/PySpark.
• Proficiency in creating AWS Lambda functions for event-based jobs.
• Knowledge of automating ETL processes using AWS Step Functions.
• Competence in building data warehouses and loading data into them.

Responsibilities:

• Understand business requirements and translate them into design.
• Assess AWS infrastructure needs for development work.
• Develop ETL jobs using Python/PySpark to meet requirements.
• Implement AWS Lambda for event-based tasks.
• Automate ETL processes using AWS Step Functions.
• Build data warehouses and manage data loading.
• Engage with customers and stakeholders to articulate the benefits of proposed solutions and frameworks.

Job Description

Mandatory Requirements 

• Experience in AWS Glue

• Experience in Apache Parquet 

• Proficient in AWS S3 and data lake 

• Knowledge of Snowflake

• Understanding of file-based ingestion best practices.

• Scripting language - Python & pyspark

CORE RESPONSIBILITIES

• Create and manage cloud resources in AWS 

• Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies 

• Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform 

• Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations 

• Develop an infrastructure to collect, transform, combine and publish/distribute customer data.

• Define process improvement opportunities to optimize data collection, insights and displays.

• Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible 

• Identify and interpret trends and patterns from complex data sets 

• Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders. 

• Key participant in regular Scrum ceremonies with the agile teams  

• Proficient at developing queries, writing reports and presenting findings 

• Mentor junior members and bring best industry practices.

QUALIFICATIONS

• 5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales) 

• Strong background in math, statistics, computer science, data science or related discipline

• Advanced knowledge one of language: Java, Scala, Python, C# 

• Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake  

• Proficient with

• Data mining/programming tools (e.g. SAS, SQL, R, Python)

• Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)

• Data visualization (e.g. Tableau, Looker, MicroStrategy)

• Comfortable learning about and deploying new technologies and tools. 

• Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines. 

• Good written and oral communication skills and ability to present results to non-technical audiences 

• Knowledge of business intelligence and analytical tools, technologies and techniques.

Familiarity and experience in the following is a plus: 

• AWS certification

• Spark Streaming 

• Kafka Streaming / Kafka Connect 

• ELK Stack 

• Cassandra / MongoDB 

• CI/CD: Jenkins, GitLab, Jira, Confluence other related tools

Roles and Responsibilities

Job Title: Data Engineer

Job Summary: As a Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure and tools necessary for data collection, storage, processing, and analysis. You will work closely with data scientists and analysts to ensure that data is available, accessible, and in a format that can be easily consumed for business insights.

Responsibilities:

• Design, build, and maintain data pipelines to collect, store, and process data from various sources.
• Create and manage data warehousing and data lake solutions.
• Develop and maintain data processing and data integration tools.
• Collaborate with data scientists and analysts to design and implement data models and algorithms for data analysis.
• Optimize and scale existing data infrastructure to ensure it meets the needs of the business.
• Ensure data quality and integrity across all data sources.
• Develop and implement best practices for data governance, security, and privacy.
• Monitor data pipeline performance / Errors and troubleshoot issues as needed.
• Stay up-to-date with emerging data technologies and best practices.

Requirements:

Bachelor's degree in Computer Science, Information Systems, or a related field.

Experience with ETL tools like Matillion,SSIS,Informatica

Experience with SQL and relational databases such as SQL server, MySQL, PostgreSQL, or Oracle.

Experience in writing complex SQL queries

Strong programming skills in languages such as Python, Java, or Scala.

Experience with data modeling, data warehousing, and data integration.

Strong problem-solving skills and ability to work independently.

Excellent communication and collaboration skills.

Familiarity with big data technologies such as Hadoop, Spark, or Kafka.

Familiarity with data warehouse/Data lake technologies like Snowflake or Databricks

Familiarity with cloud computing platforms such as AWS, Azure, or GCP.

Familiarity with Reporting tools

Teamwork/ growth contribution

• Helping the team in taking the Interviews and identifying right candidates
• Adhering to timelines
• Intime status communication and upfront communication of any risks
• Tech, train, share knowledge with peers.
• Good Communication skills
• Proven abilities to take initiative and be innovative
• Analytical mind with a problem-solving aptitude

Good to have :

Master's degree in Computer Science, Information Systems, or a related field.

Experience with NoSQL databases such as MongoDB or Cassandra.

Familiarity with data visualization and business intelligence tools such as Tableau or Power BI.

Knowledge of machine learning and statistical modeling techniques.

If you are passionate about data and want to work with a dynamic team of data scientists and analysts, we encourage you to apply for this position.

Location:  Pune/Nagpur,Goa,Hyderabad/Bangalore

Job Requirements:

• 9 years and above of total experience preferably in bigdata space.
• Creating spark applications using Scala to process data.
• Experience in scheduling and troubleshooting/debugging Spark jobs in steps.
• Experience in spark job performance tuning and optimizations.
• Should have experience in processing data using Kafka/Pyhton.
• Individual should have experience and understanding in configuring Kafka topics to optimize the performance.
• Should be proficient in writing SQL queries to process data in Data Warehouse.
• Hands on experience in working with Linux commands to troubleshoot/debug issues and creating shell scripts to automate tasks.
• Experience on AWS services like EMR.

We are looking for a Senior Data Engineer to join the Customer Innovation team, who will be responsible for acquiring, transforming, and integrating customer data onto our Data Activation Platform from customers’ clinical, claims, and other data sources. You will work closely with customers to build data and analytics solutions to support their business needs, and be the engine that powers the partnership that we build with them by delivering high-fidelity data assets.

In this role, you will work closely with our Product Managers, Data Scientists, and Software Engineers to build the solution architecture that will support customer objectives. You'll work with some of the brightest minds in the industry, work with one of the richest healthcare data sets in the world, use cutting-edge technology, and see your efforts affect products and people on a regular basis. The ideal candidate is someone that

• Has healthcare experience and is passionate about helping heal people,
• Loves working with data,
• Has an obsessive focus on data quality,
• Is comfortable with ambiguity and making decisions based on available data and reasonable assumptions,
• Has strong data interrogation and analysis skills,
• Defaults to written communication and delivers clean documentation, and,
• Enjoys working with customers and problem solving for them.

A day in the life at Innovaccer:

• Define the end-to-end solution architecture for projects by mapping customers’ business and technical requirements against the suite of Innovaccer products and Solutions.
• Measure and communicate impact to our customers.
• Enabling customers on how to activate data themselves using SQL, BI tools, or APIs to solve questions they have at speed.

What You Need:

• 4+ years of experience in a Data Engineering role, a Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field.
• 4+ years of experience working with relational databases like Snowflake, Redshift, or Postgres.
• Intermediate to advanced level SQL programming skills.
• Data Analytics and Visualization (using tools like PowerBI)
• The ability to engage with both the business and technical teams of a client - to document and explain technical problems or concepts in a clear and concise way.
• Ability to work in a fast-paced and agile environment.
• Easily adapt and learn new things whether it’s a new library, framework, process, or visual design concept.

What we offer:

• Industry certifications: We want you to be a subject matter expert in what you do. So, whether it’s our product or our domain, we’ll help you dive in and get certified.
• Quarterly rewards and recognition programs: We foster learning and encourage people to take risks. We recognize and reward your hard work.
• Health benefits: We cover health insurance for you and your loved ones.
• Sabbatical policy: We encourage people to take time off and rejuvenate, learn new skills, and pursue their interests so they can generate new ideas with Innovaccer.
• Pet-friendly office and open floor plan: No boring cubicles.

Skills- Informatica with Big Data Management

1.Minimum 6 to 8 years of experience in informatica BDM development
2.Experience working on Spark/SQL
3.Develops informtica mapping/Sql 

• Expertise in designing and implementing enterprise scale database (OLTP) and Data warehouse solutions.
• Hands on experience in implementing Azure SQL Database, Azure SQL Date warehouse (Azure Synapse Analytics) and big data processing using Azure Databricks and Azure HD Insight.
• Expert in writing T-SQL programming for complex stored procedures, functions, views and query optimization.
• Should be aware of Database development for both on-premise and SAAS Applications using SQL Server and PostgreSQL.
• Experience in ETL and ELT implementations using Azure Data Factory V2 and SSIS.
• Experience and expertise in building machine learning models using Logistic and linear regression, Decision tree  and Random forest Algorithms.
• PolyBase queries for exporting and importing data into Azure Data Lake.
• Building data models both tabular and multidimensional using SQL Server data tools.
• Writing data preparation, cleaning and processing steps using Python, SCALA, and R.
• Programming experience using python libraries NumPy, Pandas and Matplotlib.
• Implementing NOSQL databases and writing queries using cypher.
• Designing end user visualizations using Power BI, QlikView and Tableau.
• Experience working with all versions of SQL Server 2005/2008/2008R2/2012/2014/2016/2017/2019
• Experience using the expression languages MDX and DAX.
• Experience in migrating on-premise SQL server database to Microsoft Azure.
• Hands on experience in using Azure blob storage, Azure Data Lake Storage Gen1 and Azure Data Lake Storage Gen2.
• Performance tuning complex SQL queries, hands on experience using SQL Extended events.
• Data modeling using Power BI for Adhoc reporting.
• Raw data load automation using T-SQL and SSIS
• Expert in migrating existing on-premise database to SQL Azure.
• Experience in using U-SQL for Azure Data Lake Analytics.
• Hands on experience in generating SSRS reports using MDX.
• Experience in designing predictive models using Python and SQL Server.
• Developing machine learning models using Azure Databricks and SQL Server

• Total Experience of 7-10 years and should be interested in teaching and research
• 3+ years’ experience in data engineering which includes data ingestion, preparation, provisioning, automated testing, and quality checks.
• 3+ Hands-on experience in Big Data cloud platforms like AWS and GCP, Data Lakes and Data Warehouses
• 3+ years of Big Data and Analytics Technologies. Experience in SQL, writing code in spark engine using python, scala or java Language. Experience in Spark, Scala
• Experience in designing, building, and maintaining ETL systems
• Experience in data pipeline and workflow management tools like Airflow
• Application Development background along with knowledge of Analytics libraries, opensource Natural Language Processing, statistical and big data computing libraries
• Familiarity with Visualization and Reporting Tools like Tableau, Kibana.
• Should be good at storytelling in Technology