11+ Apache Flume Jobs in Hyderabad | Apache Flume Job openings in Hyderabad

Publicis Sapient Overview:

The Senior Associate People Senior Associate L1 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution 

.

Job Summary:

As Senior Associate L1 in Data Engineering, you will do technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution

The role requires a hands-on technologist who has strong programming background like Java / Scala / Python, should have experience in Data Ingestion, Integration and data Wrangling, Computation, Analytics pipelines and exposure to Hadoop ecosystem components. Having hands-on knowledge on at least one of AWS, GCP, Azure cloud platforms will be preferable.

Role & Responsibilities:

Job Title: Senior Associate L1 – Data Engineering

Your role is focused on Design, Development and delivery of solutions involving:

• Data Ingestion, Integration and Transformation

• Data Storage and Computation Frameworks, Performance Optimizations

• Analytics & Visualizations

• Infrastructure & Cloud Computing

• Data Management Platforms

• Build functionality for data ingestion from multiple heterogeneous sources in batch & real-time

• Build functionality for data analytics, search and aggregation

Experience Guidelines:

Mandatory Experience and Competencies:

# Competency

1.Overall 3.5+ years of IT experience with 1.5+ years in Data related technologies

2.Minimum 1.5 years of experience in Big Data technologies

3.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data pipeline. Working knowledge on real-time data pipelines is added advantage.

4.Strong experience in at least of the programming language Java, Scala, Python. Java preferable

5.Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery etc

Preferred Experience and Knowledge (Good to Have):

# Competency

1.Good knowledge of traditional ETL tools (Informatica, Talend, etc) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands on experience

2.Knowledge on data governance processes (security, lineage, catalog) and tools like Collibra, Alation etc

3.Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace, search & indexing and Micro services architectures

4.Performance tuning and optimization of data pipelines

5.CI/CD – Infra provisioning on cloud, auto build & deployment pipelines, code quality

6.Working knowledge with data platform related services on at least 1 cloud platform, IAM and data security

7.Cloud data specialty and other related Big data technology certifications

Job Title: Senior Associate L1 – Data Engineering

Personal Attributes:

• Strong written and verbal communication skills

• Articulation skills

• Good team player

• Self-starter who requires minimal oversight

• Ability to prioritize and manage multiple tasks

• Process orientation and the ability to define and set up processes

Data Engineering : Senior Engineer / Manager

As Senior Engineer/ Manager in Data Engineering, you will translate client requirements into technical design, and implement components for a data engineering solutions. Utilize a deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution.

Must Have skills :

1. GCP

2. Spark streaming : Live data streaming experience is desired.

3. Any 1 coding language: Java/Pyhton /Scala

Skills & Experience :

- Overall experience of MINIMUM 5+ years with Minimum 4 years of relevant experience in Big Data technologies

- Hands-on experience with the Hadoop stack - HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data pipeline. Working knowledge on real-time data pipelines is added advantage.

- Strong experience in at least of the programming language Java, Scala, Python. Java preferable

- Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery etc.

- Well-versed and working knowledge with data platform related services on GCP

- Bachelor's degree and year of work experience of 6 to 12 years or any combination of education, training and/or experience that demonstrates the ability to perform the duties of the position

Your Impact :

- Data Ingestion, Integration and Transformation

- Data Storage and Computation Frameworks, Performance Optimizations

- Analytics & Visualizations

- Infrastructure & Cloud Computing

- Data Management Platforms

- Build functionality for data ingestion from multiple heterogeneous sources in batch & real-time

- Build functionality for data analytics, search and aggregation

Company Profile :

Merilytics, an Accordion company is a fast-growing analytics firm offering advanced a and intelligent analytical solutions to clients globally. We combine domain expertise, advanced analytics, and technology to provide robust solutions for clients' business problems. You can find further details about the company at https://merilytics.com.

We partner with our clients in Private Equity, CPG, Retail, Healthcare, Media & Entertainment, Technology, Logistics industries etc. by providing analytical solutions to generate superior returns. We solve clients' business problems by analyzing large amount of data to help guide their Operations, Marketing, Pricing, Customer Strategies, and much more.

Position :

- Business Associate at Merilytics will be working on complex analytical projects and is the primary owner of the work streams involved.

- The Business Associates are expected to lead the team of Business Analysts to deliver robust analytical solutions consistently and mentor the Analysts for professional development.

Location : Hyderabad

Roles and Responsibilities :

The roles and responsibilities of a Business Associate will include the below:

- Proactively provide thought leadership to the team and have complete control on the delivery process of the project.

- Understand the client's point of view and translate it into sound judgment calls in ambiguous analytical situations.

- Highlight potential analytical issues upfront and resolve them independently.

- Synthesizes the analysis and derives insights independently.

- Identify the crux of the client problem and leverage it to draw relevant actionable insights from the analysis/work.

- Ability to manage multiple Analysts and provide customized guidance for individual development.

- Resonate with our five core values - Client First, Excellence, Integrity, Respect and Teamwork.

Pre-requisites and skillsets required to apply for this role :

- Undergraduate degree (B.E/B.Tech.) from tier-1/tier-2 colleges are preferred.

- Should have 2-4 years of experience.

- Strong leadership & proactive communication to coordinate with the project team and other internal stakeholders.

- Ability to use business judgement and a structured approach towards solving complex problems.

- Experience in client-facing/professional services environment is a plus.

- Strong hard skills on analytics tools such as R, Python, SQL, and Excel is a plus.

Why Explore a Career at Merilytics :

- High growth environment: Semi-annual performance management and promotion cycles coupled with a strong meritocratic culture, enables fast track to leadership responsibility.

- Cross Domain Exposure: Interesting and challenging work streams across industries and domains that always keep you excited, motivated, and on your toes.

- Entrepreneurial Environment: Intellectual freedom to make decisions and own them. We expect you to spread your wings and assume larger responsibilities.

- Fun culture and peer group: Non-bureaucratic and fun working environment; Strong peer environment that will challenge you and accelerate your learning curve.

Other benefits for full time employees:

(i) Health and wellness programs that include employee health insurance covering immediate family members and parents, term life insurance for employees, free health camps for employees, discounted health services (including vision, dental) for employee and family members, free doctor's consultations, counselors, etc.

(ii) Corporate Meal card options for ease of use and tax benefits.

(iii) Work dinners, team lunches, company sponsored team outings and celebrations.

(iv) Reimbursement support for travel to the office, as and when promulgated by the Company.

(v) Cab reimbursement for women employees beyond a certain time of the day.

(vi) Robust leave policy to support work-life balance. Specially designed leave structure to support woman employees for maternity and related requests.

(vii) Reward and recognition platform to celebrate professional and personal milestones.

(viii) A positive & transparent work environment including various employee engagement and employee benefit initiatives to support personal and professional learning and development.

• Minimum 1 years of relevant experience, in PySpark (mandatory)
• Hands on experience in development, test, deploy, maintain and improving data integration pipeline in AWS cloud environment is added plus 
• Ability to play lead role and independently manage 3-5 member of Pyspark development team 
• EMR ,Python and PYspark mandate.
• Knowledge and awareness working with AWS Cloud technologies like Apache Spark, , Glue, Kafka, Kinesis, and Lambda in S3, Redshift, RDS

• 5+ years of industry experience in administering (including setting up, managing, monitoring) data processing pipelines (both streaming and batch) using frameworks such as Kafka Streams, Py Spark, and streaming databases like druid or equivalent like Hive
• Strong industry expertise with containerization technologies including kubernetes (EKS/AKS), Kubeflow
• Experience with cloud platform services such as AWS, Azure or GCP especially with EKS, Managed Kafka
• 5+ Industry experience in python
• Experience with popular modern web frameworks such as Spring boot, Play framework, or Django
• Experience with scripting languages. Python experience highly desirable. Experience in API development using Swagger
• Implementing automated testing platforms and unit tests
• Proficient understanding of code versioning tools, such as Git
• Familiarity with continuous integration, Jenkins

Responsibilities

• Architect, Design and Implement Large scale data processing pipelines using Kafka Streams, PySpark, Fluentd and Druid
• Create custom Operators for Kubernetes, Kubeflow
• Develop data ingestion processes and ETLs
• Assist in dev ops operations
• Design and Implement APIs
• Identify performance bottlenecks and bugs, and devise solutions to these problems
• Help maintain code quality, organization, and documentation
• Communicate with stakeholders regarding various aspects of solution.
• Mentor team members on best practices