Data Engineer

Job Title: ELK Stack Engineer

Position Overview:

We are seeking a skilled and experienced Senior Elasticsearch Developer to join our team. The ideal candidate will be responsible for designing, developing, and maintaining data pipelines using the ELK Stack (Elasticsearch 7.1 or higher, Logstash, Kibana). The Senior Elasticsearch Developer will play a key role in managing large datasets, optimizing complex queries, and maintaining self-hosted Elastic Search clusters.

Responsibilities:

• Design, develop, and maintain data pipelines using the ELK Stack.
• Handle large datasets (3 billion+ records, 100+ fields per index) with efficiency and accuracy.
• Develop and optimize complex Elasticsearch queries for efficient data retrieval and analysis.
• Manage and maintain self-hosted Elastic Search clusters.
• Implement DevOps practices, including containerization with Docker.
• Perform backups, disaster recovery, and index migrations to ensure data security and integrity.
• Execute data processing and cleaning techniques, including writing custom scripts to extract and transform data into usable formats.
• Collaborate with the engineering team to integrate Elastic Search with other systems.

Requirements:

• Bachelor's degree in Computer Science, Engineering, or related field.
• Minimum 5+ years of experience in designing, developing, and maintaining data pipelines using the ELK Stack.
• Hands-on experience handling large datasets (3 million+ records, 50+ fields per index).
• Strong proficiency in developing and optimizing complex Elasticsearch queries.
• Experience managing and maintaining self-hosted ElasticSearch clusters.
• Proficiency in DevOps practices, including containerization with Docker.
• Knowledge of security best practices, including backups, disaster recovery, and index migrations.
• Experience with database administration and data processing techniques.
• Excellent communication and teamwork skills.

Benefits:

• Competitive salary and benefits package between 20 to 35 LPA.
• Opportunities for professional development and growth.
• Collaborative and inclusive work environment.
• Flexible work hours and remote work options.

**THIS IS A 100% WORK FROM OFFICE ROLE**

We are looking for an experienced DevOps engineer that will help our team establish DevOps practice. You will work closely with the technical lead to identify and establish DevOps practices in the company.
You will help us build scalable, efficient cloud infrastructure. You’ll implement monitoring for automated system health checks. Lastly, you’ll build our CI pipeline, and train and guide the team in DevOps practices.

ROLE and RESPONSIBILITIES:

• Understanding customer requirements and project KPIs
• Implementing various development, testing, automation tools, and IT infrastructure
• Planning the team structure, activities, and involvement in project management
activities.
• Managing stakeholders and external interfaces
• Setting up tools and required infrastructure
• Defining and setting development, test, release, update, and support processes for
DevOps operation
• Have the technical skill to review, verify, and validate the software code developed in
the project.
• Troubleshooting techniques and fixing the code bugs
• Monitoring the processes during the entire lifecycle for its adherence and updating or
creating new processes for improvement and minimizing the wastage
• Encouraging and building automated processes wherever possible
• Identifying and deploying cybersecurity measures by continuously performing
vulnerability assessment and risk management
• Incidence management and root cause analysis
• Coordination and communication within the team and with customers
• Selecting and deploying appropriate CI/CD tools
• Strive for continuous improvement and build continuous integration, continuous
development, and constant deployment pipeline (CI/CD Pipeline)
• Mentoring and guiding the team members
• Monitoring and measuring customer experience and KPIs
• Managing periodic reporting on the progress to the management and the customer

Essential Skills and Experience Technical Skills

• Proven 3+years of experience as DevOps
• A bachelor’s degree or higher qualification in computer science
• The ability to code and script in multiple languages and automation frameworks
like Python, C#, Java, Perl, Ruby, SQL Server, NoSQL, and MySQL
• An understanding of the best security practices and automating security testing and
updating in the CI/CD (continuous integration, continuous deployment) pipelines
• An ability to conveniently deploy monitoring and logging infrastructure using tools.
• Proficiency in container frameworks
• Mastery in the use of infrastructure automation toolsets like Terraform, Ansible, and command line interfaces for Microsoft Azure, Amazon AWS, and other cloud platforms
• Certification in Cloud Security
• An understanding of various operating systems
• A strong focus on automation and agile development
• Excellent communication and interpersonal skills
• An ability to work in a fast-paced environment and handle multiple projects
simultaneously

OTHER INFORMATION

The DevOps Engineer will also be expected to demonstrate their commitment:
• to gedu values and regulations, including equal opportunities policy.
• the gedu’s Social, Economic and Environmental responsibilities and minimise environmental impact in the performance of the role and actively contribute to the delivery of gedu’s Environmental Policy.
• to their Health and Safety responsibilities to ensure their contribution to a safe and secure working environment for staff, students, and other visitors to the campus.

ROLE REQUIREMENTS:

• Experience of 5+ years working as a Quality Assurance Analyst
• Experience with test management tools such as Quality Centre, JIRA, ADO.
• Knowledge of automation tools like Selenium
• Knowledge of Non-Functional testing - Accessibility, Performance
• Educated to degree level, ideally in Computer Science
• ISEB certification in Testing (intermediate level) is advantageous
• Experience in participating in process change is advantageous
• Experience testing commercial websites in a creative and/or agency environment is advantageous
• Experience testing native iOS and Android applications is advantageous

QUALIFICATIONS & CHARACTERISTICS

• 5+ years’ industry experience.
• Analytic and positive mindset with a personable and professional approach.
• Sharp attention to detail and a passion for new technology.
• Professional written and verbal communication.

Mandatory:

• 4-7 Experience in React JS,  (ReactJS 2.5yrs compulsory)
• Optional:

Knowledge in UX/UI design, Azure DevOps, Test Driven - and Domain Driven Development

• Developing of complex IT systems with various system integrations and configurations
• Data security/GDPR

Desired Candidate Profile

• Own development, design, scaling and maintenance of application and messaging engines that power the central platform of Capillary's Cloud CRM product. 
• Contribute to overall design and roadmap.
• Mentor Junior team members.

Required Skills:

• Innovative and self-motivated with passion to develop complex and scalable applications.
• 3-5 years of experience in software development with strong focus on algorithms and data structures.
• Strong coding and design skills with prior experience in developing scalable & high availability applications using Core Java/J2EE, Spring, Hibernate. 
• Work experience with Relational databases is required (Primarily MySQL)
• Prior work experience with Non-Relational databases (primarily Redis, MongoDb) is an added plus.
• Strong Analytical and Problem Solving Skills.
• BTech