Carpenter

Senior Data Engineer (Azure Databricks)

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark
• Work extensively with PySpark notebooks within Databricks for data processing and transformation
• Build and optimize batch data processing workflows
• Develop and manage data integrations using Azure Functions and Logic Apps
• Write efficient and optimized SQL queries for data extraction and transformation

Required Skills:

• Strong hands-on experience with Azure Databricks, PySpark, and SQL
• Experience working with batch processing frameworks
• Proficiency in building and managing data pipelines in Azure ecosystem

Good to Have:

• Experience with Python

Mandatory Requirement:

• Candidate must have hands-on experience working with PySpark notebooks in Databricks

Interested candidates are requested to email their resumes with the subject line "Application for [Job Title]".

Only applications received via email will be reviewed. Applications through other channels will not be considered.

Job Description

The client’s department DPS, Digital People Solutions, offers a sophisticated portfolio of IT applications, providing a strong foundation for professional and efficient People & Organization (P&O) and Business Management, both globally and locally, for a well-known German company listed on the DAX-40 index, which includes the 40 largest and most liquid companies on the Frankfurt Stock Exchange

We are seeking talented DevOps-Engineers with focus on Elastic Stack (ELK) to join our dynamic DPS team. In this role, you will be responsible for refining and advising on the further development of an existing monitoring solution based on the Elastic Stack (ELK). You will independently handle tasks related to architecture, setup, technical migration, and documentation.

The current application landscape features multiple Java web services running on JEE application servers, primarily hosted on AWS, and integrated with various systems such as SAP, other services, and external partners. DPS is committed to delivering the best digital work experience for the customers employees and customers alike.

Responsibilities:

Install, set up, and automate rollouts using Ansible/CloudFormation for all stages (Dev, QA, Prod) in the AWS Cloud for components such as Elastic Search, Kibana, Metric beats, APM server, APM agents, and interface configuration.

Create and develop regular "Default Dashboards" for visualizing metrics from various sources like Apache Webserver, application servers and databases.

Improve and fix bugs in installation and automation routines.

Monitor CPU usage, security findings, and AWS alerts.

Develop and extend "Default Alerting" for issues like OOM errors, datasource issues, and LDAP errors.

Monitor storage space and create concepts for expanding the Elastic landscape in AWS Cloud and Elastic Cloud Enterprise (ECE).

Implement machine learning, uptime monitoring including SLA, JIRA integration, security analysis, anomaly detection, and other useful ELK Stack features.

Integrate data from AWS CloudWatch.

Document all relevant information and train involved personnel in the used technologies.

Requirements:

Experience with Elastic Stack (ELK) components and related technologies.

Proficiency in automation tools like Ansible and CloudFormation.

Strong knowledge of AWS Cloud services.

Experience in creating and managing dashboards and alerts.

Familiarity with IAM roles and rights management.

Ability to document processes and train team members.

Excellent problem-solving skills and attention to detail.

Skills & Requirements

Elastic Stack (ELK), Elasticsearch, Kibana, Logstash, Beats, APM, Ansible, CloudFormation, AWS Cloud, AWS CloudWatch, IAM roles, AWS security, Automation, Monitoring, Dashboard creation, Alerting, Anomaly detection, Machine learning integration, Uptime monitoring, JIRA integration, Apache Webserver, JEE application servers, SAP integration, Database monitoring, Troubleshooting, Performance optimization, Documentation, Training, Problem-solving, Security analysis.

Responsibilities:

• Design, develop, and maintain user interfaces for the internal product suite.

• Collaborate closely with product managers and backend engineers to understand

requirements, translate them into user-friendly interfaces, and ensure a consistent

user experience across all products.

• Champion the technical feasibility of our UI/UX designs, ensuring seamless user experiences.

• Write clean, maintainable, and well-documented code adhering to best practices and coding standards.

• Conduct unit and integration tests to ensure code quality and functionality.

• Stay up-to-date with the latest UI development trends and technologies.

• Take initiative and ownership of assigned tasks, ensuring they are completed efficiently and to a high standard. 

Requirements/Qualifications

• Bachelors in any Computers stream or related fields

• Minimum 1+ years of experience as a UI Developer.

• Proficiency in Javascript,Node.js & any one of (Angular, Nextjs or React.js)

• Experience with unit and integration testing frameworks.

• Excellent communication and collaboration skills.

• Ability to work independently and as part of a team.

• A passion for modern web development practices and a drive to deliver high-quality

work.

• Good to have the Experience with responsive design and cross-browser compatibility.

• Creating and managing ETL/ELT pipelines based on requirements
• Build PowerBI dashboards and manage datasets needed.
• Work with stakeholders to identify data structures needed for future and perform any transformations including aggregations.
• Build data cubes for real-time visualisation needs and CXO dashboards.

Required Tech Skills

• Microsoft PowerBI & DAX
• Python, Pandas, PyArrow, Jupyter Noteboks, ApacheSpark
• Azure Synapse, Azure DataBricks, Azure HDInsight, Azure Data Factory

Our client is a 5 year old online gaming company with market leading games in three segments - Poker, Rummy and Cricket. Their online games are ensured to be safe, trusted and secure for their customers. They have a registered base of over a million users and are working towards the growth of online gaming in the country.

What you need to have:

• Systems: eCommerce, CMS, Transaction website, payment gateways, Omni-channel, Postman Application
• Role: Will be responsible for testing website, POS, Logistics systems. Integration testing. Automate scripts and test data. Testing of website on different screens, resolution, regression, performance & integrity testing. Build use cases, test cases and test data including API testing for request & response
• Education : BE/BTech
• Experience : 4-6 Years. Manual and/or Automation Testing, Understanding and usage of tools like Silenium, JIRA. Team player. Leadership qualities and coordination with different stakeholders

Job Responsibilities:

• Developing and installing software solutions.
• Designing, implementing and delivering high quality Software project in JAVA, SQL / Oracle, J2EE and other JAVA technologies.
• Participate in detailed level in design, coding, code walk through, peer code reviews and unit testing, System Testing, UAT, Demos, POCs, installation, maintenance of Software modules.
• Software prototype.
• System Architecture.
• Software Design Document.
• User interfaces to be developed as per UX guidelines Code files ensuring coding guidelines followed.
• Code review artefacts during peer code review.
• Test plan and Test Cases.
• Installation / Deployment document Release Document.
• Technical Documentation.

• Set up workstations with computers and necessary peripheral devices (routers, printers etc.)
• Check computer hardware (HDD, mouses, keyboards etc.) to ensure the functionality
• Install and configure appropriate software and functions according to specifications
• Develop and maintain local networks in ways that optimize performance
• Ensure security and privacy of networks and computer systems
• Provide orientation and guidance to users on how to operate new software and computer equipment
• Organize and schedule upgrades and maintenance without deterring others from completing their work
• Perform troubleshooting to diagnose and resolve problems (repair or replace parts, debugging etc.)
• Maintain records/logs of repairs and fixes and maintenance schedule
• Identify computer or network equipment shortages and place orders
• Provide technical assistance with computer hardware and software
• Resolve issues for staff via phone, in person, or electronically
• Resolve technical issues in a timely manner using available resources within the company