Data Engineer (Pyspark Developer)

Job Summary:

We are seeking a skilled DevOps Engineer to join our dynamic team. The ideal candidate will be responsible for managing, maintaining, and troubleshooting Rancher clusters, with a strong emphasis on Kubernetes operations. This role requires expertise in automation through shell scripting and proficiency in configuration management tools like Puppet and Ansible. Candidates should be highly self-motivated, capable of working on a rotating schedule, and committed to owning tasks through to delivery.

Key Responsibilities:

• Set up, operate, and maintain Rancher and Kubernetes (K8s) clusters, including on bare-metal environments.
• Perform upgrades and manage the lifecycle of Rancher clusters.
• Troubleshoot and resolve Rancher cluster issues efficiently.
• Write, maintain, and optimize shell scripts to automate Kubernetes-related tasks.
• Work collaboratively with the team to implement best practices for system automation and orchestration.
• Utilize configuration management tools like Puppet and Ansible (preferred but not mandatory).
• Participate in a rotating schedule, with the ability to work until 1 AM as required.
• Take ownership of tasks, ensuring timely delivery with high-quality standards.

Key Requirements:

• Strong expertise in Rancher and Kubernetes operations and maintenance.
• Experience in setting up and managing Kubernetes clusters on bare-metal systems is highly desirable.
• Proficiency in shell scripting for task automation.
• Familiarity with configuration management tools like Puppet and Ansible (good to have).
• Strong troubleshooting skills for Kubernetes and Rancher environments.
• Ability to work effectively in a rotating schedule and flexible hours.
• Strong ownership mindset and accountability for deliverables.

• Augmenting, improving, redesigning, and/or re-implementing Dolat's low-latency/high-throughput production trading environment, which collects data from and disseminates orders to exchanges around the world
• Optimizing this platform by using network and systems programming, as well as other advanced techniques
• Developing systems that provide easy access to historical market data and trading simulations
• Building risk-management and performance-tracking tools
• Shaping the future of Dolat through regular interviewing and infrequent campus recruiting trips
• Implementing domain-optimized data structures
• Learn and internalize the theories behind current trading system
• Participate in the design, architecture and implementation of automated trading systems
• Take ownership of system from design through implementation

Job Description:  

As a L3 analyst, you will play a key role within the Application Management team, leading services for its customer(s). This service has responsibility for the smooth daily operational running of the Client platform both in the cloud and on-premises. The Client application is business critical and centrally manages data for trading, settlement, risk, client and regulatory reporting systems. As a L3 analyst, you will be responsible for supporting complex infrastructure and application issues raised by L1/L2 support teams, and business analysts. You would also be working on client requests for enhancements, fixes, maintaining custom code, and following the best practices such as Agile, DevOps, etc.

The role is a technical role requiring a good technical knowledge of SQL/PLSQ, Core Java along with good debugging skill while supporting client implementations. You should have basic understanding on Unix system and commands. The ability to troubleshoot issues, assist users and help write specifications is essential. This position also works with product management to improve the software engineering processes and practices associated with continuously building, deploying, and updating software and environments.

• Accountability and primary responsibility/duties:
• A typical day would look like as follows: -
  • 40% support issues.
  • 40% configuration/development work.
  • 20% continuous improvement, automation, CICD, so on.
• Activities would range across the lifecycle from discovery to post-implementation support.
• Understanding client’s requirement, replicate the problem and providing the solution
• Understanding and developing new requirements coming in from BAs (Business Analysts) and customers.
• Update the existing functionality based on client needs.
• Develop changes that cater to enhancing the product and/or fixing production issues
• Prepare product releases of software components. (Java/SQL-PLSQL)
• Contribute in all phases of the development cycle. (development/Deployment/Testing, peer review)
• Create technical document for all changes made.

Required Qualifications/Knowledge/Skills

• Bachelor’s degree in Computer Science or related field
• Strong basics and working experience in following technologies: -
  • Core Java.
  • SQL / PLSQL coding.
  • Object Oriented Programming concepts and data structures.
• Working knowledge on Unix platform
• Working knowledge on XSLT and XML handling
• Basic understanding App & web server working knowledge (JBOSS, WebLogic and WebSphere) and debugging skills.
• Advantageous to have:
  • Working understanding on CICD, Dev-ops technologies.
  • Queuing technologies including Kafka, MQ, Solace.
  • Scripting including Python, Unix, java.
• Hands on knowledge of Dev ops processes and tools Good interpersonal and communication skills.
  
  

DeepSource is working on building tools that help developers ship good code. There are over 40 million developers in the world, and all of them write and review code in some form. There’s a massive opportunity to impact how software is built right from where the code is written using automation and intelligence, that not only improves developer productivity but also increases software’s robustness.

The Language Engineering team works on the source code analyzers, including both programming languages and configuration-as-code systems. As a member of the Language Engineering team, you will work on building the best, most comprehensive, Scala analyzer in the world. You will add new rules and Autofixes for finding more issues with code and automatically fixing them. You will be involved with the community to understand the problems with static analysis tools in the Scala ecosystem.

• Identify bad code practices in Scala and write new analyzers to detect them.

• Improve the coverage of automatically fixable issues.

• Ensure fewer false-positives are reported by the analyzer.

• Work on the internal tools that support analysis runtimes.

• Contribute to open-source static analysis tools.

• Strong foundational knowledge in Computer Science.

• At least 3 years of professional software development experience in Scala and Java.

• Understanding of the nuances of execution of the source code (AST, data flow graphs, etc).

• Familiarity with Scala best practices followed in the industry.

• Native experience with Linux/Unix environment.

• A focus on delivering high-quality code through strong testing practices.

We offer competitive compensation with meaningful stock options, a generous vacation policy, and a workstation of your choice, to name a few of the perks.

Role- Software Development Engineer-2

As a Software Development Engineer at Amazon, you have industry-leading technical abilities and demonstrate breadth and depth of knowledge. You build software to deliver business impact, making smart technology choices. You work in a team and drive things forward.

Top Skills

You write high quality, maintainable, and robust code, often in Java or C++ or C#

You recognize and adopt best practices in software engineering: design, testing, version control, documentation, build, deployment, and operations.

You have experience building scalable software systems that are high-performance, highly-available, highly transactional, low latency and massively distributed.

Roles & Responsibilities

You solve problems at their root, stepping back to understand the broader context.

You develop pragmatic solutions and build flexible systems that balance engineering complexity and timely delivery, creating business impact.

You understand a broad range of data structures and algorithms and apply them to deliver high-performing applications.

You recognize and use design patterns to solve business problems.

You understand how operating systems work, perform and scale.

You continually align your work with Amazon’s business objectives and seek to deliver business value.

You collaborate to ensure that decisions are based on the merit of the proposal, not the proposer.

You proactively support knowledge-sharing and build good working relationships within the team and with others in Amazon.

You communicate clearly with your team and with other groups and listen effectively.

Skills & Experience

Bachelors or Masters in Computer Science or relevant technical field.

Experience in software development and full product life-cycle.

Excellent programming skills in any object-oriented programming languages - preferably Java, C/C++/C#, Perl, Python, or Ruby.

Strong knowledge of data structures, algorithms, and designing for performance, scalability, and availability.

Proficiency in SQL and data modeling.