Database Engineer

Our product is centered around lots of data being processed, ingested and read efficiently. The underlying systems need to provide capabilities update and ingest data on the order of billions of records on a daily basis. Complex analytics queries need to run on 10s of billions of rows where a single query that can potentially touch 100+ million rows needs to finish in interactive SLAs. All of this processing happens on data with several 100s of dimensions and tens of thousands of metrics.

This leads to a very interesting and challenging use case in the emerging field of large scale distributed HTAP, which is still not mature enough to provide a solution out of the box that works for our scale and SLAs. So, we are building a solution that can handle the complexity of our use case and scale to several trillions of rows. As a "Database Engineer", you will evolve, architect, build and scale the core data warehouse that sits at the heart of Clarisights enabling large scale distributed, interactive analytics on near realtime data.

What you'll do

- Understanding and gaining expertise in existing data warehouse.
- Use the above knowledge to identify gaps in the current system and formulate strategies around what can be done to fill them
- Avail KPIs around the data warehouse.
- Find solutions to evolve and scale the data warehouse. This will involve a lot of technical research, benchmarking and testing of existing and candidate replacement systems.
- Bulid from scratch all or parts of the data warehouse to improve the KPIs.
- Ensure the SLAs and SLOs of data warehouse, which will require assuming ownership and being oncall for the same.
- Gain deep understanding into Linux and understand concepts that drive performance characteristics like IO scheduling, paging, processing scheduling, CPU instruction pipelining etc.
- Adopt/build tooling and tune the systems to extract maximum performance out of the underlying hardware.
- Build wrappers/microservices for improving visibility, control, adoption and ease of use for the data warehouse.
- Build tooling and automation for monitoring, debugging and deployment of the warehouse.
- Contribute to open source database technologies that are used at or are potential candidates for use.

What you bring

We are looking for engineers with a strong passion for solving challenging engineering problems and a burning desire to learn and grow in a fast growing startup. This is not an easy gig, it will require strong technical chops and an insatiable curiosity to make things better. We need passionate and mature engineers who can do wonders with some mentoring and don't need to be managed.

- Distributed systems: You have a good understanding of general patterns of scaling and fault-tolerance in large scale distributed systems.
- Databases: You have a good understanding of database concepts like query optimization, indexing, transactions, sharding, replication etc.
- Data pipelines: You have a working knowledge of distributed data processing systems.
- Engineer at heart: You thrive on writing great code and have a strong appreciation for modular, testable and maintainable code, and make sure to document it. You have the ability to take new initiatives and questioning status quo.
- Passion & Drive to learn and excel: You believe in our vision. You drive the product for the better, always looking to improve things, and soon become the go-to person to talk to on something that you mastered along. You love dabbling in your own side-projects and learning new skills that are not necessarily part of your normal day job.
- Inquisitiveness: You are curious to know how different modules on our platform work. You are not afraid to venture into unknown territories of code. You ask questions.
- Ownership: You are your own manager. You have the ability to implement engineering tasks on your own without a need for micro-management and take responsibility for any task that has been assigned to you.
- Teamwork: You should be helpful and work well with teams. You’re probably someone who enjoys sharing knowledge with team-mates, asking for help when they need it.
- Open Source Contribution: Bonus.