DeepIntent

What You’ll Do:

We are looking for a Staff Software Engineer based in Pune, India who can master both DeepIntent’s data architectures and pharma research and analytics methodologies to make significant contributions to how health media is analyzed by our clients. This role requires an Engineer who not only understands DBA functions but also how they impact research objectives and can work with researchers and data scientists to achieve impactful results.  

This role will be in the Analytics Organization and will require integration and partnership with the Engineering Organization. The ideal candidate is a self-starter who is inquisitive who is not afraid to take on and learn from challenges and will constantly seek to improve the facets of the business they manage. The ideal candidate will also need to demonstrate the ability to collaborate and partner with others.  

• Serve as the Engineering interface between Analytics and Engineering teams.
• Develop and standardize all interface points for analysts to retrieve and analyze data with a focus on research methodologies and data-based decision-making.
• Optimize queries and data access efficiencies, serve as an expert in how to most efficiently attain desired data points.
• Build “mastered” versions of the data for Analytics-specific querying use cases.
• Help with data ETL, table performance optimization.
• Establish a formal data practice for the Analytics practice in conjunction with the rest of DeepIntent
• Build & operate scalable and robust data architectures.
• Interpret analytics methodology requirements and apply them to data architecture to create standardized queries and operations for use by analytics teams.
• Implement DataOps practices.
• Master existing and new Data Pipelines and develop appropriate queries to meet analytics-specific objectives.
• Collaborate with various business stakeholders, software engineers, machine learning engineers, and analysts.
• Operate between Engineers and Analysts to unify both practices for analytics insight creation.

Who You Are:

• 8+ years of experience in Tech Support (Specialised in Monitoring and maintaining Data pipeline).
• Adept in market research methodologies and using data to deliver representative insights.
• Inquisitive, curious, understands how to query complicated data sets, move and combine data between databases.
• Deep SQL experience is a must.
• Exceptional communication skills with the ability to collaborate and translate between technical and non-technical needs.
• English Language Fluency and proven success working with teams in the U.S.
• Experience in designing, developing and operating configurable Data pipelines serving high-volume and velocity data.
• Experience working with public clouds like GCP/AWS.
• Good understanding of software engineering, DataOps, and data architecture, Agile and DevOps methodologies.
• Experience building Data architectures that optimize performance and cost, whether the components are prepackaged or homegrown.
• Proficient with SQL, Python or JVM-based language, Bash.
• Experience with any of Apache open-source projects such as Spark, Druid, Beam, Airflow etc. and big data databases like BigQuery, Clickhouse, etc. 
• Ability to think big, take bets and innovate, dive deep, hire and develop the best talent, learn and be curious.

What You’ll Do:

• Setting up formal data practices for the company.
• Building and running super stable and scalable data architectures.
• Making it easy for folks to add and use new data with self-service pipelines.
• Getting DataOps practices in place.
• Designing, developing, and running data pipelines to help out Products, Analytics, data scientists and machine learning engineers.
• Creating simple, reliable data storage, ingestion, and transformation solutions that are a breeze to deploy and manage.
• Writing and Managing reporting API for different products.
• Implementing different methodologies for different reporting needs.
• Teaming up with all sorts of people – business folks, other software engineers, machine learning engineers, and analysts.

Who You Are:

• Bachelor’s degree in engineering (CS / IT) or equivalent degree from a well-known Institute / University.
• 3.5+ years of experience in building and running data pipelines for tons of data.
• Experience with public clouds like GCP or AWS.
• Experience with Apache open-source projects like Spark, Druid, Airflow, and big data databases like BigQuery, Clickhouse.
• Experience making data architectures that are optimised for both performance and cost.
• Good grasp of software engineering, DataOps, data architecture, Agile, and DevOps.
• Proficient in SQL, Java, Spring Boot, Python, and Bash.
• Good communication skills for working with technical and non-technical people.
• Someone who thinks big, takes chances, innovates, dives deep, gets things done, hires and develops the best, and is always learning and curious.

What you'll do:

• Mandatory - Experience & hands-on experience in setting up, optimizing, and securing analytical distributed data sources such as ClickHouse, Druid, or similar distributed database systems. Intermidiate DBA skills required.
• Deploy, configure, and maintain Kubernetes clusters for our microservices architecture.
• Utilize Git and Helm for version control and deployment management.
• Implement and manage monitoring solutions using Prometheus and Grafana.
• Work on continuous integration and continuous deployment (CI/CD) pipelines.
• Containerize applications using Docker and manage orchestration.
• Manage and optimize AWS services, including but not limited to EC2, S3, RDS, and AWS CDN.
• Maintain and optimize MySQL databases, Airflow, and Redis instances.
• Write automation scripts in Bash or Python for system administration tasks.
• Perform Linux administration tasks and troubleshoot system issues.
• Utilize Ansible and Terraform for configuration management and infrastructure as code.
• Demonstrate knowledge of networking and load-balancing principles.
• Collaborate with development teams to ensure applications meet reliability and performance standards.

Additional Skills (Good to Know):

• Familiarity with ClickHouse and Druid for data storage and analytics.
• Experience with Jenkins for continuous integration.
• Basic understanding of Google Cloud Platform (GCP) and data center operations.

Who you are:

• Minimum 3 years of experience in a Site Reliability Engineer role or similar.
• Proven experience with Kubernetes, Git, Helm, Prometheus, Grafana, CI/CD, Docker, and microservices architecture.
• Strong knowledge of AWS services, MySQL, Airflow, Redis, AWS CDN.
• Proficient in scripting languages such as Bash or Python.
• Hands-on experience with Linux administration.
• Familiarity with Ansible and Terraform for infrastructure management.
• Understanding of networking principles and load balancing.
• Bachelor's degree in Computer Science, Information Technology, or a related field.