Cutshort logo
Consulting and Services company logo
Data Engineer
Consulting and Services company
Data Engineer
Consulting and Services company's logo

Data Engineer

at Consulting and Services company

Agency job
5 - 10 yrs
₹5L - ₹30L / yr (ESOP available)
Hyderabad, Ahmedabad
Skills
skill iconAmazon Web Services (AWS)
Apache
skill iconPython
PySpark

Data Engineer 

  

Mandatory Requirements  

  • Experience in AWS Glue 
  • Experience in Apache Parquet  
  • Proficient in AWS S3 and data lake  
  • Knowledge of Snowflake 
  • Understanding of file-based ingestion best practices. 
  • Scripting language - Python & pyspark 

 

CORE RESPONSIBILITIES 

  • Create and manage cloud resources in AWS  
  • Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies  
  • Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform  
  • Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations  
  • Develop an infrastructure to collect, transform, combine and publish/distribute customer data. 
  • Define process improvement opportunities to optimize data collection, insights and displays. 
  • Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible  
  • Identify and interpret trends and patterns from complex data sets  
  • Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders.  
  • Key participant in regular Scrum ceremonies with the agile teams   
  • Proficient at developing queries, writing reports and presenting findings  
  • Mentor junior members and bring best industry practices  

 

QUALIFICATIONS 

  • 5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)  
  • Strong background in math, statistics, computer science, data science or related discipline 
  • Advanced knowledge one of language: Java, Scala, Python, C#  
  • Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake   
  • Proficient with 
  • Data mining/programming tools (e.g. SAS, SQL, R, Python) 
  • Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum) 
  • Data visualization (e.g. Tableau, Looker, MicroStrategy) 
  • Comfortable learning about and deploying new technologies and tools.  
  • Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.  
  • Good written and oral communication skills and ability to present results to non-technical audiences  
  • Knowledge of business intelligence and analytical tools, technologies and techniques. 

 

Familiarity and experience in the following is a plus:  

  • AWS certification 
  • Spark Streaming  
  • Kafka Streaming / Kafka Connect  
  • ELK Stack  
  • Cassandra / MongoDB  
  • CI/CD: Jenkins, GitLab, Jira, Confluence other related tools 
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

DocNexus
Mahek Chhatrapati
Posted by Mahek Chhatrapati
Remote only
4 - 10 yrs
₹32L - ₹40L / yr
skill iconAngularJS (1.x)
skill iconAngular (2+)
skill iconReact.js
skill iconNodeJS (Node.js)
skill iconElastic Search
+15 more

DocNexus is revolutionizing the global medical affairs & commercial ecosystem with search. We provide a next-generation data platform that simplifies searching through millions of insights, publications, clinical trials, payments, and social media data within seconds to identify healthcare professionals (HCPs), products, manufacturers, and healthcare systems. Leveraging AI-powered Knowledge Graphs, DocNexus assists life science organizations in finding the right key opinion leaders (KOL/DOLs) who play a crucial role in developing and bringing life-saving pharmaceutical products and medical devices to market. Backed by industry leaders such as Techstars, JP Morgan, Mass Challenge, and recognized as one of the Top 200 Most Innovative Startups by TechCrunch Disrupt, we are committed to transforming healthcare insights. We are seeking a skilled and passionate DevOps Engineer to join our dynamic team and contribute to the efficient development, deployment, and maintenance of our platform.


We are looking for a visionary Sr. Full Stack Engineering Lead who is passionate about building and leading our technology department. The ideal candidate will have a solid technical background and experience in leading a team to drive innovation and growth. As Engineering Lead, you will oversee the development and dissemination of technology for external customers, vendors, and other clients to help improve and increase business.


Leadership and Strategy:

  • Lead the engineering team and make strategic decisions regarding the technology stack, project management, and resource allocation.
  • Establish the company’s technical vision and lead all aspects of technological development.

Development:

  • Develop and maintain the front-end and back-end of web applications.
  • Ensure the performance, quality, and responsiveness of applications.
  • Collaborate with a team to define, design, and ship new features.

Maintenance and Optimization:

  • Maintain code integrity and organization.
  • Identify and correct bottlenecks and fix bugs.
  • Continually work on optimizing the performance of different applications.

Security: Ensure the security of the web applications by integrating security best practices.

  • Regularly update the system to protect against vulnerabilities.

Innovation:

  • Research and implement new technologies and frameworks that can improve the performance and user experience of the platform.
  • Stay informed on emerging technologies and trends that can potentially impact the company's products and services.

Collaboration and Communication:

  • Work closely with other departments to understand their needs and translate them into technical solutions.
  • Communicate technology strategy to partners, management, investors, and employees.

Project Management:

  • Oversee and support project planning, deadlines, and progress.
  • Ensure that the technology standards and best practices are maintained across the organization.

Mentoring and Team Building:

  • Foster a culture of innovation and excellence within the technology team.
  • Mentor and guide the professional and technical development of team members.

Front-End Development:

  • HTML/CSS: For structuring and styling the web pages.
  • JavaScript/TypeScript: Core scripting language, along with frameworks like Angular, React, or Vue.js for dynamic and responsive user interfaces.

Back-End Development:

  • Python: Using frameworks like Django or Flask for server-side logic.
  • Node.js: JavaScript runtime environment for building scalable network applications.
  • Ruby on Rails: A server-side web application framework written in Ruby.

Database Management:

  • SQL Databases: MySQL, PostgreSQL for structured data storage.
  • NoSQL Databases: MongoDB, Cassandra for unstructured data or specific use cases.

Server Management:

  • Nginx or Apache: For server and reverse proxy functionalities.
  • Docker: For containerizing applications and ensuring consistency across multiple development and release cycles.
  • Kubernetes: For automating deployment, scaling, and operations of application containers.

DevOps and Continuous Integration/Continuous Deployment (CI/CD):

  • Git: For version control.
  • Jenkins, Travis CI, or CircleCI: For continuous integration and deployment.
  • Ansible, Chef, or Puppet: For configuration management.

Cloud Services:

  • AWS: For various cloud services like computing, database storage, content delivery, etc.
  • Serverless Frameworks: Such as AWS Lambda or Google Cloud Functions for running code without provisioning or managing servers.

Security:

  • OAuth, JWT: For secure authentication mechanisms.
  • SSL/TLS: For secure data transmission.
  • Various Encryption Techniques: To safeguard sensitive data.

Performance Monitoring and Testing:

  • Selenium, Jest, or Mocha: For automated testing.
  • New Relic or Datadog: For performance monitoring.

Data Science and Analytics:

  • Python Libraries: NumPy, Pandas, or SciPy for data manipulation and analysis.
  • Machine Learning Frameworks: TensorFlow, PyTorch for implementing machine learning models.

Other Technologies:

  • GraphQL: For querying and manipulating data efficiently.
  • WebSockets: For real-time bi-directional communication between web clients and servers.
Read more
Recro
at Recro
1 video
32 recruiters
Mounashree JP
Posted by Mounashree JP
Bengaluru (Bangalore)
4 - 5 yrs
Best in industry
skill iconNodeJS (Node.js)
skill iconMongoDB
Mongoose
skill iconExpress
Data Structures

Key Responsibilities:

  • Rewrite existing APIs in NodeJS.
  • Remodel the APIs into Micro services-based architecture.
  • Implement a caching layer wherever possible.
  • Optimize the API for high performance and scalability.
  • Write unit tests for API Testing.
  • Automate the code testing and deployment process.


Skills Required:

  • At least 3 years of experience developing Backends using NodeJS — should be well versed with its asynchronous nature & event loop, and know its quirks and workarounds.
  • Excellent hands-on experience using MySQL or any other SQL Database.
  • Good knowledge of MongoDB or any other NoSQL Database.
  • Good knowledge of Redis, its data types, and their use cases.
  • Experience with graph-based databases like GraphQL and Neo4j.
  • Experience developing and deploying REST APIs.
  • Good knowledge of Unit Testing and available Test Frameworks.
  • Good understanding of advanced JS libraries and frameworks.
  • Experience with Web sockets, Service Workers, and Web Push Notifications.
  • Familiar with NodeJS profiling tools.
  • Proficient understanding of code versioning tools such as Git.
  • Good knowledge of creating and maintaining DevOps infrastructure on cloud platforms.
  • Should be a fast learner and a go-getter — without any fear of trying out new things Preferences.
  • Experience building a large-scale social or location-based app.


Read more
NeoSoft Technologies (A CMMi Level 5 Organization)
Mumbai, Navi Mumbai
3 - 6 yrs
₹6L - ₹12L / yr
skill iconData Science
skill iconMachine Learning (ML)
Natural Language Processing (NLP)
Computer Vision
recommendation algorithm
+3 more
  1. Minimum 2.5 years of experience as a Python Developer.
  2. Minimum 2.5 years of experience in any framework like Django/Flask/Fast API
  3. Minimum 2.5 years of experience in SQL/ Postgress
  4. Minimum 2.5 years of experience in Git/Gitlab/Bit-Bucket
  5. Minimum 2+ years of experience in deployment (CICD with Jenkins)
  6. Minimum 2.5 years of experience in any cloud like AWS/GCP/Azure
Read more
Edyst
at Edyst
1 recruiter
Aneeq edyst
Posted by Aneeq edyst
Remote only
2 - 5 yrs
₹10L - ₹15L / yr
skill iconPython
skill iconDjango
skill iconFlask
skill iconDocker
skill iconRedis

Responsibilities:

  • Understanding requirements from the product manager and translating to technical specifications
  • DB schema development
  • API development (structuring REST endpoints + real-time endpoints like socket.io integration)
  • Integrating 3rd party services for payments, email, whatsapp, and other such services
  • Optimizing our current architecture & processes for scale (we expect usage to go up)
  • Guiding junior developers on best coding practices

 

Requirements:

  • Experience in at least one Python-based Web Framework like Flask or Django
  • Experience in SQL databases & API design
  • Experience with Docker
  • Hands on experience in full lifecycle development and deployment of a platform/system
  • Being a self-starter with the ability to thrive in ambiguous and high pressure scenarios working towards a common goal with other teams
Read more
Quinbay technologies
at Quinbay technologies
9 recruiters
VaraLakshmi Addepalli
Posted by VaraLakshmi Addepalli
Bengaluru (Bangalore)
6 - 10 yrs
₹20L - ₹30L / yr
Solr
skill iconElastic Search
skill iconSpring Boot
skill iconJava
Apache Solr
+2 more
Qualifications:

  • Bachelor’s degree in computer science, engineering, or a related field.
  • At-least more than 1+ Years of Experience in a similar role would be desired.
  • Excellent technical, diagnostic, and troubleshooting skills.
  • Strong leadership skills to drive good coding and design practices across multiple engineering teams
  • Willingness to build professional relationships with staff and clients.
  • Excellent communication, motivational, and interpersonal skills.
  • Expertise in architecting, building, and maintaining ultra-low latency, cost-efficient systems in cloud environments
  • Excellent track record in modernizing cloud-based applications using micro services, containers, and other architectures
  • Experience and working knowledge in building large-scale, data-intensive Text search applications using Solr / Elastic Search is a Must
  • Experience and working knowledge of AI/ML/ML Ops with respect to building large-scale, data-intensive applications would be a Plus
  • Experience and working knowledge in building Java based Spring Boot micro-services integrated with any messaging framework
  • Driven, Highly-motivated and passionate towards development & innovation

 

Job Role:

  • Determining project requirements and developing work schedules for the team.
  • Delegating tasks and achieving daily, weekly, and monthly goals.
  • Liaising with team members, management, and clients to ensure projects are completed to standard.
  • Identifying risks and forming contingency plans as soon as possible.
  • Analyzing existing operations and scheduling training sessions and meetings to discuss improvements.
  • Keeping up-to-date with industry trends and developments.
  • Updating work schedules and performing troubleshooting as required.
  • Motivating staff and creating a space where they can ask questions and voice their concerns.
  • Being transparent with the team about challenges, failures, and successes.
  • Writing progress reports and delivering presentations to the relevant stakeholders.
  • Extremely hands-on in delivering Development and R&D tasks
  • Design, plan and perform dev-analysis to determine effort estimates on every sprint for the team
  • Running Demo's, ensuring thorough documentation of the features built
  • Identify & Plan upgrades to technologies & frameworks from time-to-time

 

Read more
Arna Softech Pvt Ltd
at Arna Softech Pvt Ltd
2 recruiters
Dheeraj Sarsodiya
Posted by Dheeraj Sarsodiya
Indore
0 - 1 yrs
₹1L - ₹1.5L / yr
skill icon.NET
skill iconC#
ASP.NET
skill iconHTML/CSS

📌 Knowledge of development technologies including HTML, JavaScript, JQuery, AJAX, and CSS.

📌 Knowledge of  Microsoft.NET/ C#, If not Ready to learn 

📌 Comprehensive understanding of object-oriented and service-oriented application development techniques and theories.

📌 Internally motivated, able to work proficiently in a team environment.

📌 The candidate should be ready to learn multiple technologies.

Read more
Goodera
at Goodera
4 recruiters
Rahul Agarwal
Posted by Rahul Agarwal
Bengaluru (Bangalore)
1 - 4 yrs
₹2L - ₹4L / yr
Technical support
Communication Skills
Hard working
Responsibilities: - Provide technical support for customers to support product processes- Address all product-related queries on time- Train customers to use products effectively- Provide developers with customers- feedback to help identify potential new features or products- Report on product performance- Identify solutions to reduce support costs- Analyze customers- needs and suggest upgrades or additional features to meet their requirements- Liaise with the sales department to win new business and increase sales- Establish best practicesRequirements : - Proven work experience as a Technical account manager- Solid technical background with hands on experience in digital technologies- Familiarity with software and development lifecycle- An ability to gasp customers- needs and suggest timely solutions- Excellent verbal and written communication skills- Strong analytical and problem-solving skills- BSc degree in Computer Science or relevant field
Read more
10 years old Mobile & Web development firm
10 years old Mobile & Web development firm
Agency job
Kerala, Thiruvananthapuram
1 - 2 yrs
₹2L - ₹3L / yr
Business Development
Sales
B2B Marketing
B2C
IT sales
  • B2B
  • B2C( International IT Sales )
  • Knowledge of products and technologies and market conditions.
  • Excellent communications
Read more
Medlife.com
at Medlife.com
4 recruiters
Sheetal Adavani
Posted by Sheetal Adavani
Bengaluru (Bangalore)
8 - 12 yrs
₹10L - ₹35L / yr
skill iconJava
Spring
Data Structures
Algorithms
Design patterns
+1 more
You’re Part Communication – • Drive discussions to improve product across teams where ever there are inter dependencies across products Coding - • You will write performant, unit-tested code, develop object-oriented models and design data structure for new software projects taking systems aspects into account • You will translate high level business problems into scalable design and code. Create libraries & Utilities for larger consumption • Drive large refactoring exercises Design – • API Definitions and end-to-end service design and implementation for large services • Create Platforms in your product/team. Build platforms, systems and networking infrastructure using your strong background in distributed systems, OS/kernel, network system design, and large scale storage systems. Org Development – • Influence product requirements & operational plans. Instill best practices for development and champion their adoption, while working with product manager to estimate and plan projects in agile development framework. • Mentor junior engineers on software design, coding practices and TDD strategies and form a holistic mentoring plan for Mentee • You’re Array (Nice To Haves) • A multi-perspective approach to developing object-oriented models, designing data structure and building applications and platforms that can scale. • Problem solving skills for high-level business and technical problems. • Your leadership and mentoring skills to roll-up your sleeves and offer technical and project management insights and tips. You (Must Haves) • Deep expertise in at least one programming language (preferably Java) & tech stack to write maintainable, scalable, unit-tested code. • Substantial experience in building large complex business applications • Strong object-oriented design skills, knowledge of design patterns, and an uncanny ability to design intuitive module and class-level interfaces • Experience leading multi-engineer projects and mentoring junior engineers • Deep understanding of design patterns. • Prior experience in working with Agile software methodologies (XP, Scrum) • BTech or MTech in Computer Science or related technical discipline (or equivalent). • The experience of having been a software engineer for at least 6 years. You’re Cheers! • Great Work, Great People and Great environment. We call ourselves an incubator for engineers where you get all the optimal conditions to do and experience your best. Seniority Level Not Applicable Industry • Healthcare • Internet Employment Type Full-time
Read more
Betterhalfai
at Betterhalfai
5 recruiters
Pramal Tekade
Posted by Pramal Tekade
Bengaluru (Bangalore)
3 - 7 yrs
₹6L - ₹10L / yr
skill iconJavascript
skill iconPython
skill iconNodeJS (Node.js)
skill iconReact Native
skill iconMongoDB
+2 more
We are looking for a Full-Stack developer who will take a key product development role for http://betterhalf.ai/ What is the company about? We're building world's largest AI based partner prediction engine based on the past data of millions of married couples. This invention unique in its way in partner search industry has potential to offer a useful and valuable product for 500M globally. More information We’re a U.S based company with the operational centre based out of Bangalore, India. We’re currently a 8-member team and expected to grow to the size of 100 over the next few years. Why you should reach out? If you’re interested in the product and deliver on humongous responsibility to build a globally successful company (globally 500M people including 60M Indians) with the continuous thrill of ups and downs of startup, feel free to reach us at [email protected] Job Description/ Responsibilities As a Full-stack Engineer, you’ll be involved in implementing both the backend and frontend for new and existing product features. You will be joining a talented, collaborative team who is very passionate about solving this massive problem. Below is a list of several skills required to deliver on responsibilities for the Full-Stack developer role: Front-End React Native for Mobile development (Preferred) Web fundamentals like HTML, JavaScript, and CSS, Bootstrap React.js, Node.js, Angular.js Back-End API design and development CRUD (Create, Read, Update, Delete) RESTful Services Advanced in any one of the server-side languages like PHP, Python, NodeJS. Candidates having a great background in Python/Django would give a preference for this role. Basic knowledge of database technologies like MySQL/MongoDB is required. Experience with web server technologies like Nginx. Experience with AWS/Digital Ocean would be preferred. Bonus: Technical soundness on ML/AI would be a great plus. Compensation: Cash and stocks for U.S company (to be discussed on case to case basis) Perks Great company culture with a very friendly and encouraging environment Amazing location in HSR Layout Modern, comfortable workplace, with areas to chill when you need it Free lunch with lots of healthy snacks
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos