Big Data Engineer

at netmedscom

Big Data Engineer

netmedscom

Company

Home

Big Data Engineer

at netmedscom

Posted by Vijay Hemnath

2 - 5 yrs

₹6L - ₹25L / yr (ESOP available)

Chennai

Skills

Big Data

Hadoop

Apache Hive

Scala

Spark

Datawarehousing

Machine Learning (ML)

Deep Learning

SQL

Data modeling

PySpark

Python

Amazon Web Services (AWS)

Java

Cassandra

DevOps

HDFS

We are looking for an outstanding Big Data Engineer with experience setting up and maintaining Data Warehouse and Data Lakes for an Organization. This role would closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Roles and Responsibilities:

Develop and maintain scalable data pipelines and build out new integrations and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using 'Big Data' technologies.
Develop programs in Scala and Python as part of data cleaning and processing.
Assemble large, complex data sets that meet functional / non-functional business requirements and fostering data-driven decision making across the organization.
Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.
Implement processes and systems to validate data, monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Provide high operational excellence guaranteeing high availability and platform stability.
Closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Skills:

Experience with Big Data pipeline, Big Data analytics, Data warehousing.
Experience with SQL/No-SQL, schema design and dimensional data modeling.
Strong understanding of Hadoop Architecture, HDFS ecosystem and eexperience with Big Data technology stack such as HBase, Hadoop, Hive, MapReduce.
Experience in designing systems that process structured as well as unstructured data at large scale.
Experience in AWS/Spark/Java/Scala/Python development.
Should have Strong skills in PySpark (Python & SPARK). Ability to create, manage and manipulate Spark Dataframes. Expertise in Spark query tuning and performance optimization.
Experience in developing efficient software code/frameworks for multiple use cases leveraging Python and big data technologies.
Prior exposure to streaming data sources such as Kafka.
Should have knowledge on Shell Scripting and Python scripting.
High proficiency in database skills (e.g., Complex SQL), for data preparation, cleaning, and data wrangling/munging, with the ability to write advanced queries and create stored procedures.
Experience with NoSQL databases such as Cassandra / MongoDB.
Solid experience in all phases of Software Development Lifecycle - plan, design, develop, test, release, maintain and support, decommission.
Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development).
Experience building and deploying applications on on-premise and cloud-based infrastructure.
Having a good understanding of machine learning landscape and concepts.

Qualifications and Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Big Data Engineer or a similar role for 3-5 years.

Certifications:

Good to have at least one of the Certifications listed here:

AZ 900 - Azure Fundamentals

DP 200, DP 201, DP 203, AZ 204 - Data Engineering

AZ 400 - Devops Certification

Roles and Responsibilities:

Develop and maintain scalable data pipelines and build out new integrations and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using 'Big Data' technologies.
Develop programs in Scala and Python as part of data cleaning and processing.
Assemble large, complex data sets that meet functional / non-functional business requirements and fostering data-driven decision making across the organization.
Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.
Implement processes and systems to validate data, monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Provide high operational excellence guaranteeing high availability and platform stability.
Closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Skills:

Experience with Big Data pipeline, Big Data analytics, Data warehousing.
Experience with SQL/No-SQL, schema design and dimensional data modeling.
Strong understanding of Hadoop Architecture, HDFS ecosystem and eexperience with Big Data technology stack such as HBase, Hadoop, Hive, MapReduce.
Experience in designing systems that process structured as well as unstructured data at large scale.
Experience in AWS/Spark/Java/Scala/Python development.
Should have Strong skills in PySpark (Python & SPARK). Ability to create, manage and manipulate Spark Dataframes. Expertise in Spark query tuning and performance optimization.
Experience in developing efficient software code/frameworks for multiple use cases leveraging Python and big data technologies.
Prior exposure to streaming data sources such as Kafka.
Should have knowledge on Shell Scripting and Python scripting.
High proficiency in database skills (e.g., Complex SQL), for data preparation, cleaning, and data wrangling/munging, with the ability to write advanced queries and create stored procedures.
Experience with NoSQL databases such as Cassandra / MongoDB.
Solid experience in all phases of Software Development Lifecycle - plan, design, develop, test, release, maintain and support, decommission.
Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development).
Experience building and deploying applications on on-premise and cloud-based infrastructure.
Having a good understanding of machine learning landscape and concepts.

Qualifications and Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Big Data Engineer or a similar role for 3-5 years.

Certifications:

Good to have at least one of the Certifications listed here:

AZ 900 - Azure Fundamentals

DP 200, DP 201, DP 203, AZ 204 - Data Engineering

AZ 400 - Devops Certification

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Subodh Popalwar

Software Engineer, Memorres

For 2 years, I had trouble finding a company with good work culture and a role that will help me grow in my career. Soon after I started using Cutshort, I had access to information about the work culture, compensation and what each company was clearly offering.

Companies hiring on Cutshort

About netmedscom

Founded :

2015

Type

Size

Stage :

Raised funding

About

Netmeds.com is a fully licensed e-pharma portal that offers authenticated prescription and Over the Counter (OTC) medicine along with other health products. Netmeds is the “first-to-market” to offer a pan-India solution for the quick online purchase and fast delivery of prescription medications and has served more than 5.7 million customers in more than 670 cities and towns. The portal offers customers the convenience of ordering health products online via a simple process. A team of qualified pharmacists’ vets the prescription for validity and dosage that assures dispensing only appropriate medication. Through Netmeds, consumers get access to more than 70,000 prescription drugs for chronic and recurring ailments as well as enhanced lifestyle drugs and thousands of non-prescription goods for wellness, health, and personal care.

Connect with the team

Philip Assey

Connect

Vijay Hemnath

Connect

Komal Dev

Connect

Company social profiles

Similar jobs

Database Administrator

at With Reputed service based company

Agency job

via Jobdost by Saida Jabbar

Bengaluru (Bangalore)

4 - 6 yrs

₹12L - ₹15L / yr

SQL

MySQL

MySQL DBA

MariaDB

MS SQLServer

Role Description
As a Database Administrator, you will be responsible for designing, testing, planning,
implementing, protecting, operating, managing and maintaining our company’s
databases. The goal is to provide a seamless flow of information throughout the

company, considering both backend data structure and frontend accessibility for end-
users. You get to work with some of the best minds in the industry at a place where

opportunity lurks everywhere and in everything.
Responsibilities
Your responsibilities are as follows.
• Build database systems of high availability and quality depending on each end
user’s specialised role
• Design and implement database in accordance to end users’ information needs
and views
• Define users and enable data distribution to the right user, in appropriate format
and in a timely manner
• Use high-speed transaction recovery techniques and backup data
• Minimise database downtime and manage parameters to provide fast query
responses
• Provide proactive and reactive data management support and training to users
• Determine, enforce and document database policies, procedures and
standards
• Perform tests and evaluations regularly to ensure data security, privacy and
integrity
• Monitor database performance, implement changes and apply new patches
and versions when required
Required Qualifications
We are looking for individuals who are curious, excited about learning, and navigating
through the uncertainties and complexities that are associated with a growing
company. Some qualifications that we think would help you thrive in this role are:
• Minimum 4 Years of experience as a Database Administrator
• Hands-on experience with database standards and end user applications
• Excellent knowledge of data backup, recovery, security, integrity and SQL
• Familiarity with database design, documentation and coding
• Previous experience with DBA case tools (frontend/backend) and third-party
tools
• Familiarity with programming languages API
• Problem solving skills and ability to think algorithmically
• Bachelor/Masters of CS/IT Engineering, BCA/MCA, B Sc/M Sc in CS/IT

Preferred Qualifications
• Sense of ownership and pride in your performance and its impact on company’s
success
• Critical thinker and problem-solving skills
• Team player
• Good time-management skills
• Great interpersonal and communication skills.

Power BI Manager

at This opening is with an MNC

Agency job

via LK Consultants by Namita Agate

Mumbai, Malad, andheri

8 - 13 yrs

₹13L - ₹22L / yr

Spotfire

Qlikview

Tableau

PowerBI

Data Visualization

+8 more

Minimum of 8 years of experience of which, 4 years should be of applied data mining

experience in disciplines such as Call Centre Metrics.

 Strong experience in advanced statistics and analytics including segmentation, modelling, regression, forecasting etc.

 Experience with leading and managing large teams.

 Demonstrated pattern of success in using advanced quantitative analytic methods to solve business problems.

 Demonstrated experience with Business Intelligence/Data Mining tools to work with

data, investigate anomalies, construct data sets, and build models.

 Critical to share details on projects undertaken (preferably on telecom industry)

specifically through analysis from CRM.

Minimum of 8 years of experience of which, 4 years should be of applied data mining

experience in disciplines such as Call Centre Metrics.

 Strong experience in advanced statistics and analytics including segmentation, modelling, regression, forecasting etc.

 Experience with leading and managing large teams.

 Demonstrated pattern of success in using advanced quantitative analytic methods to solve business problems.

 Demonstrated experience with Business Intelligence/Data Mining tools to work with

data, investigate anomalies, construct data sets, and build models.

 Critical to share details on projects undertaken (preferably on telecom industry)

specifically through analysis from CRM.

Senior Software Engineer - Data

at 6sense

15 recruiters

Posted by Romesh Rawat

Remote only

5 - 8 yrs

₹30L - ₹45L / yr

PySpark

Data engineering

Big Data

Hadoop

Spark

+4 more

About Slintel (a 6sense company) :

Slintel, a 6sense company, the leader in capturing technographics-powered buying intent, helps companies uncover the 3% of active buyers in their target market. Slintel evaluates over 100 billion data points and analyzes factors such as buyer journeys, technology adoption patterns, and other digital footprints to deliver market & sales intelligence.

Slintel's customers have access to the buying patterns and contact information of more than 17 million companies and 250 million decision makers across the world.

Slintel is a fast growing B2B SaaS company in the sales and marketing tech space. We are funded by top tier VCs, and going after a billion dollar opportunity. At Slintel, we are building a sales development automation platform that can significantly improve outcomes for sales teams, while reducing the number of hours spent on research and outreach.

We are a big data company and perform deep analysis on technology buying patterns, buyer pain points to understand where buyers are in their journey. Over 100 billion data points are analyzed every week to derive recommendations on where companies should focus their marketing and sales efforts on. Third party intent signals are then clubbed with first party data from CRMs to derive meaningful recommendations on whom to target on any given day.

6sense is headquartered in San Francisco, CA and has 8 office locations across 4 countries.

6sense, an account engagement platform, secured $200 million in a Series E funding round, bringing its total valuation to $5.2 billion 10 months after its $125 million Series D round. The investment was co-led by Blue Owl and MSD Partners, among other new and existing investors.

Linkedin (Slintel) : https://www.linkedin.com/company/slintel/">https://www.linkedin.com/company/slintel/

Industry : Software Development

Company size : 51-200 employees (189 on LinkedIn)

Headquarters : Mountain View, California

Founded : 2016

Specialties : Technographics, lead intelligence, Sales Intelligence, Company Data, and Lead Data.

Website (Slintel) : https://www.slintel.com/slintel">https://www.slintel.com/slintel

Linkedin (6sense) : https://www.linkedin.com/company/6sense/">https://www.linkedin.com/company/6sense/

Industry : Software Development

Company size : 501-1,000 employees (937 on LinkedIn)

Headquarters : San Francisco, California

Founded : 2013

Specialties : Predictive intelligence, Predictive marketing, B2B marketing, and Predictive sales

Website (6sense) : https://6sense.com/">https://6sense.com/

Acquisition News :

https://inc42.com/buzz/us-based-based-6sense-acquires-b2b-buyer-intelligence-startup-slintel/

Funding Details & News :

Slintel funding : https://www.crunchbase.com/organization/slintel">https://www.crunchbase.com/organization/slintel

6sense funding : https://www.crunchbase.com/organization/6sense">https://www.crunchbase.com/organization/6sense

https://www.nasdaq.com/articles/ai-software-firm-6sense-valued-at-%245.2-bln-after-softbank-joins-funding-round">https://www.nasdaq.com/articles/ai-software-firm-6sense-valued-at-%245.2-bln-after-softbank-joins-funding-round

https://www.bloomberg.com/news/articles/2022-01-20/6sense-reaches-5-2-billion-value-with-softbank-joining-round">https://www.bloomberg.com/news/articles/2022-01-20/6sense-reaches-5-2-billion-value-with-softbank-joining-round

https://xipometer.com/en/company/6sense">https://xipometer.com/en/company/6sense

Slintel & 6sense Customers :

https://www.featuredcustomers.com/vendor/slintel/customers

https://www.featuredcustomers.com/vendor/6sense/customers">https://www.featuredcustomers.com/vendor/6sense/customers

About the job

Responsibilities

Work in collaboration with the application team and integration team to design, create, and maintain optimal data pipeline architecture and data structures for Data Lake/Data Warehouse
Work with stakeholders including the Sales, Product, and Customer Support teams to assist with data-related technical issues and support their data analytics needs
Assemble large, complex data sets from third-party vendors to meet business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Elastic search, MongoDB, and AWS technology
Streamline existing and introduce enhanced reporting and analysis solutions that leverage complex data sources derived from multiple internal systems

Requirements

3+ years of experience in a Data Engineer role
Proficiency in Linux
Must have SQL knowledge and experience working with relational databases, query authoring (SQL) as well as familiarity with databases including Mysql, Mongo, Cassandra, and Athena
Must have experience with Python/ Scala
Must have experience with Big Data technologies like Apache Spark
Must have experience with Apache Airflow
Experience with data pipeline and ETL tools like AWS Glue
Experience working with AWS cloud services: EC2 S3 RDS, Redshift and other Data solutions eg. Databricks, Snowflake

Desired Skills and Experience

Python, SQL, Scala, Spark, ETL

About Slintel (a 6sense company) :

Slintel's customers have access to the buying patterns and contact information of more than 17 million companies and 250 million decision makers across the world.

6sense is headquartered in San Francisco, CA and has 8 office locations across 4 countries.

Linkedin (Slintel) : https://www.linkedin.com/company/slintel/">https://www.linkedin.com/company/slintel/

Industry : Software Development

Company size : 51-200 employees (189 on LinkedIn)

Headquarters : Mountain View, California

Founded : 2016

Specialties : Technographics, lead intelligence, Sales Intelligence, Company Data, and Lead Data.

Website (Slintel) : https://www.slintel.com/slintel">https://www.slintel.com/slintel

Linkedin (6sense) : https://www.linkedin.com/company/6sense/">https://www.linkedin.com/company/6sense/

Industry : Software Development

Company size : 501-1,000 employees (937 on LinkedIn)

Headquarters : San Francisco, California

Founded : 2013

Specialties : Predictive intelligence, Predictive marketing, B2B marketing, and Predictive sales

Website (6sense) : https://6sense.com/">https://6sense.com/

Acquisition News :

https://inc42.com/buzz/us-based-based-6sense-acquires-b2b-buyer-intelligence-startup-slintel/

Funding Details & News :

Slintel funding : https://www.crunchbase.com/organization/slintel">https://www.crunchbase.com/organization/slintel

6sense funding : https://www.crunchbase.com/organization/6sense">https://www.crunchbase.com/organization/6sense

https://xipometer.com/en/company/6sense">https://xipometer.com/en/company/6sense

Slintel & 6sense Customers :

https://www.featuredcustomers.com/vendor/slintel/customers

https://www.featuredcustomers.com/vendor/6sense/customers">https://www.featuredcustomers.com/vendor/6sense/customers

About the job

Responsibilities

Work in collaboration with the application team and integration team to design, create, and maintain optimal data pipeline architecture and data structures for Data Lake/Data Warehouse
Work with stakeholders including the Sales, Product, and Customer Support teams to assist with data-related technical issues and support their data analytics needs
Assemble large, complex data sets from third-party vendors to meet business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Elastic search, MongoDB, and AWS technology
Streamline existing and introduce enhanced reporting and analysis solutions that leverage complex data sources derived from multiple internal systems

Requirements

3+ years of experience in a Data Engineer role
Proficiency in Linux
Must have SQL knowledge and experience working with relational databases, query authoring (SQL) as well as familiarity with databases including Mysql, Mongo, Cassandra, and Athena
Must have experience with Python/ Scala
Must have experience with Big Data technologies like Apache Spark
Must have experience with Apache Airflow
Experience with data pipeline and ETL tools like AWS Glue
Experience working with AWS cloud services: EC2 S3 RDS, Redshift and other Data solutions eg. Databricks, Snowflake

Desired Skills and Experience

Python, SQL, Scala, Spark, ETL

Big data Developer

at Persistent Systems

1 video

1 recruiter

Agency job

via Milestone Hr Consultancy by Haina khan

Pune, Bengaluru (Bangalore), Hyderabad, Nagpur

4 - 9 yrs

₹4L - ₹15L / yr

Spark

Hadoop

Big Data

Data engineering

PySpark

+3 more

Greetings..

We have an urgent requirements of Big Data Developer profiles in our reputed MNC company.

Location: Pune/Bangalore/Hyderabad/Nagpur
Experience: 4-9yrs

Skills: Pyspark,AWS
or Spark,Scala,AWS
or Python Aws

BI Developer

at Agiletech Info Solutions pvt ltd

Posted by Kalaithendral Nagarajan

Chennai

4 - 9 yrs

₹4L - ₹12L / yr

Data Analytics

Data Visualization

PowerBI

Tableau

Qlikview

+5 more

4 - 8 overall experience.

1-2 years’ experience in Azure Data Factory - schedule Jobs in Flows and ADF Pipelines, Performance Tuning, Error logging etc..
1+ years of experience with Power BI - designing and developing reports, dashboards, metrics and visualizations in Powe BI.
(Required) Participate in video conferencing calls - daily stand-up meetings and all day working with team members on cloud migration planning, development, and support.
Proficiency in relational database concepts & design using star, Azure Datawarehouse, and data vault.
Requires 2-3 years of experience with SQL scripting (merge, joins, and stored procedures) and best practices.
Knowledge on deploying and run SSIS packages in Azure.
Knowledge of Azure Data Bricks.
Ability to write and execute complex SQL queries and stored procedures.

4 - 8 overall experience.

1-2 years’ experience in Azure Data Factory - schedule Jobs in Flows and ADF Pipelines, Performance Tuning, Error logging etc..
1+ years of experience with Power BI - designing and developing reports, dashboards, metrics and visualizations in Powe BI.
(Required) Participate in video conferencing calls - daily stand-up meetings and all day working with team members on cloud migration planning, development, and support.
Proficiency in relational database concepts & design using star, Azure Datawarehouse, and data vault.
Requires 2-3 years of experience with SQL scripting (merge, joins, and stored procedures) and best practices.
Knowledge on deploying and run SSIS packages in Azure.
Knowledge of Azure Data Bricks.
Ability to write and execute complex SQL queries and stored procedures.

Big Data Architect

at Number Theory

3 recruiters

Posted by Nidhi Mishra

Gurugram

5 - 12 yrs

₹10L - ₹40L / yr

Spark

Hadoop

Big Data

Data engineering

PySpark

+5 more

Job Description – Big Data Architect
Number Theory is looking for experienced software/data engineer who would be focused on owning and rearchitecting dynamic pricing engineering systems
Job Responsibilities:
 Evaluate and recommend Big Data technology stack best suited for NT AI at scale Platform
and other products
 Lead the team for defining proper Big Data Architecture Design.
 Design and implement features on NT AI at scale platform using Spark and other Hadoop
Stack components.
 Drive significant technology initiatives end to end and across multiple layers of architecture
 Provides strong technical leadership in adopting and contributing to open source technologies related to Big Data across multiple engagements
 Designing /architecting complex, highly available, distributed, failsafe compute systems dealing with considerable scalable amount of data
 Identify and work upon incorporating Non-functional requirements into the solution (Performance, scalability, monitoring etc.)

Requirements:
 A successful candidate with 8+ years of experience in the role of implementation of a highend software product.
 Provides technical leadership in Big Data space (Spark and Hadoop Stack like Map/Reduc,
HDFS, Hive, HBase, Flume, Sqoop etc. NoSQL stores like Cassandra, HBase etc) across
Engagements and contributes to open-source Big Data technologies.
 Rich hands on in Spark and worked on Spark at a larger scale.
 Visualize and evangelize next generation infrastructure in Big Data space (Batch, Near
Real-time, Realtime technologies).
 Passionate for continuous learning, experimenting, applying and contributing towards
cutting edge open-source technologies and software paradigms
 Expert-level proficiency in Java and Scala.
 Strong understanding and experience in distributed computing frameworks, particularly
Apache Hadoop2.0 (YARN; MR & HDFS) and associated technologies one or more of Hive,
Sqoop, Avro, Flume, Oozie, Zookeeper, etc.Hands-on experience with Apache Spark and its
components (Streaming, SQL, MLLib)
 Operating knowledge of cloud computing platforms (AWS,Azure) –

Good to have:
 Operating knowledge of different enterprise hadoop distribution (C) –
 Good Knowledge of Design Patterns
 Experience working within a Linux computing environment, and use of command line tools
including knowledge of shell/Python scripting for automating common tasks.

Data Pipeline

at They provide both wholesale and retail funding. PM1

Agency job

via Multi Recruit by Sapna Deb

Mumbai

5 - 7 yrs

₹20L - ₹25L / yr

ETL

Talend

OLAP

Data governance

SQL

+8 more

Key responsibility is to design and develop a data pipeline including the architecture, prototyping, and development of data extraction, transformation/processing, cleansing/standardizing, and loading in Data Warehouse at real-time/near the real-time frequency. Source data can be structured, semi-structured, and/or unstructured format.
Provide technical expertise to design efficient data ingestion solutions to consolidate data from RDBMS, APIs, Messaging queues, weblogs, images, audios, documents, etc of Enterprise Applications, SAAS applications, external 3rd party sites or APIs, etc through ETL/ELT, API integrations, Change Data Capture, Robotic Process Automation, Custom Python/Java Coding, etc
Development of complex data transformation using Talend (BigData edition), Python/Java transformation in Talend, SQL/Python/Java UDXs, AWS S3, etc to load in OLAP Data Warehouse in Structured/Semi-structured form
Development of data model and creating transformation logic to populate models for faster data consumption with simple SQL.
Implementing automated Audit & Quality assurance checks in Data Pipeline
Document & maintain data lineage to enable data governance
Coordination with BIU, IT, and other stakeholders to provide best-in-class data pipeline solutions, exposing data via APIs, loading in down streams, No-SQL Databases, etc

Requirements

Programming experience using Python / Java, to create functions / UDX
Extensive technical experience with SQL on RDBMS (Oracle/MySQL/Postgresql etc) including code optimization techniques
Strong ETL/ELT skillset using Talend BigData Edition. Experience in Talend CDC & MDM functionality will be an advantage.
Experience & expertise in implementing complex data pipelines, including semi-structured & unstructured data processing
Expertise to design efficient data ingestion solutions to consolidate data from RDBMS, APIs, Messaging queues, weblogs, images, audios, documents, etc of Enterprise Applications, SAAS applications, external 3rd party sites or APIs, etc through ETL/ELT, API integrations, Change Data Capture, Robotic Process Automation, Custom Python/Java Coding, etc
Good understanding & working experience in OLAP Data Warehousing solutions (Redshift, Synapse, Snowflake, Teradata, Vertica, etc) and cloud-native Data Lake (S3, ADLS, BigQuery, etc) solutions
Familiarity with AWS tool stack for Storage & Processing. Able to recommend the right tools/solutions available to address a technical problem
Good knowledge of database performance and tuning, troubleshooting, query optimization, and tuning
Good analytical skills with the ability to synthesize data to design and deliver meaningful information
Good knowledge of Design, Development & Performance tuning of 3NF/Flat/Hybrid Data Model
Know-how on any No-SQL DB (DynamoDB, MongoDB, CosmosDB, etc) will be an advantage.
Ability to understand business functionality, processes, and flows
Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently

Functional knowledge

Data Governance & Quality Assurance
Distributed computing
Linux
Data structures and algorithm
Unstructured Data Processing

Key responsibility is to design and develop a data pipeline including the architecture, prototyping, and development of data extraction, transformation/processing, cleansing/standardizing, and loading in Data Warehouse at real-time/near the real-time frequency. Source data can be structured, semi-structured, and/or unstructured format.
Provide technical expertise to design efficient data ingestion solutions to consolidate data from RDBMS, APIs, Messaging queues, weblogs, images, audios, documents, etc of Enterprise Applications, SAAS applications, external 3rd party sites or APIs, etc through ETL/ELT, API integrations, Change Data Capture, Robotic Process Automation, Custom Python/Java Coding, etc
Development of complex data transformation using Talend (BigData edition), Python/Java transformation in Talend, SQL/Python/Java UDXs, AWS S3, etc to load in OLAP Data Warehouse in Structured/Semi-structured form
Development of data model and creating transformation logic to populate models for faster data consumption with simple SQL.
Implementing automated Audit & Quality assurance checks in Data Pipeline
Document & maintain data lineage to enable data governance
Coordination with BIU, IT, and other stakeholders to provide best-in-class data pipeline solutions, exposing data via APIs, loading in down streams, No-SQL Databases, etc

Requirements

Programming experience using Python / Java, to create functions / UDX
Extensive technical experience with SQL on RDBMS (Oracle/MySQL/Postgresql etc) including code optimization techniques
Strong ETL/ELT skillset using Talend BigData Edition. Experience in Talend CDC & MDM functionality will be an advantage.
Experience & expertise in implementing complex data pipelines, including semi-structured & unstructured data processing
Expertise to design efficient data ingestion solutions to consolidate data from RDBMS, APIs, Messaging queues, weblogs, images, audios, documents, etc of Enterprise Applications, SAAS applications, external 3rd party sites or APIs, etc through ETL/ELT, API integrations, Change Data Capture, Robotic Process Automation, Custom Python/Java Coding, etc
Good understanding & working experience in OLAP Data Warehousing solutions (Redshift, Synapse, Snowflake, Teradata, Vertica, etc) and cloud-native Data Lake (S3, ADLS, BigQuery, etc) solutions
Familiarity with AWS tool stack for Storage & Processing. Able to recommend the right tools/solutions available to address a technical problem
Good knowledge of database performance and tuning, troubleshooting, query optimization, and tuning
Good analytical skills with the ability to synthesize data to design and deliver meaningful information
Good knowledge of Design, Development & Performance tuning of 3NF/Flat/Hybrid Data Model
Know-how on any No-SQL DB (DynamoDB, MongoDB, CosmosDB, etc) will be an advantage.
Ability to understand business functionality, processes, and flows
Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently

Functional knowledge

Data Governance & Quality Assurance
Distributed computing
Linux
Data structures and algorithm
Unstructured Data Processing

Bigdata Lead Architecture

at DataMetica

1 video

7 recruiters

Posted by Nikita Aher

Pune, Hyderabad

7 - 12 yrs

₹12L - ₹33L / yr

Big Data

Hadoop

Spark

Apache Spark

Apache Hive

+3 more

Job description

Role : Lead Architecture (Spark, Scala, Big Data/Hadoop, Java)

Primary Location : India-Pune, Hyderabad

Experience : 7 - 12 Years

Management Level: 7

Joining Time: Immediate Joiners are preferred

Attend requirements gathering workshops, estimation discussions, design meetings and status review meetings
Experience of Solution Design and Solution Architecture for the data engineer model to build and implement Big Data Projects on-premises and on cloud.
Align architecture with business requirements and stabilizing the developed solution
Ability to build prototypes to demonstrate the technical feasibility of your vision
Professional experience facilitating and leading solution design, architecture and delivery planning activities for data intensive and high throughput platforms and applications
To be able to benchmark systems, analyses system bottlenecks and propose solutions to eliminate them
Able to help programmers and project managers in the design, planning and governance of implementing projects of any kind.
Develop, construct, test and maintain architectures and run Sprints for development and rollout of functionalities
Data Analysis, Code development experience, ideally in Big Data Spark, Hive, Hadoop, Java, Python, PySpark,
Execute projects of various types i.e. Design, development, Implementation and migration of functional analytics Models/Business logic across architecture approaches
Work closely with Business Analysts to understand the core business problems and deliver efficient IT solutions of the product
Deployment sophisticated analytics program of code using any of cloud application.

Perks and Benefits we Provide!

Working with Highly Technical and Passionate, mission-driven people
Subsidized Meals & Snacks
Flexible Schedule
Approachable leadership
Access to various learning tools and programs
Pet Friendly
Certification Reimbursement Policy
Check out more about us on our website below!

www.datametica.com

Job description

Role : Lead Architecture (Spark, Scala, Big Data/Hadoop, Java)

Primary Location : India-Pune, Hyderabad

Experience : 7 - 12 Years

Management Level: 7

Joining Time: Immediate Joiners are preferred

Attend requirements gathering workshops, estimation discussions, design meetings and status review meetings
Experience of Solution Design and Solution Architecture for the data engineer model to build and implement Big Data Projects on-premises and on cloud.
Align architecture with business requirements and stabilizing the developed solution
Ability to build prototypes to demonstrate the technical feasibility of your vision
Professional experience facilitating and leading solution design, architecture and delivery planning activities for data intensive and high throughput platforms and applications
To be able to benchmark systems, analyses system bottlenecks and propose solutions to eliminate them
Able to help programmers and project managers in the design, planning and governance of implementing projects of any kind.
Develop, construct, test and maintain architectures and run Sprints for development and rollout of functionalities
Data Analysis, Code development experience, ideally in Big Data Spark, Hive, Hadoop, Java, Python, PySpark,
Execute projects of various types i.e. Design, development, Implementation and migration of functional analytics Models/Business logic across architecture approaches
Work closely with Business Analysts to understand the core business problems and deliver efficient IT solutions of the product
Deployment sophisticated analytics program of code using any of cloud application.

Perks and Benefits we Provide!

Working with Highly Technical and Passionate, mission-driven people
Subsidized Meals & Snacks
Flexible Schedule
Approachable leadership
Access to various learning tools and programs
Pet Friendly
Certification Reimbursement Policy
Check out more about us on our website below!

www.datametica.com

Senior Artificial Intelligence/Machine Learning Engineer

at Artivatic

1 video

3 recruiters

Posted by Layak Singh

Bengaluru (Bangalore)

3 - 10 yrs

₹6L - ₹12L / yr

Python

Machine Learning (ML)

Artificial Intelligence (AI)

Natural Language Processing (NLP)

TensorFlow

+3 more

Responsibilities :- Define the short-term tactics and long-term technology strategy.- Communicate that technical vision to technical and non-technical partners, customers and investors.- Lead the development of AI/ML related products as it matures into lean, high performing agile teams.- Scale the AI/ML teams by finding and hiring the right mix of on-shore and off-shore resources.- Work collaboratively with the business, partners, and customers to consistently deliver business value.- Own the vision and execution of developing and integrating AI & machine learning into all aspects of the platform.- Drive innovation through the use of technology and unique ways of applying it to business problems.Experience and Qualifications :- Masters or Ph.D. in AI, computer science, ML, electrical engineering or related fields (statistics, applied math, computational neuroscience)- Relevant experience leading & building teams establishing technical direction- A well-developed portfolio of past software development, composed of some mixture of professional work, open source contributions, and personal projects.- Experience in leading and developing remote and distributed teams- Think strategically and apply that through to innovative solutions- Experience with cloud infrastructure- Experience working with machine learning, artificial intelligence, and large datasets to drive insights and business value- Experience in agents architecture, deep learning, neural networks, computer vision and NLP- Experience with distributed computational frameworks (YARN, Spark, Hadoop)- Proficiency in Python, C++. Familiarity with DL frameworks (e.g. neon, TensorFlow, Caffe, etc.)Personal Attributes :- Excellent communication skills- Strong fit with the culture- Hands-on approach, self-motivated with a strong work ethic- Ability to learn quickly (technology, business models, target industries)- Creative and inspired.Superpowers we love :- Entrepreneurial spirit and a vibrant personality- Experience with lean startup build-measure-learn cycle- Vision for AI- Extensive understanding of why things are done the way they are done in agile development.- A passion for adding business valueNote: Selected candidate will be offered ESOPs too.Employment Type : Full TimeSalary : 8-10 Lacs + ESOPFunction : Systems/Product SoftwareExperience : 3 - 10 Years

Alternative Data Programmer for Equity Fund

at Sameeksha Capital

1 recruiter

Posted by Bhavin Shah

Ahmedabad

1 - 2 yrs

₹1L - ₹3L / yr

Java

Python

Data Structures

Algorithms

C++

+1 more

Looking for Alternative Data Programmer for equity fund
The programmer should be proficient in python and should be able to work totally independently. Should also have skill to work with databases and have strong capability to understand how to fetch data from various sources, organise the data and identify useful information through efficient code.
Familiarity with Python
Some examples of work:

Text search on earnings transcripts for keywords to identify future trends.

Integration of internal and external financial database

Web scraping to capture clean and organize data

Automatic updating of our financial models by importing data from machine readable formats such as XBRL

Fetching data from public databases such as RBI, NSE, BSE, DGCA and process the same.

Back-testing of data either to test historical cause and effect relation on market performance/portfolio performance, as well as back testing of our screener criteria in devising strategy

Text search on earnings transcripts for keywords to identify future trends.

Integration of internal and external financial database

Web scraping to capture clean and organize data

Automatic updating of our financial models by importing data from machine readable formats such as XBRL

Fetching data from public databases such as RBI, NSE, BSE, DGCA and process the same.

Back-testing of data either to test historical cause and effect relation on market performance/portfolio performance, as well as back testing of our screener criteria in devising strategy

Why apply to jobs via Cutshort