Data Engineer

at prevaj consultants pvt ltd

DP
Posted by sanofar shifa
icon
Chennai
icon
5 - 15 yrs
icon
₹2L - ₹15L / yr
icon
Full time
Skills
PySpark
Data engineering
Big Data
Hadoop
Spark
Google Cloud Platform (GCP)

  • 5+ years of experience building real-time and distributed system architecture, from whiteboard to production
  • Strong programming skills in Python, Scala  and SQL.
  • Versatility. Experience across the entire spectrum of data engineering, including:
  • Data stores (e.g., AWS RDS, AWS Athena, AWS Aurora, AWS Redshift)
  • Data pipeline and workflow orchestration tools (e.g., Azkaban, Airflow)
  • Data processing technologies (e.g., Spark, Pentaho)
  • Deployment and monitoring large database clusters in public cloud platforms (e.g., Docker, Terraform, Datadog)
  • Creating ETL or ELT pipelines that transform and process petabytes of structured and unstructured data in real-time
  • Industry experience building and productionizing innovative end-to-end Machine Learning systems is a plus.
Read more

About prevaj consultants pvt ltd

Founded
2010
Type
Products & Services
Size
20-100 employees
Stage
Profitable
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Data Engineer

at Top startup of India - News App

Agency job
via Jobdost
Linux/Unix
Python
Hadoop
Apache Spark
MongoDB
Data flow
BigQuery
NOSQL Databases
Google Cloud Platform (GCP)
icon
Noida
icon
2 - 5 yrs
icon
₹20L - ₹35L / yr
Responsibilities
● Create and maintain optimal data pipeline architecture.
● Assemble large, complex data sets that meet functional / non-functional
business requirements.
● Building and optimizing ‘big data’ data pipelines, architectures and data sets.
● Maintain, organize & automate data processes for various use cases.
● Identifying trends, doing follow-up analysis, preparing visualizations.
● Creating daily, weekly and monthly reports of product KPIs.
● Create informative, actionable and repeatable reporting that highlights
relevant business trends and opportunities for improvement.

Required Skills And Experience:
● 2-5 years of work experience in data analytics- including analyzing large data sets.
● BTech in Mathematics/Computer Science
● Strong analytical, quantitative and data interpretation skills.
● Hands-on experience with Python, Apache Spark, Hadoop, NoSQL
databases(MongoDB preferred), Linux is a must.
● Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
● Experience with Google Cloud Data Analytics Products such as BigQuery, Dataflow, Dataproc etc. (or similar cloud-based platforms).
● Experience working within a Linux computing environment, and use of
command-line tools including knowledge of shell/Python scripting for
automating common tasks.
● Previous experience working at startups and/or in fast-paced environments.
● Previous experience as a data engineer or in a similar role.
Read more
Job posted by
Sathish Kumar

Senior Software Engineer Data

at Deepintent

Founded 2015  •  Product  •  20-100 employees  •  Profitable
SQL
Python
JVM
Google Cloud Platform (GCP)
Spark
icon
Pune
icon
3 - 6 yrs
icon
Best in industry
About DeepIntent:
DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioral, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences, and message them on a one-to-one basis in a privacy compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

Roles and Responsibilities
  • Establish formal data practice for the organisation.
  • Build & operate scalable and robust data architectures.
  • Create pipelines for the self-service introduction and usage of new data
  • Implement DataOps practices
  • Design, Develop, operate Data Pipelines which support Data scientists and machine learning Engineers.
  • Build simple, highly reliable Data storage, ingestion, transformation solutions which are easy to deploy and manage.
  • Collaborate with various business stakeholders, software engineers, machine learning engineers, analysts.
  •  
Desired Skills
  • Experience in designing, developing and operating configurable Data pipelines serving high volume and velocity data.
  • Experience working with public clouds like GCP/AWS.
  • Good understanding of software engineering, DataOps, and data architecture, Agile and DevOps methodologies.
  • Experience building Data architectures that optimize performance and cost, whether the components are prepackaged or homegrown
  • Proficient with SQL,Python or JVM based language, Bash.
  • Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc.and big data databases like BigQuery, Clickhouse, etc
  • Good communication skills with ability to collaborate with both technical and non technical people.
  • Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious.
 
 
 
 
 
 

 

Read more
Job posted by
Indrajeet Deshmukh

Opening for Lead Data Architect with Searce

at Searce Inc

Founded 2004  •  Products & Services  •  100-1000 employees  •  Profitable
PySpark
Data engineering
Big Data
Hadoop
Spark
Amazon Web Services (AWS)
Google Cloud Platform (GCP)
Technical Architecture
Scala
Architecture
icon
Mumbai, Pune
icon
8 - 14 yrs
icon
₹10L - ₹15L / yr
Job Responsibilities
1. Understand the business problem and translate these to data services and
engineering outcomes.
2. Expertise in working on cloud application designs, cloud approval plans, and
systems required to manage cloud storage.
3. Explore new technologies and learn new techniques to solve business problems
creatively
4. Collaborate with different teams - engineering and business, to build better data
products
5. Regularly evaluate cloud applications, hardware, and software.
6. Respond to technical issues in a professional and timely manner.
7. Identify the top cloud architecture solutions to successfully meet the strategic
needs of the company.
8. Offer guidance in infrastructure movement techniques including bulk application
transfers into the cloud.
9. Manage team and handle delivery of 2-3 projects
JD | Data Architect 24-Aug-2021
Solving for better 3 of 7

Qualifications
Is Education overrated? Yes. We believe so. But there is no way to locate you
otherwise. So we might look for at least a computer science, computer engineering,
information technology, or relevant field along with:

1. Over 4-6 years of experience in Data handling
2. Hands-on experience of any one programming language (Python, Java, Scala)
3. Understanding of SQL is must
4. Big data (Hadoop, Hive, Yarn, Sqoop)
5. MPP platforms (Spark, Presto)
6. Data-pipeline & scheduler tool (Ozzie, Airflow, Nifi)
7. Streaming engines (Kafka, Storm, Spark Streaming)
8. Any Relational database or DW experience
9. Any ETL tool experience
10. Hands-on experience in pipeline design, ETL and application development
11. Hands-on experience in cloud platforms like AWS, GCP etc.
12. Good communication skills and strong analytical skills
13. Experience in team handling and project delivery
Read more
Job posted by
Reena Bandekar

Principal Data Engineer

at AI-powered cloud-based SaaS solution provider

Agency job
via wrackle
Data engineering
Big Data
Spark
Apache Kafka
Cassandra
Apache ZooKeeper
Data engineer
Hadoop
HDFS
MapReduce
AWS CloudFormation
EMR
Amazon EMR
Amazon S3
Apache Spark
Java
PythonAnywhere
Test driven development (TDD)
Cloud Computing
Google Cloud Platform (GCP)
Agile/Scrum
OOD
Software design
Architecture
YARN
icon
Bengaluru (Bangalore)
icon
8 - 15 yrs
icon
₹25L - ₹60L / yr
Responsibilities

● Able to contribute to the gathering of functional requirements, developing technical
specifications, and test case planning
● Demonstrating technical expertise, and solving challenging programming and design
problems
● 60% hands-on coding with architecture ownership of one or more products
● Ability to articulate architectural and design options, and educate development teams and
business users
● Resolve defects/bugs during QA testing, pre-production, production, and post-release
patches
● Mentor and guide team members
● Work cross-functionally with various bidgely teams including product management, QA/QE,
various product lines, and/or business units to drive forward results

Requirements
● BS/MS in computer science or equivalent work experience
● 8-12 years’ experience designing and developing applications in Data Engineering
● Hands-on experience with Big data EcoSystems.
● Past experience with Hadoop,Hdfs,Map Reduce,YARN,AWS Cloud, EMR, S3, Spark, Cassandra,
Kafka, Zookeeper
● Expertise with any of the following Object-Oriented Languages (OOD): Java/J2EE,Scala,
Python
● Ability to lead and mentor technical team members
● Expertise with the entire Software Development Life Cycle (SDLC)
● Excellent communication skills: Demonstrated ability to explain complex technical issues to
both technical and non-technical audiences
● Expertise in the Software design/architecture process
● Expertise with unit testing & Test-Driven Development (TDD)
● Business Acumen - strategic thinking & strategy development
● Experience on Cloud or AWS is preferable
● Have a good understanding and ability to develop software, prototypes, or proofs of
concepts (POC's) for various Data Engineering requirements.
● Experience with Agile Development, SCRUM, or Extreme Programming methodologies
Read more
Job posted by
Naveen Taalanki

Database Developer

at Mobile Programming LLC

Founded 1998  •  Services  •  100-1000 employees  •  Profitable
PySpark
Data engineering
Big Data
Hadoop
Spark
Databases
Google Cloud Platform (GCP)
icon
Remote only
icon
4 - 7 yrs
icon
₹10L - ₹14L / yr

Job ID: RP100

Work Location: Remote

Required Experience: 4 to 7 years

Job Description

  • Must have Google Cloud Big Query experience
  • Strong experience with data analysis, data modeling and governance, with excellent analytical and problem-solving abilities
  • Good knowledge of Data Warehouses, data flow ETL pipelines
  • Design, configuration/administration of database software in Cloud platform.
  • Monitoring, Troubleshooting, and Performance tuning the DB objects.
  • Experience on Table Partition, Clustered Table, Materialized View, External Tables etc.

Anyone RDBMS technologies

  • Good experience in DB design with knowledge of ER Diagram, PK/FK, Stored procedure, Function, Triggers, and Indexes.
  • Understanding the requirement of the App team and creating the necessary DB objects by following the best practices.
  • Managing logins and database users, as well as database roles, application roles, and other security principles within the database.
  • Deep knowledge about Indexes, Performance tuning, and Complex SQL Query patterns.
  • Monitoring, Tuning, and Troubleshooting the database-related issues.

About Us:

Mobile programming LLC is a US-based digital transformation company. We help enterprises transform ideas into innovative and intelligent solutions, governing the Internet of Things, Digital Commerce, Business Intelligence Analytics, and Cloud Programming. Bring your challenges to us, we will give you the smartest solutions. From conceptualizing and engineering to advanced manufacturing, we help customers build and scale products fit for the global marketplace.

Mobile programming LLC has offices located in Los Angeles, San Jose, Glendale, San Diego, Phoenix, Plano, New York, Fort Lauderdale, and Boston. Mobile programming is SAP Preferred Vendor, Apple Adjunct Partner, Google Empaneled Mobile Vendor, and Microsoft Gold Certified Partner.

Read more
Job posted by
Garima Walia

Data Engineer (Azure)

at Scry Analytics

Founded 2015  •  Product  •  100-500 employees  •  Profitable
PySpark
Data engineering
Big Data
Hadoop
Spark
Windows Azure
Amazon Web Services (AWS)
Google Cloud Platform (GCP)
SQL
NOSQL Databases
Apache Kafka
icon
Remote only
icon
3 - 8 yrs
icon
₹15L - ₹20L / yr

Title: Data Engineer (Azure) (Location: Gurgaon/Hyderabad)

Salary: Competitive as per Industry Standard

We are expanding our Data Engineering Team and hiring passionate professionals with extensive

knowledge and experience in building and managing large enterprise data and analytics platforms. We

are looking for creative individuals with strong programming skills, who can understand complex

business and architectural problems and develop solutions. The individual will work closely with the rest

of our data engineering and data science team in implementing and managing Scalable Smart Data

Lakes, Data Ingestion Platforms, Machine Learning and NLP based Analytics Platforms, Hyper-Scale

Processing Clusters, Data Mining and Search Engines.

What You’ll Need:

  • 3+ years of industry experience in creating and managing end-to-end Data Solutions, Optimal

Data Processing Pipelines and Architecture dealing with large volume, big data sets of varied

data types.

  • Proficiency in Python, Linux and shell scripting.
  • Strong knowledge of working with PySpark dataframes, Pandas dataframes for writing efficient pre-processing and other data manipulation tasks.
    ● Strong experience in developing the infrastructure required for data ingestion, optimal

extraction, transformation, and loading of data from a wide variety of data sources using tools like Azure Data Factory,  Azure Databricks (or Jupyter notebooks/ Google Colab) (or other similiar tools).

  • Working knowledge of github or other version control tools.
  • Experience with creating Restful web services and API platforms.
  • Work with data science and infrastructure team members to implement practical machine

learning solutions and pipelines in production.

  • Experience with cloud providers like Azure/AWS/GCP.
  • Experience with SQL and NoSQL databases. MySQL/ Azure Cosmosdb / Hbase/MongoDB/ Elasticsearch etc.
  • Experience with stream-processing systems: Spark-Streaming, Kafka etc and working experience with event driven architectures.
  • Strong analytic skills related to working with unstructured datasets.

 

Good to have (to filter or prioritize candidates)

  • Experience with testing libraries such as pytest for writing unit-tests for the developed code.
  • Knowledge of Machine Learning algorithms and libraries would be good to have,

implementation experience would be an added advantage.

  • Knowledge and experience of Datalake, Dockers and Kubernetes would be good to have.
  • Knowledge of Azure functions , Elastic search etc will be good to have.

 

  • Having experience with model versioning (mlflow) and data versioning will be beneficial
  • Having experience with microservices libraries or with python libraries such as flask for hosting ml services and models would be great.
Read more
Job posted by
Siddarth Thakur

AI Engineer

at StatusNeo

Founded 2020  •  Products & Services  •  100-1000 employees  •  Profitable
Artificial Intelligence (AI)
Amazon Web Services (AWS)
Windows Azure
Hadoop
Scala
Python
Google Cloud Platform (GCP)
postgres
icon
Gurugram, Hyderabad, Bengaluru (Bangalore)
icon
1 - 3 yrs
icon
₹3L - ₹12L / yr


·       Build data products and processes alongside the core engineering and technology team.

·       Collaborate with senior data scientists to curate, wrangle, and prepare data for use in their advanced analytical models

·       Integrate data from a variety of sources, assuring that they adhere to data quality and accessibility standards

·       Modify and improve data engineering processes to handle ever larger, more complex, and more types of data sources and pipelines

·       Use Hadoop architecture and HDFS commands to design and optimize data queries at scale

·       Evaluate and experiment with novel data engineering tools and advises information technology leads and partners about new capabilities to determine optimal solutions for particular technical problems or designated use cases .
Read more
Job posted by
Alex P

Data Engineer

at MNC Company - Product Based

Agency job
via Bharat Headhunters
Data Warehouse (DWH)
Informatica
ETL
Python
Google Cloud Platform (GCP)
SQL
AIrflow
icon
Bengaluru (Bangalore), Chennai, Hyderabad, Pune, Delhi, Gurugram, Noida, Ghaziabad, Faridabad
icon
5 - 9 yrs
icon
₹10L - ₹15L / yr

Job Responsibilities

  • Design, build & test ETL processes using Python & SQL for the corporate data warehouse
  • Inform, influence, support, and execute our product decisions
  • Maintain advertising data integrity by working closely with R&D to organize and store data in a format that provides accurate data and allows the business to quickly identify issues.
  • Evaluate and prototype new technologies in the area of data processing
  • Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
  • High energy level, strong team player and good work ethic
  • Data analysis, understanding of business requirements and translation into logical pipelines & processes
  • Identification, analysis & resolution of production & development bugs
  • Support the release process including completing & reviewing documentation
  • Configure data mappings & transformations to orchestrate data integration & validation
  • Provide subject matter expertise
  • Document solutions, tools & processes
  • Create & support test plans with hands-on testing
  • Peer reviews of work developed by other data engineers within the team
  • Establish good working relationships & communication channels with relevant departments

 

Skills and Qualifications we look for

  • University degree 2.1 or higher (or equivalent) in a relevant subject. Master’s degree in any data subject will be a strong advantage.
  • 4 - 6 years experience with data engineering.
  • Strong coding ability and software development experience in Python.
  • Strong hands-on experience with SQL and Data Processing.
  • Google cloud platform (Cloud composer, Dataflow, Cloud function, Bigquery, Cloud storage, dataproc)
  • Good working experience in any one of the ETL tools (Airflow would be preferable).
  • Should possess strong analytical and problem solving skills.
  • Good to have skills - Apache pyspark, CircleCI, Terraform
  • Motivated, self-directed, able to work with ambiguity and interested in emerging technologies, agile and collaborative processes.
  • Understanding & experience of agile / scrum delivery methodology

 

Read more
Job posted by
Ranjini C. N

Data Analyst

at A Product development Organisation

Agency job
via Millions Advisory
Python
Big Data
Amazon Web Services (AWS)
Windows Azure
Google Cloud Platform (GCP)
Elastic Search
Linux/Unix
Shell/Bash
icon
Pune
icon
5 - 8 yrs
icon
₹10L - ₹17L / yr
  • Must have 5-8 years of experience in handling data
  • Must have the ability to interpret large amounts of data and to multi-task
  • Must have strong knowledge of and experience with programming (Python), Linux/Bash scripting, databases(SQL, etc)
  • Must have strong analytical and critical thinking to resolve business problems using data and tech
  •  Must have domain familiarity and interest of – Cloud technologies (GCP/Azure Microsoft/ AWS Amazon), open-source technologies, Enterprise technologies
  • Must have the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
  • Must have good communication skills
  • Working knowledge/exposure to ElasticSearch, PostgreSQL, Athena, PrestoDB, Jupyter Notebook
Read more
Job posted by
Vasuki N

Data Engineer

at Nisum consulting

Founded 2000  •  Products & Services  •  100-1000 employees  •  Profitable
Big Data
Hadoop
Spark
Apache Kafka
Scala
Amazon Web Services (AWS)
Windows Azure
Google Cloud Platform (GCP)
Python
icon
Hyderabad
icon
4 - 12 yrs
icon
₹1L - ₹20L / yr
  • 5+ years of experience in a Data Engineer role
  • Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
  • Experience with big data tools: Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases such as Cassandra.
  • Experience with AWS cloud services: EC2, EMR, Athena
  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
  • Advanced SQL knowledge and experience working with relational databases, query authoring (SQL) as well as familiarity with unstructured datasets.
  • Deep problem-solving skills to perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Read more
Job posted by
Sameena Shaik
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at prevaj consultants pvt ltd?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort