11+ Dataflow architecture Jobs in Pune | Dataflow architecture Job openings in Pune

CANDIDATE WILL BE DEPLOYED IN A FINANCIAL CAPTIVE ORGANIZATION @ PUNE (KHARADI)

Below are the job Details :-

Experience 10 to 18 years

Mandatory skills –

• data migration,
• data flow

The ideal candidate for this role will have the below experience and qualifications:  

• Experience of building a range of Services in a Cloud Service provider (ideally GCP)  
• Hands-on design and development of Google Cloud Platform (GCP), across a wide range of GCP services including hands on experience of GCP storage & database technologies. 
• Hands-on experience in architecting, designing or implementing solutions on GCP, K8s, and other Google technologies. Security and Compliance, e.g. IAM and cloud compliance/auditing/monitoring tools 
• Desired Skills within the GCP stack - Cloud Run, GKE, Serverless, Cloud Functions, Vision API, DLP, Data Flow, Data Fusion 
• Prior experience of migrating on-prem applications to cloud environments. Knowledge and hands on experience on Stackdriver, pub-sub, VPC, Subnets, route tables, Load balancers, firewalls both for on premise and the GCP.  
• Integrate, configure, deploy and manage centrally provided common cloud services (e.g. IAM, networking, logging, Operating systems, Containers.)  
• Manage SDN in GCP Knowledge and experience of DevOps technologies around Continuous Integration & Delivery in GCP using Jenkins.  
• Hands on experience of Terraform, Kubernetes, Docker, Stackdriver, Terraform  
• Programming experience in one or more of the following languages: Python, Ruby, Java, JavaScript, Go, Groovy, Scala  
• Knowledge or experience in DevOps tooling such as Jenkins, Git, Ansible, Splunk, Jira or Confluence, AppD, Docker, Kubernetes  
• Act as a consultant and subject matter expert for internal teams to resolve technical deployment obstacles, improve product's vision. Ensure compliance with centrally defined Security 
• Financial experience is preferred 
• Ability to learn new technologies and rapidly prototype newer concepts 
• Top-down thinker, excellent communicator, and great problem solver

Exp:- 10  to 18 years

Location:- Pune

Candidate must have experience in below.

• GCP Data Platform
• Data Processing:- Data Flow, Data Prep, Data Fusion
• Data Storage:- Big Query, Cloud Sql,
• Pub Sub, GCS Bucket

About DeepIntent:

DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioural, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences and message them on a one-to-one basis in a privacy-compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

What You’ll Do:

• Establish formal data practice for the organisation.
• Build & operate scalable and robust data architectures.
• Create pipelines for the self-service introduction and usage of new data
• Implement DataOps practices
• Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
• Engineers.
• Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
• to deploy and manage.
• Collaborate with various business stakeholders, software engineers, machine learning
• engineers, and analysts.

Who You Are:

• Experience in designing, developing and operating configurable Data pipelines serving high
• volume and velocity data.
• Experience working with public clouds like GCP/AWS.
• Good understanding of software engineering, DataOps, data architecture, Agile and
• DevOps methodologies.
• Experience building Data architectures that optimize performance and cost, whether the
• components are prepackaged or homegrown
• Proficient with SQL, Java, Spring boot, Python or JVM-based language, Bash
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
• etc. and big data databases like BigQuery, Clickhouse, etc
• Good communication skills with the ability to collaborate with both technical and non-technical
• people.
• Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

Technical/Core skills

1. Minimum 3 yrs of exp in Informatica Big data Developer(BDM) in Hadoop environment.
2. Have knowledge of informatica Power exchange (PWX).
3. Minimum 3 yrs of exp in big data querying tool like Hive and Impala.
4. Ability to designing/development of complex mappings using informatica Big data Developer.
5. Create and manage Informatica power exchange and CDC real time implementation
6. Strong Unix knowledge skills for writing shell scripts and troubleshoot of existing scripts.
7. Good knowledge of big data platforms and its framework.
8. Good to have an experience in cloudera data platform (CDP)
9. Experience with building stream processing systems using Kafka and spark
10. Excellent SQL knowledge

Soft skills :

1. Ability to work independently 
2. Strong analytical and problem solving skills
3. Attitude of learning new technology
4. Regular interaction with vendors, partners and stakeholders

Datametica is looking for talented SQL engineers who would get training & the opportunity to work on Cloud and Big Data Analytics.

Mandatory Skills:

• Strong in SQL development
• Hands-on at least one scripting language - preferably shell scripting
• Development experience in Data warehouse projects

Opportunities:

• Selected candidates will be provided training opportunities on one or more of the following: Google Cloud, AWS, DevOps Tools, Big Data technologies like Hadoop, Pig, Hive, Spark, Sqoop, Flume, and KafkaWould get a chance to be part of the enterprise-grade implementation of Cloud and Big Data systems
• Will play an active role in setting up the Modern data platform based on Cloud and Big Data
• Would be part of teams with rich experience in various aspects of distributed systems and computing

Job Role : Associate Manager (Database Development)

Key Responsibilities:

• Optimizing performances of many stored procedures, SQL queries to deliver big amounts of data under a few seconds.
• Designing and developing numerous complex queries, views, functions, and stored procedures
• to work seamlessly with the Application/Development team’s data needs.
• Responsible for providing solutions to all data related needs to support existing and new
• applications.
• Creating scalable structures to cater to large user bases and manage high workloads
• Responsible in every step from the beginning stages of the projects from requirement gathering to implementation and maintenance.
• Developing custom stored procedures and packages to support new enhancement needs.
• Working with multiple teams to design, develop and deliver early warning systems.
• Reviewing query performance and optimizing code
• Writing queries used for front-end applications
• Designing and coding database tables to store the application data
• Data modelling to visualize database structure
• Working with application developers to create optimized queries
• Maintaining database performance by troubleshooting problems.
• Accomplishing platform upgrades and improvements by supervising system programming.
• Securing database by developing policies, procedures, and controls.
• Designing and managing deep statistical systems.

Desired Skills and Experience  :

• 7+ years of experience in database development
• Minimum 4+ years of experience in PostgreSQL is a must
• Experience and in-depth knowledge in PL/SQL
• Ability to come up with multiple possible ways of solving a problem and deciding on the most optimal approach for implementation that suits the work case the most
• Have knowledge of Database Administration and have the ability and experience of using the CLI tools for administration
• Experience in Big Data technologies is an added advantage
• Secondary platforms: MS SQL 2005/2008, Oracle, MySQL
• Ability to take ownership of tasks and flexibility to work individually or in team
• Ability to communicate with teams and clients across time zones and global regions
• Good communication and self-motivated
• Should have the ability to work under pressure
• Knowledge of NoSQL and Cloud Architecture will be an advantage

• Strong Python Coding skills and OOP skills
• Should have worked on Big Data product Architecture
• Should have worked with any one of the SQL-based databases like MySQL, PostgreSQL and any one of
• NoSQL-based databases such as Cassandra, Elasticsearch etc.
• Hands on experience on frameworks like Spark RDD, DataFrame, Dataset
• Experience on development of ETL for data product
• Candidate should have working knowledge on performance optimization, optimal resource utilization, Parallelism and tuning of spark jobs
• Working knowledge on file formats: CSV, JSON, XML, PARQUET, ORC, AVRO
• Good to have working knowledge with any one of the Analytical Databases like Druid, MongoDB, Apache Hive etc.
• Experience to handle real-time data feeds (good to have working knowledge on Apache Kafka or similar tool)

• Python and Scala (Optional), Spark / PySpark, Parallel programming

ears of Exp: 3-6+ Years 
Skills: Scala, Python, Hive, Airflow, Spark

Languages: Java, Python, Shell Scripting

GCP: BigTable, DataProc,  BigQuery, GCS, Pubsub

OR
AWS: Athena, Glue, EMR, S3, Redshift

MongoDB, MySQL, Kafka

Platforms: Cloudera / Hortonworks
AdTech domain experience is a plus.
Job Type - Full Time 

• Must have 5-8 years of experience in handling data
• Must have the ability to interpret large amounts of data and to multi-task
• Must have strong knowledge of and experience with programming (Python), Linux/Bash scripting, databases(SQL, etc)
• Must have strong analytical and critical thinking to resolve business problems using data and tech
•  Must have domain familiarity and interest of – Cloud technologies (GCP/Azure Microsoft/ AWS Amazon), open-source technologies, Enterprise technologies
• Must have the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
• Must have good communication skills
• Working knowledge/exposure to ElasticSearch, PostgreSQL, Athena, PrestoDB, Jupyter Notebook