Software Engineer - Data Support

We are looking for an exceptional Software Developer for our Data Engineering India team who can-

contribute to building a world-class big data engineering stack that will be used to fuel us

Analytics and Machine Learning products. This person will be contributing to the architecture,

operation, and enhancement of:

Our petabyte-scale data platform with a key focus on finding solutions that can support

Analytics and Machine Learning product roadmap. Everyday terabytes of ingested data

need to be processed and made available for querying and insights extraction for

various use cases.

About the Organisation:

- It provides a dynamic, fun workplace filled with passionate individuals. We are at the cutting edge of advertising technology and there is never a dull moment at work.

- We have a truly global footprint, with our headquarters in Singapore and offices in Australia, United States, Germany, United Kingdom, and India.

- You will gain work experience in a global environment. We speak over 20 different languages, from more than 16 different nationalities and over 42% of our staff are multilingual.

Job Description

Position:
Software Developer, Data Engineering team
Location: Pune(Initially 100% Remote due to Covid 19 for coming 1 year)

• Our bespoke Machine Learning pipelines. This will also provide opportunities to

contribute to the prototyping, building, and deployment of Machine Learning models.

You:

• Have at least 4+ years’ Experience.
• Deep technical understanding of Java or Golang.
• Production experience with Python is a big plus, extremely valuable supporting skill for

us.

• Exposure to modern Big Data tech: Cassandra/Scylla, Kafka, Ceph, the Hadoop Stack,

Spark, Flume, Hive, Druid etc… while at the same time understanding that certain

problems may require completely novel solutions.

• Exposure to one or more modern ML tech stacks: Spark ML-Lib, TensorFlow, Keras,

GCP ML Stack, AWS Sagemaker - is a plus.

• Experience includes working in Agile/Lean model
• Experience with supporting and troubleshooting large systems
• Exposure to configuration management tools such as Ansible or Salt
• Exposure to IAAS platforms such as AWS, GCP, Azure…
• Good addition - Experience working with large-scale data
• Good addition - Good to have experience architecting, developing, and operating data

warehouses, big data analytics platforms, and high velocity data pipelines

**** Not looking for a Big Data Developer / Hadoop Developer

Title:  Data Engineer – Snowflake

Location: Mysore (Hybrid model)

Exp-2-8 yrs

Type: Full Time

Walk-in date: 25th Jan 2023 @Mysore 

Job Role: We are looking for an experienced Snowflake developer to join our team as a Data Engineer who will work as part of a team to help design and develop data-driven solutions that deliver insights to the business. The ideal candidate is a data pipeline builder and data wrangler who enjoys building data-driven systems that drive analytical solutions and building them from the ground up. You will be responsible for building and optimizing our data as well as building automated processes for production jobs. You will support our software developers, database architects, data analysts and data scientists on data initiatives

Key Roles & Responsibilities:

• Use advanced complex Snowflake/Python and SQL to extract data from source systems for ingestion into a data pipeline.
• Design, develop and deploy scalable and efficient data pipelines.
• Analyze and assemble large, complex datasets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements. For example: automating manual processes, optimizing data delivery, re-designing data platform infrastructure for greater scalability.
• Build required infrastructure for optimal extraction, loading, and transformation (ELT) of data from various data sources using AWS and Snowflake leveraging Python or SQL technologies.
• Monitor cloud-based systems and components for availability, performance, reliability, security and efficiency
• Create and configure appropriate cloud resources to meet the needs of the end users.
• As needed, document topology, processes, and solution architecture.
• Share your passion for staying on top of tech trends, experimenting with and learning new technologies

Qualifications & Experience

Qualification & Experience Requirements:

• Bachelor's degree in computer science, computer engineering, or a related field.
• 2-8 years of experience working with Snowflake
• 2+ years of experience with the AWS services.
• Candidate should able to write the stored procedure and function in Snowflake.
• At least 2 years’ experience in snowflake developer.
• Strong SQL Knowledge.
• Data injection in snowflake using Snowflake procedure.
• ETL Experience is Must (Could be any tool)
• Candidate should be aware of snowflake architecture.
• Worked on the Migration project
• DW Concept (Optional)
• Experience with cloud data storage and compute components including lambda functions, EC2s, containers.
• Experience with data pipeline and workflow management tools: Airflow, etc.
• Experience cleaning, testing, and evaluating data quality from a wide variety of ingestible data sources
• Experience working with Linux and UNIX environments.
• Experience with profiling data, with and without data definition documentation

• Familiar with Git
• Familiar with issue tracking systems like JIRA (Project Management Tool) or Trello.
• Experience working in an agile environment.

Desired Skills:

• Experience in Snowflake. Must be willing to be Snowflake certified in the first 3 months of employment.
• Experience with a stream-processing system: Snowpipe
• Working knowledge of AWS or Azure
• Experience in migrating from on-prem to cloud systems

Responsibilities :

• Involve in planning, design, development and maintenance of large-scale data repositories, pipelines, analytical solutions and knowledge management strategy
• Build and maintain optimal data pipeline architecture to ensure scalability, connect operational systems data for analytics and business intelligence (BI) systems
• Build data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
• Reporting and obtaining insights from large data chunks on import/export and communicating relevant pointers for helping in decision-making
• Preparation, analysis, and presentation of reports to the management for further developmental activities
• Anticipate, identify and solve issues concerning data management to improve data quality

Requirements :

• Ability to build and maintain ETL pipelines 
• Technical Business Analysis experience and hands-on experience developing functional spec
• Good understanding of Data Engineering principles including data modeling methodologies
• Sound understanding of PostgreSQL
• Strong analytical and interpersonal skills as well as reporting capabilities

• Collaborate with the business teams to understand the data environment in the organization; develop and lead the Data Scientists team to test and scale new algorithms through pilots and subsequent scaling up of the solutions
• Influence, build and maintain the large-scale data infrastructure required for the AI projects, and integrate with external IT infrastructure/service
• Act as the single point source for all data related queries; strong understanding of internal and external data sources; provide inputs in deciding data-schemas
• Design, develop and maintain the framework for the analytics solutions pipeline
• Provide inputs to the organization’s initiatives on data quality and help implement frameworks and tools for the various related initiatives
• Work in cross-functional teams of software/machine learning engineers, data scientists, product managers, and others to build the AI ecosystem
• Collaborate with the external organizations including vendors, where required, in respect of all data-related queries as well as implementation initiatives

Title: Data Engineer (Azure) (Location: Gurgaon/Hyderabad)

Salary: Competitive as per Industry Standard

We are expanding our Data Engineering Team and hiring passionate professionals with extensive

knowledge and experience in building and managing large enterprise data and analytics platforms. We

are looking for creative individuals with strong programming skills, who can understand complex

business and architectural problems and develop solutions. The individual will work closely with the rest

of our data engineering and data science team in implementing and managing Scalable Smart Data

Lakes, Data Ingestion Platforms, Machine Learning and NLP based Analytics Platforms, Hyper-Scale

Processing Clusters, Data Mining and Search Engines.

What You’ll Need:

• 3+ years of industry experience in creating and managing end-to-end Data Solutions, Optimal

Data Processing Pipelines and Architecture dealing with large volume, big data sets of varied

data types.

• Proficiency in Python, Linux and shell scripting.
• Strong knowledge of working with PySpark dataframes, Pandas dataframes for writing efficient pre-processing and other data manipulation tasks.
  ● Strong experience in developing the infrastructure required for data ingestion, optimal

extraction, transformation, and loading of data from a wide variety of data sources using tools like Azure Data Factory,  Azure Databricks (or Jupyter notebooks/ Google Colab) (or other similiar tools).

• Working knowledge of github or other version control tools.
• Experience with creating Restful web services and API platforms.
• Work with data science and infrastructure team members to implement practical machine

learning solutions and pipelines in production.

• Experience with cloud providers like Azure/AWS/GCP.
• Experience with SQL and NoSQL databases. MySQL/ Azure Cosmosdb / Hbase/MongoDB/ Elasticsearch etc.
• Experience with stream-processing systems: Spark-Streaming, Kafka etc and working experience with event driven architectures.
• Strong analytic skills related to working with unstructured datasets.

Good to have (to filter or prioritize candidates)

• Experience with testing libraries such as pytest for writing unit-tests for the developed code.
• Knowledge of Machine Learning algorithms and libraries would be good to have,

implementation experience would be an added advantage.

• Knowledge and experience of Datalake, Dockers and Kubernetes would be good to have.
• Knowledge of Azure functions , Elastic search etc will be good to have.

• Having experience with model versioning (mlflow) and data versioning will be beneficial
• Having experience with microservices libraries or with python libraries such as flask for hosting ml services and models would be great.

• Ensure and own Data integrity across distributed systems.
• Extract, Transform and Load data from multiple systems for reporting into BI platform.
• Create Data Sets and Data models to build intelligence upon.
• Develop and own various integration tools and data points.
• Hands-on development and/or design within the project in order to maintain timelines.
• Work closely with the Project manager to deliver on business requirements OTIF (on time in full)
• Understand the cross-functional business data points thoroughly and be SPOC for all data-related queries.
• Work with both Web Analytics and Backend Data analytics.
• Support the rest of the BI team in generating reports and analysis
• Quickly learn and use Bespoke & third party SaaS reporting tools with little documentation.
• Assist in presenting demos and preparing materials for Leadership.

• Strong experience in Datawarehouse modeling techniques and SQL queries

• A good understanding of designing, developing, deploying, and maintaining Power BI report solutions
• Ability to create KPIs, visualizations, reports, and dashboards based on business requirement
• Knowledge and experience in prototyping, designing, and requirement analysis
• Be able to implement row-level security on data and understand application security layer models in Power BI
• Proficiency in making DAX queries in Power BI desktop.
• Expertise in using advanced level calculations on data sets
• Experience in the Fintech domain and stakeholder management.