Big Data Architect

About Us:

Cognitio Analytics is an award-winning, niche service provider that offers digital transformation solutions powered by AI and machine learning. We help clients realize the full potential of their data assets and the investments made in related technologies, be it analytics and big data platforms, digital technologies or your people. Our solutions include Health Analytics powered by Cognitio’s Health Data Factory that drives better care outcomes and higher ROI on care spending. We specialize in providing Data Governance solutions that help effectively use data in a consistent and compliant manner. Additionally, our smart intelligence solutions enable a deeper understanding of operations through the use of data science and advanced solutions like process mining technologies. We have offices in New Jersey and Connecticut in the USA and in Gurgaon in India.

What we're looking for:

• Ability in data modelling, design, build and deploy DW/BI systems for Insurance, Health Care, Banking, etc.
• Performance tuning of ETL process and SQL queries and recommend & implement ETL and query tuning techniques.
• Develop and create transformation queries, views, and stored procedures for ETL processes, and process automations.
• Translate business needs into technical specifications, evaluate, and improve existing BI systems.
• Use business intelligence and visualization software (e.g., Tableau, Qlik Sense, Power BI etc.) to empower customers to drive their analytics and reporting
• Develop and update technical documentation for BI systems.

Key Technical Skills:

• Hands on experience in MS SQL Server & MSBI (SSIS/SSRS/SSAS), with understanding of database concepts, star schema, SQL Tuning, OLAP, Databricks, Hadoop, Spark, cloud technologies.
• Experience in designing and building complete ETL / SSIS processes and transforming data for ODS, Staging, and Data Warehouse
• Experience building self-service reporting solutions using business intelligence software (e.g., Tableau, Qlik Sense, Power BI etc.)

About Telstra

Telstra is Australia’s leading telecommunications and technology company, with operations in more than 20 countries, including In India where we’re building a new Innovation and Capability Centre (ICC) in Bangalore.

We’re growing, fast, and for you that means many exciting opportunities to develop your career at Telstra. Join us on this exciting journey, and together, we’ll reimagine the future.

Why Telstra?

• We're an iconic Australian company with a rich heritage that's been built over 100 years. Telstra is Australia's leading Telecommunications and Technology Company. We've been operating internationally for more than 70 years.
• International presence spanning over 20 countries.
• We are one of the 20 largest telecommunications providers globally
• At Telstra, the work is complex and stimulating, but with that comes a great sense of achievement. We are shaping the tomorrow's modes of communication with our innovation driven teams.

Telstra offers an opportunity to make a difference to lives of millions of people by providing the choice of flexibility in work and a rewarding career that you will be proud of!

About the team

Being part of Networks & IT means you'll be part of a team that focuses on extending our network superiority to enable the continued execution of our digital strategy.

With us, you'll be working with world-leading technology and change the way we do IT to ensure business needs drive priorities, accelerating our digitisation programme.

Focus of the role

Any new engineer who comes into data chapter would be mostly into developing reusable data processing and storage frameworks that can be used across data platform.

About you

To be successful in the role, you'll bring skills and experience in:-

Essential 

• Hands-on experience in Spark Core, Spark SQL, SQL/Hive/Impala, Git/SVN/Any other VCS and Data warehousing
• Skilled in the Hadoop Ecosystem(HDP/Cloudera/MapR/EMR etc)
• Azure data factory/Airflow/control-M/Luigi
• PL/SQL
• Exposure to NOSQL(Hbase/Cassandra/GraphDB(Neo4J)/MongoDB)
• File formats (Parquet/ORC/AVRO/Delta/Hudi etc.)
• Kafka/Kinesis/Eventhub

Highly Desirable

Experience and knowledgeable on the following:

• Spark Streaming
• Cloud exposure (Azure/AWS/GCP)
• Azure data offerings - ADF, ADLS2, Azure Databricks, Azure Synapse, Eventhubs, CosmosDB etc.
• Presto/Athena
• Azure DevOps
• Jenkins/ Bamboo/Any similar build tools
• Power BI
• Prior experience in building or working in team building reusable frameworks,
• Data modelling.
• Data Architecture and design principles. (Delta/Kappa/Lambda architecture)
• Exposure to CI/CD
• Code Quality - Static and Dynamic code scans
• Agile SDLC      

If you've got a passion to innovate, succeed as part of a great team, and looking for the next step in your career, we'd welcome you to apply!

___________________________

We’re committed to building a diverse and inclusive workforce in all its forms. We encourage applicants from diverse gender, cultural and linguistic backgrounds and applicants who may be living with a disability. We also offer flexibility in all our roles, to ensure everyone can participate.

To learn more about how we support our people, including accessibility adjustments we can provide you through the recruitment process, visit tel.st/thrive.

Should have Passion to learn and adapt new technologies, understanding,

solving/troubleshooting issues and risks, able to make informed decisions and ability to

lead the projects.

Your Qualifications

• 2-5 Years’ Experience with functional programming
• Experience with functional programming using Scala with Spark framework.
• Strong understanding of Object-oriented programming, data structures and algorithms
• Good experience in any of the cloud platforms (Azure, AWS, GCP) etc.,
• Experience with distributed (multi-tiered) systems, relational databases and NoSql storage solutions
• Desire to learn new technologies and languages
• Participation in software design, development, and code reviews
• High level of proficiency with Computer Science/Software Engineering knowledge and contribution to the technical skills growth of other team members

Your Responsibility

• Design, build and configure applications to meet business process and application requirements
• Proactively identify and communicate potential issues and concerns and recommend/implement alternative solutions as appropriate.
• Troubleshooting & Optimization of existing solution

BRIEF DESCRIPTION:

At-least 1 year of Python, Spark, SQL, data engineering experience

Primary Skillset: PySpark, Scala/Python/Spark, Azure Synapse, S3, RedShift/Snowflake

Relevant Experience: Legacy ETL job Migration to AWS Glue / Python & Spark combination

ROLE SCOPE:

Reverse engineer the existing/legacy ETL jobs

Create the workflow diagrams and review the logic diagrams with Tech Leads

Write equivalent logic in Python & Spark

Unit test the Glue jobs and certify the data loads before passing to system testing

Follow the best practices, enable appropriate audit & control mechanism

Analytically skillful, identify the root causes quickly and efficiently debug issues

Take ownership of the deliverables and support the deployments

REQUIREMENTS:

Create data pipelines for data integration into Cloud stacks eg. Azure Synapse

Code data processing jobs in Azure Synapse Analytics, Python, and Spark

Experience in dealing with structured, semi-structured, and unstructured data in batch and real-time environments.

Should be able to process .json, .parquet and .avro files

PREFERRED BACKGROUND:

Tier1/2 candidates from IIT/NIT/IIITs

However, relevant experience, learning attitude takes precedence

Job ID: RP100

Work Location: Remote

Required Experience: 4 to 7 years

Job Description

• Must have Google Cloud Big Query experience
• Strong experience with data analysis, data modeling and governance, with excellent analytical and problem-solving abilities
• Good knowledge of Data Warehouses, data flow ETL pipelines
• Design, configuration/administration of database software in Cloud platform.
• Monitoring, Troubleshooting, and Performance tuning the DB objects.
• Experience on Table Partition, Clustered Table, Materialized View, External Tables etc.

Anyone RDBMS technologies

• Good experience in DB design with knowledge of ER Diagram, PK/FK, Stored procedure, Function, Triggers, and Indexes.
• Understanding the requirement of the App team and creating the necessary DB objects by following the best practices.
• Managing logins and database users, as well as database roles, application roles, and other security principles within the database.
• Deep knowledge about Indexes, Performance tuning, and Complex SQL Query patterns.
• Monitoring, Tuning, and Troubleshooting the database-related issues.

About Us:

Mobile programming LLC is a US-based digital transformation company. We help enterprises transform ideas into innovative and intelligent solutions, governing the Internet of Things, Digital Commerce, Business Intelligence Analytics, and Cloud Programming. Bring your challenges to us, we will give you the smartest solutions. From conceptualizing and engineering to advanced manufacturing, we help customers build and scale products fit for the global marketplace.

Mobile programming LLC has offices located in Los Angeles, San Jose, Glendale, San Diego, Phoenix, Plano, New York, Fort Lauderdale, and Boston. Mobile programming is SAP Preferred Vendor, Apple Adjunct Partner, Google Empaneled Mobile Vendor, and Microsoft Gold Certified Partner.

• Strong Python Coding skills and OOP skills
• Should have worked on Big Data product Architecture
• Should have worked with any one of the SQL-based databases like MySQL, PostgreSQL and any one of
• NoSQL-based databases such as Cassandra, Elasticsearch etc.
• Hands on experience on frameworks like Spark RDD, DataFrame, Dataset
• Experience on development of ETL for data product
• Candidate should have working knowledge on performance optimization, optimal resource utilization, Parallelism and tuning of spark jobs
• Working knowledge on file formats: CSV, JSON, XML, PARQUET, ORC, AVRO
• Good to have working knowledge with any one of the Analytical Databases like Druid, MongoDB, Apache Hive etc.
• Experience to handle real-time data feeds (good to have working knowledge on Apache Kafka or similar tool)

• Python and Scala (Optional), Spark / PySpark, Parallel programming

To be considered as a candidate for a Senior Data Engineer position, a person must have a proven track record of architecting data solutions on current and advanced technical platforms. They must have leadership abilities to lead a team providing data centric solutions with best practices and modern technologies in mind. They look to build collaborative relationships across all levels of the business and the IT organization. They possess analytic and problem-solving skills and have the ability to research and provide appropriate guidance for synthesizing complex information and extract business value. Have the intellectual curiosity and ability to deliver solutions with creativity and quality. Effectively work with business and customers to obtain business value for the requested work. Able to communicate technical results to both technical and non-technical users using effective story telling techniques and visualizations. Demonstrated ability to perform high quality work with innovation both independently and collaboratively.