Big data Jobs

Fast growing SA Technology Company is seeking an experienced and highly skilled Senior Hadoop Developer with expertise in Hadoop, Spark, and Python to join their team.

The ideal candidate should have extensive hands-on experience with Hadoop ecosystem technologies, strong programming skills in Python, and a deep understanding of distributed systems, big data processing, and Spark.

Responsibilities:

• Design, develop, and optimize Hadoop-based solutions for large-scale data processing and analysis.
• Collaborate with stakeholders to understand business requirements and translate them into technical specifications and Hadoop architecture designs.
• Write high-quality, scalable, and efficient code in Python and other relevant programming languages for data processing, data ingestion, and data transformations.
• Implement and maintain data processing workflows using Apache Spark, leveraging its distributed computing capabilities for efficient data analysis and machine learning tasks.
• Collaborate with data engineers and data scientists to integrate machine learning models and algorithms into Hadoop-based systems.
• Optimize performance and troubleshoot issues related to Hadoop, Spark, and Python-based applications and workflows.
• Ensure data security and compliance by implementing appropriate access controls, data encryption, and data governance practices.
• Monitor Hadoop clusters and Spark applications, proactively identifying and addressing performance bottlenecks, scalability issues, and system failures.
• Stay updated with the latest advancements and best practices in Hadoop, Spark, Python, and big data technologies, and provide guidance and mentorship to junior team members.

Requirements:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 4+ years of professional experience as a Hadoop Developer, with a focus on Hadoop, Spark, and Python.
• Strong hands-on experience with Hadoop ecosystem technologies, such as HDFS, MapReduce, Hive, Pig, HBase, and YARN.
• In-depth knowledge and practical experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark MLlib.
• Proficiency in programming languages such as Python, Java, or Scala, with a strong emphasis on Python.
• Solid understanding of distributed systems, parallel computing, and big data processing concepts.
• Experience with data ingestion, ETL processes, and data integration using tools like Apache Nifi, Kafka, or Sqoop.
• Familiarity with cloud-based big data platforms, such as AWS EMR or Google Cloud Dataproc, is a plus.
• Strong problem-solving skills and the ability to work independently as well as part of a collaborative team.
• Excellent communication skills and the ability to effectively articulate complex technical concepts to both technical and non-technical stakeholders.

Qualifications & Experience:

▪ 2 - 4 years overall experience in ETLs, data pipeline, Data Warehouse development and database design
▪ Software solution development using Hadoop Technologies such as MapReduce, Hive, Spark, Kafka, Yarn/Mesos etc.
▪ Expert in SQL, worked on advanced SQL for at least 2+ years
▪ Good development skills in Java, Python or other languages
▪ Experience with EMR, S3
▪ Knowledge and exposure to BI applications, e.g. Tableau, Qlikview
▪ Comfortable working in an agile environment

• KSQL
• Data Engineering spectrum (Java/Spark)
• Spark Scala / Kafka Streaming
• Confluent Kafka components
• Basic understanding of Hadoop

We are seeking an experienced Senior Data Platform Engineer to join our team. The ideal candidate should have extensive experience with Pyspark, Airflow, Presto, Hive, Kafka and Debezium, and should be passionate about developing scalable and reliable data platforms.

Responsibilities:

• Design, develop, and maintain our data platform architecture using Pyspark, Airflow, Presto, Hive, Kafka, and Debezium.
• Develop and maintain ETL processes to ingest, transform, and load data from various sources into our data platform.
• Work closely with data analysts, data scientists, and other stakeholders to understand their requirements and design solutions that meet their needs.
• Implement and maintain data governance policies and procedures to ensure data quality, privacy, and security.
• Continuously monitor and optimize the performance of our data platform to ensure scalability, reliability, and cost-effectiveness.
• Keep up-to-date with the latest trends and technologies in the field of data engineering and share knowledge and best practices with the team.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 5+ years of experience in data engineering or related fields.
• Strong proficiency in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium.
• Experience with data warehousing, data modeling, and data governance.
• Experience working with large-scale distributed systems and cloud platforms (e.g., AWS, GCP, Azure).
• Strong problem-solving skills and ability to work independently and collaboratively.
• Excellent communication and interpersonal skills.

If you are a self-motivated and driven individual with a passion for data engineering and a strong background in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium, we encourage you to apply for this exciting opportunity. We offer competitive compensation, comprehensive benefits, and a collaborative work environment that fosters innovation and growth.

Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.

Job responsibilities:

• You will partner with teammates to create complex data processing pipelines in order to solve our clients' most complex challenges
• You will collaborate with Data Scientists in order to design scalable implementations of their models
• You will pair to write clean and iterative code based on TDD
• Leverage various continuous delivery practices to deploy, support and operate data pipelines
• Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
• Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
• Create data models and speak to the tradeoffs of different modeling approaches
• Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process
• Assure effective collaboration between Thoughtworks' and the client's teams, encouraging open communication and advocating for shared outcomes

Job qualifications:

Technical skills:

• You have a good understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
• You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
• Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions
• You are comfortable taking data-driven approaches and applying data security strategy to solve business problems
• Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
• You're genuinely excited about data infrastructure and operations with a familiarity working in cloud environments

Professional skills:

• You're resilient and flexible in ambiguous situations and enjoy solving problems from technical and business perspectives
• An interest in coaching, sharing your experience and knowledge with teammates
• You enjoy influencing others and always advocate for technical excellence while being open to change when needed
• Presence in the external tech community: you willingly share your expertise with others via speaking engagements, contributions to open source, blogs and more

Other things to know:

L&D:

There is no one-size-fits-all career path at Thoughtworks: however you want to develop your career is entirely up to you. But we also balance autonomy with the strength of our cultivation culture. This means your career is supported by interactive tools, numerous development programs and teammates who want to help you grow. We see value in helping each other be our best and that extends to empowering our employees in their career journeys.

About Thoughtworks:

Thoughtworks is a global technology consultancy that integrates strategy, design and engineering to drive digital innovation. For 28+ years, our clients have trusted our autonomous teams to build solutions that look past the obvious. Here, computer science grads come together with seasoned technologists, self-taught developers, midlife career changers and more to learn from and challenge each other. Career journeys flourish with the strength of our cultivation culture, which has won numerous awards around the world.

Join Thoughtworks and thrive. Together, our extra curiosity, innovation, passion and dedication overcomes ordinary.

Role: Data Engineer

Total Experience: 5 to 8 Years

Job Location: Gurgaon

Budget -26 28 LPA

Must have - Technical & Soft Skills:

• Python: Data Structures, List, Libraries, Data engineering basics
• SQL: Joins, Groups, Aggregations, Windowing functions, analytic functions etc.
• Worked in AWS services S3, EC2, Glue, Data Pipeline, Athena and Redshift
• Solid hands-on working experience in Big-Data Technologies
• Strong hands-on experience of programming languages like Python, Scala with Spark.
• Good command and working experience on Hadoop/Map Reduce, HDFS, Hive, HBase, and No-SQL Databases
• Hands on working experience on any of the data engineering/analytics platform AWS preferred
• Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Ozzie
• Hands on working experience of data processing at scale with event driven systems, message queues (Kafka/ Flink/Spark Streaming)
• Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API
• Gateway, Lake Foundation
• Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.)
• Feature Engineering/Data Processing to be used for Model development
• Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)
• Hands-on working experience in analyzing source system data and data flows, working with structured and unstructured data

Lead Data Engineer

Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.

Job responsibilities

·      You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems

·      You will partner with teammates to create complex data processing pipelines in order to solve our clients' most ambitious challenges

·      You will collaborate with Data Scientists in order to design scalable implementations of their models

·      You will pair to write clean and iterative code based on TDD

·      Leverage various continuous delivery practices to deploy, support and operate data pipelines

·      Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available

·      Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions

·      Create data models and speak to the tradeoffs of different modeling approaches

·      On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product

·      Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process

·      Assure effective collaboration between Thoughtworks' and the client's teams, encouraging open communication and advocating for shared outcomes

Job qualifications Technical skills

·      You are equally happy coding and leading a team to implement a solution

·      You have a track record of innovation and expertise in Data Engineering

·      You're passionate about craftsmanship and have applied your expertise across a range of industries and organizations

·      You have a deep understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop

·      You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting

·      Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions

·      You are comfortable taking data-driven approaches and applying data security strategy to solve business problems

·      You're genuinely excited about data infrastructure and operations with a familiarity working in cloud environments

·      Working with data excites you: you have created Big data architecture, you can build and operate data pipelines, and maintain data storage, all within distributed systems

Professional skills

·      Advocate your data engineering expertise to the broader tech community outside of Thoughtworks, speaking at conferences and acting as a mentor for more junior-level data engineers

·      You're resilient and flexible in ambiguous situations and enjoy solving problems from technical and business perspectives

·      An interest in coaching others, sharing your experience and knowledge with teammates

·      You enjoy influencing others and always advocate for technical excellence while being open to change when needed

Role: Principal Software Engineer

We looking for a passionate Principle Engineer - Analytics to build data products that extract valuable business insights for efficiency and customer experience. This role will require managing, processing and analyzing large amounts of raw information and in scalable databases. This will also involve developing unique data structures and writing algorithms for the entirely new set of products. The candidate will be required to have critical thinking and problem-solving skills. The candidates must be experienced with software development with advanced algorithms and must be able to handle large volume of data. Exposure with statistics and machine learning algorithms is a big plus. The candidate should have some exposure to cloud environment, continuous integration and agile scrum processes.

Responsibilities:

• Lead projects both as a principal investigator and project manager, responsible for meeting project requirements on schedule

• Software Development that creates data driven intelligence in the products which deals with Big Data backends

• Exploratory analysis of the data to be able to come up with efficient data structures and algorithms for given requirements

• The system may or may not involve machine learning models and pipelines but will require advanced algorithm development

• Managing, data in large scale data stores (such as NoSQL DBs, time series DBs, Geospatial DBs etc.)

• Creating metrics and evaluation of algorithm for better accuracy and recall

• Ensuring efficient access and usage of data through the means of indexing, clustering etc.

• Collaborate with engineering and product development teams.

Requirements:

• Master’s or Bachelor’s degree in Engineering in one of these domains - Computer Science, Information Technology, Information Systems, or related field from top-tier school

• OR Master’s degree or higher in Statistics, Mathematics, with hands on background in software development.

• Experience of 8 to 10 year with product development, having done algorithmic work

• 5+ years of experience working with large data sets or do large scale quantitative analysis

• Understanding of SaaS based products and services.

• Strong algorithmic problem-solving skills

• Able to mentor and manage team and take responsibilities of team deadline.

Skill set required:

• In depth Knowledge Python programming languages

• Understanding of software architecture and software design

• Must have fully managed a project with a team

• Having worked with Agile project management practices

• Experience with data processing analytics and visualization tools in Python (such as pandas, matplotlib, Scipy, etc.)

• Strong understanding of SQL and querying to NoSQL database (eg. Mongo, Casandra, Redis

Minimum 4 to 10 years of experience in testing distributed backend software architectures/systems.

• 4+ years of work experience in test planning and automation of enterprise software

• Expertise in programming using Java or Python and other scripting languages.

• Experience with one or more public clouds is expected.

• Comfortable with build processes, CI processes, and managing QA Environments as well as working with build management tools like Git, and Jenkins

. • Experience with performance and scalability testing tools.

• Good working knowledge of relational databases, logging, and monitoring frameworks is expected.

Familiarity with system flow like how they interact with an application Eg. Elasticsearch, Mongo, Kafka, Hive, Redis, AWS

RequiredSkills:

• Minimum of 4-6 years of experience in data modeling (including conceptual, logical and physical data models. • 2-3 years of experience inExtraction, Transformation and Loading ETLwork using data migration tools like Talend, Informatica, Datastage, etc. • 4-6 years of experience as a database developerinOracle, MS SQLor another enterprise database with a focus on building data integration process • Candidate should haveanyNoSqltechnology exposure preferably MongoDB. • Experience in processing large data volumes indicated by experience with BigDataplatforms (Teradata, Netezza, Vertica or Cloudera, Hortonworks, SAP HANA, Cassandra, etc.). • Understanding of data warehousing concepts and decision support systems.

• Ability to deal with sensitive and confidential material and adhere to worldwide data security and • Experience writing documentation for design and feature requirements. • Experience developing data-intensive applications on cloud-based architectures and infrastructures such as AWS, Azure etc. • Excellent communication and collaboration skills.

KEY RESPONSIBILITIES

• Building a website based on the given requirements and ensure it’s successfully deployed
• Responsible for designing, planning, and testing new web pages and site features
• A propensity for brainstorming and coming up with solutions to open-ended problems
• Work closely with other teams, and project managers, to understand all stakeholders’ requirements and ensure that all specifications and requirements are met in final development
• Troubleshoot and solve problems related to website functionality
• Takes ownership of initiatives and drives them to completion.
• Desire to learn and dive deep into new technologies on the job, especially around modern data storage and streaming open source systems
• Responsible for creating, optimizing, and managing REST APIs
• Create website content and enhance website usability and visibility
• Ensure cross-browser compatibility and testing for mobile responsiveness
• Ability to integrate payment processing and search functionality software solutions
• Stay up-to-date with technological advancements and the latest coding practices
• Collaborate with the team of designers, content managers, and developers to determine site goals, functionality, and layout
• Monitor website traffic and overall system’s health with Google analytics to ensure high GTmetrix score
• Build the front-end of applications through appealing visual design
• Design client-side and server-side architecture
• Develop server-side logic and APIs that integrate with front-end applications.
• Architect and design complex database structures and data models.
• Develop and implement backend systems to support scalable and high-performance web applications.
• Create automated tests to ensure system stability and performance.
• Ensure security and data privacy measures are maintained throughout the development process.
• Maintain an up-to-date changelog for all new, updated, and fixed changes.
• Ability to document and manage all the software design, requirements, reusable & transferable code, and other technical aspects of the project.
• Create and convert storyboards and wireframes into high-quality full-stack code
• Write, execute, and maintain clean, reusable, and scalable code
• Design and implement low-latency, high-availability, and performant applications
• Implement security and data protection
• Ensure code that is platform and device-agnostic

EDUCATION & SKILLS REQUIREMENT

• B.Tech. / BE / MS degree in Computer Science or Information Technology
• Expertise in MERN stack (MongoDB, Express.js, React.js, Node.js)
• Should have prior working experience of at least 3 years as web developer or full stack developer
• Should have done projects in e-commerce or have preferably worked with companies operating in e-commerce
• Should have expert-level knowledge in implementing frontend technologies
• Should have worked in creating backend and have deep understanding of frameworks
• Experience in the complete product development life cycle
• Hands-on experience with JavaScript, HTML, CSS, JQuery, JSON, PHP, XML
• Proficiency in databases, including analytical (e.g., mySQL, MongoDB, PostgreSQL, DynamoDB, Redis, Hive, Elastic etc.)
• Knowledge of architecting or implementing search APIs
• Great understanding of data modeling and RESTful APIs
• Strong knowledge of CS fundamentals, data structures, algorithms, and design patterns
• Strong analytical, consultative, and communication skills
• Excellent understanding of Microsoft office tools : excel, word, powerpoint etc.
• Excellent organizational and time management skills
• Experience with responsive and adaptive design (Web, Mobile & App)
• Should be a self starter and have ability to work without being supervised
• Excellent debugging and optimization skills
• Experience building high throughput/low latency systems.
• Knowledge of big data systems such as Cassandra, Elastic, Kafka, Kubernetes, and Docker
• Should be willing to be a part of a small team and working in fast-paced environment
• Should be highly passionate about building products that create a significant impact.
• Should have experience in user experience design, website optimization techniques and different PIM tools

Responsibilities:

● Designing, building and maintaining efficient, reusable, and reliable architecture and code. ● Participate in the architecture and system design discussions

● Independently perform hands on development/coding and unit testing of the applications

● Collaborate with the development and AI teams and build individual components into complex enterprise web systems

● Work in a team environment with product, frontend design, production operation, QE/QA and cross functional teams to deliver a project throughout the whole software development cycle

● Architect and implement CI/CD strategy for EDP

● Implement high velocity streaming solutions using Amazon Kinesis, SQS, and Kafka (preferred) ● Designing, building and maintaining efficient, reusable, and reliable architecture and code.

● Ensure the best possible performance and quality of high scale web applications and services ● To identify and resolve any performance issues

● Keep up to date with new technology development and implementation

● Participate in code review to make sure standards and best practices are met

● Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift

● Migrate data from AWS Dynamodb to relational database such as PostgreSQL

● Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift

● Work closely with the Data Scientist leads, CTO, Product, Engineering, DevOps and other members of the Ai Science teams

● Collaborate with the product team, share feedback from project implementations and influence the product roadmap.

● Be comfortable in a highly dynamic, agile environment without sacrificing the quality of work products.

Position Requirements:

● Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience

● 5+ years of experience as Data application developer

● AWS Solutions Architect or AWS Developer Certification preferred

● Experience implementing software applications supporting data lakes, data warehouses and data applications on AWS for large enterprises

● Solid Programming experience with Python, Shell scripting and SQL

● Solid experience of AWS services such as CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DataSync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.

● Solid experience implementing solutions on AWS based data lakes.

● Experience in AWS data lake/data warehouse/business analytics/

● Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS

● Knowledge of ETL/ELT

● End-to-end data solutions (ingest, storage, integration, processing, access) on AWS ● Experience developing  business applications using NoSQL/SQL databases.

● Experience working with Object stores(S3) and JSON is must have

● Should have good experience with AWS Services – Glue, Lambda, Step Functions, SQS, DynamoDB, S3, Redshift, RDS, Cloudwatch and ECS.

● Should have hands-on experience with Python, Django

● Great knowledge of Data Science models

● Plus to have knowledge on Snowflake

Nice to have:

● Solid experience in AWS AI solutions such Recognition, Comperhind and Transcribe

● Python, NodeJS, .NetCore, C#, Reactjs, RestAPI, Microservices, Postman, GraphQL, Mongo, Linux, Javascript, HTML5, CSS, Django

For direct application fill the Form: https://forms.gle/z1Zhz32oHkNmANFV8

Job description

• Engage with the business team and stakeholder at different levels to understand business needs, analyze, document, prioritize the requirements, and make recommendations on the solution and implementation.
• Delivering the product that meets business requirements, reliability, scalability, and performance goals
• Work with Agile scrum team and create the scrum team strategy roadmap/backlog, develop minimal viable product and Agile user stories that drive a highly effective and efficient project development and delivery scrum team.
• Work on Data mapping/transformation, solution design, process diagram, acceptance criteria, user acceptance testing and other project artifacts.
• Work effectively with the technical/development team and help them understand the specifications/requirements for technical development, testing and implementation.
• Ensure solutions promote simplicity, efficiency, and conform to enterprise and architecture standards and guidelines.
• Partner with the support organization to provide training, support and technical assistance to operation team and end users as necessary
• Product/Application Developer
• Designs and develops software applications based on user requirements in a variety of coding environments such as graphical user interface, database query languages, report writers, and specific development languages
• Consult on the use and implementation of software products and applications and specialize in the business development environment, including the selection of development tools and methodology

Primary / Mandatory skills:

• Overall Experience: Overall 4 to 6 years of IT development experience
• Design and Code NodeJS based Microservices, API Webservices, NoSql technologies (Cassandra/MongoDb)
• Expert in developing code for Node-JS based Microservice in TypeScript
• Good Experience in understanding the data Transmission through pug/sub mechanism like Event Hub and Kafka
• Good Understanding of Analytics and clickstream data capture is HUGE Plus
• Good understanding of frameworks like Java Spring Boot, Python is preferred
• Good understanding of Microsoft Azure principles and services is preferred
• Able to write Unit test cases
• Familiarity with performance testing tools such as Akamai SOASTA is preferred
• Good knowledge on Source Code control like GIT, code clout, etc and understanding of CI/CD(Jenkins and Kubernetes)
• Solid technical background with understanding and/or experience in software development and web technologies
• Strong analytical skills and the ability to convert consumer insights and performance data into high impact initiatives
• Experience working within scaled agile development team
• Excellent written and verbal communication skills with demonstrated ability to present complex technical information in a clear manner to peers, developers, and senior leaders
• The desire to be continually learning about emerging technologies/industry trends

We are looking for a DevOps Engineer (individual contributor) to maintain and build upon our next-generation infrastructure. We aim to ensure that our systems are secure, reliable and high-performing by constantly striving to achieve best-in-class infrastructure and security by:

• Leveraging a variety of tools to ensure all configuration is codified (using tools like Terraform and Flux) and applied in a secure, repeatable way (via CI)
• Routinely identifying new technologies and processes that enable us to streamline our operations and improve overall security
• Holistically monitoring our overall DevOps setup and health to ensure our roadmap constantly delivers high-impact improvements
• Eliminating toil by automating as many operational aspects of our day-to-day work as possible using internally created, third party and/or open-source tools
• Maintain a culture of empowerment and self-service by minimizing friction for developers to understand and use our infrastructure through a combination of innovative tools, excellent documentation and teamwork

Tech stack: Microservices primarily written in JavaScript, Kotlin, Scala, and Python. The majority of our infrastructure sits within EKS on AWS, using Istio. We use Terraform and Helm/Flux when working with AWS and EKS (k8s). Deployments are managed with a combination of Jenkins and Flux. We rely heavily on Kafka, Cassandra, Mongo and Postgres and are increasingly leveraging AWS-managed services (e.g. RDS, lambda).

8+ years of Experience including –

MS SQL Server

Big Data – Hadoop Stack (HDFS, Hive)

Big Data – Basics of Hadoop Cluster Set Up / Job Monitoring / Troubleshooting

Data Pipeline Design – Experience in designing data processing flows / pipelines

Complex Query Writing (Joins, aggregations, analytical functions, etc.)

Exposure to Stored Procedure Development

Basics of Unix Shell Scripting

Excellent Written and Verbal Communication Skills (Regular Client Interaction Expected)

Team-leading experience is required

Ability to work independently along with leading the team, take technical ownership, mentor team members

Good Problem solving skills

Good to Have:

Experience / Exposure to Apache Spark

Knowledge of Programming (any language Java / Scala / Python)

Team Lead Role – Take Technical Ownership

Design & Maintain Data Pipeline Solutions in SQL Server / Big Data technologies as per customer requirement

Mentor and groom Team Members

Perform individual tasks along with the lead role

Regularly interact with Customer and their partners over e-mails and meetings etc.

About Kloud9:

Kloud9 exists with the sole purpose of providing cloud expertise to the retail industry. Our team of cloud architects, engineers and developers help retailers launch a successful cloud initiative so you can quickly realise the benefits of cloud technology. Our standardised, proven cloud adoption methodologies reduce the cloud adoption time and effort so you can directly benefit from lower migration costs.

Kloud9 was founded with the vision of bridging the gap between E-commerce and cloud. The E-commerce of any industry is limiting and poses a huge challenge in terms of the finances spent on physical data structures.

At Kloud9, we know migrating to the cloud is the single most significant technology shift your company faces today. We are your trusted advisors in transformation and are determined to build a deep partnership along the way. Our cloud and retail experts will ease your transition to the cloud.

Our sole focus is to provide cloud expertise to retail industry giving our clients the empowerment that will take their business to the next level. Our team of proficient architects, engineers and developers have been designing, building and implementing solutions for retailers for an average of more than 20 years.

We are a cloud vendor that is both platform and technology independent. Our vendor independence not just provides us with a unique perspective into the cloud market but also ensures that we deliver the cloud solutions available that best meet our clients' requirements.

What we are looking for:

● 3+ years’ experience developing Data & Analytic solutions

● Experience building data lake solutions leveraging one or more of the following AWS, EMR, S3, Hive& Spark

● Experience with relational SQL

● Experience with scripting languages such as Shell, Python

● Experience with source control tools such as GitHub and related dev process

● Experience with workflow scheduling tools such as Airflow

● In-depth knowledge of scalable cloud

● Has a passion for data solutions

● Strong understanding of data structures and algorithms

● Strong understanding of solution and technical design

● Has a strong problem-solving and analytical mindset

● Experience working with Agile Teams.

● Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders

● Able to quickly pick up new programming languages, technologies, and frameworks

● Bachelor’s Degree in computer science

Why Explore a Career at Kloud9:

With job opportunities in prime locations of US, London, Poland and Bengaluru, we help build your career paths in cutting edge technologies of AI, Machine Learning and Data Science. Be part of an inclusive and diverse workforce that's changing the face of retail technology with their creativity and innovative solutions. Our vested interest in our employees translates to deliver the best products and solutions to our customers.

Requirements:

● Understanding our data sets and how to bring them together.

● Working with our engineering team to support custom solutions offered to the product development.

● Filling the gap between development, engineering and data ops.

● Creating, maintaining and documenting scripts to support ongoing custom solutions.

● Excellent organizational skills, including attention to precise details

● Strong multitasking skills and ability to work in a fast-paced environment

● 5+ years experience with Python to develop scripts.

● Know your way around RESTFUL APIs.[Able to integrate not necessary to publish]

● You are familiar with pulling and pushing files from SFTP and AWS S3.

● Experience with any Cloud solutions including GCP / AWS / OCI / Azure.

● Familiarity with SQL programming to query and transform data from relational Databases.

● Familiarity to work with Linux (and Linux work environment).

● Excellent written and verbal communication skills

● Extracting, transforming, and loading data into internal databases and Hadoop

● Optimizing our new and existing data pipelines for speed and reliability

● Deploying product build and product improvements

● Documenting and managing multiple repositories of code

● Experience with SQL and NoSQL databases (Casendra, MySQL)

● Hands-on experience in data pipelining and ETL. (Any of these frameworks/tools: Hadoop, BigQuery,

RedShift, Athena)

● Hands-on experience in AirFlow

● Understanding of best practices, common coding patterns and good practices around

● storing, partitioning, warehousing and indexing of data

● Experience in reading the data from Kafka topic (both live stream and offline)

● Experience in PySpark and Data frames

Responsibilities:

You’ll

● Collaborating across an agile team to continuously design, iterate, and develop big data systems.

● Extracting, transforming, and loading data into internal databases.

● Optimizing our new and existing data pipelines for speed and reliability.

● Deploying new products and product improvements.

● Documenting and managing multiple repositories of code.

About DeepIntent:

DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioural, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences and message them on a one-to-one basis in a privacy-compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

What You’ll Do:

• Establish formal data practice for the organisation.
• Build & operate scalable and robust data architectures.
• Create pipelines for the self-service introduction and usage of new data
• Implement DataOps practices
• Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
• Engineers.
• Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
• to deploy and manage.
• Collaborate with various business stakeholders, software engineers, machine learning
• engineers, and analysts.

Who You Are:

• Experience in designing, developing and operating configurable Data pipelines serving high
• volume and velocity data.
• Experience working with public clouds like GCP/AWS.
• Good understanding of software engineering, DataOps, data architecture, Agile and
• DevOps methodologies.
• Experience building Data architectures that optimize performance and cost, whether the
• components are prepackaged or homegrown
• Proficient with SQL, Python or JVM-based language, Bash.
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
• etc. and big data databases like BigQuery, Clickhouse, etc
• Good communication skills with the ability to collaborate with both technical and non-technical
• people.
• Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

Senior Software Engineer - Data 

Job Description:

We are looking for a tech savvy Data Engineer to join our growing data team. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support our software developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. The hire must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. 

What You’ll Do:

• Establish formal data practices for the organization
• Build & operate scalable and robust data architectures
• Create pipelines for the self-service introduction and usage of new data
• Implement data ops practices
• Design, develop, operate data Pipelines which support data scientists and machine learning Engineers
• Build simple, highly reliable data storage, ingestion, transformation solutions which are easy to deploy and manage
• Collaborate with various business stakeholders, software engineers, machine learning engineers, analysts

Who You Are:

• Experience in designing, developing and operating configurable data pipelines serving high volume and velocity data
• Experience working with public clouds like GCP/AWS
• Good understanding of software engineering, data ops, and data architecture, Agile and DevOps methodologies
• Experience building data architectures that optimize performance and cost, whether the components are prepackaged or homegrown
• Proficient with SQL, Python or JVM based language, Bash
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc. and big data databases like BigQuery, Clickhouse, etc
• Good communication skills with ability to collaborate with both technical and non technical teams
• Ability to think big, take bets and innovate, learn and be curious.

Why LiftOff? 

We at LiftOff specialize in product creation, for our main forte lies in helping Entrepreneurs realize their dream. We have helped businesses and entrepreneurs launch more than 70 plus products.

Many on the team are serial entrepreneurs with a history of successful exits.

As a Data Engineer, you will work directly with our founders and alongside our engineers on a variety of software projects covering various languages, frameworks, and application architectures.

About the Role

If you’re driven by the passion to build something great from scratch, a desire to innovate, and a commitment to achieve excellence in your craft, LiftOff is a great place for you.

• Architecture/design / configure the data ingestion pipeline for data received from 3rd party vendors
• Data loading should be configured with ease/flexibility for adding new data sources & also refresh of the previously loaded data
• Design & implement a consumer graph, that provides an efficient means to query the data via email, phone, and address information (using any one of the fields or combination)
• Expose the consumer graph/search capability for consumption by our middleware APIs, which would be shown in the portal
• Design / review the current client-specific data storage, which is kept as a copy of the consumer master data for easier retrieval/query for subsequent usage

Please Note that this is for a Consultant Role

Candidates who are okay with freelancing/Part-time can apply

We are looking for a Big Data Engineer with java for Chennai Location

Location : Chennai 

Exp : 11 to 15 Years 

Job description

Required Skill:

1. Candidate should have minimum 7 years of experience as total

2. Candidate should have minimum 4 years of experience in Big Data design and development

3. Candidate should have experience in Java, Spark, Hive & Hadoop, Python 

4. Candidate should have experience in any RDBMS.

Roles & Responsibility:

1. To create work plans, monitor and track the work schedule for on time delivery as per the defined quality standards.

2. To develop and guide the team members in enhancing their technical capabilities and increasing productivity.

3. To ensure process improvement and compliance in the assigned module, and participate in technical discussions or review.

4. To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalation

Regards,

Priyanka S

7P8R9I9Y4A0N8K8A7S7

• Design the architecture of our big data platform
• Perform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries
• Design and implement data stores that support the scalable processing and storage of our high-frequency data
• Maintain our data pipeline
• Customize and oversee integration tools, warehouses, databases, and analytical systems
• Configure and provide availability for data-access tools used by all data scientists

Requirements:

• 5+ years of experience in backend software technologies like Java and Spring boot
• Experience in working with MYSQL Database and prior knowledge on db modeling
• Strong background in JS(Java script)
• Strong background in responsive and multi-platform design.
• Experience in JPA 2.6.3 (Java Persistent API) frameworks
• Prior experience working with firebase for authentication
• Prior experience working with any of analytics tool integration ( GA / Firebase/ Mix Panel)
• Experience in integrating with web service APIs / Rest APIs
• Experience in working with micro-service architecture
• Experience integrating with backend infrastructure and manipulating data structures
• Proficient understanding of client-side scripting and JavaScript frameworks and front-end software design patterns
• Should have worked in a minimum of Agile projects (SCRUM)
• Experience in Unit testing/integration testing of backend APIs using Junit or any unit testing framework
• Prior experience in working with MVC Architecture
• Prior experience working with containerized applications and CI/CD architecture
• Prior experience working with cloud platforms, preferably AWS (Amazon web service)
• Should be able to work in a dynamic and fast-moving environment
• Working proficiency and communication skills in verbal and written English
• Startup experience is a huge plus.

Data Engineer- Senior

Cubera is a data company revolutionizing big data analytics and Adtech through data share value principles wherein the users entrust their data to us. We refine the art of understanding, processing, extracting, and evaluating the data that is entrusted to us. We are a gateway for brands to increase their lead efficiency as the world moves towards web3.

What are you going to do?

Design & Develop high performance and scalable solutions that meet the needs of our customers.

Closely work with the Product Management, Architects and cross functional teams.

Build and deploy large-scale systems in Java/Python.

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Create data tools for analytics and data scientist team members that assist them in building and optimizing their algorithms.

Follow best practices that can be adopted in Bigdata stack.

Use your engineering experience and technical skills to drive the features and mentor the engineers.

What are we looking for ( Competencies) :

Bachelor’s degree in computer science, computer engineering, or related technical discipline.

Overall 5 to 8 years of programming experience in Java, Python including object-oriented design.

Data handling frameworks: Should have a working knowledge of one or more data handling frameworks like- Hive, Spark, Storm, Flink, Beam, Airflow, Nifi etc.

Data Infrastructure: Should have experience in building, deploying and maintaining applications on popular cloud infrastructure like AWS, GCP etc.

Data Store: Must have expertise in one of general-purpose No-SQL data stores like Elasticsearch, MongoDB, Redis, RedShift, etc.

Strong sense of ownership, focus on quality, responsiveness, efficiency, and innovation.

Ability to work with distributed teams in a collaborative and productive manner.

Benefits:

Competitive Salary Packages and benefits.

Collaborative, lively and an upbeat work environment with young professionals.

Job Category: Development

Job Type: Full Time

Job Location: Bangalore

Roles and Responsibilities

Skills

About us

Classplus is India's largest B2B ed-tech start-up, enabling 1 Lac+ educators and content creators to create their digital identity with their own branded apps. Starting in 2018, we have grown more than 10x in the last year, into India's fastest-growing video learning platform.

Over the years, marquee investors like Tiger Global, Surge, GSV Ventures, Blume, Falcon, Capital, RTP Global, and Chimera Ventures have supported our vision. Thanks to our awesome and dedicated team, we achieved a major milestone in March this year when we secured a “Series-D” funding.

Now as we go global, we are super excited to have new folks on board who can take the rocketship higher🚀. Do you think you have what it takes to help us achieve this? Find Out Below!

What will you do?

· Define the overall process, which includes building a team for DevOps activities and ensuring that infrastructure changes are reviewed from an architecture and security perspective

· Create standardized tooling and templates for development teams to create CI/CD pipelines

· Ensure infrastructure is created and maintained using terraform

· Work with various stakeholders to design and implement infrastructure changes to support new feature sets in various product lines.

· Maintain transparency and clear visibility of costs associated with various product verticals, environments and work with stakeholders to plan for optimization and implementation

· Spearhead continuous experimenting and innovating initiatives to optimize the infrastructure in terms of uptime, availability, latency and costs

You should apply, if you

1. Are a seasoned Veteran: Have managed infrastructure at scale running web apps, microservices, and data pipelines using tools and languages like JavaScript(NodeJS), Go, Python, Java, Erlang, Elixir, C++ or Ruby (experience in any one of them is enough)

2. Are a Mr. Perfectionist: You have a strong bias for automation and taking the time to think about the right way to solve a problem versus quick fixes or band-aids.

3. Bring your A-Game: Have hands-on experience and ability to design/implement infrastructure with GCP services like Compute, Database, Storage, Load Balancers, API Gateway, Service Mesh, Firewalls, Message Brokers, Monitoring, Logging and experience in setting up backups, patching and DR planning

4. Are up with the times: Have expertise in one or more cloud platforms (Amazon WebServices or Google Cloud Platform or Microsoft Azure), and have experience in creating and managing infrastructure completely through Terraform kind of tool

5. Have it all on your fingertips: Have experience building CI/CD pipeline using Jenkins, Docker for applications majorly running on Kubernetes. Hands-on experience in managing and troubleshooting applications running on K8s

6. Have nailed the data storage game: Good knowledge of Relational and NoSQL databases (MySQL,Mongo, BigQuery, Cassandra…)

7. Bring that extra zing: Have the ability to program/script is and strong fundamentals in Linux and Networking.

8. Know your toys: Have a good understanding of Microservices architecture, Big Data technologies and experience with highly available distributed systems, scaling data store technologies, and creating multi-tenant and self hosted environments, that’s a plus

Being Part of the Clan

At Classplus, you’re not an “employee” but a part of our “Clan”. So, you can forget about being bound by the clock as long as you’re crushing it workwise😎. Add to that some passionate people working with and around you, and what you get is the perfect work vibe you’ve been looking for!

It doesn’t matter how long your journey has been or your position in the hierarchy (we don’t do Sirs and Ma’ams); you’ll be heard, appreciated, and rewarded. One can say, we have a special place in our hearts for the Doers! ✊🏼❤️

Are you a go-getter with the chops to nail what you do? Then this is the place for you.

Senior Software Developer

Location: Hyderabad (Work From Office, Hybrid)

Permanent Position

Technical skills in most of the following areas:

• Node.js , ReactJS and Redux, Saga, GraphQL, REST APIs , HTML, CSS, CSS3, JavaScript , TypeScript & Serverless.com framework or (Knowledge or Experience with AWS SAM, Lambda , S3,  CloudWatch & DynamoDB.) , Knowledge or Experience with Cassandra , Mysql Databases.
• Should be able to communicate vision with enthusiasm
• Demonstrated ability to recognize problems, recommend solutions, and collaboratively implement changes
• Excellent interpersonal and communication skills, strong analytical skills, and the ability to interface effectively with all levels within the organization, including executive and senior management teams.
• Should be capable to undertake innovative & critical thinking that challenges the status quo with ideas that create significant business value.
• Independent thinker and researcher

Title: Cloud DevOps Engineer 

Role: Individual Contributor (4-8 yrs)  

Requirements: 

• Energetic self-starter, a fast learner, with a desire to work in a startup environment  
• Experience working with Public Clouds like AWS 

• Operating and Monitoring cloud infrastructure on AWS. 
• Primary focus on building, implementing and managing operational support 
• Design, Develop and Troubleshoot Automation scripts (Configuration/Infrastructure as code or others) for Managing Infrastructure. 
• Expert at one of the scripting languages – Python, shell, etc  
• Experience with Nginx/HAProxy, ELK Stack, Ansible, Terraform, Prometheus-Grafana stack, etc 

• Handling load monitoring, capacity planning, and services monitoring. 
• Proven experience With CICD Pipelines and Handling Database Upgrade Related Issues. 
• Good Understanding and experience in working with Containerized environments like Kubernetes and Datastores like Cassandra, Elasticsearch, MongoDB, etc

What will you do?

You will help build cutting-edge products in various verticals.

You will have to understand the solution domain and understand/architect the data flow.

You will be accountable for the data models and data pipelines driving the solution.

You will also be researching, and iterating for better solutions and this would involve staying up to speed with the latest technologies in the data space.

Skills Required:

Should have a clear understanding of one or more of the below technologies –

• Database: PostgreSQL, MySQL etc.

• BI Reporting: QlikView, Qliksense, SSRS, Tableau & Power BI.

• Cloud – One of AWS, Azure, GCP

• Big Data – Spark SQL, Scala, pySpark, Red Shift, Hive, HDFS, Cloudera 

About us:

Zemoso Technologies is a Software Product Market Fit Studio that brings silicon valley style rapid prototyping and rapid application builds to Entrepreneurs and Corporate innovation. We offer Innovation as a service and work on ideas from scratch and take it to the Product Market Fit stage using Design Thinking->Lean Execution->Agile Methodology.

We were featured as one of Deloitte's Fastest 50 growing tech companies from India thrice (2016, 2018 and 2019). We were also featured in Deloitte Technology Fast 500 Asia Pacific both in 2016 and 2018.

We are located in Hyderabad, India, and Dallas, US. We have recently incorporated another office in Waterloo, Canada.

Our founders have had past successes - founded a decision management company acquired by SAP AG (now part of Hana Big data stack & NetWeaver BPM), the early engineering team of Zoho (a leading billion $ SaaS player) & some Private Equity experience.

Marquee customers along with some exciting start-ups are part of our clientele.

About RARA NOW :

• RaRa Now is revolutionizing instant delivery for e-commerce in Indonesia through data-driven logistics.

• RaRa Now is making instant and same-day deliveries scalable and cost-effective by leveraging a differentiated operating model and real-time optimization technology. RaRa makes it possible for anyone, anywhere to get same-day delivery in Indonesia. While others are focusing on - one-to-one- deliveries, the company has developed proprietary, real-time batching tech to do - many-to-many- deliveries within a few hours. RaRa is already in partnership with some of the top eCommerce players in Indonesia like Blibli, Sayurbox, Kopi Kenangan, and many more.

• We are a distributed team with the company headquartered in Singapore, core operations in Indonesia, and a technology team based out of India.

Future of eCommerce Logistics :

• Data driven logistics company that is bringing in same-day delivery revolution in Indonesia

• Revolutionizing delivery as an experience

• Empowering D2C Sellers with logistics as the core technology

About the Role :

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Prior experience on working on Big Query, Redshift or other data warehouses

Bigdata with cloud:

Experience : 5-10 years

Location : Hyderabad/Chennai

Notice period : 15-20 days Max

1.  Expertise in building AWS Data Engineering pipelines with AWS Glue -> Athena -> Quick sight

2.  Experience in developing lambda functions with AWS Lambda

3.  Expertise with Spark/PySpark – Candidate should be hands on with PySpark code and should be able to do transformations with Spark

4.  Should be able to code in Python and Scala.

5.  Snowflake experience will be a plus

Must Have skills :

Experience in Linux Administration

Experience in building, deploying, and monitoring distributed apps using container systems (Docker) and container orchestration (Kubernetes, EKS)

Ability to read and understand code (Java / Python / R / Scala)

Experience AWS and tools

Nice to have skills:

Experience in SAS Viya administration

Experience managing large Big Data clusters

Experience in Big Data tools like Hue, Hive, Spark, Jupyter, SAS and R-Studio

Roles and

Responsibilities

Seeking AWS Cloud Engineer /Data Warehouse Developer for our Data CoE team to

help us in configure and develop new AWS environments for our Enterprise Data Lake,

migrate the on-premise traditional workloads to cloud. Must have a sound

understanding of BI best practices, relational structures, dimensional data modelling,

structured query language (SQL) skills, data warehouse and reporting techniques.

 Extensive experience in providing AWS Cloud solutions to various business

use cases.

 Creating star schema data models, performing ETLs and validating results with

business representatives

 Supporting implemented BI solutions by: monitoring and tuning queries and

data loads, addressing user questions concerning data integrity, monitoring

performance and communicating functional and technical issues.

Job Description: -

This position is responsible for the successful delivery of business intelligence

information to the entire organization and is experienced in BI development and

implementations, data architecture and data warehousing.

Requisite Qualification

Essential

-

AWS Certified Database Specialty or -

AWS Certified Data Analytics

Preferred

Any other Data Engineer Certification

Requisite Experience

Essential 4 -7 yrs of experience

Preferred 2+ yrs of experience in ETL & data pipelines

Skills Required

Special Skills Required

 AWS: S3, DMS, Redshift, EC2, VPC, Lambda, Delta Lake, CloudWatch etc.

 Bigdata: Databricks, Spark, Glue and Athena

 Expertise in Lake Formation, Python programming, Spark, Shell scripting

 Minimum Bachelor’s degree with 5+ years of experience in designing, building,

and maintaining AWS data components

 3+ years of experience in data component configuration, related roles and

access setup

 Expertise in Python programming

 Knowledge in all aspects of DevOps (source control, continuous integration,

deployments, etc.)

 Comfortable working with DevOps: Jenkins, Bitbucket, CI/CD

 Hands on ETL development experience, preferably using or SSIS

 SQL Server experience required

 Strong analytical skills to solve and model complex business requirements

 Sound understanding of BI Best Practices/Methodologies, relational structures,

dimensional data modelling, structured query language (SQL) skills, data

warehouse and reporting techniques

Preferred Skills

Required

 Experience working in the SCRUM Environment.

 Experience in Administration (Windows/Unix/Network/Database/Hadoop) is a

plus.

 Experience in SQL Server, SSIS, SSAS, SSRS

 Comfortable with creating data models and visualization using Power BI

 Hands on experience in relational and multi-dimensional data modelling,

including multiple source systems from databases and flat files, and the use of

standard data modelling tools

 Ability to collaborate on a team with infrastructure, BI report development and

business analyst resources, and clearly communicate solutions to both

technical and non-technical team members

Casandra Architeture- 7+ Yrs- Bangalore

Strong knowledge of Cassandra Architecture including read/write paths, hinted handoffs, read repairs, compaction, cluster/replication strategies, client drivers, caching, GC Tuning.

                Experience in writing queries and performance tuning.

                Experience in handling real time Cassandra clusters, debugging and resolution of issues.

                Experience in implementing Keyspaces, Table, Indexes, security, data models & access administration.

                Knowledge in cassandra backup and recovery.

                Good communication skills.

About the role:

An exciting opportunity to work with an established company in Asia with our global clients, the team needs to expand their capabilities in all things Back End. We especially want to hear from you if you have JAVA and subsequent frameworks. This is a great opportunity for a leader to shine and further their career on a global stage.

What YOU do - job description

• Understanding business requirements and business process of the client’s request
• Delivery and deployment of features on the server
• Fix defects requested by clients
• Writing analysis documents and related documents of the implementation
• Providing unit testing and integration testing
• Self-starting on new technologies related to business requirements
• Work with business and system analysts to design and develop technical requirements.
• Develop accurate and efficient programs with unit tests according to the requirements.
• Maintain current knowledge of the standard language, coding conventions, and operations requirements.
• Build and deploy code to several environments (i.e. Development, SIT, UAT).
• Support, investigate and analyze the root causes of reported issues/defects/problems of the developed application.
• Consult with users, analyze requirements and recommend technical specifications.

What YOU Bring - must have qualifications/ skills

• Bachelor’s Degree in Computer Science, IT or other related fields.
• 10+ years of technical experience 
• Experience in Spring Boot or Spring MVC framework is a plus
• Strong problem-solving skills, good attitude and teamwork.
• Experience and knowledge in Web service development based on J2EE framework including
  • Java 8 and Java 11
  • Spring Boot, SQL (MySQL, PostGres etc)
  • NoSQL databases (Redis, Cassandra, couchbase etc)
  • Queues (Kafka, PubSub etc), Architecture: SOA
  • Microservices
• Experience in developing applications with Docker, Kubernetes and cloud-like Google GCP and AWS.
• Using enterprise-level databases (e.g., Oracle, MSSQL, Postgresql)
• Using NoSQL databases (e.g., Redis, Cassandra)
• Well-versed with Git.
• Strong knowledge of OOP software and REST/SOAP/gRPC web services design and implementation.
• Fluent in deploying and troubleshooting applications in the Linux OS environment.
• A full understanding of what the clean code principles mean and the implications of adhering to them.

 Good to have qualifications/ skills

• Experience in developing applications from at least more 2 programming languages e.g. Node.JS, React Native, Angular, React.JS, Ruby, Golang would be an advantage including Servlet, Java beans, EJB, JMS, JavaMail, Web Services, HTML, XML, UML etc.

• Using enterprise-level database (e.g. Oracle, MSSQL) Eclipse, Netbeans or Jetbrain IDE.
• Basic knowledge of computer networks relating to building web applications (i.e. frequently used protocols in TCP/IP stack such as FTP, SMTP, DNS etc.).
• Experience in developing software in an Agile process.

About Bluebik Tech Center
Bluebik Technology Center (India) Private Limited (Bluebik Tech Center) is a subsidiary of Bluebik Group Public Limited Company, a top leading strategic and digital transformation consulting firm with a net worth of USD180M in Thailand. Striving to serve the high demands of digital transformation from our multi-national clients, Bluebik Tech Center has been established to be the source of state-of-the-art technology development and to produce world-class IT professionals in the world and share innovative technology wisdom. We are keen on innovation and R&D and have unique dynamic training programs to upskill and reskill our talents.

With rapidly expanding business across the globe, we are looking for a candidate who is passionate about building and developing new software products and enhancements by excelling at large-scale applications and frameworks with outstanding analytical and communication skills

Responsibilities:

• Lead simultaneous development for multiple business verticals.
• Design & develop highly scalable, reliable, secure, and fault-tolerant systems.
• Ensure that exceptional standards are maintained in all aspects of engineering.
• Collaborate with other engineering teams to learn and share best practices.
• Take ownership of technical performance metrics and strive actively to improve them.
• Mentors junior members of the team and contributes to code reviews.

Requirements:

• A passion to solve tough engineering/data challenges.
• Be well versed with cloud computing platforms AWS/GCP
• Experience with SQL technologies (MySQL, PostgreSQL)
• Experience working with NoSQL technologies (MongoDB, ElasticSearch)
• Excellent Programming skills in Python/Java/GoLang
• Big Data streaming services (Kinesis, Kafka, RabbitMQ)
• Distributed cache systems(Redis, Memcache)
• Advanced data solutions(BigQuery, RedShift, DynamoDB, Cassandra)
• Automated testing frameworks and CI/CD pipelines Infrastructure orchestration(Docker/Kubernetes/Nginx)
• Cloud-native tech like Lambda, ASG, CDN, ELB, SNS/SQS, S3 Route53 SES