Sr Data Engineer
As a Data Engineer you have to understand Organisation data sets and how to bring them together. You have to work with sales engineering team to support custom solutions offered to the client. Filling the gap between development, sales engineering and data ops and creating, maintaining and documenting scripts to support ongoing custom solutions.
Job Responsibilities:
- Collaborating across an agile team to continuously design, iterate, and develop big data systems.
- Extracting, transforming, and loading data into internal databases.
- Optimizing our new and existing data pipelines for speed and reliability.
- Deploying new products and product improvements
- Documenting and managing multiple repositories of code.
Mandatory Requirements:
- Experience with Pandas to process the data and Jupyter notebooks to keep it all together.
- Familiar with pulling and pushing files from SFTP and AWS S3. Familiarity with AWS Athena and Redshift is mandatory.
- Familiarity with SQL programming to query and transform data from relational Databases.
- Familiarities with AWS Cloud and Linux (and Linux work environment) are mandatory.
- Excellent written and verbal communication skills.
Desired Requirements:
- Excellent organizational skills, including attention to precise details.
- Strong multitasking skills and ability to work in a fast-paced environment Know your way around REST APIs (Able to integrate not necessary to publish)
Qualities:
- Python
- Sql
- API
- AWS
- GCP
- OCI
- Azure
- Redshift
Eligibility Criteria:
- 5 years experience in database systems
- 3 years experience with Python to develop scripts
What’s for the Candidate:
12 LPA
Job Location(s)
Hyderabad / Remote
About
Similar jobs
Job Tittle:- AWS Solution Architect
CTC:- 22 LPA
Experience:- 7+Years
Location:- Goregaon, Thane
Working Mode:- WFO
Primary Skills
- Proven experience as an AWS Solution Architect or a similar role, with a strong understanding of AWS services and architecture.
- AWS certifications such as AWS Certified Solutions Architect - Associate or AWS Certified Solutions Architect - Professional.
- Strong knowledge of cloud computing concepts, distributed systems, and architectural best practices.
- Proficiency in designing and implementing AWS solutions for computing, storage, networking, database, and security.
- Familiarity with automation and orchestration tools such as AWS CloudFormation or Terraform.
Qualification:- BCA, MCA, B.Tech, M.Tech
Responsibilities:-
- Collaborate with clients and internal teams to understand business requirements and design effective solutions using AWS services.
- Assess infrastructure needs and design scalable and resilient architectures that leverage AWS best practices.
- Assist with the migration of on-premises applications and infrastructure to AWS, ensuring minimal downtime and maximum efficiency.
- Optimize application performance and cost by analyzing and fine-tuning AWS resources.
- Ensure security and compliance of AWS solutions by implementing appropriate security controls, encryption, and access management.
- Create and maintain technical documentation, including architecture diagrams, deployment guides, and best practices.
- Provide guidance and support to development teams in implementing AWS services and integrating them into applications.
- Stay updated with the latest AWS services, features, and industry trends, and evaluate their applicability to the organization's needs.
- Collaborate with cross-functional teams, including developers, operations, and project managers, to deliver successful AWS solutions.
The Impact you will create on the Job
Understand and handle deploying, troubleshooting issues with Compute, Networking, Storage, Database services on AWS & Azure.
IT experience in a team handling the cloud, infrastructure and Linux, Windows operating systems.
Working on different services in AWS & Azure Cloud Platform
In depth knowledge of a wide range of AWS & Azure services in Compute, Storage, Networking, Infrastructure as a code, Serverless computing, IAM, CI/CD pipelines
Possess a thorough understanding of Internet based technologies (DNS, Security, IP Routing, SSH, FTP, HTTP/HTTPS etc.)
CoreStack, an AI-powered multi-cloud governance solution, empowers enterprises to unleash the power of the cloud on their terms by helping them rapidly achieve continuous and autonomous cloud governance at scale. CoreStack enables enterprises to realize outcomes across FinOps, SecOps and CloudOps such as a 40% decrease in cloud costs and a 50% increase in operational efficiencies by governing operations, security, cost, access, and resources. CoreStack also assures 100% compliance with standards such as ISO, FedRAMP, NIST, HIPAA, PCI-DSS, AWS CIS & Well-Architected Framework (WAF). CoreStack works with many large global customers across multiple industries including Financial Services, Healthcare, Retail, Education, Telecommunications, Technology and Government.
The company is backed by industry-leading venture investors. CoreStack is a recent recipient of the 2021 Gold Stevie American Business Awards in the Cloud Infrastructure category and the 2021 Gold Globee Winner of the Most Innovative Company of the Year in IT Cloud/SaaS. In addition, CoreStack won the 2021 Best New Products American Business Award in Cloud Governance as well as Golden Bridge Awards for Cloud Computing/SaaS Innovation and Cloud Security Innovation. CoreStack was recognized as IDC Innovator in Cloud Management Solutions and in the Gartner Magic Quadrant for Cloud Management Platforms in 2020. The Company is a three-time TiE50 Winner and an Emerge 50 League-10 NASSCOM award recipient in Enterprise Software. CoreStack is a Google Cloud Build Partner, Microsoft Azure Gold & Co-Sell Partner, and Amazon AWS Advanced Technology Competency Partner.
Responsibilities:
- Part of a product team responsible for the Quality of the product used by Global customers
- Drive the automation efforts to reduce the time taken to identify issues
- Contribute towards the process improvement involving the entire Product development life cycle.
- Should be able to capture the user behavior and identify the gaps in UX flows.
- Responsible for the performance and usability of the Product.
Requirements
- Minimum 4+ Years of experience in testing domain.
- Should be well versed with Software Testing Methodologies and Techniques
- Should have experience in AWS or Azure or Google Cloud
- Should have experience in Manual Testing and identifying the critical scenario.
- Should have experience in Data Base testing
- Should have experience in Test Management tools.
- Should have experience in performance testing, load testing and basic knowledge on security testing.
- API testing using Soap UI is an added advantage.
- Selenium with Java is an added advantage.
- CI & CD using Jenkins is an added advantage.
- Prior work experience with Cloud domain and cloud based products is a Plus
CoreStack Offers
- Competitive salary
- Competitive benefit package with appreciable equity
- Exciting, fast-paced and entrepreneurial culture
- Health insurance and other company benefits
Role : Senior Customer Scientist
Experience : 6-8 Years
Location : Chennai (Hybrid)
Who are we?
A young, fast-growing AI and big data company, with an ambitious vision to simplify the world’s choices. Our clients are top-tier enterprises in the banking, e-commerce and travel spaces. They use our core AI-based choice engine http://maya.ai/">maya.ai, to deliver personal digital experiences centered around taste. The http://maya.ai/">maya.ai platform now touches over 125M customers globally. You’ll find Crayon Boxes in Chennai and Singapore. But you’ll find Crayons in every corner of the world. Especially where our client projects are – UAE, India, SE Asia and pretty soon the US.
Life in the Crayon Box is a little chaotic, largely dynamic and keeps us on our toes! Crayons are a diverse and passionate bunch. Challenges excite us. Our mission drives us. And good food, caffeine (for the most part) and youthful energy fuel us. Over the last year alone, Crayon has seen a growth rate of 3x, and we believe this is just the start.
We’re looking for young and young-at-heart professionals with a relentless drive to help Crayon double its growth. Leaders, doers, innovators, dreamers, implementers and eccentric visionaries, we have a place for you all.
Can you say “Yes, I have!” to the below?
- Experience with exploratory analysis, statistical analysis, and model development
- Knowledge of advanced analytics techniques, including Predictive Modelling (Logistic regression), segmentation, forecasting, data mining, and optimizations
- Knowledge of software packages such as SAS, R, Rapidminer for analytical modelling and data management.
- Strong experience in SQL/ Python/R working efficiently at scale with large data sets
- Experience in using Business Intelligence tools such as PowerBI, Tableau, Metabase for business applications
Can you say “Yes, I will!” to the below?
- Drive clarity and solve ambiguous, challenging business problems using data-driven approaches. Propose and own data analysis (including modelling, coding, analytics) to drive business insight and facilitate decisions.
- Develop creative solutions and build prototypes to business problems using algorithms based on machine learning, statistics, and optimisation, and work with engineering to deploy those algorithms and create impact in production.
- Perform time-series analyses, hypothesis testing, and causal analyses to statistically assess the relative impact and extract trends
- Coordinate individual teams to fulfil client requirements and manage deliverable
- Communicate and present complex concepts to business audiences
- Travel to client locations when necessary
Crayon is an equal opportunity employer. Employment is based on a person's merit and qualifications and professional competences. Crayon does not discriminate against any employee or applicant because of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, marital status, pregnancy or related.
More about Crayon: https://www.crayondata.com/">https://www.crayondata.com/
More about http://maya.ai/">maya.ai: https://maya.ai/">https://maya.ai/
Basic Qualifications
- Over 8 years of experience and 5+ in API/mulesoft
- Expert with API technology including (REST, JSON, XML, Web Security, API Management, Public/Private APIs)
- Strong experience with MuleSoft cloud hub/AnyPoint Platform with proven experience architecting, designing, implementing enterprise services, and platform operations.
- Should be proficient in developing enterprise standards, guidelines, best practices and policies for the Mulesoft platform and drive adoption
- Should be able to provide technical expertise in bringing-in automation across API life cycle ( code quality, migration, deployment, platform monitoring etc) using tools such as App Dynamics, puppet, chef or any similar tools
- Should be hands-on and proficient in DevOps methodologies such as CICD, test automation etc
- Should be able to evangelize and work with enterprise architects on API driven architecture, platform roadmap
- Should be very proficient in planning, stakeholder management, team mgmt. and communication at all levels
- Provides overall API technical leadership working with Business, BAs, Tech lead, Scrum Master to clarify requirements and set technical direction for the team
Should be having overall years of experience of 7-12 years and having Team management ability.
Hands-on experience of minimum 5 years into Golang.
Proficient exp of minimum 2-3 years into DevOps* - end to end project implementation.
Strong expertise on DevOps concepts like Continuous Integration (CI), Continuous delivery (CD) and Infrastructure as Code, Cloud deployments.
Good experience of minimum 1-3 year into "Terraform".
Good experience of min 6months - 1 year into frontend technology like Vue.js
Good experience or working knowledge in AWS, Docker, Kubernetes, Helm etc.
Job Description:
- Design, implement and deliver custom solutions using the existing robotics framework.
- Debug issues, do root-cause analysis and apply fixes.
- Design and implement tools to facilitate application development and testing.
- Participate in architectural improvements.
- Work with team members in deployment and field testing.
Requirements:
- Bachelor Degree / Masters in Engineering (ECE or CSE preferred)
- Work experience of 3+ years in software programming.
- Proficiency in Python programming for Linux based systems.
- Full understanding of software engineering.
- Basic Knowledge of Robot Operating System(ROS) is a plus.
- Good understanding of the algorithms and control loops.
- Working knowledge of Git: creating, merging branches, cherry-picking commits, examining the diff between two hashes. Advanced Git usage is a plus.
- Knowledge of video streaming from edge devices is a plus.
- Thrive in a fast-paced environment and have the ability to own the project’s tasks end-to-end with minimum hand-holding
- Learn and adapt new technologies & skills. Work on projects independently with timely delivery & defect free approach.
2. Sound knowledge of Coding & Programming.
3. Have done any technical training.
4. Excellent Communication skills.
5. Positive attitude.
6. Have the zeal to learn new technologies.
- BE Computer Science, MCA or equivalent
- Cloud app development experience
- Strong Troubleshooting/Debugging experience
- Expert in RabbitMQ internals
- Experience with SQL and NOSQL databases
- Strong communication skills
Experience:
- Min 5 year experience
- Not more than 15 year experience.
- Startup experience is a must.
Location
● Remotely, anywhere in India
1
Timings:
- 40 hours a week but with 4 hours a day overlapping with client timezone. Typically
clients are in California PST Timezone.
Position:
- Full time/Direct
- We have great benefits such as PF, medical insurance, 12 annual company holidays, 12
PTO leaves per year, annual increments, Diwali bonus, spot bonuses and other
incentives etc.
- We dont believe in locking in people with large notice periods. You will stay here
because you love the company. We have only a 15 days notice period.