Issue # 35: April 28th, 2023 to May 5th, 2023
[Read the browser version right here]
Welcome to Issue # 35 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from April 28th, 2023 to May, 5th, 2023.
This week, I received the incredible news that I will be part of the AWS Community Builders Program for a second year.
It's incredible to read that the work we are doing with this newsletter is being recognized by many people out there.
From the bottom of my heart, THANK YOU.
Back to business.
The recommended resources for this week?
- Read how you can use AWS Graviton 2 to optimize PyTorch inference
- How Instructure uses Graviton3 instances on video now
- the Cloud Cost Report for Q1 2023 by Vantage
Enjoy the content of this week
Are you looking for amazing professionals ready for your company?
Your search has ended with the Interesting Data Gigs Talent Collective.
People from Google, Interbank, Ivanti, Novartis and more global organizations are looking for a new gig today.
Sign up as a business today, and as an additional benefit, you can post at least two open roles in our Jobs Board, that can be seen by more than 17k people around all LinkedIn, Twitter, and our partner newsletter.
Graviton3-based instances provide up to a 30% performance
improvement and up to a 27%
price/performance improvement over Graviton2-based instances on RDS
for open-source databases depending on database engine, version, and workload. You can launch these database instances when using Amazon RDS for MySQL
, Amazon RDS for PostgreSQL
, and Amazon RDS for MariaDB
We are excited to announce the availability of ml.inf2 and ml.trn1 family of instances on Amazon SageMaker for deploying machine learning (ML) models for Real-time and Asynchronous inference.
You can use these instances on SageMaker to achieve high performance at a low cost for generative artificial intelligence (AI) models, including large language models (LLMs) and vision transformers. In addition, you can use SageMaker Inference Recommender to help you run load tests and evaluate the price-performance benefits of deploying your model on these instances.
Amazon Managed Streaming for Apache Kafka (Amazon MSK)
now supports Apache Kafka version 3.4.0 for new and existing clusters. Apache Kafka 3.4.0 includes several bug fixes and new features that improve performance. Key features include a fix to improve stability to fetch from the closest replica. Amazon MSK will continue to use and manage Zookeeper for quorum management in this release. For a complete list of improvements and bug fixes, see the Apache Kafka release notes for 3.4.0
An image worths more than 1,000 words
ARTICLES AND TUTORIALS
AWS measured up to 50% cost savings for PyTorch inference with AWS Graviton3-based Amazon Elastic Cloud Compute C7g instances across Torch Hub Resnet50, and multiple Hugging Face models relative to comparable EC2 instances.
These instances are available on SageMaker and Amazon EC2.
The AWS Graviton Technical Guide provides the list of optimized libraries and best practices that will help you achieve cost benefits with Graviton instances across different workloads.
In this post, we showcased the newly launched capability of SageMaker, which now supports ml.inf2 and ml.trn1 instances for hosting generative AI models. We demonstrated how to deploy GPT4ALL-J, a generative AI model, on AWS Inferentia2 using SageMaker and the LMI container, without writing any code. We also showcased how you can use DJLServing and
transformers-neuronx to load a model, partition it, and serve.
Before, running the job required an engineer to constantly monitor it; now, it runs in less than an hour with no support needed. In addition, the new implementation with AWS Step Functions Distributed Map costs less than what it did originally.
I am going to compare the following ARM-based systems:
- c7g.large: Amazon Graviton 3 running Ubuntu 22.04 (GCC 11)
- macBook Air 2022: Apple M2 LLVM 14
- Windows Dev Kit 2023: Qualcomm 8cx 3rd gen running Ubuntu 22.04 (GCC 11) inside WSL (Windows 11)
AWS Graviton processors are ARM-based, purpose-built processors for Cloud Native workloads. They leverage Nitro technology to provide maximum performance at any scale.
Recently, AWS introduced Graviton 3 instance types (m7g and R7g) for RDS Open-source databases. Those who are already part of the AWS ecosystem know the performance and cost benefits of Graviton instance types.
Furthermore, this new Graviton 3 instance type has many more benefits than the previous version. So we wanted to give a try on Graviton 3 and compared its performance to the Graviton 2. We did the benchmark using sysbench on MySQL 8.0.
SLIDES, VIDEOS, and AUDIO
Education technology company Instructure needed to increase its compute throughput while managing its scaling costs. The online learning solution provider needed to manage server load caused by massive remote learning traffic spikes as more students and teachers move their class online. Instructure wanted a viable solution for scaling compute power without degrading performance.
After migrating to AWS Graviton3-based Amazon EC2 instances, the company realized as much as 20% improvement in throughput and in cost savings versus older instances types.
When adopting AWS Graviton-based EC2 instances, a best practice is to measure the performance of your applications before and after the migration to ensure that you are getting the best out of the new processors. During this session, we'll cover the AWS Graviton processors architecture, explain the influence it has on application performance, and how to measure and analyse the performance of your applications.
Find more open roles here
Wednesday, May31 2023 | 8:30AM - 5:00PM PDTAmazon SJC18, 2100 University Avenue, East Palo Alto, CA94303
About the event
AWS presents the Graviton Essentials Developer Day. Graviton Essentials Developer Day is a free in-person immersion event where AWS Graviton experts deliver technically-focused Graviton-based Amazon EC2 training. This full-day event helps attendees learn best practices to accelerate migration and development of their workloads on Graviton-based instances. Attendees can expect to leave Graviton Essentials Developer Day feeling confident they will know how to achieve performance gains and cost reductions from using Graviton-based instances. Sessions will cover topics such as Introduction to Graviton, programming language and focused deep dives, testing and optimization techniques, and workload deployments. Space is limited, so sign up today.
Presenters & Hosts
AWS Graviton Specialist Solution Architects
About Graviton: AWS Graviton processors are custom Arm-based processors designed by Amazon Web Services (AWS). Graviton-based instances deliver up to 20% lower cost, up to 40% better price performance, and up to 60% less energy consumption than comparable x86-based instances. With Graviton-based instances, you can:
1. Increase and scale your application performance to enhance your customer experience
2. Reduce your operational cost
3. Support running workloads sustainably in the cloud
Who should attend
- Software Engineers
- Cloud Architects
- DevOps Engineers
- Platform Engineers
What to expect
- Introduction to Graviton
- Programming language & focused deep dives
- Testing & optimization techniques
- Workload deployments
- Hands-on working sessions
- Interaction with AWS Graviton SMEs
- Light breakfast, lunch, and happy hour
- Participants need to supply their own laptop and power supply
- Some coding knowledge
No prior Graviton experience is necessary
May 4, 2023 | SINGAPORE
Sands Expo & Convention Centre at Marina Bay Sands
Moving from x86-based Amazon EC2 instances to AWS Graviton Arm-based processors can save you a lot of money, with up to 40 percent better price performance. Can you simply update your AWS CloudFormation templates from c5 to c6g and reap the savings? In this session, learn about customer-proven strategies that can help you make the move to AWS Graviton confidently while minimising uncertainty and risk. Gain insights on identifying candidate workloads, performance testing, maintaining availability and flexibility, monitoring tools and release management.
A modern enterprise demands best-in-class solutions to remain secure, operational, and deliver digital-first experiences to drive growth. Modernisation is challenging, and moving from a traditional data centre to a hybrid cloud environment is a complex exercise. In this session, hear from Red Hat and AWS on how their customers overcame these complexities to build and implement a secure, robust and scalable cloud environment to match customer demands. Discover how this lead to sustainable long term cost savings, while enabling their technology teams to architect for the future by leveraging managed app platforms.
- Paul Whiten, Emerging Sales Specialist, Cloud Services, Red Hat
- Wayne Toh, Sr. Specialist SA, EC2 Graviton, AWS
From the ARM Ecosystem
Tweet of the week