Issue # 35: April 28th, 2023 to May 5th, 2023
[Read the browser version right here]
In partnership with the Interesting Data Gigs Talent Collective
Hey Reader
Welcome to Issue # 35 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from April 28th, 2023 to May, 5th, 2023.
This week, I received the incredible news that I will be part of the AWS Community Builders Program for a second year.
It's incredible to read that the work we are doing with this newsletter is being recognized by many people out there.
From the bottom of my heart, THANK YOU.
Back to business.
The recommended resources for this week?
- Read how you can use AWS Graviton 2 to optimize PyTorch inference
- How Instructure uses Graviton3 instances on video now
- the Cloud Cost Report for Q1 2023 by Vantage
Enjoy the content of this week
Brought to you by the Interesting Data Gigs Talent Collective
Are you looking for amazing professionals ready for your company?
Your search has ended with the Interesting Data Gigs Talent Collective.
People from Google, Interbank, Ivanti, Novartis and more global organizations are looking for a new gig today.
Sign up as a business today, and as an additional benefit, you can post at least two open roles in our Jobs Board, that can be seen by more than 17k people around all LinkedIn, Twitter, and our partner newsletter.
| Get access today |
NEWS
Amazon RDS now supports M7g and R7g database instances
AWS Graviton3-based M7g and R7g database instances are now generally available for Amazon Relational Database Service (RDS).
Graviton3-based instances provide up to a 30% performance improvement and up to a 27% price/performance improvement over Graviton2-based instances on RDS for open-source databases depending on database engine, version, and workload. You can launch these database instances when using Amazon RDS for MySQL, Amazon RDS for PostgreSQL, and Amazon RDS for MariaDB.
Learn more:
https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-rds-m7g-r7g-database-instances/
SageMaker announces ml.inf2 and ml.trn1 instances for model deployment
We are excited to announce the availability of ml.inf2 and ml.trn1 family of instances on Amazon SageMaker for deploying machine learning (ML) models for Real-time and Asynchronous inference.
You can use these instances on SageMaker to achieve high performance at a low cost for generative artificial intelligence (AI) models, including large language models (LLMs) and vision transformers. In addition, you can use SageMaker Inference Recommender to help you run load tests and evaluate the price-performance benefits of deploying your model on these instances.
Learn more:
https://github.com/aws/deep-learning-containers/blob/master/available_images.md
Amazon MSK adds support for Apache Kafka version 3.4.0
Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 3.4.0 for new and existing clusters. Apache Kafka 3.4.0 includes several bug fixes and new features that improve performance. Key features include a fix to improve stability to fetch from the closest replica. Amazon MSK will continue to use and manage Zookeeper for quorum management in this release. For a complete list of improvements and bug fixes, see the Apache Kafka release notes for 3.4.0.
Learn more:
https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-msk-apache-kafka-version-3-4-0/
Vantage Cloud Cost Report: Q1 2023
An image worths more than 1,000 words
ARTICLES AND TUTORIALS
Optimized PyTorch 2.0 inference with AWS Graviton processors, by Sunita Nadampalli (Software Development Manager at AWS)
AWS measured up to 50% cost savings for PyTorch inference with AWS Graviton3-based Amazon Elastic Cloud Compute C7g instances across Torch Hub Resnet50, and multiple Hugging Face models relative to comparable EC2 instances.
These instances are available on SageMaker and Amazon EC2.
The AWS Graviton Technical Guide provides the list of optimized libraries and best practices that will help you achieve cost benefits with Graviton instances across different workloads.
Learn more:
https://github.com/aws/aws-graviton-getting-started
Achieve high performance with lowest cost for generative AI inference using AWS Inferentia2 and AWS Trainium on Amazon SageMaker, by Vivek Gangasani (Senior Machine Learning Solutions Architect at AWS), Hiroshi Tokoyo ( Solutions Architect at AWS Annapurna Labs), Dhawal Patel (Principal Machine Learning Architect at AWS), Qing Lan (Software Development Engineer in AWS), Qingwei Li (Machine Learning Specialist at Amazon Web Services), Alan Tan (Senior Product Manager with SageMaker) and Varun Syal (Software Development Engineer with AWS Sagemaker)
In this post, we showcased the newly launched capability of SageMaker, which now supports ml.inf2 and ml.trn1 instances for hosting generative AI models. We demonstrated how to deploy GPT4ALL-J, a generative AI model, on AWS Inferentia2 using SageMaker and the LMI container, without writing any code. We also showcased how you can use DJLServing and
transformers-neuronx to load a model, partition it, and serve.Learn more:
https://huggingface.co/nomic-ai/gpt4all-j
How CyberGRX cut ML processing time from 8 days to 56 minutes with AWS Step Functions Distributed Map, by Marcia Villalba (Principal Developer Advocate for Amazon Web Services)
Last December, Sébastien Stormacq wrote about the availability of a distributed map state for AWS Step Functions, a new feature that allows you to orchestrate large-scale parallel workloads in the cloud. That’s when Charles Burton, a data systems engineer for a company called CyberGRX, found out about it and refactored his workflow, reducing the processing time for his machine learning (ML) processing job from 8 days to 56 minutes.
Before, running the job required an engineer to constantly monitor it; now, it runs in less than an hour with no support needed. In addition, the new implementation with AWS Step Functions Distributed Map costs less than what it did originally.
Learn more:
Graviton 3, Apple M2 and Qualcomm 8cx 3rd gen: a URL parsing benchmark, by Daniel Lemire (Computer Science Professor at Université du Québec (TÉLUQ))
I am going to compare the following ARM-based systems:
- c7g.large: Amazon Graviton 3 running Ubuntu 22.04 (GCC 11)
- macBook Air 2022: Apple M2 LLVM 14
- Windows Dev Kit 2023: Qualcomm 8cx 3rd gen running Ubuntu 22.04 (GCC 11) inside WSL (Windows 11)
Read more:
AWS Graviton 3 vs Graviton 2 — A SysBench on RDS MySQL, by Bhuvanesh R (CTO and co-founder of ShellKode)
AWS Graviton processors are ARM-based, purpose-built processors for Cloud Native workloads. They leverage Nitro technology to provide maximum performance at any scale.
Recently, AWS introduced Graviton 3 instance types (m7g and R7g) for RDS Open-source databases. Those who are already part of the AWS ecosystem know the performance and cost benefits of Graviton instance types.
Furthermore, this new Graviton 3 instance type has many more benefits than the previous version. So we wanted to give a try on Graviton 3 and compared its performance to the Graviton 2. We did the benchmark using sysbench on MySQL 8.0.
Read more:
https://blog.shellkode.com/aws-graviton-3-vs-graviton-2-a-sysbench-on-rds-mysql-629bce9b56e2
SLIDES, VIDEOS, and AUDIO
[VIDEO] Instructure used Graviton3 to increase performance while keeping costs down, with Zach Pendleton (Chief Architect at Instructure)
Education technology company Instructure needed to increase its compute throughput while managing its scaling costs. The online learning solution provider needed to manage server load caused by massive remote learning traffic spikes as more students and teachers move their class online. Instructure wanted a viable solution for scaling compute power without degrading performance.
After migrating to AWS Graviton3-based Amazon EC2 instances, the company realized as much as 20% improvement in throughput and in cost savings versus older instances types.
Read more:
https://aws.amazon.com/solutions/case-studies/instructure-case-study/
[VIDEO] Introduction to performance measurement, analysis & optimisation on AWS EC2 Graviton-based instances, by Arthur Petitpierre (EC2 Graviton Principal Specialist Solutions Architect at AWS)
When adopting AWS Graviton-based EC2 instances, a best practice is to measure the performance of your applications before and after the migration to ensure that you are getting the best out of the new processors. During this session, we'll cover the AWS Graviton processors architecture, explain the influence it has on application performance, and how to measure and analyse the performance of your applications.
Learn more:
https://aws.amazon.com/ec2/graviton/fast-start/
JOBS
EVENTS
Graviton Essentials - Developer Day
Wednesday, May31 2023 | 8:30AM - 5:00PM PDTAmazon SJC18, 2100 University Avenue, East Palo Alto, CA94303
About the event
AWS presents the Graviton Essentials Developer Day. Graviton Essentials Developer Day is a free in-person immersion event where AWS Graviton experts deliver technically-focused Graviton-based Amazon EC2 training. This full-day event helps attendees learn best practices to accelerate migration and development of their workloads on Graviton-based instances. Attendees can expect to leave Graviton Essentials Developer Day feeling confident they will know how to achieve performance gains and cost reductions from using Graviton-based instances. Sessions will cover topics such as Introduction to Graviton, programming language and focused deep dives, testing and optimization techniques, and workload deployments. Space is limited, so sign up today.
Presenters & Hosts
AWS Graviton Specialist Solution Architects
About Graviton: AWS Graviton processors are custom Arm-based processors designed by Amazon Web Services (AWS). Graviton-based instances deliver up to 20% lower cost, up to 40% better price performance, and up to 60% less energy consumption than comparable x86-based instances. With Graviton-based instances, you can:
About Graviton: AWS Graviton processors are custom Arm-based processors designed by Amazon Web Services (AWS). Graviton-based instances deliver up to 20% lower cost, up to 40% better price performance, and up to 60% less energy consumption than comparable x86-based instances. With Graviton-based instances, you can:
1. Increase and scale your application performance to enhance your customer experience
1. Increase and scale your application performance to enhance your customer experience
2. Reduce your operational cost
3. Support running workloads sustainably in the cloud
3. Support running workloads sustainably in the cloud
Who should attend
- Software Engineers
- Cloud Architects
- DevOps Engineers
- Platform Engineers
What to expect
- Introduction to Graviton
- Programming language & focused deep dives
- Testing & optimization techniques
- Workload deployments
- Hands-on working sessions
- Interaction with AWS Graviton SMEs
- Light breakfast, lunch, and happy hour
Requirements
- Participants need to supply their own laptop and power supply
- Some coding knowledge
No prior Graviton experience is necessary
AWS Summit ASEAN 2023
May 4, 2023 | SINGAPORE
Sands Expo & Convention Centre at Marina Bay Sands
Improving price-performance with AWS Graviton-based instances
Moving from x86-based Amazon EC2 instances to AWS Graviton Arm-based processors can save you a lot of money, with up to 40 percent better price performance. Can you simply update your AWS CloudFormation templates from c5 to c6g and reap the savings? In this session, learn about customer-proven strategies that can help you make the move to AWS Graviton confidently while minimising uncertainty and risk. Gain insights on identifying candidate workloads, performance testing, maintaining availability and flexibility, monitoring tools and release management.
Speakers:
- Chetan Suri, Enterprise Support Lead, AWS
- Marc Venturini, Senior Software Engineering Manager, Build Automation, Grab
Unlocking innovation from 24x7 Operations to 9x5 Innovation
A modern enterprise demands best-in-class solutions to remain secure, operational, and deliver digital-first experiences to drive growth. Modernisation is challenging, and moving from a traditional data centre to a hybrid cloud environment is a complex exercise. In this session, hear from Red Hat and AWS on how their customers overcame these complexities to build and implement a secure, robust and scalable cloud environment to match customer demands. Discover how this lead to sustainable long term cost savings, while enabling their technology teams to architect for the future by leveraging managed app platforms.
Speakers:
- Paul Whiten, Emerging Sales Specialist, Cloud Services, Red Hat
- Wayne Toh, Sr. Specialist SA, EC2 Graviton, AWS
From the ARM Ecosystem
- The fastest Arm desktop PC on the planet by Jeff Geerling
- High IPO valuation for Arm would be a warning for investors, by Financial Times (from my perspective, Arm will be OK)
- How Arm is making it easier to build platforms that support Confidential Computing, by Mark Knight (Director of Architecture Product Management at Arm)
- What's new in LLVM 16, by Pablo Barrio (Staff software engineer and Product Owner at Arm)
Tweet of the week
April 28th 2023
|
