Arrow Left
profile

Marcos Ortiz

AWS Graviton Weekly # 35

Published about 1 year ago • 9 min read

Issue # 35: April 28th, 2023 to May 5th, 2023

​[Read the browser version right here]​

​

​In partnership with the I​nteresting Data Gigs Talent Collective

Hey Reader

Welcome to Issue # 35 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from April 28th, 2023 to May, 5th, 2023.

This week, I received the incredible news that I will be part of the AWS Community Builders Program for a second year.

It's incredible to read that the work we are doing with this newsletter is being recognized by many people out there.

From the bottom of my heart, THANK YOU.

Back to business.

The recommended resources for this week?

  • Read how you can use AWS Graviton 2 to optimize PyTorch inference
  • How Instructure uses Graviton3 instances on video now
  • the Cloud Cost Report for Q1 2023 by Vantage

​

Enjoy the content of this week

Brought to you by the Interesting Data Gigs Talent Collective​

Are you looking for amazing professionals ready for your company?

Your search has ended with the Interesting Data Gigs Talent Collective.

People from Google, Interbank, Ivanti, Novartis and more global organizations are looking for a new gig today.

Sign up as a business today, and as an additional benefit, you can post at least two open roles in our Jobs Board, that can be seen by more than 17k people around all LinkedIn, Twitter, and our partner newsletter.

​

​

​

NEWS

​Amazon RDS now supports M7g and R7g database instances​

AWS Graviton3-based M7g and R7g database instances are now generally available for Amazon Relational Database Service (RDS).
Graviton3-based instances provide up to a 30% performance improvement and up to a 27% price/performance improvement over Graviton2-based instances on RDS for open-source databases depending on database engine, version, and workload. You can launch these database instances when using Amazon RDS for MySQL, Amazon RDS for PostgreSQL, and Amazon RDS for MariaDB.

Learn more:

​https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-rds-m7g-r7g-database-instances/​

​

​SageMaker announces ml.inf2 and ml.trn1 instances for model deployment​

We are excited to announce the availability of ml.inf2 and ml.trn1 family of instances on Amazon SageMaker for deploying machine learning (ML) models for Real-time and Asynchronous inference.
You can use these instances on SageMaker to achieve high performance at a low cost for generative artificial intelligence (AI) models, including large language models (LLMs) and vision transformers. In addition, you can use SageMaker Inference Recommender to help you run load tests and evaluate the price-performance benefits of deploying your model on these instances.

Learn more:

​https://aws.amazon.com/about-aws/whats-new/2023/05/sagemaker-ml-inf2-ml-trn1-instances-model-deployment/​

​https://github.com/aws/deep-learning-containers/blob/master/available_images.md​

​

​Amazon MSK adds support for Apache Kafka version 3.4.0​

​Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 3.4.0 for new and existing clusters. Apache Kafka 3.4.0 includes several bug fixes and new features that improve performance. Key features include a fix to improve stability to fetch from the closest replica. Amazon MSK will continue to use and manage Zookeeper for quorum management in this release. For a complete list of improvements and bug fixes, see the Apache Kafka release notes for 3.4.0.

Learn more:

​https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-msk-apache-kafka-version-3-4-0/​

​

​Vantage Cloud Cost Report: Q1 2023​

An image worths more than 1,000 words

​

ARTICLES AND TUTORIALS

​Optimized PyTorch 2.0 inference with AWS Graviton processors, by Sunita Nadampalli (Software Development Manager at AWS)

AWS measured up to 50% cost savings for PyTorch inference with AWS Graviton3-based Amazon Elastic Cloud Compute C7g instances across Torch Hub Resnet50, and multiple Hugging Face models relative to comparable EC2 instances.
​
These instances are available on SageMaker and Amazon EC2.
The AWS Graviton Technical Guide provides the list of optimized libraries and best practices that will help you achieve cost benefits with Graviton instances across different workloads.

Learn more:

​https://aws.amazon.com/blogs/machine-learning/optimized-pytorch-2-0-inference-with-aws-graviton-processors/​

​https://github.com/aws/aws-graviton-getting-started​

​

​

​Achieve high performance with lowest cost for generative AI inference using AWS Inferentia2 and AWS Trainium on Amazon SageMaker, by Vivek Gangasani (Senior Machine Learning Solutions Architect at AWS), Hiroshi Tokoyo ( Solutions Architect at AWS Annapurna Labs), Dhawal Patel (Principal Machine Learning Architect at AWS), Qing Lan (Software Development Engineer in AWS), Qingwei Li (Machine Learning Specialist at Amazon Web Services), Alan Tan (Senior Product Manager with SageMaker) and Varun Syal (Software Development Engineer with AWS Sagemaker)

In this post, we showcased the newly launched capability of SageMaker, which now supports ml.inf2 and ml.trn1 instances for hosting generative AI models. We demonstrated how to deploy GPT4ALL-J, a generative AI model, on AWS Inferentia2 using SageMaker and the LMI container, without writing any code. We also showcased how you can use DJLServing and transformers-neuronx to load a model, partition it, and serve.

Learn more:

​https://aws.amazon.com/blogs/machine-learning/achieve-high-performance-with-lowest-cost-for-generative-ai-inference-using-aws-inferentia2-and-aws-trainium-on-amazon-sagemaker/​

​https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/benchmarks/inf2/inf2-performance.html#inf2-performance​

​https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/generativeai/llm-workshop/lab8-Inferentia2-gpt4all-j/inferentia2-llm-GPT4allJ.ipynb​

​https://huggingface.co/nomic-ai/gpt4all-j​

​

​How CyberGRX cut ML processing time from 8 days to 56 minutes with AWS Step Functions Distributed Map, by Marcia Villalba (Principal Developer Advocate for Amazon Web Services)

Last December, Sébastien Stormacq wrote about the availability of a distributed map state for AWS Step Functions, a new feature that allows you to orchestrate large-scale parallel workloads in the cloud. That’s when Charles Burton, a data systems engineer for a company called CyberGRX, found out about it and refactored his workflow, reducing the processing time for his machine learning (ML) processing job from 8 days to 56 minutes.
Before, running the job required an engineer to constantly monitor it; now, it runs in less than an hour with no support needed. In addition, the new implementation with AWS Step Functions Distributed Map costs less than what it did originally.

Learn more:

​https://aws.amazon.com/blogs/aws/how-cybercrx-cut-ml-processing-time-from-8-days-to-56-minutes-with-aws-step-functions-distributed-map/​

​https://aws.amazon.com/blogs/aws/step-functions-distributed-map-a-serverless-solution-for-large-scale-parallel-data-processing/​

​https://www.cybergrx.com/​

​

​

​Graviton 3, Apple M2 and Qualcomm 8cx 3rd gen: a URL parsing benchmark, by Daniel Lemire (Computer Science Professor at Université du Québec (TÉLUQ))

I am going to compare the following ARM-based systems:
- c7g.large: Amazon Graviton 3 running Ubuntu 22.04 (GCC 11)
​
- macBook Air 2022: Apple M2 LLVM 14
​
- Windows Dev Kit 2023: Qualcomm 8cx 3rd gen running Ubuntu 22.04 (GCC 11) inside WSL (Windows 11)

Read more:

​https://lemire.me/blog/2023/05/03/graviton-3-apple-m2-and-qualcomm-8cx-3rd-gen-a-url-parsing-benchmark/​

​

​

​

​AWS Graviton 3 vs Graviton 2 — A SysBench on RDS MySQL, by Bhuvanesh R (CTO and co-founder of ShellKode)

AWS Graviton processors are ARM-based, purpose-built processors for Cloud Native workloads. They leverage Nitro technology to provide maximum performance at any scale.
Recently, AWS introduced Graviton 3 instance types (m7g and R7g) for RDS Open-source databases. Those who are already part of the AWS ecosystem know the performance and cost benefits of Graviton instance types.
Furthermore, this new Graviton 3 instance type has many more benefits than the previous version. So we wanted to give a try on Graviton 3 and compared its performance to the Graviton 2. We did the benchmark using sysbench on MySQL 8.0.

Read more:

​https://blog.shellkode.com/aws-graviton-3-vs-graviton-2-a-sysbench-on-rds-mysql-629bce9b56e2​

​

​

SLIDES, VIDEOS, and AUDIO

​[VIDEO] Instructure used Graviton3 to increase performance while keeping costs down, with Zach Pendleton (Chief Architect at Instructure)

Education technology company Instructure needed to increase its compute throughput while managing its scaling costs. The online learning solution provider needed to manage server load caused by massive remote learning traffic spikes as more students and teachers move their class online. Instructure wanted a viable solution for scaling compute power without degrading performance.
​
After migrating to AWS Graviton3-based Amazon EC2 instances, the company realized as much as 20% improvement in throughput and in cost savings versus older instances types.

Read more:

​https://aws.amazon.com/solutions/case-studies/instructure-case-study/​

​

​[VIDEO] Introduction to performance measurement, analysis & optimisation on AWS EC2 Graviton-based instances, by Arthur Petitpierre (EC2 Graviton Principal Specialist Solutions Architect at AWS)

When adopting AWS Graviton-based EC2 instances, a best practice is to measure the performance of your applications before and after the migration to ensure that you are getting the best out of the new processors. During this session, we'll cover the AWS Graviton processors architecture, explain the influence it has on application performance, and how to measure and analyse the performance of your applications.

Learn more:

​https://aws.amazon.com/ec2/graviton/fast-start/​

​

​

JOBS

​Find more open roles here​

​

EVENTS

​Graviton Essentials - Developer Day​

Wednesday, May31 2023 | 8:30AM - 5:00PM PDTAmazon SJC18, 2100 University Avenue, East Palo Alto, CA94303

About the event

AWS presents the Graviton Essentials Developer Day. Graviton Essentials Developer Day is a free in-person immersion event where AWS Graviton experts deliver technically-focused Graviton-based Amazon EC2 training. This full-day event helps attendees learn best practices to accelerate migration and development of their workloads on Graviton-based instances. Attendees can expect to leave Graviton Essentials Developer Day feeling confident they will know how to achieve performance gains and cost reductions from using Graviton-based instances. Sessions will cover topics such as Introduction to Graviton, programming language and focused deep dives, testing and optimization techniques, and workload deployments. Space is limited, so sign up today.

​
​Presenters & Hosts

AWS Graviton Specialist Solution Architects

​
​About Graviton: AWS Graviton processors are custom Arm-based processors designed by Amazon Web Services (AWS). Graviton-based instances deliver up to 20% lower cost, up to 40% better price performance, and up to 60% less energy consumption than comparable x86-based instances. With Graviton-based instances, you can:
​
1. Increase and scale your application performance to enhance your customer experience
2. Reduce your operational cost
3. Support running workloads sustainably in the cloud

​
Who should attend
​

  • Software Engineers
  • Cloud Architects
  • DevOps Engineers
  • Platform Engineers

​

What to expect
​

  • Introduction to Graviton
  • Programming language & focused deep dives
  • Testing & optimization techniques
  • Workload deployments
  • Hands-on working sessions
  • Interaction with AWS Graviton SMEs
  • Light breakfast, lunch, and happy hour

​

Requirements
​

  • Participants need to supply their own laptop and power supply
  • Some coding knowledge

​
​No prior Graviton experience is necessary

​

​AWS Summit ASEAN 2023

May 4, 2023 | SINGAPORE​
Sands Expo & Convention Centre at Marina Bay Sands

​Improving price-performance with AWS Graviton-based instances​

Moving from x86-based Amazon EC2 instances to AWS Graviton Arm-based processors can save you a lot of money, with up to 40 percent better price performance. Can you simply update your AWS CloudFormation templates from c5 to c6g and reap the savings? In this session, learn about customer-proven strategies that can help you make the move to AWS Graviton confidently while minimising uncertainty and risk. Gain insights on identifying candidate workloads, performance testing, maintaining availability and flexibility, monitoring tools and release management.

Speakers:

  • ​Chetan Suri, Enterprise Support Lead, AWS
  • ​Marc Venturini, Senior Software Engineering Manager, Build Automation, Grab

​

​Unlocking innovation from 24x7 Operations to 9x5 Innovation​

A modern enterprise demands best-in-class solutions to remain secure, operational, and deliver digital-first experiences to drive growth. Modernisation is challenging, and moving from a traditional data centre to a hybrid cloud environment is a complex exercise. In this session, hear from Red Hat and AWS on how their customers overcame these complexities to build and implement a secure, robust and scalable cloud environment to match customer demands. Discover how this lead to sustainable long term cost savings, while enabling their technology teams to architect for the future by leveraging managed app platforms.

Speakers:

  • ​Paul Whiten, Emerging Sales Specialist, Cloud Services, Red Hat
  • ​Wayne Toh, Sr. Specialist SA, EC2 Graviton, AWS

​

​

From the ARM Ecosystem

Tweet of the week

​

twitter profile avatar
AWS Developers
Twitter Logo
@awsdevelopers
Want to use AWS Graviton Processors in Amazon CodeCatalyst, but need x86 too? Check out this post in the #AWS #DevOps blog about multi-architecture container builds in CodeCatalyst, and get the best of both worlds! 🌎🏆 go.aws/3LlonGg
link visual
Multi-Architecture Container Builds with CodeCat...
AWS Graviton Processors are designed by AWS to deliver the best price performance for your cloud wor...
go.aws
April 28th 2023
1
Retweets
6
Likes
facebooklinkedin

​

Marcos Ortiz

I'm a Data Engineer by day at Riot Games (via X-Team ) and by night, I curate the last news/product announcements/resources about AWS Silicon (Graviton, AWS Nitro, Inferentia, and Trainium).

Read more from Marcos Ortiz
AWS Graviton Weekly # 97: How Amazon’s New CPU Fights Cybersecurity Threats?

Issue # 97: July 19, 2024 to July 26, 2024 Hey Reader. Welcome to Issue # 97 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from July 19, 2024 to July 26, 2024. Enjoy. Recommendation of the week: CAST.AI NEWS AWS Step Functions now supports Customer Managed Keys Llama 3.1 models from Meta are now available on AWS, offering more options for building generative AI applications AWS Lambda now supports Amazon MQ for...

about 17 hours ago • 1 min read

Issue # 96: July 11, 2024 to July 19, 2024 Hey Reader. Welcome to Issue # 96 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from July 11, 2024 to July 19, 2024. Before continuing with the regular content: #hugops for CrowdStrike for the Microsoft Windows BSOD issue today. BTW, Cristian Măgherușan-Stanciu worked on some Terraform automation tool to fix this issue on AWS. He is looking for testers. Back to business now...

8 days ago • 1 min read

Issue # 95: July 5, 2024 to July 11, 2024 Hey Reader. Welcome to Issue # 95 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from July 5, 2024 to July 11, 2024. Enjoy. Recommendation of the week: CAST.AI NEWS Amazon EC2 R8g instances powered by AWS Graviton4 now generally available AWS Neuron introduces Flash Attention kernel enabling high performance and large sequence lengths Announcing availability of AWS Outposts in...

15 days ago • 1 min read