profile

Marcos Ortiz

AWS Graviton Weekly # 28: PyTorch 2.0 Optimized for Graviton3 and a Graviton-focused virtual event you can't miss

Published about 1 year ago • 7 min read

You're receiving this because you subscribed here OR here​
​

This email may contain affiliate links. I receive a small commission for recommending products I use & love at no extra cost to you.

​[Read the browser version right here]​

​

​Brought to you in partnership with C​onvertKit​

​

Issue # 28: March 10th, 2023 to March 17th, 2023

Hey Reader

Welcome to Issue # 28 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from March 10th, 2023 to March 17th, 2023.

Enjoy the content of this week.

The recommended readings for this week? 2 things

  • PyTorch 2.0 was highly optimized to take advantage of Graviton3
  • The Graviton-focused virtual event you can't miss

Brought to you by ConvertKit​

Get your newsletter recommended by like-minded creators. Connect, collaborate, and build your business–together.

​

NEWS

​

​PyTorch 2.0 Optimized for Graviton3​

Amazon AWS optimizes the PyTorch CPU inference on AWS Graviton3 based C7g instances. PyTorch 2.0 improves inference performance on Graviton compared to the previous releases, including improvements for Resnet50 and Bert.
The optimizations focused on three key areas: GEMM kernels, bfloat16 support, primitive caching and the memory allocator. For aarch64 platforms, PyTorch supports Arm Compute Library (ACL) GEMM kernels via Mkldnn(OneDNN) backend. The ACL library provides Neon/SVE GEMM kernels for fp32 and bfloat16 formats. The bfloat16 support on c7g allows efficient deployment of bfloat16 trained, AMP (Automatic Mixed Precision) trained, or even the standard fp32 trained models. The standard fp32 models leverage bfloat16 kernels via OneDNN fast math mode, without any model quantization. Next we implemented primitive caching for conv, matmul and inner product operators. More information on the updated PyTorch user guide with the upcoming 2.0 release improvements and TorchBench benchmark details can be found

Learn more here:

​https://pytorch.org/blog/pytorch-2.0-release/​

​

​

​Announcing Amazon Linux 2023​

Today, we are announcing the general availability of Amazon Linux 2023(AL2023), our new Linux-based operating system for AWS that is designed to provide a secure, stable, high-performance environment to develop and run your cloud applications.
AL2023 provides seamless integration with various AWS services and development tools and offers optimized performance for Amazon Elastic Compute Cloud (EC2) Graviton-based instances and AWS Support at no additional licensing cost.
Starting with AL2023, a new Amazon Linux major release will be available every 2 years. This cadence provides you with a more predictable release cycle and up to 5 years of support, making it easier for you to plan your upgrades.

Learn more here:

​https://aws.amazon.com/linux/amazon-linux-2023/​

​

​

​

​API Gateway Trends behind Features: Apache APISIX 3.0 vs. Kong 3.0​

ARM64 has become a very mainstream server architecture for cloud vendors. Various cloud vendors have begun rolling out servers based on Arm architecture like AWS Graviton and GCP Tau T2AA.
Apache APISIX has done a comprehensive CI regression test on ARM64 to ensure smoothness when users run Apache APISIX under the Arm architecture. Users care a lot about this.
​

Learn more here:

​https://api7.ai/blog/apisix-vs-kong-3-0​

​https://apisix.apache.org/blog/2022/07/22/how-is-google-cloud-tau-t2a-performing/​

​

​

Articles and Tutorials

​Tehama leverages Graviton cost efficiency to strengthen business core competency, by Cedric Hu (AWS Solutions Architect at Amazon Web Services (AWS)), Ahmed Elhosary (Technical Account Manager at Amazon Web Services (AWS)) and Ken Bantoft​

Tehama adopted Graviton technology from a strategic level to greatly improve business core competency. In about 6 months, Tehama has successfully migrated their whole technology stack to Graviton. The end result is that Tehama’s solution can run on both Intel and Graviton platforms, benefiting from the latest advances in silicon technologies.

Learn more here:

​https://aws.amazon.com/blogs/industries/tehama-leverages-graviton-cost-efficiency-to-strengthen-business-core-competency/​

​

​

​Ultra-Fast AWS Graviton Instances in Altinity.Cloud, by Alexander Zaitsev (CTO at Altinity.Cloud)

Last month, AWS introduced new instance type families, powered by Graviton3 ARM processors: m7g and r7g. Since our go-to instance types for Altinity.Cloud are m5, m6i and m6g, I could not resist testing the performance of m7g. TLDR – it is outstanding!

Learn more here:

​https://altinity.com/blog/ultra-fast-aws-graviton-instances-in-altinity-cloud​

​

​

​Moving AWS Lambda functions with Golang to Graviton chips, by Stathis Peioglou (Senior Solution Architect at AWS)

AWS Lambda allows you to configure new and existing functions to run on Arm-based AWS Graviton2 processors in addition to x86-based functions.
On top, with this choice you can save money in two ways. First, your functions run more efficiently due to the Graviton2 architecture. Second, you pay less for the time that they run. In fact, Lambda functions powered by Graviton2 are designed to deliver up to 19 percent better performance at 20 percent lower cost.

Learn more here:

​https://medium.com/@spei/moving-aws-lambda-functions-with-golang-on-graviton-based-chips-1996f794b82e​

​

Slides, Videos and Audio

​[VIDEO] SoftServe's Cloud Cost Optimization Accelerator Powered by AWS Graviton​

With increasingly constrained budgets, understanding how to optimize spending without degrading performance is vital. SoftServe's Cost Optimization Accelerator can identify cloud savings opportunities in seconds.
Watch the re-play of SoftServe's Cloud Cost Optimization Accelerator Powered by AWS Graviton and learn more about SoftServe's no-cost tool that runs on AMI, is easy to use, and identifies immediate savings wins by workload.
Learn about our AWS Funded Assessment that empowers your engineers with a roadmap to achieve cost savings and performance enhancements through modernization.

​

​

​[VIDEO] How do you save cost with ECS? by Eoin Shanaghy and Luciano Mammino​

AWS ECS is a powerful service that allows you to run containerized applications at scale. It's suitable for a variety of use cases, including web applications, microservices, and background processing. In this episode, we'll provide an introduction to the main concepts of ECS and then dive into cost-optimization strategies.
We'll explore the different options for running containers on ECS, including EC2, Fargate, and ECS Anywhere. We'll discuss various opportunities for saving money, such as using Arm (Graviton) instances, Spot instances, Compute Savings Plans, and RIs or EC2 Saving Plans. Finally, we'll cover how to set up ECS to use Spot instances, including how to create capacity providers and specify a capacity provider strategy. We'll also discuss whether it's always best to use EC2 instead of Fargate for cost optimization and recommend some tools that can help you find other opportunities to save on container costs.

​

​

​[VIDEO] AWS CFM Talks - Tackle 4 common questions to manage and optimize your cloud costs, with Savanna Jensen, Rosa Corley and Lisa Harnett​

In this AWS CFM Talks webinar, our experts answer four common customer questions related to managing and optimizing cloud costs, so you can kickstart your optimization efforts with confidence.

​

​

​[VIDEO] Improving performance with Amazon RDS Optimized Writes & Optimized Reads - AWS Databases in 15, with Komal Pal​

At re:Invent 2022, Amazon RDS customers gained access to a number of exciting new features and capabilities. Two of these were Amazon RDS Optimized Writes and Amazon RDS Optimized Reads. Optimized Writes, built on top of the new AWS Nitro System Torn Write Prevention feature, allow you to improve your database’s write transaction throughput by up to 2x in RDS for MySQL at no additional cost. For customers looking for faster query processing capabilities can use Optimized Reads to achieve up to 2x faster query processing in Amazon RDS for MySQL and Amazon RDS for MariaDB at no additional cost.

​

Jobs

​

Events

[EVENT] Thursday, March 23th 9:00 AM - 9:50 AM GMT -05 A Demonstration of AI and HPC Applications for NVIDIA Grace CPU [S51880]

It's often said that “Porting to Arm is boring,” but how easy is it, really? We'll demonstrate top machine learning frameworks, HPC applications, and tools for data science on the NVIDIA Arm HPC DevKit. The DevKit is an on-ramp platform for NVIDIA Grace CPU that incorporates dual NVIDIA A100 GPUs, an NVIDIA BlueField DPU, and an 80-core Ampere Altra Arm CPU, in a standards-compliant Arm server. We'll walk through the complete installation process for key applications and their dependencies including codes like TensorFlow, OpenRADIOSS, WRF, GROMACS, BWA-MEM2, and Jupyter Notebook. We'll also show how codes incorporating x86 AVX and SSE SIMD instructions can be trivially ported to Arm with freely available tools. We'll conclude with a general guide to porting to NVIDIA Grace and links to downloadable resources and tutorials that fully replicate our demonstrations. This session is a strong starting point for anyone targeting NVIDIA Grace Hopper or the NVIDIA Grace CPU Superchips.

​

​

​[Virtual Event] Accelerate SaaS revenue and improve price-performance with AWS Graviton​

Thursday, March30 2023 | 9:00AM - 11:00AM PDT​
​

AWS Container Services, including Amazon Elastic Kubernetes Service (EKS), Amazon Elastic Container Service (ECS), and AWS Fargate, streamline the deployment and management of software applications on AWS. This leads to a significant reduction in operational workload during the development, testing, and large-scale deployment of enterprise software products.
​
Additionally, these services fully support the use of AWS Graviton-based EC2 instances, which enables customers to benefit from the cost savings associated with Graviton instances. Graviton offers the best price performance of all EC2 instances, so you can accelerate growth while you improve operating margins.
​
Join us to learn how customers have realized price-performance improvements by switching to AWS Graviton for their container workloads. We will share best-practices and considerations when moving workloads to AWS Graviton

Speakers:

​

​

​AWS at KubeCon + CloudNativeCon Europe 2023​

RAI Amsterdam, Europaplein 24 1078 GZ Amsterdam Nederland

April 18th - April 21st, 2023, 9am - 6pm CEST

​

Future proof your Kubernetes cluster for cost optimization

Cost optimization is a common priority. In this session, learn about the many factors that can increase costs in your Kubernetes usage beyond compute, such as compute efficiencies at the node and pod level, scaling parameters, networking cost, multi-architecture image creation, security posture, and more. Discover how to change your application components and migrate from x86-based instances to AWS Graviton to achieve cost efficiency at higher performance. Also, learn how to use open-source tools and AWS services to optimize these costs and make your Kubernetes cluster more resilient to economic instability.

​

Speakers:

​
​

​

Tweet of the week

twitter profile avatar
theCUBE
Twitter Logo
@theCUBE
March 13th 2023
3
Retweets
7
Likes
facebooklinkedin

​

Marcos Ortiz

I'm a Data Engineer by day at Riot Games (via X-Team ) and by night, I curate the last news/product announcements/resources about AWS Silicon (Graviton, AWS Nitro, Inferentia, and Trainium).

Read more from Marcos Ortiz
AWS Silicon Content Library

Issue # 83: April 12, 2024 to April 19, 2024 Hey Reader. Welcome to Issue # 83 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from April 12, 2024, to April 19, 2024. I shared a little bit about last week's issue, and today is the day to finally reveal to you what I was working on a long time ago. It's called the Cloud Silicon Content Library But first, let me talk about the PROBLEM When I started working on this...

4 days ago • 2 min read

Issue # 82: April 5, 2024 to April 12, 2024 Hey Reader. Welcome to Issue # 82 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from April 5, 2024, to April 12, 2024. This week has been very quiet related to AWS Silicon because most headlines were occupied by Google Next, especially the new Arm-based CPU called Axion, which looks very interesting and promising. But one thing I noticed was that more and more times we will...

11 days ago • 3 min read

Issue # 81: March 29, 2024 to April 5, 2024 Hey Reader. Welcome to Issue # 81 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from March 29, 2024, to April 5, 2024. Things you can't miss in this issue: How Quora modernized its model serving with NVIDIA Triton and Amazon EKS Now, back to business. Tool of the week: Antimetal New website, new product; same incredible possible outcome if your company uses this tool to...

18 days ago • 1 min read
Share this post