Amazon AWS optimizes the PyTorch CPU inference on AWS Graviton3 based C7g instances. PyTorch 2.0 improves inference performance on Graviton compared to the previous releases, including improvements for Resnet50 and Bert.
The optimizations focused on three key areas: GEMM kernels, bfloat16 support, primitive caching and the memory allocator. For aarch64 platforms, PyTorch supports Arm Compute Library (ACL) GEMM kernels via Mkldnn(OneDNN) backend. The ACL library provides Neon/SVE GEMM kernels for fp32 and bfloat16 formats. The bfloat16 support on c7g allows efficient deployment of bfloat16 trained, AMP (Automatic Mixed Precision) trained, or even the standard fp32 trained models. The standard fp32 models leverage bfloat16 kernels via OneDNN fast math mode, without any model quantization. Next we implemented primitive caching for conv, matmul and inner product operators. More information on the updated PyTorch user guide with the upcoming 2.0 release improvements and TorchBench benchmark details can be found
Today, we are announcing the general availability of Amazon Linux 2023(AL2023), our new Linux-based operating system for AWS that is designed to provide a secure, stable, high-performance environment to develop and run your cloud applications.
AL2023 provides seamless integration with various AWS services and development tools and offers optimized performance for Amazon Elastic Compute Cloud (EC2) Graviton-based instances and AWS Support at no additional licensing cost.
Starting with AL2023, a new Amazon Linux major release will be available every 2 years. This cadence provides you with a more predictable release cycle and up to 5 years of support, making it easier for you to plan your upgrades.
Tehama adopted Graviton technology from a strategic level to greatly improve business core competency. In about 6 months, Tehama has successfully migrated their whole technology stack to Graviton. The end result is that Tehama’s solution can run on both Intel and Graviton platforms, benefiting from the latest advances in silicon technologies.
Last month, AWS introduced new instance type families, powered by Graviton3 ARM processors: m7g and r7g. Since our go-to instance types for Altinity.Cloud are m5, m6i and m6g, I could not resist testing the performance of m7g. TLDR – it is outstanding!
AWS Lambda allows you to configure new and existing functions to run on Arm-based AWS Graviton2 processors in addition to x86-based functions.
On top, with this choice you can save money in two ways. First, your functions run more efficiently due to the Graviton2 architecture. Second, you pay less for the time that they run. In fact, Lambda functions powered by Graviton2 are designed to deliver up to 19 percent better performance at 20 percent lower cost.
With increasingly constrained budgets, understanding how to optimize spending without degrading performance is vital. SoftServe's Cost Optimization Accelerator can identify cloud savings opportunities in seconds.
Watch the re-play of SoftServe's Cloud Cost Optimization Accelerator Powered by AWS Graviton and learn more about SoftServe's no-cost tool that runs on AMI, is easy to use, and identifies immediate savings wins by workload.
Learn about our AWS Funded Assessment that empowers your engineers with a roadmap to achieve cost savings and performance enhancements through modernization.
AWS ECS is a powerful service that allows you to run containerized applications at scale. It's suitable for a variety of use cases, including web applications, microservices, and background processing. In this episode, we'll provide an introduction to the main concepts of ECS and then dive into cost-optimization strategies.
We'll explore the different options for running containers on ECS, including EC2, Fargate, and ECS Anywhere. We'll discuss various opportunities for saving money, such as using Arm (Graviton) instances, Spot instances, Compute Savings Plans, and RIs or EC2 Saving Plans. Finally, we'll cover how to set up ECS to use Spot instances, including how to create capacity providers and specify a capacity provider strategy. We'll also discuss whether it's always best to use EC2 instead of Fargate for cost optimization and recommend some tools that can help you find other opportunities to save on container costs.
At re:Invent 2022, Amazon RDS customers gained access to a number of exciting new features and capabilities. Two of these were Amazon RDS Optimized Writes and Amazon RDS Optimized Reads. Optimized Writes, built on top of the new AWS Nitro System Torn Write Prevention feature, allow you to improve your database’s write transaction throughput by up to 2x in RDS for MySQL at no additional cost. For customers looking for faster query processing capabilities can use Optimized Reads to achieve up to 2x faster query processing in Amazon RDS for MySQL and Amazon RDS for MariaDB at no additional cost.
It's often said that “Porting to Arm is boring,” but how easy is it, really? We'll demonstrate top machine learning frameworks, HPC applications, and tools for data science on the NVIDIA Arm HPC DevKit. The DevKit is an on-ramp platform for NVIDIA Grace CPU that incorporates dual NVIDIA A100 GPUs, an NVIDIA BlueField DPU, and an 80-core Ampere Altra Arm CPU, in a standards-compliant Arm server. We'll walk through the complete installation process for key applications and their dependencies including codes like TensorFlow, OpenRADIOSS, WRF, GROMACS, BWA-MEM2, and Jupyter Notebook. We'll also show how codes incorporating x86 AVX and SSE SIMD instructions can be trivially ported to Arm with freely available tools. We'll conclude with a general guide to porting to NVIDIA Grace and links to downloadable resources and tutorials that fully replicate our demonstrations. This session is a strong starting point for anyone targeting NVIDIA Grace Hopper or the NVIDIA Grace CPU Superchips.
AWS Container Services, including Amazon Elastic Kubernetes Service (EKS), Amazon Elastic Container Service (ECS), and AWS Fargate, streamline the deployment and management of software applications on AWS. This leads to a significant reduction in operational workload during the development, testing, and large-scale deployment of enterprise software products. Additionally, these services fully support the use of AWS Graviton-based EC2 instances, which enables customers to benefit from the cost savings associated with Graviton instances. Graviton offers the best price performance of all EC2 instances, so you can accelerate growth while you improve operating margins. Join us to learn how customers have realized price-performance improvements by switching to AWS Graviton for their container workloads. We will share best-practices and considerations when moving workloads to AWS Graviton
RAI Amsterdam, Europaplein 24 1078 GZ Amsterdam Nederland
April 18th - April 21st, 2023, 9am - 6pm CEST
Future proof your Kubernetes cluster for cost optimization
Cost optimization is a common priority. In this session, learn about the many factors that can increase costs in your Kubernetes usage beyond compute, such as compute efficiencies at the node and pod level, scaling parameters, networking cost, multi-architecture image creation, security posture, and more. Discover how to change your application components and migrate from x86-based instances to AWS Graviton to achieve cost efficiency at higher performance. Also, learn how to use open-source tools and AWS services to optimize these costs and make your Kubernetes cluster more resilient to economic instability.