vLLM

Deploy Meta Llama 3.1-8B on AWS Inferentia using Amazon EKS and vLLM | Amazon Web Services

With the rise of large language models (LLMs) like Meta Llama 3.1, there is an increasing need for scalable, reliable, and cost-effective solutions to deploy and serve these models. AWS Trainium and AWS Inferentia based instances, combined with Amazon Elastic Kubernetes Service (Amazon EKS), provide a performant and low costContinue Reading

Serving LLMs using vLLM and Amazon EC2 instances with AWS AI chips | Amazon Web Services

In: Artificial Intelligence

The use of large language models (LLMs) and generative AI has exploded over the last year. With the release of powerful publicly available foundation models, tools for training, fine tuning and hosting your own LLM have also become democratized. Using vLLM on AWS Trainium and Inferentia makes it possible toContinue Reading

Deploy Meta Llama 3.1-8B on AWS Inferentia using Amazon EKS and vLLM | Amazon Web Services

Serving LLMs using vLLM and Amazon EC2 instances with AWS AI chips | Amazon Web Services

Add Zoom as a data accessor to your Amazon Q index | Amazon Web Services

The future of quality assurance: Shift-left testing with QyrusAI and Amazon Bedrock | Amazon Web Services