项目简介：

接下来，小李哥将会每天介绍一个基于亚马逊云科技AWS云计算平台的全球前沿AI技术解决方案，帮助大家快速了解国际上最热门的云计算平台亚马逊云科技AWS AI最佳实践，并应用到自己的日常工作里。本次介绍的是如何在Amazon SageMaker上使用大语言模型Meta Llama 7B，提供可扩展和安全的AI解决方案。通过Amazon API Gateway和AWS Lambda将应用程序与AI模型集成。本方案的解决方案架构图如下：

本方案将主要使用亚马逊云科技AWS上的大模型/机器学习模型托管服务Amazon SageMaker，下面我们介绍一下该服务。

什么是Amazon SageMaker？

Amazon SageMaker是一款由亚马逊云科技提供的全面模型托管服务，旨在帮助开发者和数据科学家快速、轻松地构建、训练和部署机器学习模型。SageMaker原生集成了机器行业热门的工具和框架，使用户能够专注于模型的开发和优化，而无需担心基础设施的管理。

通过SageMaker，用户可以使用预构建的算法和框架，或者将自己的自定义代码带入平台进行训练。其自动化的数据处理和模型训练功能，可以大大减少模型开发的时间和复杂性。此外，SageMaker还提供了云托管Jupyter Notebook，用户可以使用本地熟悉的训练模型工具，以便无缝从本地迁移到云端模型训练。

在模型训练完成后，SageMaker可以将模型部署为托管的推理服务，配置API节点，确保高可用性和可扩展性。借助SageMaker，用户可以无缝地集成机器学习模型到应用程序中，例如通过API Gateway和AWS Lambda来实现实时推理服务。

Amazon SageMaker还具备内置的监控和调优功能，使得模型的性能优化和管理更加高效。无论是初学者还是专业数据科学家，SageMaker都是一种理想的解决方案，帮助企业快速实现人工智能和机器学习的价值。

本方案包括的内容：

本方案主要包括如下内容：

1. 使用Amazon SageMaker部署基础AI/ML模型(Meta Llama 7B)作为推理的应用节点

  1. 在云原生代码托管服务AWS Lambda部署代码以调用SageMaker推理。

3. 使用测试应用程序为已部署的模型进行功能测试。

项目搭建具体步骤：

下面跟着小李哥手把手搭建一个亚马逊云科技AWS上的生成式AI模型（Meta Llama 7B）的软件应用，并且测试大模型的多种应用场景下的表现。

  1. 首先进入SageMarker

  2. 点击进入Jumpstart->Foundation Models，查看目前SageMaker上现有的开源ML模型

  3. 进入Studio，点击已经创建好的“Open Studio”

  4. 点击“Studio Classic”，再点击右侧Open

  5. 打开SageMaker Studio控制台

  6. 下载存放在S3中的ML模型工程代码：

aws s3 sync s3://<Replace with lab-code bucket name> .

  1. 配置ML模型训练、预测的Python工程环境

  2. 在第一个代码单元中更新和安装SageMaker SDK

!pip install sagemaker --quiet --upgrade --force-reinstall

  1. 配置需要的ML开源模型ID和版本（Meta Llama 7B）

model_id, model_version, = ( "huggingface-llm-falcon-7b-instruct-bf16", "*", )

  1. 创建SageMaker上的API端点，供应用访问

``` %%time from sagemaker.jumpstart.model import JumpStartModel

mymodel = JumpStartModel(modelid=modelid,instancetype="ml.g5.2xlarge" ) predictor = my_model.deploy() ```

项目测试阶段：

11. 利用编写好的模型提示词，对部署的模型进行问答测试。

``` %%time

prompt = "Tell me about Amazon SageMaker."

payload = { "inputs": prompt, "parameters": { "dosample": True, "topp": 0.9, "temperature": 0.8, "maxnewtokens": 1024, "stop": ["<|endoftext|>", ""] } }

response = predictor.predict(payload) print(response[0]["generated_text"]) ```

得到测试结果

Amazon SageMaker is a machine learning platform provided by Amazon Web Services that enables users to train and deploy machine learning models without having to build, train, and manage a machine learning infrastructure. CPU times: user 13.5 ms, sys: 6.66 ms, total: 20.1 ms Wall time: 1.38 s

12. 对大模型代码生成功能进行测试：

payload = {"inputs": "Write a program to compute factorial in python:", "parameters":{"max_new_tokens": 200}} query_endpoint(payload)

得到测试结果

``` Input: Write a program to compute factorial in python: Output: Here is a Python program to compute factorial:

```python def factorial(n): if n == 0: return 1 else: return n * factorial(n-1)

print(factorial(5)) # Output: 120

13. 对大模型进行推理、任务完成能力测试

payload = { "inputs": "Building a website can be done in 10 simple steps:", "parameters":{ "max_new_tokens": 110, "no_repeat_ngram_size": 3 } } query_endpoint(payload)

得到测试结果

Input: Building a website can be done in 10 simple steps: Output: 1. Choose a domain name 2. Register a domain name 3. Choose a web hosting provider 4. Create a website design 5. Add content to your website 6. Test your website 7. Optimize your website for search engines 8. Promote your website 9. Update your website regularly 10. Monitor your website for security

14. 对大模型代码进行语言翻译测试：

``` payload = { "inputs": """Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese =>""",
"parameters":{
    "max_new_tokens": 3
}

}

query_endpoint(payload) ```

得到测试结果

``` Input: Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese =>

Output: fromage ```

15. 对大模型的基于文字的情绪分析能力进行测试

payload = { "inputs": """"I hate it when my phone battery dies." Sentiment: Negative ### Tweet: "My day has been :+1:" Sentiment: Positive ### Tweet: "This is the link to the article" Sentiment: Neutral ### Tweet: "This new music video was incredibile" Sentiment:""", "parameters": { "max_new_tokens":2 } } query_endpoint(payload)

得到测试结果

Input: "I hate it when my phone battery dies." Sentiment: Negative ### Tweet: "My day has been :+1:" Sentiment: Positive ### Tweet: "This is the link to the article" Sentiment: Neutral ### Tweet: "This new music video was incredibile" Sentiment: Output: Positive

16. 对大模型的问答能力进行测试2

payload = { "inputs": "Could you remind me when was the C programming language invented?", "parameters":{ "max_new_tokens": 50 } } query_endpoint(payload)

得到测试结果

Input: Could you remind me when was the C programming language invented? Output: The C programming language was invented in 1972 by Dennis Ritchie at Bell Labs.

17.利用大模型生成食谱

payload = {"inputs": "What is the recipe for a delicious lemon cheesecake?", "parameters":{"max_new_tokens": 400}} query_endpoint(payload)

得到测试结果

``` Input: What is the recipe for a delicious lemon cheesecake? Output: Here is a recipe for a delicious lemon cheesecake:

Ingredients: - 1 1/2 cups graham cracker crumbs - 4 tablespoons butter, melted - 2 (8 ounce) packages cream cheese, softened - 1/2 cup granulated sugar - 2 eggs - 1/2 cup lemon juice - 1/2 teaspoon salt - 1/2 teaspoon vanilla extract - 1/2 cup heavy cream - 1/2 cup granulated sugar - 1/2 teaspoon lemon zest

Instructions: 1. Preheat oven to 350 degrees F. 2. In a medium bowl, mix together the graham cracker crumbs and melted butter. Press the mixture onto the bottom and sides of a 9-inch springform pan. 3. In a large bowl, beat the cream cheese and sugar until smooth. Add the eggs, lemon juice, salt, vanilla, and heavy cream. Beat until well combined. 4. Pour the mixture into the prepared pan. 5. Bake for 30 minutes or until the cheesecake is set. 6. Let cool for 10 minutes before serving. 7. In a small bowl, mix together the lemon zest and sugar. Sprinkle over the cheesecake before serving. Step 4.2.7 : Test summarization¶ ```

18. 测试大模型的文字总结能力

payload = { "inputs":"""Amazon SageMaker is a fully managed machine learning service. With SageMaker, data scientists and developers can quickly and easily build and train machine learning models, and then directly deploy them into a production-ready hosted environment. It provides an integrated Jupyter authoring notebook instance for easy access to your data sources for exploration and analysis, so you don't have to manage servers. It also provides common machine learning algorithms that are optimized to run efficiently against extremely large data in a distributed environment. With native support for bring-your-own-algorithms and frameworks, SageMaker offers flexible distributed training options that adjust to your specific workflows. Deploy a model into a secure and scalable environment by launching it with a few clicks from SageMaker Studio or the SageMaker console. Summarize the article above:""", "parameters":{ "max_new_tokens":200 } } query_endpoint(payload)

得到测试结果

Input: Amazon SageMaker is a fully managed machine learning service. With SageMaker, data scientists and developers can quickly and easily build and train machine learning models, and then directly deploy them into a production-ready hosted environment. It provides an integrated Jupyter authoring notebook instance for easy access to your data sources for exploration and analysis, so you don't have to manage servers. It also provides common machine learning algorithms that are optimized to run efficiently against extremely large data in a distributed environment. With native support for bring-your-own-algorithms and frameworks, SageMaker offers flexible distributed training options that adjust to your specific workflows. Deploy a model into a secure and scalable environment by launching it with a few clicks from SageMaker Studio or the SageMaker console. Summarize the article above: Output: SageMaker is a cloud-based machine learning platform that provides a range of tools and services to help data scientists and developers build, train, and deploy machine learning models. It offers an integrated Jupyter notebook environment, optimized algorithms, and flexible distributed training options.

  1. 完成所有测试后，我们获取大模型在SageMaker上的URL端点，作为API供应用使用。

以上就是在亚马逊云科技上部署开源大模型，并且多场景测试的全部步骤。欢迎大家关注小李哥，未来获取更多国际前沿的生成式AI开发方案。