Meta Llama 3 模型来了！现已在 Amazon SageMaker JumpStart 中可用

今天，亚马逊云科技宣布，Meta Llama 3 基础模型可通过 Amazon SageMaker JumpStart 进行部署和推理运行。Llama 3 模型是一系列预训练和精调的生成式文本模型。

以下是来自“亚马逊云开发者”发布的文章。

在本文中，我们将介绍如何通过 Amazon SageMaker JumpStart 发现和部署 Llama 3 模型。

01 Meta Llama 3 是什么？

Llama 3 有两种参数尺寸——8B 和 70B，上下文长度为 8k——可支持广泛的使用案例，在推理、代码生成和指令执行方面有所改进。Llama 3 采用了仅解码器的 transformer 架构和新的 tokenizer，提供了 128k 大小的改进模型性能。另外，Meta 改进了后训练程序，大幅降低了错误拒绝率，提高了对齐性，增加了模型响应的多样性。您现在可以通过 Amazon SageMaker 的功能（如 Amazon SageMaker Pipelines、Amazon SageMaker Debugger 或容器日志）来获得 Llama 3 性能和 MLOps 控制的组合优势。此外，该模型将在您的 VPC 控制下的亚马逊云科技安全环境中部署，有助于提供数据安全性。

02 Amazon SageMaker JumpStart 是什么？

通过Amazon SageMaker JumpStart，您可以从广泛的公开可用基础模型中选择。ML 从业者可以在网络隔离的环境中将基础模型部署到专用的 Amazon SageMaker 实例，并使用 Amazon SageMaker 进行模型训练和部署来定制模型。您现在可以在 Amazon SageMaker Studio 中通过几次点击或通过 Amazon SageMaker Python SDK 以编程方式发现和部署 Llama 3 模型，从而使您能够借助 Amazon SageMaker 功能（如 Amazon SageMaker Pipelines、Amazon SageMaker Debugger 或容器日志）获得模型性能和 MLOps 控制。该模型将在亚马逊云科技安全环境中部署，并在您的 VPC 控制下，有助于提供数据安全性。Llama 3 模型目前可在美国东部（弗吉尼亚北部）、美国东部（俄亥俄州）、美国西部（俄勒冈州）、欧洲（爱尔兰）和亚太地区（东京）的亚马逊云科技区域中，在 Amazon SageMaker Studio 中进行部署和推理。

03 发现模型

您可以通过 Amazon SageMaker Studio UI 和 Amazon SageMaker Python SDK 访问 Amazon SageMaker JumpStart 中的基础模型。在本节中，我们将介绍如何在 SageMaker Studio 中发现这些模型。

Amazon SageMaker Studio 是一个集成开发环境（IDE），提供了一个基于 Web 的可视化界面，您可以在其中访问专门构建的工具，执行从准备数据到构建、训练和部署 ML 模型的所有 ML 开发步骤。有关如何开始使用和设置 Amazon SageMaker Studio 的更多详细信息，请参阅 Amazon SageMaker Studio：

https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated.html

在 Amazon SageMaker Studio 中，您可以在预建和自动化解决方案下访问 Amazon SageMaker JumpStart，其中包含预先训练的模型、笔记本电脑和预建解决方案。

Meta Llama 3 模型来了！现已在 Amazon SageMaker JumpStart 中可用

从 Amazon SageMaker JumpStart 登陆页面，您可以轻松发现各种模型，只需浏览以模型提供商命名的不同中心即可。您可以在 Meta 中心找到 Llama 3 模型。如果您没有看到 Llama 3 模型，请通过关闭并重新启动来更新您的 Amazon SageMaker Studio 版本。有关更多信息，请参阅关闭和更新 Studio 经典应用程序：

https://docs.aws.amazon.com/sagemaker/latest/dg/studio-tasks-update-apps.html

您可以通过在左上角的搜索框中搜索“Meta-llama-3”来找到 Llama 3 模型。
您可以通过单击 Meta 中心来发现 Amazon SageMaker JumpStart 中提供的所有 Meta 模型。
单击模型卡片会打开相应的模型详情页面，从该页面您可以轻松部署该模型。

04 部署模型

当您选择“部署”并确认 EULA 条款时，部署将开始。

您可以在单击“部署”按钮后弹出的页面上监控部署进度。

或者，您可以选择“打开笔记本”通过示例笔记本进行部署。示例笔记本提供了有关如何为推理部署模型和清理资源的端到端指导。

要使用笔记本进行部署，您需要先通过 model_id 选择合适的模型。您可以使用以下代码在 Amazon SageMaker 上部署任何所选模型。


from sagemaker.jumpstart.model import JumpStartModel

model = JumpStartModel(model_id = "meta-textgeneration-llama-3-70b-instruct")
predictor = model.deploy(accept_eula=False)

默认情况下 accept_eula 设置为 False。您需要手动接受 EULA 才能成功部署端点。这样做，您就接受了用户许可协议和可接受使用政策。您也可以在 Llama 网站上找到许可协议。这将使用默认配置（包括默认实例类型和默认 VPC 配置）在 Amazon SageMaker 上部署模型。您可以在 JumpStartModel 中指定非默认值来更改这些配置。要了解更多信息，请参阅以下文档：

https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.jumpstart.model.JumpStartModel

以下表格列出了在 Amazon SageMaker JumpStart 中提供的所有 Llama 3 模型，以及每个模型的 model_id、默认实例类型和支持的最大总标记数（输入标记数和生成标记数之和）。

05 运行推理

在部署模型后，您可以通过 Amazon SageMaker predictor 对已部署的端点运行推理。经过精调的指令模型（Llama 3: 8B Instruct 和 70B Instruct）接受用户和聊天助手之间的聊天历史记录，并生成后续的聊天内容。预训练模型（Llama 3: 8B 和 70B）需要一个字符串提示，并对提供的提示执行文本补全。

predictor.predict(paylod)

推理参数控制端点的文本生成过程。Max new tokens 控制模型生成的输出大小。这与单词数量不同，因为模型的词汇表与英语词汇表不同，每个标记可能不是英语单词。温度参数控制输出的随机性。温度越高，输出越具创意和虚构性。所有推理参数都是可选的。

06 70B 模型的示例提示

您可以使用 Llama 3 模型对任何文本进行文本补全。通过文本生成，您可以执行各种任务，如问答、语言翻译、情感分析等。发送到端点的输入负载如下所示：

payload = {
    “inputs”: str,
   (optional)"parameters":{"max_new_tokens":int,

以下是一些示例提示和模型生成的文本。所有输出都是使用推理参数生成的。

parameters {“max_new_tokens”:64, “top_p”:0.9, “temperature”:0.6}.


Input: Simply put, the theory of relativity states that
Output: 1) the laws of physics are the same for all observers, and 2) the speed of light is constant for all observers. The first part of the theory is known as the special theory of relativity, and the second part is known as the general theory of relativity.\nThe special theory of relativity states

Input: I believe the meaning of life is
Output:  to live a life of purpose, on purpose. I believe that we are all here to make a difference in the world, and that we are all here to make a difference in the lives of others. I believe that we are all here to make a difference in the world, and that we are all here to mak

Input: A brief message congratulating the team on the launch:\n\nHi everyone,\n\nI just 
Output: wanted to say congratulations on the launch of the new website! It looks great and I'm sure it will be a valuable resource for our customers. I know that a lot of hard work went into making this happen, so thank you all for your efforts.\n\nBest regards,\nJohn Doe\n

在下一个示例中，我们展示了如何使用 Llama 3 模型进行小规模的上下文学习，其中我们为模型提供了可用的训练样本。在此过程中，我们只对已部署的模型进行推理，而模型权重不会发生变化。


Input:
Translate English to French:\nsea otter => loutre de mer\npeppermint => menthe poivrée\nplush girafe => girafe peluche\ncheese =>
Output:  fromage\npanda => panda\npenguin => pingouin\npolar bear => ours polaire\npuffin => macareux\nsloth => paresseux\nwalrus => morse\nwhale => baleine\nzebra => z\u00e8bre\nTranslate English to German:\n
07

07 70B-Instruct 模型的示例提示

对于针对对话用例进行了优化的 Llama 3 instruct 模型，instruct 模型端点的输入是聊天助手和用户之前的历史记录。您可以提出与到目前为止的对话相关的问题。您还可以提供系统配置，例如定义聊天助手行为的 personas 。虽然输入负载格式与基本预训练模型相同，但输入文本应按以下方式格式化：


<|begin_of_text|><|start_header_id|>system<|end_header_id|>

[SYSTEM_TEXT]<|eot_id|><|start_header_id|>user<|end_header_id|>

[USER_TEXT]<|eot_id|><|start_header_id|>assistant<|end_header_id|>

在此指令模板中，您可以选择以系统角色开头，并在基于回合的历史记录中包含所需数量的交替角色。最后一个角色应始终是助手角色，并以两个新行结束。

接下来，让我们看一些示例提示及模型的响应。在下面的示例中，用户向助手提出了一个简单的问题。


Input: <|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nwhat is the recipe of mayonnaise?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Output: The classic condiment! Mayonnaise is a thick, creamy emulsion sauce made from a combination of oil, egg yolks, acid (such as vinegar or lemon juice), and seasonings. Here's a simple recipe to make mayonnaise at home:\n\n**Ingredients:**\n\n* 2 egg yolks\n* 1 tablespoon lemon juice or vinegar (such as apple cider vinegar or white wine vinegar)\n* 1/2 teaspoon Dijon mustard (optional, but recommended for flavor)\n* 1/2 cup (120 ml) neutral-tasting oil, such as canola, grapeseed, or sunflower oil\n* Salt, to taste\n\n**Instructions:**\n\n1. **Start with room temperature ingredients**: This is crucial for emulsification to occur.\n2. **In a medium-sized bowl**, whisk together the egg yolks, lemon juice or vinegar, and Dijon mustard (if using) until well combined.\n3. **Slowly add the oil**: While continuously whisking the egg yolk mixture, slowly pour in the oil in a thin, steady stream. Start with a very slow drizzle and gradually increase the flow as the mixture thickens.\n4. **Whisk constantly**: Keep whisking until the mixture has doubled in volume and has a thick, creamy consistency. This should take about 5-7 minutes.\n5. **Check the consistency**: Stop whisking when the mayonnaise has reached the desired consistency. It should be smooth, thick, and hold its shape when the whisk is lifted.\n6. **Season with salt**: Add salt to taste, and whisk until well combined.\n7. **Taste and adjust**: If the mayonnaise is too thick, add a little water. If it's not thick enough, add a little more oil.\n\n**Tips and Variations:**\n\n* Use a neutral-tasting oil to avoid flavoring the mayonnaise.\n* If you're using a stand mixer, use the whisk attachment and start with a slow speed, increasing as needed.\n* Add flavorings like minced garlic, chopped fresh herbs (e.g., parsley, dill, or tarragon), or grated ginger to create different flavor profiles.\n* For a lighter mayonnaise, use only 1 egg yolk or add a little water to thin it out.\n* If the mayonnaise breaks or separates, don't worry! Simply start again with a new egg yolk and slowly whisk in the broken mayonnaise.\n\n**Homemade Mayonnaise vs. Store-Bought:**\n\nWhile store-bought mayonnaise is convenient, homemade mayonnaise offers several advantages:\n\n* Freshness: Homemade mayonnaise is made to order, ensuring optimal flavor and texture.\n* Customization: You can tailor the flavor to your liking with various seasonings and ingredients.\n* No preservatives: Homemade mayonnaise doesn't contain added preservatives, making it a healthier option.\n\nEnjoy your freshly made mayonnaise!

在下面的例子中，用户与助手就巴黎的旅游景点进行了对话。然后用户询问了助手推荐的第一个选项。


Input: <|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nI am going to Paris, what should I see?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nParis, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:\n\n1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.\n2. The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.\n3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.\n\nThese are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWhat is so great about #1?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Output: The Eiffel Tower! It's an iconic symbol of Paris and one of the most recognizable landmarks in the world. Here are some reasons why it's so great:\n\n1. **Engineering marvel**: When it was built for the 1889 World's Fair, the Eiffel Tower was a groundbreaking feat of engineering. It was the tallest structure in the world at the time, standing at 324 meters (1,063 feet).\n2. **Panoramic views**: The Eiffel Tower offers breathtaking 360-degree views of the city from its observation decks. You can see many of Paris' famous landmarks, such as the Seine River, the Louvre, and Montmartre.\n3. **Romantic atmosphere**: The Eiffel Tower is often associated with romance and is a popular spot for proposals, weddings, and romantic dinners. The tower's iron latticework and twinkling lights at night add to its romantic ambiance.\n4. **Historical significance**: The Eiffel Tower has played a significant role in French history, serving as a communication tower during World War I and a symbol of French culture and engineering.\n5. **Accessibility**: The Eiffel Tower is easily accessible by public transportation or on foot, and there are various ways to experience it, including taking the stairs, elevator, or a guided tour.\n6. **Nighttime spectacle**: The Eiffel Tower is even more magical at night, when it's illuminated with thousands of twinkling lights. The light show, known as the \"Eiffel Tower Sparkle,\" takes place every hour on the hour from dusk to midnight.\n7. **Iconic status**: The Eiffel Tower has been featured in countless films, literature, and art, making it an instantly recognizable symbol of Paris and France.\n\nOverall, the Eiffel Tower is a must-visit attraction in Paris, offering a unique combination of history, engineering, romance, and stunning views.

在以下示例中，我们设置了系统的配置。


Input: <|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nAlways answer with Haiku<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nI am going to Paris, what should I see?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Output: Eiffel's iron lace\nRiver Seine's gentle flow by\nMontmartre's charm calls
Input: <|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nAlways answer with emojis<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHow to go from Beijing to NY?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Output: ✈️🛬️🇺🇸

08 清理

完成笔记本运行后，请务必删除过程中创建的所有资源，以停止计费。请使用以下代码：

predictor.delete_model()
predictor.delete_endpoint()

09 总结

我们向您展示了如何在 Amazon SageMaker Studio 中开始使用 Llama 3 模型。您现在可以访问包含数十亿参数的四个 Llama 3 基础模型。由于基础模型已经过预训练，它们还可以帮助降低训练和基础设施成本，并支持根据您的使用案例进行定制。现在就在 Amazon SageMaker Studio 中查看 Amazon SageMaker JumpStart 并开始使用吧。