这个人工智能打破了所有记录

Gemma2–2B 是一颗冉冉升起的新星，它超越了175亿的竞争对手。以下是如何在您的电脑上本地运行它的方法。

金质奖章记录 — 由作者修改自

那里有一个新的模型。低于30亿参数是我测试过的最好的！ Gemma2 总结，提出新话题，提出问题，并提供基于真实的回答。可以像多语者一样切换语言，并保持对话流畅进行。谷歌在人工智能领域一直是先锋，开发了多种在不同任务中表现出色的模型。在他们最引人注目的创造中，我们必须提到BERT（双向编码器表示从变换器）。这种开创性的模型在推进自然语言理解方面发挥了重要作用。BERT能够双向处理文本，同时考虑前后文，显著提高了其在各种自然语言处理任务上的性能。 BigG 已经探索了变换器架构，试图从编码器和解码器模型中榨取最佳性能。最后一个出生的，并且具有开放权重的是Gemma 2。它是几天前才出现的，并且已经

breaking all the records

感叹号！ * 这个生成式人工智能拥有26.1亿参数，足够小，可以在任何消费级硬件上运行 * 即使在仅使用CPU的情况下，您也拥有不错的速度 * 8k tokens的上下文长度使我们能够进行RAG（Retrieval-Augmented Generation，检索增强生成）、上下文学习以及思维链 * 能够相当流利地说多种语言：英语、意大利语、法语、德语

注意：可能出于安全原因（并且因为还有

ShieldGemma

在Gemma2系列中，该模型没有系统提示。在这篇文章中，我们将学习如何在您的个人电脑上完全运行这个拥有20亿参数的模型，不需要API调用，也不需要互联网连接。一个功能强大的人工智能，免费安装在你的笔记本电脑上！

Gemma2 在您的笔记本电脑上 — 图片由

lexica.art

吉玛家族

在六月份，谷歌

released Gemma 2

，一种新的同类最佳开放模型，具有270亿（27B）和90亿（9B）参数规模。自首次亮相以来，27B模型迅速成为排名最高的开放模型之一。

LMSYS Chatbot Arena leaderboard

在真实对话中，甚至超过了其规模两倍以上的流行模型。

但Gemma不仅仅是关于性能。它建立在负责任的人工智能基础之上，优先考虑安全性和可访问性。为了支持这一承诺，我们在7月31日收到了一个令人兴奋的公告：Gemma 2家族新增了三个成员，对我们来说是免费的： 1.

Gemma 2 2B

我们广受欢迎的20亿（2B）参数模型的全新版本，具有内置的安全进步和强大的性能与效率平衡。 2.

ShieldGemma

— 一套基于Gemma 2构建的安全内容分类器模型，用于过滤人工智能模型的输入和输出，确保用户安全。 3.

Gemma Scope

一种新的模型可解释性工具，为我们的模型内部工作原理提供了无与伦比的洞察力。

这张图表显示了从最新的Gemma 2检查点开始的性能以及基准预训练指标。从

Source

Gemma 2B：人工智能的小巨人

让我们开始大声说出来：Gemma 2B 是一个口袋大小的强大设备，正在撼动人工智能世界。这款Gemma家族的最新成员虽然体积小巧，但性能出色，令人印象深刻。建立在负责任的人工智能基础之上，Gemma 2B 不仅仅是关于原始能力；它是关于利用这种能力做好事。谷歌已经融入了最先进的安全特性，以确保 Gemma 2B 不仅智能而且安全。几天后，那些家伙在

@GoogleDeepMind

成功实现了不可能的事情！刚刚发布的Gemma-2 2B在聊天机器人竞技场超越了所有GPT-3.5模型。 看这个！一个2B参数模型超越了GPT-3.5（175B+参数）——几乎难以置信。

Gemma 2 2B：2亿参数模型在聊天机器人竞技场上的表现超过了GPT-3.5。来自

Source

有关于本地运行Gemma2 2b - fabiomatricardi/Gemma2-2b-it-chatbot

github.com

https://github.com/fabiomatricardi/Gemma2-2b-it-chatbot)> 请注意，Streamlit 1.36.0 不支持 Python 3.10 及更早的版本。 ```

create a new directory

mkdir Gemma2-2b

browse the new directory

cd Gemma2-2b

Create a virtual environment

python -m venv venv

Activate the venv (on windows)

venvScriptsactivate ```

一旦激活了venv，我们就可以安装包：我保留了功能所需的最小集合（图形界面、计数令牌和速度）。 pip install streamlit==1.36.0 llama-cpp-python==0.2.85 tiktoken

现在唯一缺少的部分是模型：我使用了托管在Hugging Face上的模型。

Bartowski repo

我们在名为model的子目录中保存模型。请注意，如果您克隆

my GitHub repo

你已经找到了为你创建的模型和图像的子文件夹，但你仍然需要自己下载GGUF文件。您可以从那里下载它

HF page model card

点击 gemma-2–2b-it-Q5_K_M.gguf 下载按钮图标或者从终端 wget https://huggingface.co/bartowski/gemma-2-2b-it-GGUF/resolve/main/gemma-2-2b-it-Q5_K_M.gguf -OutFile model/gemma-2-2b-it-Q5_K_M.gguf

我们都准备好了。基本推理

在任何 llama-cpp-python 模型调用的核心，实际上只有3行代码。其他一切都是微调和接口。我们导入了库 from llama_cpp import Llama

我们加载模型（我使用了多行以便于阅读，但它可以是单行）。 llm = Llama( model_path='model/gemma-2-2b-it-Q5_K_M.gguf', #n_gpu_layers=0, temperature=0.24, n_ctx=8196, max_tokens=600, repeat_penalty=1.176, stop=['<eos>'], verbose=False, )

最重要的参数包括 model_path（模型路径）、n_ctx（模型的上下文长度）和 stop（一个令牌或单词列表，用于避免模型生成无限序列……）。最后我们调用推理：有几种方法可以调用生成。如果你有一个聊天模型，你会使用 create_chat_completion 方法，但基本的 __call__ 也是可以接受的。主要的区别在于，对于聊天完成，模型将自动应用官方分词器中的聊天模板格式，在 __call__ 方法中，您必须相应地正确格式化提示。让我们来看一下。使用聊天完成进行推理

您可以在o中找到解释

fficial documentation for llama-cpp-python,

但是为了简化，你需要类似这样的东西： print(llm.create_chat_completion( messages=[{'role':'user','content':'What is Science?'}], stop=['<eos>']))

我仅使用换行符来提高可读性。重要的是聊天消息，它是一个遵循的字典列表。

the standard chat template format

您可以在这里阅读更多信息： [聊天模型的模板]

我们正在踏上一条通过开源和开放科学推进和民主化人工智能的旅程。

Hugging Face

https://huggingface.co/docs/transformers/main/chat_templating)我移除了所有其他调整参数，比如temperature和repeat_penalty，但我保留了stop（见上面的注释）。代码将打印一个特殊对象，而不仅仅是回复。让我们来看一下： {'id': 'chatcmpl-1b0f37ba-c6e0-4f04-ad65-ae16417b460a', 'object': 'chat.completion', 'created': 1723417873, 'model': 'model/gemma-2-2b-it-Q5_K_M.gguf', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "Science is a systematic and organized way of understanding the world around us. It's a process of asking questions, gathering evidence, and forming explanations based on that evidence. Here's a breakdown of what science is: **Key Characteristics:** * **Empirical:** Science relies on observation and experimentation. It's not based on faith or personal beliefs. * **Objective:** Scientists strive to be unbiased and avoid personal opinions influencing their work. * **Testable:** Scientific claims must be falsifiable, meaning they can be proven wrong through evidence. * **Replicable:** Experiments should be repeatable by other scientists to ensure the results are reliable. * **Predictive:** Science aims to make predictions about future events based on existing knowledge. * **Dynamic:** Science is constantly evolving as new discoveries are made and existing theories are revised. **What Science Does:** * **Explains the natural world:** Science seeks to understand the laws that govern the universe, from the smallest particles to the largest galaxies. * **Develops technologies:** Scientific discoveries often lead to new technologies that improve our lives. * **Addresses societal issues:** Science plays a crucial role in solving problems like climate change, disease, and poverty. * **Promotes critical thinking:** Science encourages us to question assumptions, analyze evidence, and develop logical reasoning skills. **Different Branches of Science:** Science is not a single field but rather a vast collection of disciplines, each focusing on a specific area of study. Some examples include: * **Physics:** The study of matter, energy, and their interactions. * **Chemistry:** The study of matter and its properties. * **Biology:** The study of living organisms. * **Astronomy:** The study of celestial objects and phenomena. * **Geology:** The study of Earth's physical structure and history. * **Psychology:** The study of the human mind and behavior. **In Conclusion:** Science is a powerful tool for understanding the world and solving problems. It's a process of continuous discovery, driven by curiosity, observation, and critical thinking. Let me know if you'd like to learn more about a specific aspect of science! "}, 'logprobs': None, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 13, 'completion_tokens': 454, 'total_tokens': 467}}

llama-cpp-python 的文档将输出对象描述为

来自

read the docs

要访问我们所需的文本，我们需要一些路径查找能力。因为我们需要消息内容，我们向后查找父元素，看起来像这样： print(output['choices'][0]['message']['content'])

这将给你一个更好、更美丽的文本： ``` Science is a systematic and organized way of understanding the world around us. It's a process of asking questions, gathering evidence, and forming explanations based on that evidence.

Here's a breakdown of what science is:

Key Characteristics:

Empirical: Science relies on observation and experimentation. It's not based on faith or personal beliefs.
Objective: Scientists strive to be unbiased and avoid personal opinions influencing their work.
Testable: Scientific claims must be falsifiable, meaning they can be proven wrong through evidence.
Replicable: Experiments should be repeatable by other scientists to ensure the results are reliable.
Predictive: Science aims to make predictions about future events based on existing knowledge.
Dynamic: Science is constantly evolving as new discoveries are made and existing theories are revised.

What Science Does:

Explains the natural world: Science seeks to understand the laws that govern the universe, from the smallest particles to the largest galaxies.
Develops technologies: Scientific discoveries often lead to new technologies that improve our lives.
Addresses societal issues: Science plays a crucial role in solving problems like climate change, disease, and poverty.
Promotes critical thinking: Science encourages us to question assumptions, analyze evidence, and develop logical reasoning skills.

Different Branches of Science:

Science is not a single field but rather a vast collection of disciplines, each focusing on a specific area of study. Some examples include:

Physics: The study of matter, energy, and their interactions.
Chemistry: The study of matter and its properties.
Biology: The study of living organisms.
Astronomy: The study of celestial objects and phenomena.
Geology: The study of Earth's physical structure and history.
Psychology: The study of the human mind and behavior.

In Conclusion:

Science is a powerful tool for understanding the world and solving problems. It's a process of continuous discovery, driven by curiosity, observation, and critical thinking.

Let me know if you'd like to learn more about a specific aspect of science! ```

这种方法是可行的，如果模型（比如 Gemma2–2B）在分词器中设置了聊天模板。使用普通调用方法进行推理

有时候你不能使用聊天模板（模型没有附带一个分词器聊天模板）。在这些情况下，我们使用一个普通的生成调用。我们已经有了所有适当的格式化提示的解释：

提示格式 — 来自截图

model card

对于Gemma，

Bartowski

在模型卡上做得非常出色。无论如何，看起来有一些重复，所以我查看了原始的模型卡，正确的提示格式应该像这样：

从

official model card

好的，我会按照您提供的格式来生成内容。以下是关于“科学是什么？”的简单生成内容：

科学是什么？

科学是一种系统性的知识体系，它通过观察、实验和理论分析来解释自然现象。科学方法包括以下几个步骤：

观察：科学家首先观察自然现象，收集数据。
假设：基于观察，提出可能的解释或假设。
实验：设计实验来测试假设，收集更多数据。
分析：分析数据，验证假设是否成立。
理论：如果假设得到验证，形成理论，进一步指导未来的研究。

科学的目标是发现自然界的规律，推动技术进步，改善人类生活。

prompt = """<start_of_turn>user What is Science?<end_of_turn> <start_of_turn>model """

我们使用较小的输出长度（250）进行生成调用。 out = llm(prompt,stop=['<eos>'],max_tokens=250) print(out)

您可能也注意到这里的输出是一个复杂的对象，与之前的略有不同，尽管如此： ```

print(out) {'id': 'cmpl-fd4b82fa-dbe1-4447-b070-813e3c2f94c1', 'object': 'text_completion', 'created': 1723419513, 'model': 'model/gemma-2-2b-it-Q5_K_M.gguf', 'choices': [{'text': "Science is a systematic and organized approach to understanding the world around us. It's not just about facts, but also about: Key Characteristics: Empirical Evidence: Science relies on observations and experimentation to gather data and test hypotheses. Testable Hypotheses: Scientific theories are well-defined, testable ideas that can be verified through research. * Peer Review: Scientific findings are rigorously scrutinized by experts in the field before being accepted as valid. * Falsifiable: Scientific theories are open to being disproven by evidence. This is a crucial part of the scientific process, as it allows for continuous improvement and refinement. * Objective & Transparent: Science strives for objectivity and transparency, using clear methodologies and data to avoid bias. What Science Does: Explains Natural Phenomena: Science seeks to understand how the universe works, from the smallest particles to the largest galaxies. Develops Technology: Scientific discoveries often lead to technological advancements that improve our lives. Predicts Future Events: Based on existing knowledge, scientists can make predictions about future events and phenomena. * Addresses Societal Questions:* Science plays a crucial role in addressing societal", 'index': 0, 'logprobs': None, 'finish_reason': 'length'}], 'usage': {'prompt_tokens': 13, 'completion_tokens': 250, 'total_tokens': 263}}

```

请注意，从这个对象中，您还可以看到为什么文本被截断的原因……‘finish_reason’: ‘length’。看起来250个标记对于Gemma2来说还不够！要仅访问文本，你现在可以猜测我们需要做类似这样的事情： print(out['choices'][0]['text'])

它会很好地展现出来 ``` Science is a systematic and organized approach to understanding the world around us. It's not just about facts, but also about:

Key Characteristics:

Empirical Evidence: Science relies on observations and experimentation to gather data and test hypotheses.
Testable Hypotheses: Scientific theories are well-defined, testable ideas that can be verified through research.
Peer Review: Scientific findings are rigorously scrutinized by experts in the field before being accepted as valid.
Falsifiable: Scientific theories are open to being disproven by evidence. This is a crucial part of the scientific process, as it allows for continuous improvement and refinement.
Objective & Transparent: Science strives for objectivity and transparency, using clear methodologies and data to avoid bias.

What Science Does:

Explains Natural Phenomena: Science seeks to understand how the universe works, from the smallest particles to the largest galaxies.
Develops Technology: Scientific discoveries often lead to technological advancements that improve our lives.
Predicts Future Events: Based on existing knowledge, scientists can make predictions about future events and phenomena.
Addresses Societal Questions: Science plays a crucial role in addressing societal ```

最终应用程序 Streamlit 应用程序

我设置了一个Streamlit应用程序，它使用了上述描述的所有基本概念。如果您需要代码的逐步解释，您可以在本文中使用相同的方法：外星人工智能就在我们中间

这么小，你简直不敢相信它是一个人工智能——而且它正在工作！

博客.stackademic.com

https://blog.stackademic.com/oute-worlder-ai-are-among-us-28c85884e036)基本结构是相同的，差异是在前几节中讨论的那些。要使用Streamlit应用程序，您还需要从仓库下载所有图片（并将它们放入images子文件夹）——如果您已经克隆了仓库，它们将自动下载。有两种版本（一种支持GPU，另一种不支持）

stappFULL

.py 仅适用于 CPU）。要运行它，请在激活了 venv 的终端中启动： streamlit run stappFULL.py