mirror of
https://github.com/moeru-ai/airi.git
synced 2026-10-29 18:48:55 +08:00
feat(stage-ui,agents,core-agent): let the model pick a Tabler icon for each tool call (#3051)
## Description A tool call row shows the summary of the call, but every call of a tool without its own icon shows the same magic wand. With this PR, the model picks an icon for each call, from the Tabler icon set. - Every tool gets a second argument `call_icon` after `call_summary`: up to three Tabler icon names for what the call does, best first. The tool never gets it, the same as `call_summary`. - The chat draws the first name that exists. A name that does not exist goes through a search of the icon names, so a close name still finds an icon. Without a match, the row keeps the default icon of the tool. - The picked icon replaces the `--un-icon` mask of the icon element that the row and the pill already have, so size and color stay. There is no extra request, and nothing is stored outside the recorded arguments. - Cards can use any Tabler icon. The card model and the chat model for reply cards get `builtIn_searchIcons`. It searches the words of the icon names, and a variant, for example `bookmark-off`, goes with its icon. The card validator rejects a name that does not exist. - The default tool icons, the skill icons, and the error icon of the pill are Tabler outline icons too, so one row of icons uses one set. - A task beside the chat, for example the card model, gets only its own tools. It does not get the tools of the chat, or the web search that OpenAI Responses adds from the provider settings. Why the model writes names: I compared names with local matching on two test sets. One set has 400 English tool call labels, in four kinds. The other has 104 labels in the 10 languages of AIRI. `gpt-6-sol` judged each icon. The first icon from names fit 89% and 91% of the labels. The best local method fit 74%. That method was a search of the Tabler tags with a trained ranker, or EmbeddingGemma on the label itself. Names cost a few tokens per call. In the 1437 labels, the names held a real icon name 1427 times, so the search that corrects a name runs rarely. ## Changes | Part | Change | | --- | --- | | `stores/ai/chat-llm/tool-call-summary.ts` | Adds `call_icon` after `call_summary`, and removes both before the tool gets its input. `readCallSummary` reads the line and the icon names, each with its own fallback. | | `libs/tabler-icons.ts` | New. Loads the Tabler icons on first use. Draws an icon by name for cards, searches icons by the words of their names, and turns `call_icon` candidates into the `--un-icon` mask of a row. | | `tools/search-icons.ts` | New. `builtIn_searchIcons` for the card model and for reply cards. | | `tool-call-group.vue` | The row and the pill draw the picked icon. The error icon of the pill is Tabler. | | `tools/icons.ts` | The default tool icons and the skill icons are Tabler outline icons. | | `libs/cards/catalog.ts`, `libs/cards/program.ts` | Cards draw Tabler icons, and the validator checks the names against the Tabler set. | | `stores/tool-cards.ts`, `stores/chat.ts` | The card model gets the icon search. A chat that draws reply cards gets it with the card tool. | | `composables/text-generation.ts`, `stores/ai/chat-llm/llm.ts`, `@proj-airi/agents` | `generateText` takes the tools of a task. A request with task tools gets no tools of the chat and no tools of the provider. | | `@proj-airi/core-agent` | New stream option `providerTools`. When it is `false`, OpenAI Responses adds no web search. | | `package.json` files | `@iconify-json/tabler` for stage-ui and for the UnoCSS classes of the desktop, mobile, and Kirie apps. It is the only source of the icons. | ## Verification - The pre-commit hook linted the staged files of each commit. I did not run the typecheck, the tests, or the full lint locally. CI runs them. - `.vishot/capture-tool-call-icons.mjs` (local and ignored) puts one reply with three calls into the stage-web chat. Each call has a summary and icon names: a web search and a weather call that finished, and a page read that failed. It opens the pill and captures at 1440×1600, light, English. The same script ran against `main` (`c8943f47c`) and against this branch. ## Visual changes | Before | After | |---|---| |  |  | | Chat / Tool call panel | Chat / Tool call panel | ## Additional Context - Each tool schema in a request grows by one more property, and each call carries up to three short names. - The icon search matches the words of the icon names only. A search for an idea, for example "save", finds fewer icons than a search of the Tabler tags. The model searches again with a thing that the icon draws, for example "device floppy". The names that a model writes in `call_icon` almost never need the search. - A tool card saved before this PR keeps the MingCute icon names that its model wrote. A name that Tabler does not have draws no icon. Making the card again fixes it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This commit is contained in:
蓝莓🫐
@@ -119,6 +119,7 @@
|
||||
"@iconify-json/simple-icons": "catalog:",
|
||||
"@iconify-json/solar": "catalog:",
|
||||
"@iconify-json/svg-spinners": "catalog:",
|
||||
"@iconify-json/tabler": "catalog:",
|
||||
"@iconify-json/vscode-icons": "catalog:",
|
||||
"@intlify/unplugin-vue-i18n": "catalog:",
|
||||
"@proj-airi/cap-vite": "workspace:*",
|
||||
|
||||
@@ -140,6 +140,7 @@
|
||||
"@iconify-json/simple-icons": "catalog:",
|
||||
"@iconify-json/solar": "catalog:",
|
||||
"@iconify-json/svg-spinners": "catalog:",
|
||||
"@iconify-json/tabler": "catalog:",
|
||||
"@iconify-json/vscode-icons": "catalog:",
|
||||
"@iconify/utils": "catalog:",
|
||||
"@intlify/unplugin-vue-i18n": "catalog:",
|
||||
|
||||
@@ -173,6 +173,7 @@
|
||||
"@iconify-json/simple-icons": "catalog:",
|
||||
"@iconify-json/solar": "catalog:",
|
||||
"@iconify-json/svg-spinners": "catalog:",
|
||||
"@iconify-json/tabler": "catalog:",
|
||||
"@iconify-json/vscode-icons": "catalog:",
|
||||
"@iconify/utils": "catalog:",
|
||||
"@intlify/unplugin-vue-i18n": "catalog:",
|
||||
|
||||
@@ -8,7 +8,7 @@ Each agent owns one prompt, one request shape, and one reply parser. An agent do
|
||||
|
||||
| Agent | Entry | Use |
|
||||
| --- | --- | --- |
|
||||
| Tool card | `@proj-airi/agents/tool-card` | Writes an OpenUI Lang card that shows the results of one tool. |
|
||||
| Tool card | `@proj-airi/agents/tool-card` | Writes an OpenUI Lang card that shows the results of one tool. It can search icons with the tools that the caller passes. |
|
||||
| Voice end judge | `@proj-airi/agents/voice-end-judge` | Decides whether a paused spoken message is finished. |
|
||||
| Voice transcription rephrase | `@proj-airi/agents/voice-transcription-rephrase` | Cleans up the speech recognition text of one spoken message. |
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { Conversation } from '@proj-airi/core-agent'
|
||||
import type { Conversation, StreamOptions } from '@proj-airi/core-agent'
|
||||
import type { GenerationProvider } from '@proj-airi/provider-inference'
|
||||
|
||||
import { streamFrom } from '@proj-airi/core-agent'
|
||||
@@ -8,8 +8,11 @@ import { streamFrom } from '@proj-airi/core-agent'
|
||||
*
|
||||
* Agents receive this function instead of a provider, so each runtime keeps its own request policy.
|
||||
* For example, stage-ui passes its LLM store, which remembers provider compatibility.
|
||||
*
|
||||
* `tools` are the only tools of the request, for example an icon search. The provider adds none of its own, such as a
|
||||
* web search. Text before a tool call only leads up to the call, so the reply is the text after the last call.
|
||||
*/
|
||||
export type GenerateText = (conversation: Conversation, signal: AbortSignal) => Promise<string>
|
||||
export type GenerateText = (conversation: Conversation, signal: AbortSignal, tools?: Extract<StreamOptions['tools'], unknown[]>) => Promise<string>
|
||||
|
||||
/**
|
||||
* Creates a {@link GenerateText} that streams with the core-agent runtime and collects the text deltas.
|
||||
@@ -18,19 +21,23 @@ export type GenerateText = (conversation: Conversation, signal: AbortSignal) =>
|
||||
* - A caller has no runtime of its own, for example an eval or a script.
|
||||
*
|
||||
* Returns:
|
||||
* - A generator that requests no tools.
|
||||
* - A generator that requests only the tools that a call passes.
|
||||
*/
|
||||
export function generateTextFrom(model: string, provider: GenerationProvider): GenerateText {
|
||||
return async (conversation, signal) => {
|
||||
return async (conversation, signal, tools) => {
|
||||
let reply = ''
|
||||
await streamFrom({
|
||||
model,
|
||||
chatProvider: provider,
|
||||
conversation,
|
||||
options: {
|
||||
supportsTools: false,
|
||||
supportsTools: Boolean(tools),
|
||||
tools,
|
||||
providerTools: false,
|
||||
abortSignal: signal,
|
||||
onStreamEvent: (event) => {
|
||||
if (event.type === 'tool-call')
|
||||
reply = ''
|
||||
if (event.type === 'text-delta')
|
||||
reply += event.text
|
||||
},
|
||||
|
||||
@@ -11,7 +11,7 @@ import type { GenerateText } from '../generate-text'
|
||||
export const TOOL_CARD_INSTRUCTIONS = [
|
||||
'You design a card that shows the result of one tool call in a chat app.',
|
||||
'The app saves the card and shows every later result of the same tool with it, so the card must work for other results of the same shape.',
|
||||
'You cannot call tools. The user message gives the result of one call as a sample. A result can be an object, a list, or text.',
|
||||
'You cannot call the tool of the card. The user message gives the result of one call as a sample. A result can be an object, a list, or text.',
|
||||
'Read the result with exactly one Query, for example `data = Query("tool_name", {}, {items: []})`. The app answers that Query with the result of each call.',
|
||||
'Read every value from the Query. Never copy values of the sample into the program.',
|
||||
'Do not use Mutation, forms, or $variables.',
|
||||
@@ -96,6 +96,8 @@ export interface ToolCardAgentOptions {
|
||||
* The model reads each problem in the fix round, so a problem names the statement and the fix.
|
||||
*/
|
||||
validate: (program: string) => string[] | Promise<string[]>
|
||||
/** Tools that the model can use while it writes, for example an icon search. */
|
||||
tools?: Parameters<GenerateText>[2]
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -112,7 +114,7 @@ export interface ToolCardAgentOptions {
|
||||
export function createToolCardAgent(options: ToolCardAgentOptions) {
|
||||
async function make(request: ToolCardRequest, signal: AbortSignal): Promise<ToolCard> {
|
||||
const conversation = createToolCardConversation(request)
|
||||
const reply = await options.generateText(conversation, signal)
|
||||
const reply = await options.generateText(conversation, signal, options.tools)
|
||||
const card = readToolCardReply(reply)
|
||||
const problems = await options.validate(card.program)
|
||||
if (problems.length === 0)
|
||||
@@ -129,7 +131,7 @@ export function createToolCardAgent(options: ToolCardAgentOptions) {
|
||||
const fixed = readToolCardReply(await options.generateText({ turns: [
|
||||
...conversation.turns,
|
||||
{ id: 'tool-card-fix', type: 'user', content: [{ type: 'text', text: fix }] },
|
||||
] }, signal))
|
||||
] }, signal, options.tools))
|
||||
const remaining = await options.validate(fixed.program)
|
||||
if (remaining.length > 0)
|
||||
throw new Error(`The card program still has problems: ${remaining.join('; ')}`)
|
||||
|
||||
@@ -143,7 +143,7 @@ async function streamOnce({
|
||||
|
||||
try {
|
||||
const streamResult = request.protocol === 'responses'
|
||||
? streamResponses({ config: request.config, webSearch: supportedTools && request.webSearch, conversation, scope, options, tools, initialStep, onEvent })
|
||||
? streamResponses({ config: request.config, webSearch: supportedTools && options?.providerTools !== false && request.webSearch, conversation, scope, options, tools, initialStep, onEvent })
|
||||
: streamChatCompletions({ config: request.config, conversation, scope, options, tools, initialStep, onEvent, supportsContentArray: contentArraySupported })
|
||||
|
||||
// NOTICE:
|
||||
|
||||
@@ -221,7 +221,7 @@ export function streamResponses(input: {
|
||||
maxOutputTokens: next.maxTokens ?? input.options?.maxTokens,
|
||||
headers: mergeRequestHeaders(nextRequest.config.headers, next.headers),
|
||||
tools: toolsSupported
|
||||
? nextRequest.webSearch ? [...(next.tools ?? []), { type: 'web_search' as const }] : next.tools
|
||||
? nextRequest.webSearch && input.options?.providerTools !== false ? [...(next.tools ?? []), { type: 'web_search' as const }] : next.tools
|
||||
: undefined,
|
||||
toolChoice: undefined,
|
||||
})
|
||||
|
||||
@@ -105,6 +105,12 @@ export interface StreamOptions {
|
||||
/** Provider tool-selection directive for one request. */
|
||||
toolChoice?: ToolChoice
|
||||
tools?: Tool[] | (() => Promise<Tool[] | undefined>)
|
||||
/**
|
||||
* Lets the provider add its own tools, for example the web search of OpenAI Responses that its settings turn on.
|
||||
* A task beside the chat sets it to `false`, so its request carries only the tools that it passes.
|
||||
* @default true
|
||||
*/
|
||||
providerTools?: boolean
|
||||
/**
|
||||
* Per-model runtime cache of whether the provider accepts content-part arrays
|
||||
* (e.g. `[{type:'text',...},{type:'image_url',...}]`) for `messages[].content`.
|
||||
|
||||
@@ -80,6 +80,7 @@
|
||||
"@formkit/auto-animate": "catalog:",
|
||||
"@huggingface/transformers": "catalog:",
|
||||
"@iconify-json/mingcute": "catalog:",
|
||||
"@iconify-json/tabler": "catalog:",
|
||||
"@iconify/utils": "catalog:",
|
||||
"@moeru/eventa": "catalog:",
|
||||
"@moeru/std": "catalog:",
|
||||
|
||||
@@ -15,6 +15,7 @@ import CardView from '../../../../libs/cards/card-view.vue'
|
||||
import HitPill from './hit-pill.vue'
|
||||
import ToolCallPayload from './tool-call-payload.vue'
|
||||
|
||||
import { tablerIconMask } from '../../../../libs/tabler-icons'
|
||||
import { readCallSummary } from '../../../../stores/ai/chat-llm/tool-call-summary'
|
||||
import { useLlmToolsStore } from '../../../../stores/ai/chat-llm/tools'
|
||||
import { canRerunToolCall } from '../../../../stores/tool-call-rerun'
|
||||
@@ -51,8 +52,12 @@ const toolCards = useToolCardsStore()
|
||||
|
||||
const calls = computed(() => {
|
||||
return props.items.map((item) => {
|
||||
// The model says in the call what it does and why. The row shows it, and a new card of the tool learns from it.
|
||||
const summary = readCallSummary(item.toolCall.args)
|
||||
// The model says in the call what it does and why, with an icon for it. The row shows both, and a new card of the
|
||||
// tool learns from the line.
|
||||
const { text: summary, icons } = readCallSummary(item.toolCall.args)
|
||||
// The icon draws in the element of the tool icon, so it keeps the size and the color of that icon. The tool icon
|
||||
// shows when no candidate leads to an icon.
|
||||
const iconMask = tablerIconMask(icons)
|
||||
const result = props.resultOf(item.sliceIndex)
|
||||
const widget = props.widgets[item.toolCall.toolName]
|
||||
const reading = widget?.read(result?.result, item.toolCall.args)
|
||||
@@ -69,6 +74,7 @@ const calls = computed(() => {
|
||||
// A widget title already says the state of the call.
|
||||
titled: Boolean(title),
|
||||
icon: toolIconOf(item.toolCall.toolName, llmTools.tools.findLast(tool => tool.function.name === item.toolCall.toolName)?.icon),
|
||||
iconStyle: iconMask ? { '--un-icon': iconMask } : undefined,
|
||||
state,
|
||||
result: result?.result,
|
||||
widget: widget && reading ? { component: widget.component, ...reading } : undefined,
|
||||
@@ -128,7 +134,8 @@ function popLabelIn(el: Element, done: () => void) {
|
||||
function hitIcon(call: { state: ToolCallState, icon: string }) {
|
||||
if (call.state === 'executing')
|
||||
return 'i-svg-spinners:ring-resize'
|
||||
return call.state === 'error' ? 'i-mingcute:close-circle-line' : call.icon
|
||||
// The error icon takes the place of a tool icon, so it comes from the same set.
|
||||
return call.state === 'error' ? 'i-tabler:circle-x' : call.icon
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -223,7 +230,7 @@ function rerun(item: ToolCallGroupItem) {
|
||||
<span v-for="call in visibleHits" :key="call.key" :class="hitClass">
|
||||
<!-- A call casts with a spinner. The spinner shrinks away, then the call lands as its own icon with a bounce. -->
|
||||
<Transition mode="out-in" :css="false" @enter="popIn" @leave="popOut">
|
||||
<span :key="hitIcon(call)" :class="[hitIcon(call), 'text-[11px]']" />
|
||||
<span :key="hitIcon(call)" :class="[hitIcon(call), 'text-[11px]']" :style="hitIcon(call) === call.icon ? call.iconStyle : undefined" />
|
||||
</Transition>
|
||||
</span>
|
||||
</TransitionGroup>
|
||||
@@ -253,7 +260,10 @@ function rerun(item: ToolCallGroupItem) {
|
||||
@click="toggleCall(call.key)"
|
||||
>
|
||||
<span :class="['relative shrink-0 inline-flex']" aria-hidden="true">
|
||||
<span :class="[call.state === 'executing' ? 'i-svg-spinners:ring-resize' : call.icon, 'text-sm', 'text-primary-500 dark:text-primary-300']" />
|
||||
<span
|
||||
:class="[call.state === 'executing' ? 'i-svg-spinners:ring-resize' : call.icon, 'text-sm', 'text-primary-500 dark:text-primary-300']"
|
||||
:style="call.state === 'executing' ? undefined : call.iconStyle"
|
||||
/>
|
||||
<!-- The ring takes the panel color, so the dot reads apart from the icon under it. -->
|
||||
<span
|
||||
v-if="STATE_DOTS[call.state]"
|
||||
|
||||
@@ -7,21 +7,26 @@ import { useProviderStore } from '../stores/providers/provider'
|
||||
* Sends one conversation to a chat model for a task beside the chat, for example a voice end judgement or a tool card.
|
||||
*
|
||||
* Returns:
|
||||
* - `generateText`, which sends a conversation through the LLM store with reasoning off and without tools.
|
||||
* The store remembers Provider compatibility.
|
||||
* - `generateText`, which sends a conversation through the LLM store with reasoning off. A request gets only the tools
|
||||
* that the task passes, never the tools of the chat. The store remembers provider compatibility.
|
||||
*/
|
||||
export function useTextGeneration() {
|
||||
const llm = useLLM()
|
||||
const providers = useProviderStore()
|
||||
|
||||
function generateText(selection: { providerId: string, model: string }): GenerateText {
|
||||
return async (conversation, signal) => {
|
||||
return async (conversation, signal, tools) => {
|
||||
const provider = await providers.getChatProviderInstance(selection.providerId, { reasoning: 'disabled' })
|
||||
let reply = ''
|
||||
await llm.stream(selection.model, provider, conversation, {
|
||||
supportsTools: false,
|
||||
// A task with tools still follows the learned compatibility of the model, so a model without tools answers without them.
|
||||
supportsTools: tools ? undefined : false,
|
||||
taskTools: tools,
|
||||
abortSignal: signal,
|
||||
onStreamEvent: (event) => {
|
||||
// Text before a tool call only leads up to the call. The reply is the text after the last call.
|
||||
if (event.type === 'tool-call')
|
||||
reply = ''
|
||||
if (event.type === 'text-delta')
|
||||
reply += event.text
|
||||
},
|
||||
|
||||
@@ -1,51 +1,18 @@
|
||||
import type { IconifyJSON } from '@iconify-json/mingcute'
|
||||
import type { ElementNode } from '@openuidev/lang-core'
|
||||
import type { FunctionalComponent, VNodeChild } from 'vue'
|
||||
import type { VNodeChild } from 'vue'
|
||||
|
||||
import { getIconData, iconToSVG, replaceIDs } from '@iconify/utils'
|
||||
import { createLibrary, defineComponent } from '@openuidev/lang-core'
|
||||
import { h, shallowRef } from 'vue'
|
||||
import { h } from 'vue'
|
||||
import { z } from 'zod'
|
||||
|
||||
import { SEARCH_ICONS_TOOL_NAME } from '../../tools/search-icons'
|
||||
import { TablerIcon } from '../tabler-icons'
|
||||
|
||||
// The parts that a model can use in a card. They copy the look of the chat's tool widgets, so a card that a model
|
||||
// writes reads like a widget of the app. A part holds its own look, and the program only arranges parts.
|
||||
|
||||
/** The icon set of every card icon. It loads on first use, so the app does not carry 1.5 MB of icons until a card needs one. */
|
||||
let iconSet: Promise<IconifyJSON> | undefined
|
||||
const loadedIconSet = shallowRef<IconifyJSON>()
|
||||
|
||||
/**
|
||||
* Loads the MingCute icons, once. A program can name any of them, so they do not go through UnoCSS, which only
|
||||
* generates icons whose class a scanned file writes out.
|
||||
*/
|
||||
export function loadCardIcons() {
|
||||
iconSet ??= import('@iconify-json/mingcute').then(({ icons }) => {
|
||||
loadedIconSet.value = icons
|
||||
return icons
|
||||
})
|
||||
return iconSet
|
||||
}
|
||||
|
||||
/** Draws a MingCute icon by name. An unknown name, or an icon set that is still loading, draws nothing. */
|
||||
const CardIcon: FunctionalComponent<{ name?: string }> = (props, { attrs }) => {
|
||||
if (!props.name)
|
||||
return null
|
||||
if (!loadedIconSet.value) {
|
||||
void loadCardIcons()
|
||||
return null
|
||||
}
|
||||
const data = getIconData(loadedIconSet.value, props.name)
|
||||
if (!data)
|
||||
return null
|
||||
const { attributes, body } = iconToSVG(data, { height: '1em' })
|
||||
return h('svg', { ...attributes, ...attrs, 'aria-hidden': 'true', 'innerHTML': replaceIDs(body) })
|
||||
}
|
||||
CardIcon.props = ['name']
|
||||
// The caller's class goes on the svg through `attrs`, so Vue must not add it a second time.
|
||||
CardIcon.inheritAttrs = false
|
||||
|
||||
function iconOf(name: string | undefined, extra: string) {
|
||||
return name ? h(CardIcon, { name, class: extra }) : null
|
||||
return name ? h(TablerIcon, { name, class: extra }) : null
|
||||
}
|
||||
|
||||
/** As in the web search widget, only a web page becomes a link. Another scheme leaves the part without a link. */
|
||||
@@ -147,7 +114,7 @@ const Tag = definePart({
|
||||
|
||||
const Row = definePart({
|
||||
name: 'Row',
|
||||
description: 'One item with its own title: small gray details under it, an icon, tags, and a link, for example an issue or a message. Several items with the same fields read better as a Table. The icon is a MingCute icon name, for example "github-line".',
|
||||
description: `One item with its own title: small gray details under it, an icon, tags, and a link, for example an issue or a message. Several items with the same fields read better as a Table. The icon is a Tabler icon name that \`${SEARCH_ICONS_TOOL_NAME}\` found.`,
|
||||
props: z.object({
|
||||
title: z.string(),
|
||||
details: z.array(z.string()).optional(),
|
||||
@@ -411,7 +378,7 @@ const Stack = definePart({
|
||||
|
||||
const Card = definePart({
|
||||
name: 'Card',
|
||||
description: 'The whole card, with an optional title line and a MingCute icon name for it. Every other part goes inside it.',
|
||||
description: `The whole card, with an optional title line and an icon for it: a Tabler icon name that \`${SEARCH_ICONS_TOOL_NAME}\` found. Every other part goes inside it.`,
|
||||
props: z.object({
|
||||
children: z.array(z.union([...leaves, Stack.ref])),
|
||||
title: z.string().optional(),
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import type { ElementNode, OpenUIError, ParseResult } from '@openuidev/lang-core'
|
||||
import type { VNodeChild } from 'vue'
|
||||
|
||||
import { getIconData } from '@iconify/utils'
|
||||
import { errorMessageFrom } from '@moeru/std'
|
||||
import { createParser, createStore, evaluate, evaluateElementProps } from '@openuidev/lang-core'
|
||||
|
||||
import { cardLibrary, drawCard, isPartNode, loadCardIcons, partProblem } from './catalog'
|
||||
import { SEARCH_ICONS_TOOL_NAME } from '../../tools/search-icons'
|
||||
import { hasTablerIcon } from '../tabler-icons'
|
||||
import { cardLibrary, drawCard, isPartNode, partProblem } from './catalog'
|
||||
|
||||
const programParser = createParser(cardLibrary.toJSONSchema(), 'Card')
|
||||
/** A tool name in a Query is a string literal, so it evaluates without state or references. */
|
||||
@@ -94,12 +95,11 @@ export async function problemsOfProgram(program: string, query?: { key: string,
|
||||
if (parsed.mutationStatements.length > 0)
|
||||
problems.push('Remove every Mutation. A card only shows a result.')
|
||||
|
||||
// A part draws nothing for an icon that MingCute does not have. The model can still fix the name now.
|
||||
const icons = await loadCardIcons()
|
||||
// A part draws nothing for an icon that Tabler does not have. The model can still fix the name now.
|
||||
for (const part of partsOf(root)) {
|
||||
const icon = part.props.icon
|
||||
if (typeof icon === 'string' && icon && !getIconData(icons, icon))
|
||||
problems.push(`"${icon}" is not a MingCute icon name. Use a MingCute name such as "github-line", or leave the icon out.`)
|
||||
if (typeof icon === 'string' && icon && !await hasTablerIcon(icon))
|
||||
problems.push(`"${icon}" is not a Tabler icon name. Find one with \`${SEARCH_ICONS_TOOL_NAME}\`, or leave the icon out.`)
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
import type { IconifyJSON } from '@iconify-json/tabler'
|
||||
import type { FunctionalComponent } from 'vue'
|
||||
|
||||
import { encodeSvgForCss, getIconData, iconToHTML, iconToSVG, replaceIDs } from '@iconify/utils'
|
||||
import { h, shallowRef } from 'vue'
|
||||
|
||||
// The Tabler icons that a model can name: in a card, or for a tool call. A model can name any of them, so they do not
|
||||
// go through UnoCSS, which only generates icons whose class a scanned file writes out.
|
||||
|
||||
/** The icon set. It loads on first use, so the app does not carry the icons until a model names one. */
|
||||
let iconSet: Promise<IconifyJSON> | undefined
|
||||
const loadedIconSet = shallowRef<IconifyJSON>()
|
||||
|
||||
/** Loads the Tabler icons, once. */
|
||||
export function loadTablerIcons() {
|
||||
iconSet ??= import('@iconify-json/tabler').then(({ icons }) => {
|
||||
loadedIconSet.value = icons
|
||||
return icons
|
||||
})
|
||||
return iconSet
|
||||
}
|
||||
|
||||
/** Whether Tabler has an icon of this name, for example `cloud-rain` or `star-filled`. */
|
||||
export async function hasTablerIcon(name: string) {
|
||||
return Boolean(getIconData(await loadTablerIcons(), name))
|
||||
}
|
||||
|
||||
/** Draws a Tabler icon by name. An unknown name, or an icon set that is still loading, draws nothing. */
|
||||
export const TablerIcon: FunctionalComponent<{ name?: string }> = (props, { attrs }) => {
|
||||
const icons = loadedIconSet.value
|
||||
if (!props.name)
|
||||
return null
|
||||
if (!icons) {
|
||||
void loadTablerIcons()
|
||||
return null
|
||||
}
|
||||
const data = getIconData(icons, props.name)
|
||||
if (!data)
|
||||
return null
|
||||
const { attributes, body } = iconToSVG(data, { height: '1em' })
|
||||
return h('svg', { ...attributes, ...attrs, 'aria-hidden': 'true', 'innerHTML': replaceIDs(body) })
|
||||
}
|
||||
TablerIcon.props = ['name']
|
||||
// The caller's class goes on the svg through `attrs`, so Vue must not add it a second time.
|
||||
TablerIcon.inheritAttrs = false
|
||||
|
||||
/** One outline icon of the set, with the words of its name, for example `cloud-rain` with `cloud` and `rain`. */
|
||||
interface SearchEntry {
|
||||
name: string
|
||||
words: string[]
|
||||
}
|
||||
|
||||
/** One search hit, with the variants of the icon that the query also matches, for example `bookmark-plus`. */
|
||||
export interface TablerIconHit {
|
||||
name: string
|
||||
variants: string[]
|
||||
}
|
||||
|
||||
let searchEntries: SearchEntry[] | undefined
|
||||
|
||||
function wordsOf(text: string) {
|
||||
return text.toLowerCase().split(/[^a-z0-9]+/).filter(word => word && !/^\d+$/.test(word))
|
||||
}
|
||||
|
||||
/** The outline icons of the set. The chat draws outline icons, and Tabler names a filled one `<name>-filled`. */
|
||||
function searchEntriesOf(icons: IconifyJSON) {
|
||||
searchEntries ??= Object.keys(icons.icons).filter(name => !name.endsWith('-filled')).map(name => ({ name, words: wordsOf(name) }))
|
||||
return searchEntries
|
||||
}
|
||||
|
||||
/**
|
||||
* How well one word of a query names one word of an icon. A word matches whole, or as the start of the other word, so
|
||||
* "search" finds "searching" and "cloud" finds "clouds".
|
||||
*/
|
||||
function wordScore(query: string, word: string) {
|
||||
if (query === word)
|
||||
return 2
|
||||
// A short word, for example `up`, would start too many other words.
|
||||
if ((query.length >= 3 && word.startsWith(query)) || (word.length >= 3 && query.startsWith(word)))
|
||||
return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
/**
|
||||
* Ranks the icons for a query, best first. An icon ranks by the query words that its name matches, then by how few
|
||||
* other words its name has, so `cloud` comes before `cloud-rain`.
|
||||
*/
|
||||
function rank(entries: SearchEntry[], query: string): SearchEntry[] {
|
||||
const words = wordsOf(query)
|
||||
if (words.length === 0)
|
||||
return []
|
||||
|
||||
const ranked = entries.flatMap((entry) => {
|
||||
let score = 0
|
||||
let matched = 0
|
||||
for (const word of words) {
|
||||
const best = Math.max(0, ...entry.words.map(entryWord => wordScore(word, entryWord)))
|
||||
score += best
|
||||
if (best > 0)
|
||||
matched += 1
|
||||
}
|
||||
return matched > 0 ? [{ entry, score, matched, extra: entry.words.length - matched }] : []
|
||||
})
|
||||
ranked.sort((left, right) =>
|
||||
right.matched - left.matched
|
||||
|| right.score - left.score
|
||||
|| left.extra - right.extra
|
||||
|| left.entry.name.length - right.entry.name.length
|
||||
|| left.entry.name.localeCompare(right.entry.name))
|
||||
|
||||
return ranked.map(({ entry }) => entry)
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds Tabler icons by the English words of their names, best first. Tabler names an icon after what it draws, so a
|
||||
* query names a thing, for example "cloud rain" or "device floppy". A variant, for example `bookmark-off`, goes with
|
||||
* its icon instead of taking a place of its own, so the hits cover more icons.
|
||||
*
|
||||
* @example
|
||||
* await searchTablerIcons('cloud rain')
|
||||
* // => [{ name: 'cloud-rain', variants: [] }, { name: 'cloud', variants: ['cloud-off', ...] }, ...]
|
||||
*/
|
||||
export async function searchTablerIcons(query: string, limit = 10): Promise<TablerIconHit[]> {
|
||||
const hits: TablerIconHit[] = []
|
||||
for (const entry of rank(searchEntriesOf(await loadTablerIcons()), query)) {
|
||||
const icon = hits.find(hit => entry.name.startsWith(`${hit.name}-`))
|
||||
if (icon)
|
||||
icon.variants.push(entry.name)
|
||||
else if (hits.length < limit)
|
||||
hits.push({ name: entry.name, variants: [] })
|
||||
}
|
||||
return hits
|
||||
}
|
||||
|
||||
/**
|
||||
* The outline icon of the best candidate. A candidate is an icon name, or English words that the search reads. A real
|
||||
* name among the candidates comes first, because a search for a wrong name finds only one of its words. The search
|
||||
* then corrects a name that is close.
|
||||
*/
|
||||
function pickIcon(icons: IconifyJSON, candidates: readonly string[]) {
|
||||
for (const candidate of candidates) {
|
||||
const exact = getIconData(icons, candidate.trim().toLowerCase().replace(/-filled$/, ''))
|
||||
if (exact)
|
||||
return exact
|
||||
}
|
||||
for (const candidate of candidates) {
|
||||
const [best] = rank(searchEntriesOf(icons), candidate)
|
||||
if (best)
|
||||
return getIconData(icons, best.name)
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* The outline icon of the best candidate, as the value of `--un-icon`. The chat draws every tool icon as an outline.
|
||||
*
|
||||
* UnoCSS draws a one-color icon class as a mask of its `--un-icon` variable, at the size and color of the class
|
||||
* (`@unocss/preset-icons`). So an element with an icon class draws this icon instead when its style sets the variable,
|
||||
* and it keeps its own icon when no candidate leads to one.
|
||||
*
|
||||
* Returns `undefined` while the icons load. It reads reactive state, so a view draws again once they arrive.
|
||||
*
|
||||
* @example
|
||||
* tablerIconMask(['currency-baht', 'coin'])
|
||||
* // => 'url("data:image/svg+xml;utf8,...")', the shape of `currency-baht`
|
||||
*/
|
||||
export function tablerIconMask(candidates: readonly string[]) {
|
||||
if (candidates.length === 0)
|
||||
return undefined
|
||||
const icons = loadedIconSet.value
|
||||
if (!icons) {
|
||||
void loadTablerIcons()
|
||||
return undefined
|
||||
}
|
||||
const data = pickIcon(icons, candidates)
|
||||
if (!data)
|
||||
return undefined
|
||||
const { attributes, body } = iconToSVG(data)
|
||||
return `url("data:image/svg+xml;utf8,${encodeSvgForCss(iconToHTML(replaceIDs(body), attributes))}")`
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { Conversation, StreamOptions } from '@proj-airi/core-agent'
|
||||
import type { GenerationProvider } from '@proj-airi/provider-inference'
|
||||
import type { Tool } from '@xsai/shared-chat'
|
||||
|
||||
import type { DescribeToolImage } from './tool-images'
|
||||
|
||||
@@ -17,6 +18,11 @@ export { isContentArrayRelatedError, isToolRelatedError } from '@proj-airi/core-
|
||||
export interface LlmStreamOptions extends StreamOptions {
|
||||
/** Reads the images in tool results as text. See {@link resolveLlmTools}. */
|
||||
describeToolImage?: DescribeToolImage
|
||||
/**
|
||||
* The only tools of a task beside the chat, for example the icon search of a tool card. The request gets them in
|
||||
* place of the tools of the chat, and without the tools of the provider, so a task runs nothing else.
|
||||
*/
|
||||
taskTools?: Tool[]
|
||||
}
|
||||
|
||||
export const useLLM = defineStore('llm', () => {
|
||||
@@ -28,8 +34,9 @@ export const useLLM = defineStore('llm', () => {
|
||||
// `prepareConversation` runs for every resolved request, so the key follows a step that changes the model.
|
||||
let key = modelKey(model, chatProvider.generation(model))
|
||||
let toolExecutionStarted = false
|
||||
const { tools: customTools, describeToolImage, ...streamOptions } = options ?? {}
|
||||
const builtinToolsResolver = () => resolveLlmTools({ customTools, describeImage: describeToolImage })
|
||||
const { tools: customTools, describeToolImage, taskTools, ...streamOptions } = options ?? {}
|
||||
const resolveTools = async () => taskTools ?? await resolveLlmTools({ customTools, describeImage: describeToolImage })
|
||||
const builtinToolsResolver = resolveTools
|
||||
const resolveStep = streamOptions.resolveStep
|
||||
|
||||
const runStream = () => coreStreamFrom({
|
||||
@@ -38,9 +45,10 @@ export const useLLM = defineStore('llm', () => {
|
||||
conversation: context,
|
||||
options: {
|
||||
...streamOptions,
|
||||
...(taskTools ? { providerTools: false } : {}),
|
||||
// Core uses only the tools of a resolved step, so each step carries the complete tool list.
|
||||
resolveStep: resolveStep
|
||||
? async () => ({ ...await resolveStep(), tools: await resolveLlmTools({ customTools, describeImage: describeToolImage }) })
|
||||
? async () => ({ ...await resolveStep(), tools: await resolveTools() })
|
||||
: undefined,
|
||||
prepareConversation: async (conversation, target) => {
|
||||
key = modelKey(target.model, target.request)
|
||||
|
||||
@@ -1,45 +1,54 @@
|
||||
import type { Tool } from '@xsai/shared-chat'
|
||||
|
||||
import { isPlainObject, omit } from 'es-toolkit'
|
||||
import { looseObject, parseJson, pipe, safeParse, string } from 'valibot'
|
||||
import { array, fallback, looseObject, optional, parseJson, pipe, safeParse, string } from 'valibot'
|
||||
|
||||
/**
|
||||
* The argument that the model writes first in every tool call: one line that tells the owner what the call does and
|
||||
* why. The recorded call keeps it, so the chat shows it on the row of the call and a new tool card learns the purpose
|
||||
* of the call from it. The tool never gets it.
|
||||
* The arguments that the model writes first in every tool call: one line that tells the owner what the call does and
|
||||
* why, and the icon of that line. The recorded call keeps them, so the chat shows them on the row of the call, and a
|
||||
* new tool card learns the purpose of the call from the line. The tool never gets them.
|
||||
*/
|
||||
export const CALL_SUMMARY_ARGUMENT = 'call_summary'
|
||||
export const CALL_ICON_ARGUMENT = 'call_icon'
|
||||
|
||||
const callSummaryProperty = {
|
||||
type: 'string',
|
||||
// A model that reads the line as words to the user also writes it in the reply, where speech and copy pick it up.
|
||||
// So the description calls it a label and keeps it out of the reply.
|
||||
description: 'Write this first. A short label shown on this call in the chat: what the call does and why, in the language of your reply. It is not part of your reply, so do not repeat it in your text. No final period.',
|
||||
const summaryProperties = {
|
||||
[CALL_SUMMARY_ARGUMENT]: {
|
||||
type: 'string',
|
||||
// A model that reads the line as words to the user also writes it in the reply, where speech and copy pick it up.
|
||||
// So the description calls it a label and keeps it out of the reply.
|
||||
description: 'Write this first. A short label shown on this call in the chat: what the call does and why, in the language of your reply. It is not part of your reply, so do not repeat it in your text. No final period.',
|
||||
},
|
||||
[CALL_ICON_ARGUMENT]: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
description: 'Write this second. Up to three Tabler icon names (tabler.io/icons) for what this call does, best first, for example ["currency-baht", "coin"]. A name says what the icon draws: "app-window" is a program window, and "window" is a window of a house. The chat draws the first one that exists next to the label.',
|
||||
},
|
||||
}
|
||||
const summaryArguments = Object.keys(summaryProperties)
|
||||
|
||||
/**
|
||||
* The parameters of a tool with the summary as their first argument. It is the first property, so the model writes it
|
||||
* before the other arguments. It is required, because a strict provider rejects a property that is not required.
|
||||
* The parameters of a tool with the summary arguments first. They are the first properties, so the model writes them
|
||||
* before the other arguments. They are required, because a strict provider rejects a property that is not required.
|
||||
*
|
||||
* Returns `undefined` for a tool that declares the argument itself, or whose parameters are no object.
|
||||
* Returns `undefined` for a tool that declares one of the arguments itself, or whose parameters are no object.
|
||||
*/
|
||||
function parametersWithSummary(parameters: Record<string, unknown> | undefined): Record<string, unknown> | undefined {
|
||||
if (!parameters || (parameters.type !== undefined && parameters.type !== 'object'))
|
||||
return undefined
|
||||
const properties = isPlainObject(parameters.properties) ? parameters.properties : {}
|
||||
if (CALL_SUMMARY_ARGUMENT in properties)
|
||||
if (summaryArguments.some(name => name in properties))
|
||||
return undefined
|
||||
const required = Array.isArray(parameters.required) ? parameters.required : []
|
||||
return {
|
||||
...parameters,
|
||||
type: 'object',
|
||||
properties: { [CALL_SUMMARY_ARGUMENT]: callSummaryProperty, ...properties },
|
||||
required: [CALL_SUMMARY_ARGUMENT, ...required],
|
||||
properties: { ...summaryProperties, ...properties },
|
||||
required: [...summaryArguments, ...required],
|
||||
}
|
||||
}
|
||||
|
||||
function withoutSummary(input: unknown) {
|
||||
return isPlainObject(input) && CALL_SUMMARY_ARGUMENT in input ? omit(input, [CALL_SUMMARY_ARGUMENT]) : input
|
||||
return isPlainObject(input) ? omit(input, summaryArguments) : input
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -53,7 +62,7 @@ function withoutSummary(input: unknown) {
|
||||
* without the summary, so a strict schema of the tool still accepts it.
|
||||
*
|
||||
* Returns:
|
||||
* - The tool with the summary argument. A tool that declares the argument itself stays as it is.
|
||||
* - The tool with the summary arguments. A tool that declares one of them itself stays as it is.
|
||||
*/
|
||||
export function withCallSummary(tool: Tool): Tool {
|
||||
const parameters = parametersWithSummary(tool.function?.parameters)
|
||||
@@ -69,19 +78,34 @@ export function withCallSummary(tool: Tool): Tool {
|
||||
}
|
||||
}
|
||||
|
||||
// A finished call holds its arguments as JSON text, as the model wrote them.
|
||||
const recordedSummarySchema = pipe(string(), parseJson(), looseObject({ [CALL_SUMMARY_ARGUMENT]: string() }))
|
||||
// A finished call holds its arguments as JSON text, as the model wrote them. A malformed argument reads as missing, so
|
||||
// the other one still shows.
|
||||
const recordedSummarySchema = pipe(string(), parseJson(), looseObject({
|
||||
[CALL_SUMMARY_ARGUMENT]: fallback(optional(string()), undefined),
|
||||
[CALL_ICON_ARGUMENT]: fallback(array(string()), []),
|
||||
}))
|
||||
|
||||
/** The summary of one call, as the model wrote it. */
|
||||
export interface CallSummary {
|
||||
/** What the call does and why. Missing when the model wrote none. */
|
||||
text?: string
|
||||
/** The icon candidates of the line, best first. The chat draws the first one that it finds. */
|
||||
icons: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads the summary from the recorded arguments of a call.
|
||||
*
|
||||
* Returns the summary, or `undefined` when the model wrote none.
|
||||
*
|
||||
* @example
|
||||
* readCallSummary('{"call_summary":"Check the weather in Bangkok","city":"Bangkok"}')
|
||||
* // => 'Check the weather in Bangkok'
|
||||
* readCallSummary('{"call_summary":"Check the weather in Bangkok","call_icon":["cloud-rain"],"city":"Bangkok"}')
|
||||
* // => { text: 'Check the weather in Bangkok', icons: ['cloud-rain'] }
|
||||
*/
|
||||
export function readCallSummary(args: string): string | undefined {
|
||||
export function readCallSummary(args: string): CallSummary {
|
||||
const recorded = safeParse(recordedSummarySchema, args)
|
||||
return recorded.success ? recorded.output[CALL_SUMMARY_ARGUMENT].trim() || undefined : undefined
|
||||
if (!recorded.success)
|
||||
return { icons: [] }
|
||||
return {
|
||||
text: recorded.output[CALL_SUMMARY_ARGUMENT]?.trim() || undefined,
|
||||
icons: recorded.output[CALL_ICON_ARGUMENT],
|
||||
}
|
||||
}
|
||||
|
||||
@@ -606,10 +606,10 @@ describe('chat store contract', () => {
|
||||
expect(getChatProviderInstanceMock).toHaveBeenCalledWith('mock-provider', { reasoning: 'disabled' })
|
||||
expect(() => structuredClone(result)).not.toThrow()
|
||||
// The session character's wake word tool joins every turn. It is not a selectable stored tool.
|
||||
// Reply cards are on by default, so the owner's chat also gets the card tool.
|
||||
// Reply cards are on by default, so the owner's chat also gets the card tool and the icon search of cards.
|
||||
expect(resolvedToolNames).toEqual([
|
||||
['stage_widgets', 'configure_wake_words', 'builtIn_readMemory', 'builtIn_writeMemory', 'builtIn_forgetMemory', 'builtIn_useRecipe', 'builtIn_proposeRecipe', 'builtIn_drawCard'],
|
||||
['stage_widgets', 'configure_wake_words', 'builtIn_readMemory', 'builtIn_writeMemory', 'builtIn_forgetMemory', 'builtIn_useRecipe', 'builtIn_proposeRecipe', 'builtIn_drawCard'],
|
||||
['stage_widgets', 'configure_wake_words', 'builtIn_readMemory', 'builtIn_writeMemory', 'builtIn_forgetMemory', 'builtIn_useRecipe', 'builtIn_proposeRecipe', 'builtIn_drawCard', 'builtIn_searchIcons'],
|
||||
['stage_widgets', 'configure_wake_words', 'builtIn_readMemory', 'builtIn_writeMemory', 'builtIn_forgetMemory', 'builtIn_useRecipe', 'builtIn_proposeRecipe', 'builtIn_drawCard', 'builtIn_searchIcons'],
|
||||
])
|
||||
})
|
||||
|
||||
|
||||
@@ -37,6 +37,7 @@ import { composeDrawCardPrompt, createDrawCardTool, DRAW_CARD_TOOL_NAME, drawCar
|
||||
import { createJudgeRecipeTool, judgmentSkillText } from '../tools/judge-recipe'
|
||||
import { composeMemoryPrompt, createMemoryTools } from '../tools/memory'
|
||||
import { createProposeRecipeTool } from '../tools/propose-recipe'
|
||||
import { createSearchIconsTool } from '../tools/search-icons'
|
||||
import { createUseRecipeTool } from '../tools/use-recipe'
|
||||
import { createWakeWordTool } from '../tools/wake-words'
|
||||
import { useLLM } from './ai/chat-llm/llm'
|
||||
@@ -634,8 +635,9 @@ export const useChatStore = defineStore('chat', () => {
|
||||
...(ownerOnly ? await createUseRecipeTool({ recipes: () => recipes.recipes, start: (recipe, task) => startRecipe(recipe, { parentSessionId: sessionId, task }) }) : []),
|
||||
// The owner can turn proposals off. Every proposal waits for the owner's approval.
|
||||
...(ownerOnly && recipes.proposalsEnabled ? await createProposeRecipeTool({ propose: recipe => recipes.propose(recipe) }) : []),
|
||||
// Cards show in the owner's own chat only. A scene, such as a Discord channel, gets text.
|
||||
...(drawsCards(sessionId) ? await createDrawCardTool() : []),
|
||||
// Cards show in the owner's own chat only. A scene, such as a Discord channel, gets text. A card names its icons
|
||||
// from a search, so the model never guesses an icon name.
|
||||
...(drawsCards(sessionId) ? [...await createDrawCardTool(), ...await createSearchIconsTool()] : []),
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ import { toolCardDataRepo } from '../database/repos/tool-card-data.repo'
|
||||
import { problemsOfProgram } from '../libs/cards/program'
|
||||
import { cardLanguagePrompt } from '../libs/cards/prompt'
|
||||
import { MCP_CALL_TOOL_NAME } from '../tools/mcp'
|
||||
import { createSearchIconsTool } from '../tools/search-icons'
|
||||
import { useLlmToolsStore } from './ai/chat-llm/tools'
|
||||
import { useConsciousnessStore } from './modules/consciousness'
|
||||
|
||||
@@ -166,6 +167,8 @@ const useToolCardsStateStore = defineStore('tool-cards-state', () => {
|
||||
const agent = createToolCardAgent({
|
||||
generateText: generateText({ providerId, model }),
|
||||
validate: program => problemsOfProgram(program, { key: request.key, data: request.data }),
|
||||
// The model searches the icons that it names, so a card never guesses one.
|
||||
tools: await createSearchIconsTool(),
|
||||
})
|
||||
return agent.make({
|
||||
catalogPrompt: ['You write a card in openui-lang, a declarative UI language, as the program of an app card.', '', cardLanguagePrompt()].join('\n'),
|
||||
|
||||
@@ -1,34 +1,39 @@
|
||||
import { JUDGE_RECIPE_TOOL_NAME } from './judge-recipe'
|
||||
import { FORGET_MEMORY_TOOL_NAME, READ_MEMORY_TOOL_NAME, WRITE_MEMORY_TOOL_NAME } from './memory'
|
||||
import { PROPOSE_RECIPE_TOOL_NAME } from './propose-recipe'
|
||||
import { SEARCH_ICONS_TOOL_NAME } from './search-icons'
|
||||
import { USE_RECIPE_TOOL_NAME } from './use-recipe'
|
||||
|
||||
// UnoCSS generates an icon only when its class appears literally in a file that it scans. Every app scans stage-ui,
|
||||
// but an app scans only some of its own folders. So every tool icon class lives here, and a tool registration
|
||||
// refers to one of these values instead of writing a class of its own.
|
||||
// UnoCSS generates an icon only for a class that a scanned file writes out. Every app scans stage-ui, but only some of
|
||||
// its own folders. So every tool icon class lives here, and a tool registration refers to one of these values.
|
||||
|
||||
/** The icon of each kind of skill. The skills settings page and the chat show the same icons. */
|
||||
export const SKILL_ICONS = {
|
||||
instructions: 'i-mingcute:document-fill',
|
||||
decision: 'i-mingcute:git-branch-fill',
|
||||
instructions: 'i-tabler:file-text',
|
||||
decision: 'i-tabler:git-branch',
|
||||
} as const
|
||||
|
||||
/** Each value matches the icon that the same feature shows in Settings or in the chat composer, in the MingCute set. */
|
||||
/**
|
||||
* Tabler outline icons, as are the icons that the model picks for each call, so a row of tool icons uses one set. Each
|
||||
* value draws the same thing as the icon of the feature in Settings or in the chat composer.
|
||||
*/
|
||||
export const TOOL_ICONS = {
|
||||
weather: 'i-mingcute:sun-cloudy-fill',
|
||||
widgets: 'i-mingcute:layout-4-fill',
|
||||
imageJournal: 'i-mingcute:palette-fill',
|
||||
computerUse: 'i-mingcute:monitor-fill',
|
||||
homeAssistant: 'i-mingcute:home-4-fill',
|
||||
memory: 'i-mingcute:book-2-fill',
|
||||
wakeWords: 'i-mingcute:mic-fill',
|
||||
spark: 'i-mingcute:send-plane-fill',
|
||||
weather: 'i-tabler:cloud-rain',
|
||||
widgets: 'i-tabler:layout-grid',
|
||||
imageJournal: 'i-tabler:palette',
|
||||
computerUse: 'i-tabler:device-desktop',
|
||||
homeAssistant: 'i-tabler:home',
|
||||
memory: 'i-tabler:book-2',
|
||||
wakeWords: 'i-tabler:microphone',
|
||||
spark: 'i-tabler:send',
|
||||
/** A search, for example of the icons that a card names. */
|
||||
search: 'i-tabler:search',
|
||||
/** Every tool that an MCP server provides. */
|
||||
mcp: 'i-mingcute:server-fill',
|
||||
mcp: 'i-tabler:server',
|
||||
/** Every tool of the skills feature that has no kind of its own. */
|
||||
skill: SKILL_ICONS.instructions,
|
||||
/** A tool that matches nothing else. */
|
||||
generic: 'i-mingcute:magic-2-fill',
|
||||
generic: 'i-tabler:wand',
|
||||
} as const
|
||||
|
||||
/** Tools that stage-ui and core-agent create for each send. They never enter the tool registry, so their icons live here. */
|
||||
@@ -40,6 +45,7 @@ const SEND_TOOL_ICONS: Record<string, string> = {
|
||||
[WRITE_MEMORY_TOOL_NAME]: TOOL_ICONS.memory,
|
||||
[FORGET_MEMORY_TOOL_NAME]: TOOL_ICONS.memory,
|
||||
configure_wake_words: TOOL_ICONS.wakeWords,
|
||||
[SEARCH_ICONS_TOOL_NAME]: TOOL_ICONS.search,
|
||||
builtIn_mcpListTools: TOOL_ICONS.mcp,
|
||||
builtIn_mcpCallTool: TOOL_ICONS.mcp,
|
||||
builtIn_emitSparkCommand: TOOL_ICONS.spark,
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import type { Tool } from '@xsai/shared-chat'
|
||||
|
||||
import { rawTool } from '@xsai/tool'
|
||||
import { description, pipe, safeParse, strictObject, string } from 'valibot'
|
||||
import { toJsonSchema } from 'xsschema'
|
||||
|
||||
import { searchTablerIcons } from '../libs/tabler-icons'
|
||||
|
||||
export const SEARCH_ICONS_TOOL_NAME = 'builtIn_searchIcons'
|
||||
|
||||
const searchIconsParameters = strictObject({
|
||||
query: pipe(string(), description('English words for what the icon draws, for example "cloud rain" or "device floppy".')),
|
||||
})
|
||||
|
||||
/**
|
||||
* Finds Tabler icons for a model, so it picks a name that exists instead of guessing one.
|
||||
*
|
||||
* Use when:
|
||||
* - A model names an icon: in a card, or for a tool call.
|
||||
*
|
||||
* Returns:
|
||||
* - The tool. It answers with the best icons first, one icon on each line with its variants.
|
||||
*/
|
||||
export async function createSearchIconsTool(): Promise<Tool[]> {
|
||||
return [
|
||||
rawTool({
|
||||
name: SEARCH_ICONS_TOOL_NAME,
|
||||
description: 'Searches the Tabler icons by the English words of their names, best first. A name says what the icon draws, so search for a thing that it can draw.',
|
||||
parameters: await toJsonSchema(searchIconsParameters),
|
||||
execute: async (rawInput) => {
|
||||
const parsed = safeParse(searchIconsParameters, rawInput)
|
||||
const query = parsed.success ? parsed.output.query.trim() : ''
|
||||
const hits = query ? await searchTablerIcons(query) : []
|
||||
// Icon names say what an icon draws, so an idea such as "weather" can find little, but "sun cloud" finds more.
|
||||
if (hits.length === 0)
|
||||
return `No Tabler icon matches "${query}". Search again with words for something that the icon can draw.`
|
||||
return [
|
||||
`Tabler icons for "${query}", best first:`,
|
||||
...hits.map(hit => `${hit.name}${hit.variants.length > 0 ? ` (variants: ${hit.variants.join(', ')})` : ''}`),
|
||||
].join('\n')
|
||||
},
|
||||
}),
|
||||
]
|
||||
}
|
||||
Generated
+12
@@ -1749,6 +1749,9 @@ importers:
|
||||
'@iconify-json/svg-spinners':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.4
|
||||
'@iconify-json/tabler':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.38
|
||||
'@iconify-json/vscode-icons':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.74
|
||||
@@ -2236,6 +2239,9 @@ importers:
|
||||
'@iconify-json/svg-spinners':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.4
|
||||
'@iconify-json/tabler':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.38
|
||||
'@iconify-json/vscode-icons':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.74
|
||||
@@ -2772,6 +2778,9 @@ importers:
|
||||
'@iconify-json/svg-spinners':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.4
|
||||
'@iconify-json/tabler':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.38
|
||||
'@iconify-json/vscode-icons':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.74
|
||||
@@ -4896,6 +4905,9 @@ importers:
|
||||
'@iconify-json/mingcute':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.8
|
||||
'@iconify-json/tabler':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.38
|
||||
'@iconify/utils':
|
||||
specifier: 'catalog:'
|
||||
version: 3.1.4
|
||||
|
||||
Reference in New Issue
Block a user