* ai: add embeddings basic support
- add Embeddings service and its local implementation
- add Embedding type and proto message
- add nodeEmbeddingCollector tracking nodes
- add NodeEmbeddingWatcher watching for events adn sending them to the
collector
- add the Embedder interface and its openai implementation
* ai: adapt embeddings to the vector index
* fixup! ai: adapt embeddings to the vector index
* fixup! fixup! ai: adapt embeddings to the vector index
* Update lib/service/service.go
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
* address feedback pt.1
* address feedback pt.2: store protobuf message in backend
* address feedback pt.3: have GetEmbeddings return a stream
* Update lib/services/embeddings.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* address feedback pt.4: extract embedding logic out of Embeddings service
* fixup! address feedback pt.4: extract embedding logic out of Embeddings service
* address feedback pt.5: simpler error handling when embedding fails
* fix tests pt.1
* fix tests pt.2
* fix tests pt.3
* [Assist] Replace embedding watcher (#27953)
Change the way how the embeddings are calculated. Instead of creating a watcher in Auth, we will process all nodes every hour and process embeddings if any embeddings are missing or any node has been updated.
---------
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
fheinecke