14 Commits
Author SHA1 Message Date
william chang(張仲威) 38a0d0e255 fix(docker): remove redis (#418) 2024-06-14 20:02:38 +08:00
Andy YenandShimin f4e4551498 Epic: integrate with ibis-server (#403)
* Chore: integrate with ibis server (#306)

* feat: Add ibis adaptor for querying supported data sources

* feat: Add testIbis resolver for querying supported data sources

* feat: use queryService for preview and describe statement

* feat: Refactor ibisAdaptor to use DataSourceName enum consistently

* test ibis using tpch query

* rm test ibis

* feat: Add previewSql resolver for SQL preview

* rm debug log

* use projectId and manifest to generate hash
Implement getMDL api

* pass mdl to ibis server to preview

* transform manifest data type to json object when retrieved from database

* better error message from Ibis server, rewrite host when development and using docker

* use base64 str to transfer mdl

* update README

* change env name to fit naming style.
update docker-compose.x.yaml and .env.example file

* fix yarn build issue

* update docker-compose file: fix incorrent ibis server endpoint

* port forward to ibis server

* fix incorrect migration rollback function

* update engine version

* Chore: refactor database schema to reflect how we use them. (#341)

* add connectionInfo column and do data migration

* drop deprecated columns that store connection info, use connectionInfo column instead

* do data migration to store tableReference in model's properties

* fix incorrect migration rollback function

* use tableReference in mdl instead of refSql

* chore: Update variable names in ibisAdaptor and queryService

* chore: Update variable names in ibisAdaptor and queryService

* fix lint

* rename variable in ibis adaptor

* - remove duplicate logic to parse string
- add comment

* release 0.4.0-rc.1 (#378)

* fix: fix nested resolver to format sql (#379)

* feat(wren-ui): Provide limit property in preview data API for excel-add-in used (#373)

* fix(wren-ui): add displayName to all view API which share ViewInfo type in graphQL

* feat(wren-ui): provide limit property of preview API for usage in excel-add-in

* Chore: minimized the diversity of data sources from wren-ui  (#384)

* chore: Update ibisAdaptor to include methods for getting tables and constraints

* remove strategies and connectors from codebase, use ibis server and wrenEngine to fetch metadata of data source

* add validation rule api in ibisAdapter

* replace invalid char when generating referenceName

* fix comments

* Feature: support dry run in preview API (#402)

* can dry-run  with previewSql

* update env.example and docker-compose-dev.yaml

---------

Co-authored-by: Shimin <andreashimin@gmail.com>
2024-06-12 21:19:05 +08:00
Pao Sheng b86cd6f3e2 feature(ai-service): wren-ui dry-run SQL API instead of wren-engine API (#400)
* feat(ai-service): change dryrun api

* feat: change the docker env
2024-06-12 20:20:43 +08:00
Chih-Yu Yeh 8d8bccfbd7 Chore/ai service/minor update (#382)
* update

* use redis

* update

* fix document store duplicate collection issue

* update timer setting

* fix pre_run and comment out qurey_understanding pipeline

* add redis container

* update redis config

* update redis config

* update redis config

* fix bug

* update dummy api

* add DummyAskUser

* update

* add load test observation notes

* refine wording

* update

* fix missing env

* fix bug
2024-06-06 18:29:30 +08:00
Chih-Yu Yeh e96c928b36 Chore/ai service/minor updates (#377)
* restructure test folder

* fix WREN_ENGINE_ENDPOINT issue

* add async timer

* update

* use logger.info

* use asyncio.sleep

* use asyncio.sleep

* fix timer to async_timer

* add dummy endpoint

* fix time.sleep

* add WORKER

* add timer

* add LOGGING_LEVEL env

* simplify code

* change to async text/document embedder

* simplify code

* update configs and fix timer
2024-06-05 14:40:35 +08:00
HoupusandChih-Yu Yeh 7decf5fbc4 Support 3rd party openai api (#365)
* Support for specifying third-party OPENAI proxy services

* Remove unnecessary changes

* Support for specifying third-party OPENAI proxy services

* update get_generator method

---------

Co-authored-by: Chih-Yu Yeh <chihyu.jimmy.yeh@gmail.com>
2024-06-04 21:38:23 +08:00
Chih-Yu Yeh e724bd8ad0 add WREN_ENGINE_VERSION (#303) 2024-05-22 18:28:09 +08:00
Chih-Yu Yeh 1fac9b76f3 chore/small refinements to launcher (#269)
* fix missing envs to posthog

* fix .env.example

* update add openai key instructions

* add missing env vars
2024-05-20 16:17:02 +08:00
Chih-Yu Yeh 66bfe8a95e Chore(wren-ui): record openai generation model used by user (#254)
* record openai generation model used by user

* remove unused comments
2024-05-15 11:20:10 +08:00
Chih-Yu Yehandwilliam chang d75f6d1ed7 allow users to change openai model in launcher (#244)
* ..

* add logs for init_providers

* fix ci

* add openai model url instructions

* fix empty generation model if OPENAI_GENERATION_MODEL is not provided

* Update wren-launcher/utils/docker.go

Co-authored-by: william chang(張仲威) <wwwy3y3@gmail.com>

---------

Co-authored-by: william chang(張仲威) <wwwy3y3@gmail.com>
2024-05-14 08:49:15 +08:00
william chang(張仲威) a59bafc7f4 feat(wren-launcher): add pg_username & password to accounts file at bootstrap 2024-04-16 11:52:55 +08:00
Chih-Yu Yeh 69263e6dd9 Chore/docker compose update (#69)
* update docker related settings

* change WREN_AI_SERVICE_PORT default 5555

* add pull_policy, restart condition and update wren-ai-service port

* fix default port for wren-ai-service
2024-04-03 15:57:09 +08:00
Chih-Yu YehandPao Sheng 32cd3df311 Feature/ai service/fix failed sql queries + add error code (#50)
* resolve conflict

* allow visualization using one file

* remove default argparse option value

* resolve conflict

* resolve conflict

* resolve conflict

* update setup instructions

* resolve conflict

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* add anthropic model pricing

* allow change model for anthropic

* resolve conflicts

* resolve conflicts

* remove invalid sql generation results

* WIP: add sql correction

* add sql correction pipeline

* update ai service ci yaml

* update eval

* add instructions for running tests

* add comments for cte_name validation for last step

* use wren-engine api to validate sql instead of wren-engine psql

* simplify testing instructions

* update testing step

* create missing dir

* fix commands

* add WREN_ENGINE_ENDPOINT

* update WREN_ENGINE_ENDPOINT

* add container:ubuntu

* undo

* changed to localhost

* adjust docker network

* test ci

* test ci

* update

* update

* update

* revert

* update

* revert

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix bug

* update

* remove contest

* Feature/ai service/add error code (#40)

* resolve conflict

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* resolve conflicts

* refine

* remove unused files

* resolve conflicts

* add error code and post processor to ask details

* add volume for qdrant and WREN_ENGINE_ENDPOINT for wren-ai-service

* add status check for ask service and fix tests

* Update wren-ai-service/tests/services/test_ask_details.py

Co-authored-by: Pao Sheng <paooap.oappao@gmail.com>

* Update wren-ai-service/tests/services/test_ask_details.py

Co-authored-by: Pao Sheng <paooap.oappao@gmail.com>

---------

Co-authored-by: Pao Sheng <paooap.oappao@gmail.com>

---------

Co-authored-by: Pao Sheng <paooap.oappao@gmail.com>
2024-03-28 15:11:30 +08:00
william chang(張仲威) ffbb2c4084 Feature: build wren-ui docker image & docker-compose to start all service we need (#27)
* feat(dockerfile): add standalone option. fix eslint issues at next.config.js

* feat(docker): add docker-compose file

* feat(docker): add QDRANT_HOST

* feat(docker): modify docker-compose to add a bootstrap runner

* feat(docker): bootstrap service

* feat(wren-ui): add dockerfile for wren-ui

* fix(wren-ui): rm docker entry script

* feat(docker): publish ui & bootstrap packages

* feat(docker): remove comment in dockerfile & put up a how to start in readme

* fix(docker): change ai-service port to 80
2024-03-22 16:39:48 +08:00