- Updated subproject commit to indicate a dirty state.
- Removed the `create_new_adapter` field from `TrainSftArgs` class in `config_models.py` to streamline training configuration.
- Added `train_pt` command to continue pre-training using prepared text datasets.
- Introduced `TrainPtArgs` class for pre-training parameters, extending `TrainSftArgs`.
Address Sourcery review feedback - use client.base_url instead of
hardcoding http://127.0.0.1:8005 so the error message stays in sync
if the base URL configuration changes.
Add _check_api_server() that probes GET /v1/models before the test loop
starts. When the server is not running, the user now gets a clear error
message with the exact command to start it instead of a cryptic
connection-refused traceback.
Co-Authored-By: Octopus <liyuan851277048@icloud.com>
- Updated the guided decoding parameter logic to support dynamic imports, improving compatibility with different configurations.
- Simplified the creation of guided decoding parameters by introducing a helper function.
- Modified the retry mechanism in the OpenAI API call to retry on all exceptions, enhancing robustness in error handling.
- Replaced the previous batch processing logic with a more robust implementation that utilizes guided decoding for improved result parsing.
- Enhanced error logging for failed parsing attempts, providing clearer warnings for issues encountered during inference.
- Updated the score mapping to utilize the new parsed results structure, ensuring accurate score assignment to QA pairs.
- Replaced the previous parsing logic with enhanced error handling for JSON extraction from LLM outputs.
- Introduced detailed logging for failed requests and unexpected result types, improving traceability of issues during inference.
- Updated the import statement to reflect the new utility function for JSON extraction.
- Updated Python version requirement in `pyproject.toml` to 3.12.
- Adjusted dependency specifications for `llamafactory`, `torch`, `torchvision`, and `torchaudio` to include version constraints.
- Added logic to handle the "<begin_chat>" marker in user messages, allowing for improved context in conversation flows.
- Updated the message processing to include the assistant's response when the user initiates a chat, ensuring more coherent QA pairs.
- Refactored the handling of conversation messages to maintain clarity and structure in the generated QA data.
- Updated the `qa_generator.py` to include a new mechanism for managing chat member relationships, allowing the addition of contextual information about the relationship between users in conversations.
- Refactored the CSV loading function to support loading user relationship data from a `users.json` file, improving the context provided during QA generation.
- Added a new configuration option `add_relation` to the dataset settings, enabling users to toggle this feature.
- Updated the `.gitignore` to exclude additional data directories and cache files for better repository hygiene.
- Bumped version to 0.3.03 in `pyproject.toml` to reflect these changes.
Enhance error messages when JSON parsing of LLM outputs fails in
offline inference. The system now dynamically extracts relevant
text snippets based on the response object type (e.g., RequestOutput,
ChatCompletion), making error logs more informative.
Additionally, suppress verbose debug/info logs from the OpenAI client
and httpx to reduce console noise during online inference.
Update the 'WC-exp' submodule and add 'models_final/*' to .gitignore
for repository hygiene.
Introduces a unified utility to parse and validate JSON-structured outputs
from both vLLM and OpenAI API inference results using Pydantic models.
This enables guided decoding for the OnlineLLM.chat method.
The existing guided decoding logic in vllm_infer is refactored to use
this new shared utility, improving consistency and error handling across
inference modes.
Expands the list of message types to be skipped or ignored by the application.
This ensures proper handling for user-uploaded GIF stickers and a new sticker
type identified as 'sticker2', preventing potential processing errors.
Introduce a ThreadPoolExecutor in the OnlineLLM class to enable
concurrent API calls, significantly boosting throughput for LLM operations.
Refactor the OlineLLMCleaningStrategy to leverage this new batching
capability, allowing multiple data points to be processed in parallel.
Increase the default `clean_batch_size` in settings and enhance
`n_process`/`batch_size` for PII detection to optimize for the new
concurrency. Simplify prompt management by removing a dedicated prompt
for online LLM cleaning. Add context manager support to OnlineLLM for
reliable resource cleanup. Ensure LLM chat responses explicitly request
JSON format.
Adds a call to empty the CUDA cache immediately after the LLM object is deleted, ensuring prompt release of GPU memory. This helps prevent out-of-memory issues and improves resource utilization for subsequent operations.
Allow `vllm_infer` to return `None` for failed JSON parsing results,
clarifying the sequence index for such failures. This enables
`LLMCleaningStrategy` to robustly align parsed LLM scores with original data,
correctly handling examples where JSON parsing failed by assigning a default
score of 0.
Adds an option to disable data cleaning during training setup.
Refactors dataset management to consistently append '-vl' for vision-language
datasets and dynamically name cleaned datasets (e.g., 'dataset-cleaned').
Enforces vLLM as a strict dependency for LLM-based data cleaning, exiting if
unavailable. Integrates 'enable_thinking' option for LLM cleaning and enables
cleaning by default in relevant test configurations.
Adds torchvision dependency for vision models and streamlines the cleaning call
in training to centralize decision-making.
xming