- Updated Python version requirement in `pyproject.toml` to 3.12.
- Adjusted dependency specifications for `llamafactory`, `torch`, `torchvision`, and `torchaudio` to include version constraints.
- Updated the `qa_generator.py` to include a new mechanism for managing chat member relationships, allowing the addition of contextual information about the relationship between users in conversations.
- Refactored the CSV loading function to support loading user relationship data from a `users.json` file, improving the context provided during QA generation.
- Added a new configuration option `add_relation` to the dataset settings, enabling users to toggle this feature.
- Updated the `.gitignore` to exclude additional data directories and cache files for better repository hygiene.
- Bumped version to 0.3.03 in `pyproject.toml` to reflect these changes.
Refactors dataset management to consistently append '-vl' for vision-language
datasets and dynamically name cleaned datasets (e.g., 'dataset-cleaned').
Enforces vLLM as a strict dependency for LLM-based data cleaning, exiting if
unavailable. Integrates 'enable_thinking' option for LLM cleaning and enables
cleaning by default in relevant test configurations.
Adds torchvision dependency for vision models and streamlines the cleaning call
in training to centralize decision-making.
Updates the Qwen3 model in test configurations from 4B to the smaller 0.6B version.
This change helps improve test performance and reduce resource requirements.
Ensures the data generation pipeline correctly includes the expected number of <image> tags for multimodal configurations, specifically for Qwen2.5-VL datasets.
Adds a blocked words filter to the dataset creation process.
This prevents undesirable words from appearing in the generated
datasets, enhancing data quality and safety.
A test case is added to verify the filter is working correctly.
Also, skips "text" type in qa_generator to avoid duplicated filter.
Refactors the test environment setup to use pytest fixtures
for session-level setup and teardown. This ensures a clean
and consistent environment before and after each test session.
It also improves the robustness by backing up and restoring
existing 'model_output' and 'dataset/csv' directories.
xming521