UnifiedTask is a Python package for automatic task transfer in neural networks using diff-based techniques. This approach is conceived by Jeonghwan Park (maywell), the main committer of InstructKR.
- You can check the example of ipynb here: https://github.com/StableFluffy/EasyLLMFeaturePorter
pip install UnifiedTask
fromUnifiedTask.diff_transferimportcalculate_model_diffs, calculate_sigmoid_ratios, apply_model_diffsfromUnifiedTask.models.llamaimportload_llama_modelinformative_model=load_llama_model("gradientai/Llama-3-8B-Instruct-262k")
base_model=load_llama_model("kuotient/Meta-Llama-3-8B-Instruct")
target_model=load_llama_model("beomi/Llama-3-Open-Ko-8B-Instruct-preview")
model_diffs=calculate_model_diffs(informative_model, base_model)
sigmoid_ratios=calculate_sigmoid_ratios(base_model, target_model)
apply_model_diffs(target_model, model_diffs, sigmoid_ratios)
...This project is licensed under the MIT License.