Update peft requirement from <0.13,>=0.10.0 to >=0.10.0,<0.14

[dependabot]

Updates the requirements on peft to permit the latest version.

Release notes

Sourced from peft's releases.

LoRA+, VB-LoRA, and more

Highlights

New methods

LoRA+

@​kallewoof added LoRA+ to PEFT (#1915). This is a function that allows to initialize an optimizer with settings that are better suited for training a LoRA adapter.

VB-LoRA

@​leo-yangli added a new method to PEFT called VB-LoRA (#2039). The idea is to have LoRA layers be composed from a single vector bank (hence "VB") that is shared among all layers. This makes VB-LoRA extremely parameter efficient and the checkpoints especially small (comparable to the VeRA method), while still promising good fine-tuning performance. Check the VB-LoRA docs and example.

Enhancements

New Hugging Face team member @​ariG23498 added the helper function rescale_adapter_scale to PEFT (#1951). Use this context manager to temporarily increase or decrease the scaling of the LoRA adapter of a model. It also works for PEFT adapters loaded directly into a transformers or diffusers model.

@​ariG23498 also added DoRA support for embedding layers (#2006). So if you're using the use_dora=True option in the LoraConfig, you can now also target embedding layers.

For some time now, we support inference with batches that are using different adapters for different samples, so e.g. sample 1-5 use "adapter1" and samples 6-10 use "adapter2". However, this only worked for LoRA layers so far. @​saeid93 extended this to also work with layers targeted by modules_to_save (#1990).

When loading a PEFT adapter, you now have the option to pass low_cpu_mem_usage=True (#1961). This will initialize the adapter with empty weights ("meta" device) before loading the weights instead of initializing on CPU or GPU. This can speed up loading PEFT adapters. So use this option especially if you have a lot of adapters to load at the same time or if these adapters are very big. Please let us know if you encounter issues with this option, as we may make this the default in the future.

Changes

Safe loading of PyTorch weights

Unless indicated otherwise, PEFT adapters are saved and loaded using the secure safetensors format. However, we also support the PyTorch format for checkpoints, which relies on the inherently insecure pickle protocol from Python. In the future, PyTorch will be more strict when loading these files to improve security by making the option weights_only=True the default. This is generally recommended and should not cause any trouble with PEFT checkpoints, which is why with this release, PEFT will enable this by default. Please open an issue if this causes trouble.

What's Changed

• Bump version to 0.12.1.dev0 by @​BenjaminBossan in huggingface/peft#1950
• CI Fix Windows permission error on merge test by @​BenjaminBossan in huggingface/peft#1952
• Check if past_key_values is provided when using prefix_tuning in peft_model by @​Nidhogg-lyz in huggingface/peft#1942
• Add lora+ implementation by @​kallewoof in huggingface/peft#1915
• FIX: New bloom changes breaking prompt learning by @​BenjaminBossan in huggingface/peft#1969
• ENH Update VeRA preconfigured models by @​BenjaminBossan in huggingface/peft#1941
• fix: lora+: include lr in optimizer kwargs by @​kallewoof in huggingface/peft#1973
• FIX active_adapters for transformers models by @​BenjaminBossan in huggingface/peft#1975
• FIX Loading adapter honors offline mode by @​BenjaminBossan in huggingface/peft#1976
• chore: Update CI configuration for workflows by @​XciD in huggingface/peft#1985
• Cast to fp32 if using bf16 weights on cpu during merge_and_unload by @​snarayan21 in huggingface/peft#1978
• AdaLora: Trigger warning when user uses 'r' inplace of 'init_r' by @​bhargavyagnik in huggingface/peft#1981
• [Add] scaling LoRA adapter weights with a context manager by @​ariG23498 in huggingface/peft#1951
• DOC Small fixes for HQQ and section title by @​BenjaminBossan in huggingface/peft#1986
• Add docs and examples for X-LoRA by @​EricLBuehler in huggingface/peft#1970
• fix: fix docker build gpus by @​XciD in huggingface/peft#1987
• FIX: Adjust transformers version check for bloom by @​BenjaminBossan in huggingface/peft#1992
• [Hotfix] Fix BOFT mixed precision by @​Edenzzzz in huggingface/peft#1925

... (truncated)

Commits

• f0b066e Release v0.13.0 (#2093)
• 8f39708 ENH: Better DoRA check in mixed adapter batch inference (#2089)
• f4cf170 DOC Docstring of load_adapter, type annotation (#2087)
• b67c9b6 FIX: Bug in find_minimal_target_modules (#2083)
• 5efeba1 ENH: Add default target layers for gemma2 architecture (#2078)
• af275d2 ENH: Allow empty initialization of adapter weight (#1961)
• 9bc670e MNT Update author email in setup.py (#2086)
• 5d94458 ENH Expose bias of ModulesToSaveWrapper (#2081)
• 152ed70 ENH PiSSA/OLoRA: Preserve original config on save (#2077)
• f5dd2ac TST Skip some quantization tests on XPU (#2074)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)