How to resolve the following error

  File "/venv/Lib/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/venv/Lib/site-packages/transformers/generation/utils.py", line 1744, in generate
    model_kwargs["past_key_values"] = self._get_cache(
                                      ^^^^^^^^^^^^^^^^
  File "/venv/Lib/site-packages/transformers/generation/utils.py", line 1435, in _get_cache
    self._cache = cache_cls(
                  ^^^^^^^^^^
  File "/venv/Lib/site-packages/transformers/cache_utils.py", line 1012, in __init__
    torch._dynamo.mark_static_address(new_layer_key_cache)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'torch._dynamo' has no attribute 'mark_static_address'

Cause

Your version of PyTorch may be out of date.

Check the PyTorch version installed in your current environment

You can check the versions of installed packages with the following command:

pip list

For example, with an older PyTorch such as the one below, the error above will occur.

torch                         2.0.1+cu117
torchaudio                    2.0.2+cu117
torchvision                   0.15.2+cu117

Fix

Update PyTorch to the latest version.

Using the pip install --upgrade command, you can update to the latest version.

Linux

pip install --upgrade torch torchvision torchaudio

Windows

RuntimeError: No GPU found. A GPU is needed for quantization.

If you see an error like the one above, install with an explicit CUDA version (11.8 in this example) as follows:

pip3 install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118