![Tensor Operations Worth Mastering: tensor.unsqueeze(0) vs. array[None]](https://images.unsplash.com/photo-1618912617207-0c06851b2dc9?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDQ1fHxzcXVlZXplfGVufDB8fHx8MTcyNjExNjYxNHww&ixlib=rb-4.0.3&q=80&w=600)
NumPy/PyTorch
Today we will walk through unsqueeze(0), and along the way touch on a familiar problem many of us run into. Can you tell "value.unsqueeze(0)" and "value[None]" apart at a glance? The former is primarily a PyTorch operation, while the latter belongs to
NumPy/PyTorch
NumPy reshape: Taking Full Control of Your Data's Shape NumPy's reshape function is a powerful tool for changing the shape of multidimensional arrays. In this article, we cover reshape in detail, from basic usage to more advanced applications, with concrete examples. 1. reshape Basics reshape changes
Python
Hello! If you use Anaconda, have you ever accidentally installed extra packages into the base environment? I did exactly that recently—I installed FastAPI into the base environment, and my dependencies ended up in a tangled mess. In this article, we walk through how to safely restore Anaconda's
NumPy/PyTorch
This series is for machine learning engineers just getting started: it helps you build a clear mental picture of the tensor operations you use all the time in PyTorch and NumPy. Rather than a reference that aims for strict formal rigor, we will learn through the kinds of code you
Daily Dev Tips
Hello! When working with ONNX Runtime, you may run into an error like the following [E:onnxruntime:Default, provider_bridge_ort.cc:1744 TryGetProviderInfo_CUDA] Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory [W:onnxruntime:
Daily Dev Tips
When you encounter an error like the following: RuntimeError: implement_array_function method already has a docstring This error message appears to stem from a problem inside the numpy package. Specifically, the error says that the implement_array_function method already has documentation attached, and it typically seems to occur
Daily Dev Tips
How to resolve the following error File "/venv/Lib/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/venv/Lib/site-packages/transformers/generation/utils.py", line 1744, in generate model_kwargs["past_key_values"] = self._get_cache( ^^^^^^^^^^^^^^^^ File
Daily Dev Tips
This is a list of NVIDIA GPU Compute Capability levels. The various acceleration features supported by inference engines depend on the Compute Capability level, since the hardware accelerators and dedicated functions available differ by level. Data Center / Professional GPUs GeForce GPU Capability Level Architecture - GeForce RTX 5090 120 Blackwell
Daily Dev Tips
In this article, we introduce sample code for using the OpenAI API. The OpenAI API is not limited to OpenAI's own service; other inference engines such as vLLM also expose OpenAI-compatible API servers, and it is becoming the de facto standard for LLM serving APIs. So it is
NumPy/PyTorch
Optimizing PyTorch Tensor Operations: Understanding and Resolving a Warning Message Hello! This is the Qualiteg Product Development Team. While working with PyTorch 1.13, we ran into the following warning message: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach(
Daily Dev Tips
Hello from the Product Development Team at Qualiteg. TensorRT-LLM is an inference engine from NVIDIA that can be regarded as the successor to FasterTransformer, and it is available as one of the inference engines for our ChatStream. Like vLLM, it adds support for new models quickly and already supports a
![[ChatStream] Pin the Revision to Guard Against Unexpected Changes in Input/Output Prompts](/content/images/size/w600/2024/05/-----TIPs--1--1.png)
Daily Dev Tips
Hello, this is the Product Development Department at Qualiteg Inc. During Japan's Golden Week holidays, the tokenizer.json of microsoft/Phi-3-mini-128k-instruct was changed. As a result, prompt parsing failed and chat streaming stopped working. Specifically, the following change was made: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/commit/8a362e755d2faf8cec2bf98850ce2216023d178a Originally,
Daily Dev Tips
This post explains how to deal with the following error when you try to pip install bitsandbytes. Could not load bitsandbytes native library: libcusparse.so.11: cannot open shared object file: No such file or directory Traceback (most recent call last): File "/home/mlu/.virtualenvs/ChatStream/lib/python3.10/
Daily Dev Tips
This post explains how to deal with the error "triu_tril_cuda_template" not implemented for 'BFloat16', which can occur when you specify torch_dtype=torch.bfloat16 while loading a model. Below is the log from when this occurred with Llama 3. File "/home/mlu/
Daily Dev Tips
Hello, this is the Qualiteg Product Development team. Overview * There have been reports that installing PyTorch, which ChatStream requires, does not work properly on Python 3.12. * Installation succeeded on Python 3.11 and earlier. Error message pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 Running
Daily Dev Tips
When building LLM services, we often need to move an LLM that has already been downloaded once to a different server. (When you want to use the same LLM model on servers with an identical configuration, this is faster than downloading it again from Hugging Face, for example.) When a