GPU not used

[igorpanjan-redbumerang]

I've uninstalled faiss-cpu and installed faiss-gpu as you mentioned in the docs. But when I run st.run_index() GPU is not utilized.

GPU utilization is very important to me, because I have to train cca. 500 GB of images. My GPU is Nvidia GTX 1080 Ti.

Nvidia drivers are installed correctly - Nvidia-smi returns list of GPUs. I've also run torch.cuda.is_available(), which returns True.

I would appreciate any pointers.

I also have a feature request - match batch of similar images.

Library is great BTW!

[TechyNilesh]

Thank you for informing me about the issue you are facing with GPU utilization. I understand the importance of GPU in your work, especially when dealing with a large number of images to be trained. It's great to know that you have already checked the necessary requirements for running faiss-gpu such as Nvidia drivers and torch.cuda.is_available(). I will look into this and provide some pointers to resolve the issue. Additionally, thank you for the feature request of matching a batch of similar images. I'll take note of this for future updates. Lastly, I'm glad to know that you find the library great! *Thanks & Regards* Nilesh Verma Data Scientist and AI Researcher *Amlgo Labs, India*

…

On Fri, 28 Apr 2023 at 13:42, Igor Panjan ***@***.***> wrote: I've uninstalled faiss-cpu and installed faiss-gpu as you mentioned in the docs. But when I run st.run_index() GPU is not utilized. GPU utilization is very important to me, because I have to train cca. 500 GB of images. My GPU is Nvidia GTX 1080 Ti. Nvidia drivers are installed correctly - Nvidia-smi returns list of GPUs. I've also run torch.cuda.is_available(), which returns True. I would appreciate any pointers. I also have a feature request - match batch of similar images. Library is great BTW! — Reply to this email directly, view it on GitHub <#19>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALYDGNSSQAGYS3G4QBFWXWDXDN3VBANCNFSM6AAAAAAXO4WWLQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[igorpanjan-redbumerang]

I did some digging on my own: first goal significantly increase speed of Search_Setup.

Since torch can see my GPU I had to enable it as device in self.model = torch.nn.Sequential(*list(base_model.children())[:-1]).cuda(0)

where .cuda(0) is my first GPU.

The speed of indexing is increased 10x - from cca. 4,5 it/s to 45+ - 57 it/s.

But than I get an Exception - I basically break the formation of *.pkl and *.idx files in metadata-files - files are couple of bytes:

Metadata and Features are already present, Do you want Extract Again? Enter yes or no
yes
  1%|          | 6/1000 [00:00<00:17, 57.16it/s]/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/PIL/Image.py:992: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
  warnings.warn(
100%|██████████| 1000/1000 [00:22<00:00, 43.62it/s]
Traceback (most recent call last):
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/indexes/range.py", line 345, in get_loc
    return self._range.index(new_key)
ValueError: 0 is not in range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/stoneball/PycharmProjects/DeepImageSearch_GPU_V2/DeepImageSearch_GPU_V2.py", line 11, in <module>
    st.run_index()
  File "/home/stoneball/PycharmProjects/DeepImageSearch_GPU_V2/DeepImageSearch.py", line 168, in run_index
    self._start_indexing(data)
  File "/home/stoneball/PycharmProjects/DeepImageSearch_GPU_V2/DeepImageSearch.py", line 140, in _start_indexing
    d = len(image_data['features'][0])  # Length of item vector that will be indexed
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/series.py", line 1007, in __getitem__
    return self._get_value(key)
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/series.py", line 1116, in _get_value
    loc = self.index.get_loc(label)
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/indexes/range.py", line 347, in get_loc
    raise KeyError(key) from err
KeyError: 0
 Image Meta Information Saved: [metadata-files/vgg19/image_data_features.pkl]

[igorpanjan-redbumerang]

This is something out of my understanding. In this thread LINK I found a tip to add Flatten layer.

So the code in Search_Setup class looks like this (it works):

self.model = torch.nn.Sequential(*list(base_model.children())[:-1]).cuda(0)
self.model = torch.nn.Flatten()

It supper fast - peak it/s are in the range of 200+ it/s, *.pkl and *.idx files for the batch fo 10.000 images are in the range of 6 GB. GPU RAM usage for 10.000 images is cca. 730 MB. This means running this code on one 1080 TI is 50x faster than running it on 2 40 cores CPUs. Processing time for 10.000 images using 40 core CPU is the range of 75 minutes, using one1080 Ti processing time is slashed to the range of 1 minute.

The problem is, that I definitely broke the accuracy - plotting similar images are all over the place. So if I understand correctly I broke the last layer - somehow, by using CUDA.

This is something out of my scope. :(

Bottom line:

• to speed things up using CUDA in torch part of the library is the key. It is not faiss, as I thought in the beginning.

I'm adding some screenshots:

[StatsAI]

Hi, I ran across the same issue on Colab. However, I think I figured out the no GPU solution per page 7 of this https://wangzwhu.github.io/home/file/acmmm-t-part3-ann.pdf

Within the function "start_indexing", the CPU function, rather than the GPU function is being called. Thus, we need to call the GPUIndex method, rather than the IndexFlatL2 method.

Admittingly, I am unsure of why this library works in the first place given that I followed the instructions to uninstall faiss_cpu and then installed faiss_gpu.

[StatsAI]

I tried the following solution and still encountered Value error: 0 not in the range at the end of execution D

1. Within the "init" method, do this:

#self.model = torch.nn.Sequential(*list(base_model.children())[:-1])
self.model = torch.nn.Sequential(*list(base_model.children())[:-1]).cuda(0)

2. Within the "start_indexing" method, do this:

#index = faiss.IndexFlatL2(d)
index = faiss.GpuIndexFlatL2(d)

3. Speedups obtained

Using CPU:

Using GPU:

[ergin]

@StatsAI you haven't moved the variables which are defined at CPU to GPU. You receive the error "Value error: 0" because your index is empty and trying to access first item of a zero sized list occurs that error which means none of the images are processed successfully.

[cyberlabz]

Это что-то за пределами моего понимания. В этой теме ССЫЛКА я нашел совет по добавлению слоя Flatten.

Итак, код в классе Search_Setup выглядит следующим образом (это работает):

self.model = torch.nn.Sequential(*list(base_model.children())[:-1]).cuda(0)
self.model = torch.nn.Flatten()

Он работает очень быстро - пиковые значения ит / с находятся в диапазоне 200 + ит / с, файлы *.pkl и * .idx для пакета из 10000 изображений имеют размер в диапазоне 6 ГБ. Использование оперативной памяти графического процессора для 10000 изображений составляет около 730 МБ. Это означает, что выполнение этого кода на одном 1080 TI в 50 раз быстрее, чем на двух процессорах с 40 ядрами. Время обработки 10 000 изображений с использованием 40-ядерного процессора составляет 75 минут, при использовании one1080 Ti время обработки сокращается до 1 минуты.

Проблема в том, что я определенно нарушил точность - повсюду строятся похожие изображения. Итак, если я правильно понимаю, я нарушил последний слой - каким-то образом, с помощью CUDA.

Это выходит за рамки моей компетенции. :(

Итог:

• для ускорения работы необходимо использовать CUDA в torch-части библиотеки. Это не так просто, как я думал вначале.

Я добавляю несколько скриншотов:

Good afternoon! As a result, did you manage to run on the GPU? I've already broken my whole brain, it's not working.