we will need to make some structural changes in the data logging code so that it becomes RAM independent. Currently it reserves the said space in ram before capturing the frames, which is reaching it's limits as we are capturing twice the number of images (raw and cropped) thus limiting out data collection. I think we will need to save the images as they are captured instead of storing them in a queue and then returning it to the main thread. I tried doing that but it bombs the code nuclear. same goes for data loader in pyotrch, we will need to load batches of data as they are needed and not store the whole dataset in memory. this will lead to a major slow down in runtime but at least the network will capture and train on any data size.
we will need to make some structural changes in the data logging code so that it becomes RAM independent. Currently it reserves the said space in ram before capturing the frames, which is reaching it's limits as we are capturing twice the number of images (raw and cropped) thus limiting out data collection. I think we will need to save the images as they are captured instead of storing them in a queue and then returning it to the main thread. I tried doing that but it bombs the code nuclear. same goes for data loader in pyotrch, we will need to load batches of data as they are needed and not store the whole dataset in memory. this will lead to a major slow down in runtime but at least the network will capture and train on any data size.