Thread safety

[alimoeeny]

Hi @galeone
Thanks for all the hard work.
Was wondering if tfgo.Model is "thread safe".
Specifically can I call tfgo.Exec from different goroutines ? are there any considerations ? I assume I at least need to keep the scopes separate?
Thank you again for keeping this repo up to date.

[galeone]

Hi @alimoeeny thank you for the kind words!

Honestly, I have never tested multiple inferences of the same model in multiple go routines. I guess, being the forward pass handled by the C library, the TensorFlow backend should already handle the concurrency - but it's just a guess.

Now that I'm thinking about it (keep this message as a stream of consciousness from my side lol), I guess it's instead the exact opposite. The thread safety is left to the user in the C library - and this would make sense.

Because the nature of the SavedModel is complex. There could be SavedModel stateful (where, for example, the TensorFlow program updates internal tf.Variable nodes), or SavedModel stateless (the serialization of some tf.* operation without a state).

For this reason, the best thing to do is to left to the user the hassle of handling the multi-threading stuff.

If your SavedModel is stateless, who cares about thread safety, otherwise you should handle it 🤔

Keep this message for what it is - just some thoughts about this topic. The best thing to do is to experiment. If you can do it, everyone that uses tfgo will benefit from your tests (right now I'm busy working on a different Go software 😅 )

[alimoeeny]

thank you very much @galeone
I'll post to this thread if I discovered any concrete information, either from the docs or by experimentation :)