[WIP] Add artifact repository storage capabilities

[amotl]

About

Evaluate whether CrateDB's blob store capabilities with a few additional polyfills can actually yield a reasonable and capable object store implementation.

Details

• Typical software tests like provided by other backends succeed.
• Caveat: Filesystem information is not persistent yet.
  I wonder if the path hierarchy analyzer can be used sensibly here, as presented by @marijaselakovic at
  https://community.crate.io/t/storing-and-querying-hierarchical-data-in-cratedb/1147.

[FrancescoL96]

Hello,
I am using CrateDB as my main storage solution and would love to integrate it fully with MLflow. Artifact storage is the only missing component for this integration, and this pull request seems to implement exactly that. However, I noticed this has been open for a while without recent activity.

Is there a specific reason it hasn’t been merged yet? Is it still under development?

Looking forward to any updates. Thank you for your work!

[amotl]

Dear Francesco,

thank you for writing in. You are right, activity became lower on this repository, and it is a bit behind the MLflow releases, because the fluent update process driven by Dependabot had a hiccup, and we haven't been able to dedicate a few cycles to resolve that yet, see #191.

We will try to improve on this matter, and will consider integrating this patch on the next development iteration, if it's ready. Thank you for signaling interest about it.

Did you have the chance to test it already, if it works well for you? Knowing about that will be tremendously helpful for us to evaluate the situation if that patch would be ready to be included into the next release, or if it will need additional iterations beforehand, to make it ready for prime time.

With kind regards,
Andreas.

[amotl]

Q & A

Did you have the chance to test it already, if it works well for you?

I can spot in the PR description that an important detail might be missing:

Caveat: Filesystem information is not persistent yet.

Evaluation

That's of course a significant bummer, so it doesn't make much sense to bring this patch into user testing just yet ;]. I agree it would be a very interesting feature to complete the story of using CrateDB as a unified data store, that's exactly the reason why I was staging this patch here, as I can gradually remember now after refreshing memories a bit.

I can't promise anything due to capacity reasons, but sure enough your inquiry sparked my interest to complete this patch. Until then, I am humbly asking for your patience. Of course, contributions are always welcome, and can possibly speed up the process of shipping new features.

To all reading who would be interested to see this feature materialize: Please upvote by leaving a 👍 on the canonical ticket GH-195 I've just created.

[FrancescoL96]

Dear Francesco,

thank you for writing in. You are right, activity became lower on this repository, and it is a bit behind the MLflow releases, because the fluent update process driven by Dependabot had a hiccup, and we haven't been able to dedicate a few cycles to resolve that yet, see #191.

We will try to improve on this matter, and will consider integrating this patch on the next development iteration, if it's ready. Thank you for signaling interest about it.

Did you have the chance to test it already, if it works well for you? Knowing about that will be tremendously helpful for us to evaluate the situation if that patch would be ready to be included into the next release, or if it will need additional iterations beforehand, to make it ready for prime time.

With kind regards, Andreas.

Thank you for the update and sorry for the late response. I did not have a chance to try it, however, I will try to get to it as soon as I can. I will let you know the results.

Thanks again.

[amotl]

Hi Francesco, please don't worry about delays. Apparently, there is nothing here ready for testing, yet:

I can spot in the PR description that an important detail might be missing:

Caveat: Filesystem information is not persistent yet.

[amotl]

Hi @FrancescoL96. I am hereby un-assigning myself. Please let @florinutz or @bgunebakan know if you still have interest in this implementation. Maybe @WalBeh can also lend a hand with their clanker to complete it.