Add https endpoint support#267
Conversation
mil-ad
left a comment
There was a problem hiding this comment.
These look even less like a file system. At least buckets have the file system hierarchy, even though it's fake, but not sure about HTTP URLs.
Isn't it enough to instead change filesystem_from_path() to have some sort of HTTP filesystem?
|
but this is fine if you imagine a lot of URL path manipulation |
its just that it would make life easier for two reasons:
``filesystem_from_path() |
|
I thought of another potential approach and wrote a working initial example in PR #275 . I am sure it's very possible I am missing important context, but I figured it was worth exploring to get to know the code base a bit more, and I'd learn from the negative case where I find out I am missing something too. |
@ReadyPlayerEmma is there a different solution for this https url issue that you are working on ? Should I close this PR ? |
Public buckets on cloudflare have
https://...prefixes.fsspeccan handle them easily. We can then read from cloudflare r2 public buckets without needing the R2 secrets which are currently on GCP and OSS users wont have access to that.