Implement proper retrieval receipt validation

[volmedo]

Consolidation of an egress record implies fetching the receipts batch it refers to and processing each of the receipts in it. As part of this processing, each receipt should be validated (

etracker/internal/consolidator/consolidator.go

Lines 229 to 230 in 35cc8d6

	// TODO: do more validation here.
	// At the very least that the invocation is a retrieval invocation and the audience is the node

).

At the very least, the validator should be invoked to check the delegation chain (see Alan's comment), but there are likely other validations we can implement.

[travis]

This should either be implemented before we launch OR do this as a fast follow - we open ourselves up to potentially unlimited payments to node providers until we get this in.

[frrist]

To validate a receipt we must (at least):

1. Assert the receipt+invocation chain is "valid": @alanshaw to confirm this is the correct package for providing such validation: https://github.com/storacha/go-ucanto/blob/main/validator/lib.go; Retrieval UCAN flow outlined here: https://github.com/storacha/specs/blob/main/w3-retrieval.md
   • Seems like we might be able to define a ValidationContext specific to our validation needs? (https://github.com/storacha/go-ucanto/blob/main/validator/lib.go#L189)
2. Assert the issuer of the invocation is a customer of the warm storage network authorized for making retrievals. Or that the invocation contains a proof from said customer.
3. Assert the invocation causing a retrieval is https://github.com/storacha/go-libstoracha/blob/main/capabilities/space/content/retrieve.go#L20, retrievals spawned from replication or indexing are not considered valid per the spec: https://github.com/storacha/specs/blob/main/w3-egress-tracking.md#retrieval-receipts
   • This may require modifications within Piri - e.g. https://github.com/storacha/piri/blob/main/pkg/fx/retrieval/ucan/handlers/provider.go#L71 - though I am unsure. Per the aformentioned spec Piri should only be submitting receipts corresponding to space/content/retrieve for egress tracking.

[alanshaw]

This sounds pretty much correct to me.

For 1), yes, as I mentioned in chat: We need to verify the receipt issuer is the node that submitted the batch and that the signature verifies. You then need to validate the delegation chain, by passing the invocation referenced by the receipt to the validator e.g.validator.Access(ctx, receipt.Ran(), validationContext). Example of us already doing this in Piri https://github.com/storacha/piri/blob/main/pkg/service/storage/ucan/access_grant.go#L164

For 2) the delegation chain will prove this... but I think the one additional thing we should do is verify that the resource (with - i.e. the space) has been provisioned by the upload service. i.e. the space has a provider of did:web:up.storacha.network (or the upload service DID for the network) and a corresponding subscription to an account. There's no point counting egress for a space that the upload service is not aware of.

IIRC for 3) we're only logging space/content/retrieve receipts but I might be wrong.

[frrist]

IIRC for 3) we're only logging space/content/retrieve receipts but I might be wrong.

As I understand it, this is the entry point to the egress service on line 77, but will wait for @volmedo to clarify.

[volmedo]

For 1), yes, as I mentioned in chat: We need to verify the receipt issuer is the node that submitted the batch and that the signature verifies. You then need to validate the delegation chain, by passing the invocation referenced by the receipt to the validator e.g.validator.Access(ctx, receipt.Ran(), validationContext).

I assumed that checking the signature was part of what validator.Access did? Does it need to be done independently from validation of the delegation chain?

For 2) the delegation chain will prove this... but I think the one additional thing we should do is verify that the resource (with - i.e. the space) has been provisioned by the upload service. i.e. the space has a provider of did:web:up.storacha.network (or the upload service DID for the network) and a corresponding subscription to an account. There's no point counting egress for a space that the upload service is not aware of.

Is this to prevent a malicious user from making up a non-existent space? Can spaces have other providers?

I guess verifying the space was provisioned by the upload service implies going to some upload-service tables, right?