Owning references (&own T) - #4000
Conversation
|
To create the appropriate cross-links to relevant issues:
|
There was a problem hiding this comment.
Here's my thought on this: I like the concept, but feel like requiring a third reference type is probably not what we want for API explosion issues.
Here's another thought: what if this were a property of lifetimes instead? Really, what you want is to be able say is that a lifetime is maximal for a given type. This would mean that generally, as long as the lifetime is kept the same through methods, you don't have to worry about "ownership" being tracked through all involved methods, only ones that would explicitly need to know about the drop behaviour.
We can bikeshed the syntax however we want, but how I see it, the main distinction is that whenever a lifetime is explicitly tagged as "maximal", it has ownership mechanics. This also allows an interesting case not covered by this RFC: mutability of a reference simply controls whether the data is allowed to be modified before it's moved, rather than after. While I'm struggling to imagine API scenarios where this is useful, it is technically an interesting option.
Also commenting a bit on the syntax bikeshed, maybe something like:
fn box_move<final 'a>(data: &'a T) -> Box<T> {
// ...
}Which could maybe have some shorthand syntax like:
fn box_move(data: &final T) -> Box<T> {
// ...
}Obviously requires a lot of extra detail/bikeshedding, but, I feel like focusing on the lifetime having properties rather than the reference itself makes a lot more sense. We could also maybe use it to extend some existing logic to allow "move and put back" type actions:
let value = *x;
*x = value + 1;Where the difference on a maximal lifetime is that you are not allowed to "put back" the value; it has to be moved out and dropped.
There was a problem hiding this comment.
This isn't really a property of lifetimes, which only considers for how long a value lives. This is a property of ownership, similar to & vs. &mut. As an example, what is the meaning of Type: final 'a?
There was a problem hiding this comment.
I don't think this has any more to do with lifetimes than the difference between & and &mut has to do with lifetimes.
In the past, I have considered the question of writing methods that are generic over reference type, and one possibility I considered is to express the difference between & and &mut as a lifetime (i.e. you can consider a mutable reference to be one that has a mutable lifetime). It kind-of works, so I'd expect expressing the difference between & and &own as a lifetime, or between &mut and &own as a lifetime, to also kind-of work. But it doesn't really capture the fundamental differences between the types of references.
In any case, most methods that could reasonably be & or &mut are not meaningful with &own. For example, you can borrow an &Vec<T> as an &[T], or an &mut Vec<T> as an &mut [T]. It does not make sense to borrow a &own Vec<T> as a &own [T] (because you cannot drop the Vec while keeping the memory it owns alive). Most other methods with an & and an &mut version would at minimum create a memory leak if written as an &own version. (The "put the value back" idea in the original post is actually an exception to this, but I think it corresponds to take_mut rather than to &own and the two are significantly different features which should be handled by different RFCs.)
A good way to think about it is that any method that takes an &'a own (to some type) should probably also return an &'a own (to some, possibly different, type) – if it does not, the memory that contained it will be unusable for the rest of the lifetime 'a. This means that the APIs that make sense for owning references usually look significantly different from those that make sense for mutable references. (The iterator example in the RFC is a good one: a mutable-borrow iterator looks like fn next(&mut self) -> Option<T>, whereas an owning-reference iterator looks like fn next(&own self) -> Option<(T, &own Self)> or perhaps even fn next(&own self) -> Result<(T, &own Self), &mut MaybeUninit<Self>> so that the memory could be reused after the iterator finishes.)
There was a problem hiding this comment.
I agree that lifetimes are mostly unrelated to this. Especially since here, it would mean that lifetimes affect post-mono behavior (whether to drop), which just doesn't work in Rust.
Regarding API explosion: This is of course a valid concern, like it is with most pointer types. However, I also believe that it is nowhere near as bad as with & and &mut: After all, e.g., writing an accessor does not really make sense with &own, at least not in the current formulation. I imagine that &own will see significantly less API compared with other references.
I am happy to mention API explosion in drawbacks though, if there are others with similar concern?
There was a problem hiding this comment.
This interacts heavily with the Beyond the & project goals family. See also the #t-lang/custom-refs Zulip channel. People have been discussing this for a long time now, I don't think we should ignore all that discussion (while the RFC has a section for the history of &own/&move, which is good, it does not include those).
There was a problem hiding this comment.
I have been somewhat involved in those discussions as well, so I believe I have a mostly complete picture of said proposals. My argument here (which I thought I put into the RFC, but maybe forgot) is that we would likely want an owning reference with the proposed semantics in any case, even if we support custom reference types. This just means that this proposal would change from a lang feature to a libs API, with the same contents (except syntax).
| However, in a possible future where Rust splits immovability from the drop guarantee, `&own` would play a central role in passing ownership of immobile types. | ||
|
|
||
| > The "Immobile types and guaranteed destructors" project goal[^move-trait] is (among other options) considering a combination of `T: !Move + !Forget` to replace `Pin<T>`. | ||
|
|
There was a problem hiding this comment.
This RFC is missing what is to me the most important use for &own: creating references that can change the type of their target. For me this is an extremely strong motivation for &own, and so I'd like to see it mentioned.
A toy example of this sort of API would be fn try_as_nonzero(n: &own usize) -> Result<&own NonZeroUsize, &own usize>: you give it an owning reference to a usize and get back an owning reference to it as a NonZeroUsize, if possible.
More useful examples include things like fn drop_in_place(t: &own T) -> &own MaybeUninit<T> (i.e. "drop a value we own, while keeping the memory containing it"). This makes it possible for safe code to use the same memory to store multiple different types of objects; currently, doing that in safe code without a memory leak requires storing the object via MaybeUninit::<Option<T>>::write (allowing Option::take to drop it), but this has both the memory overhead of Option and the "it's hard to prove this program doesn't panic" overhead of Option).
There was a problem hiding this comment.
fn drop_in_place(t: &own T) -> &own MaybeUninit<T> seems unsound, the &mut T equivalent is unsound because you can use it to overwrite an enum tag that was niche-optimized so is in the same memory as T, which you can then overwrite by writing MaybeUninit::uninit()
There was a problem hiding this comment.
@programmerjake This was discussed recently at rust-lang/unsafe-code-guidelines#618. It has not yet been decided for &mut whether or not it is legal to overwrite an enum discriminant from within one of its own variants, if the original reference is never used again (but I was hoping for a decision to be made that it would be).
For &own I think there is less downside than with &mut in allowing it (creating an &own reference to an enum's fields conceptually requires destroying the enum and thus it no longer has a discriminant to overwrite), and more upside than with &mut in allowing it (because writing this sort of code would otherwise require unsafe Rust).
There was a problem hiding this comment.
One thing we did talk about was having some type, let me call it AlreadyDropped<T>, which behaves like ManuallyDrop but is known to be dropped. With such a type, I could see APIs like drop_in_place<T>(&own T) -> &own AlreadyDropped<T> and AlreadyDropped<T>::write(&own self, val: T) -> &own T.
Here, AlreadyDropped would guarantee a valid bit pattern for T, so that these operations can be safe and sound.
I'm definitely happy to put fn try_as_nonzero(n: &own usize) -> Result<&own NonZeroUsize, &own usize> as an interesting API in the appendix, but if you think this should receive a bigger spotlight in the Motivation section, I'm sure we can come up with something compelling there as well.
There was a problem hiding this comment.
Yeah, the output would need to be OutRef<'_, T> or this &'_ own/mut AlreadyDropped<T>.
- Nit: I'd even suggest having
&mut AlreadyDropped<T>rather than&own AlreadyDropped<T>, as&owndoes not offer anything else, other than silly things such asdrop_in_place(drop_in_place(drop_in_place(own_ref)))…
Indeed, let's not forget the case of T: Copy, wherein the original value will be accessible beyond the lifetime of this &own/&out-chain:
// `mut` required to take `&own` to a known-`Copy` place.
let mut b = false;
let r: &own bool = &own b;
// If we had the following API:
let r2: &mut MU<bool> = drop_in_place(r);
r2.write(MU::uninit());
dbg!(b); // Uh-ohThere was a problem hiding this comment.
We can't allow both:
- The first one allows overwriting all bytes behind
&ownincluding niches, since that's howMaybeUninitworks. - The second one requires keeping niches intact, or at least restoring them before the callback returns, which you cannot guarantee safely.
Sure, both can be used soundly, but they can't be simultaneously safe.
There was a problem hiding this comment.
The second is effectively an owning-reference version of take_mut (rust-lang/rust#161168), whose soundness has been disputed for a while (and is still not stabilized), but for which the general sentiment seems to be leaning in the direction of considering it sound.
take_mut aborts if the inner closure panics, and your replace_with_own would also have to do that, but that has nothing to do with whether overwriting niches is legal or not (replace_with_own could overwrite the niche of the mut while it was running, restore it from the T returned by the FnOnce, and abort if the FnOnce panicked, regardless of whether or not take existed). So I think they can be simultaneously safe (and in fact, if rust-lang/rust#161452 is accepted, the soundness of replace_with_own will have nothing to do with niches because temporarily overwriting a niche will be explicitly legal).
The most difficult problem with implementing this specific type signature would be that the return value of the FnOnce might or might not alias its argument. You would probably have to write it, from the Rust abstract machine point of view, as moving out of the return value into a temporary, and then from the temporary back into the &mut. The optimizer would likely be able to convert that into a memmove (which would have a fast path for the common case where the return value had the same address as the &mut).
There was a problem hiding this comment.
I'd very much like for &mut and Box to be unified opsem-wise, and it seems like it's a feasible way forward that also makes life easier for us in defining Box semantics more broadly. I do believe in that case both Box::take and replace_with_own would be entirely sound, even in combination
There was a problem hiding this comment.
fn drop_in_place(t: &own T) -> &own MaybeUninit<T>seems unsound, the&mut Tequivalent is unsound because you can use it to overwrite an enum tag that was niche-optimized so is in the same memory asT, which you can then overwrite by writingMaybeUninit::uninit()
&own and &mut can and maybe even should have different rules, given their different roles. (&mut and Box already have subtly different rules today so if &own is like Box it'd be surprising if &own was exactly like &mut.) So nothing we discuss about &mut should be taken as binding for &mut.
There was a problem hiding this comment.
So nothing we discuss about
&mutshould be taken as binding for&mut.
Did you mean: So nothing we discuss about &own should be taken as binding for &mut.
|
|
||
| The type `&'a own T` behaves similar like other references, except it owns the value of the pointee: | ||
|
|
||
| - It is **covariant** in both `'a` and `T`. |
There was a problem hiding this comment.
I believe that, unlike &'a mut T, &'a own T should not have the well-formedness condition that T: 'a (in other words, it should be legal to have an &'a own T for which the T is not valid for entire lifetime 'a. With &'a mut T, at least in safe code the reference has to store a T for the entire lifetime 'a, so it makes sense to require T to live that long. With &'a own T, it is possible to drop the T before the lifetime 'a ends, and so it is both reasonable and useful to store a short-lived value of type T into long-lived memory that lasts for 'a (many of my programs that want to do this are temporarily storing short-lived values into long-lived memory, but want to remember that the memory is long-lived).
I think, but am not totally sure, that this is compatible with the requirement to drop the T when an &'a own T is dropped (because you can't drop a type outside the lifetime of any of its generic parameters, so even though an &'a own T could be alive during parts of the lifetime 'a where T is dead, the reference couldn't be dropped at such times, only leaked or forgotten).
If adopted, the "there is no requirement that T: 'a" condition should be added to this list, as it is a difference from mutable references that is not currently listed.
There was a problem hiding this comment.
you can drop things where their internal lifetimes have expired, that's what #[may_dangle] does (used in container types in std): https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=42e06a037f1711bed836023f2b6fb163
There was a problem hiding this comment.
I forgot about #[may_dangle], but I don't think it matters here: if a wrapper Wrapper does not use #[may_dangle] (regardless of what else might be using it), you can't drop a Wrapper<T> after T's lifetime has expired except in cases where it's safe to drop a T after T's lifetime has expired. Thus, in any situation where you drop an &'a own T after T's lifetime has expired, it should be safe to drop the T.
There was a problem hiding this comment.
This briefly came up in discussions prior to this RFC, though I don't think we elaborated much on it. One thing is sure though: Even if we drop the implicit bound &'a own T implies T: 'a, in many cases this will be required by dropcheck. In particular, any function that takes a generic &'a own T and drops it inside its scope must require that T: 'a, since it otherwise calling drop(T) would be called on dangling lifetimes.
This can be relaxed only if we know we do not need a valid T for dropping, e.g. if it is copy or marked with may_dangle in the appropriate drop implementation. (Or we never drop it in the generic function.)
There was a problem hiding this comment.
I don't agree that this is required by dropcheck. If you take a generic &'a own T, then even without the well-formedness requirement, you must be both inside the lifetime 'a and inside the lifetime of T (because a function/method can only be called if all its generic parameters are alive, both types and lifetimes). Thus, it should be OK to drop it.
This is making me think that Own<'a, T> may be the right name for this sort of reference, because it makes it more obvious that the lifetime of the reference is tied to both 'a and T rather than just 'a.
There was a problem hiding this comment.
Ok, actually I agree now, after reading a bit into it. It seems to be true that &'a own T does likely not need T: 'a in order to be well-formed. And for dropping, we need both T and 'a to be live, but that does not mean that we need the outlives condition.
I'm not sure I agree with the argument about Own<'a, T> being clearer in that sense, but it may be true.
There was a problem hiding this comment.
Hm... I'm still struggling with this. I'm trying to figure out why &'a T and &'a mut T both require T: 'a but &'a own T does not. My problem is that I don't understand the requirement in the first place. T: 'a is neeed in order for &'a T to be well-formed, but why? Is this documented somewhere?
There was a problem hiding this comment.
As far as I understand it, &'a T and &'a mut T don't require T: 'a for soundness. Rather, there's no way to safely create them except when T: 'a (something which wouldn't be true for &own – &own admits safe APIs that produce short-lived values in long-lived memory, and we would probably want to add some). If something generic in 'a and T happens to require T: 'a, then normally you would need to write a where clause, where T: 'a, but the well-formedness rule means that if you're taking an &'a T or &'a mut T as an argument, you can leave the where clause out and you still get the constraint on the generics.
So it's one of those obscure ease-of-use rules that increases the number of situations in which the obvious/natural code ends up doing what the programmer wants (similar to the way that, e.g., a match on a reference allows you to match fields of the thing you're matching as references, rather than needing to explicitly write match *x at the top and ref on each of the fields), in this case by meaning that you can in some circumstances omit where clauses that would otherwise be required. It makes code easier to write, but makes the rules of the language harder to learn and reason about (because other types in Rust don't work like that).
There was a problem hiding this comment.
Yeah, I actually think that, in hindsight, T : 'a required for &'a [mut] T is not only unnecessary, but that it was a design mistake, which greatly hampers the expressibility of the language in for<'a>-quantified contexts. I think it was an overzealous excess of caution which caused this, as it "felt right" at the time, without realizing that it should have only come to exist alongside for<'a where … : 'a> quantifications. And since these do not exist, we run into that category of limitations (c.f. GATs and the "entails : 'static" limitation).
In practice, it does come with one advantage, but which boils down to the very same "lack of explicit for<'a where … : 'a> quantification" in the language: &'a T results in an implicit T : 'a bound which in turn, in certain contexts, is able to "retroactively" constrain a for<'a> quantification down to a for<'a where T : 'a> one.
Nowadays, however, we cannot remove this design constraint because the language very much makes use of &'a T entailing T : 'a, and then making use of the latter property. Notably, given a &'static mut T, you can query the TypeId of that T, and when T: Send, you can send an owned T instance to a thread::spawn().
Back to &'r own T, we could go either way, but once we were to commit to a choice, there would be no going back.
- With no
T : 'r, we'd be one step closer toBox<T, CovariantNoop<'r>>; - With
T : 'r, we'd be closer to / more consistent with&'r [mut] T, which would result in a principle of least surprise.
So in a vacuum I'd be tempted to suggest we stick to the latter, unless we manages to come up with an API usage of &owns which is limited by this (can you think of one @ais523?). But precisely my gut feeling tells me such an API is quite plausible, actually, which in turn would make me lean towards the former 😅. So it all hinges on our foreseeing ability to come up with such an API or a lack thereof, I guess
There was a problem hiding this comment.
One program that I've recently been trying to write requires &own to not require well-formedness. In that program, the ill-formed &own gets created by an API for transmuting MaybeUninit, which looks like this:
/// Panics if `T` and `U` have different sizes or if `U` requires more alignment than `T`.
fn transmute_uninit_own<'a, T, U>(old: &'a own MaybeUninit<T>) -> &'a own MaybeUninit<U>;This is safe for the same reason that align_to_uninit_mut is, but (if you don't enforce well-formedness) is able to produce a U that has a shorter lifetime than T (and thus potentially a shorter lifetime than a).
My program uses the API in question to store short-lived references into a static variable (it needs to be able to find the references in memory without having a reference to them, so they need to be stored somewhere which has a known address). It's useful more generally, though (in particular, it can be used to write allocators in safe code).
| - Especially lifetime extension is missing. | ||
| - `&own self` receivers are not (currently) possible. | ||
| - It is non-straightforward to write functions are "allocation-agnostic", i.e., work with both `Box` and `StackBox`. | ||
|
|
There was a problem hiding this comment.
Another drawback to implementing this with a library implementation (based on experience with trying to implement it): there is no way to convey the fact that an &own is owned to the compiler, so it will generate code assuming that it might have aliases. This leads to worse-quality code generation.
There was a problem hiding this comment.
Good point, though this could be fixed by using Box<T, Noop<'a>> as the pointer type, which currently provides noalias.
There was a problem hiding this comment.
Strangely enough, it doesn't: putting noalias on a Box<T, A> is actually unsound in the current Rust operational semantics unless A allocates only fresh memory from entirely outside the Rust abstract machine, and is also unsound in the current LLVM operational semantics unless A is entirely opaque to LLVM. (The current resolution to these problems are that LLVM does not inline or otherwise optimize around calls to global memory-allocation functions, and Rust does not put noalias on Box unless it is using the global allocator). There's at least one RFC on LLVM's side to try to address the situation (and I've been working on a not-yet-posted proposal to do so from the Rust side, too).
There was a problem hiding this comment.
Hm that seems... odd...
Oh I see, you mean that when we allocated from e.g. an arena, we're technically aliasing? Do I understand this right? Otherwise I'm a bit confused by your statement.
I would assume that &own is noalias, and if we can't express this with Box (Unique doesn't provide this either), then that is indeed a hint that we may need a library type. However, the arena example would likely apply to &own as well, people will want to do that operation
There was a problem hiding this comment.
The issue is that you can pass a box to a function as an argument, free the box within the function, then (still within the same function) allocate a new box that happens to occupy the same memory. The new box and old box are at the same memory location, so when you access the new box, you end up accessing memory that was pointed to by a noalias argument (i.e. the old box) without going through that argument, which is undefined behavior. The current workaround is for LLVM to pretend that the two boxes actually point to different memory (despite being at the same address), but that model would break any other references to the same thing (e.g. the backing allocation of an arena).
I consider this to primarily be a bug in LLVM (which is probably ultimately the fault of the C standard), and am hopeful that the situation will be possible to fix, although it will likely need changes to both LLVM and to Rust's aliasing model.
(This is my third attempt to post this comment, hopefully it's in the right place now.)
There was a problem hiding this comment.
I wrote up my thoughts on making Box and &own work in the aliasing model here on IRLO.
| - Doubles the amount of typing and adds visual clutter | ||
| - `Own<'_, T>` could use normal type syntax, avoiding additional parsing complexity. | ||
| - This would visually more closely resemble `Box` rather than other reference types | ||
| - This would likely need a macro to perform (re-)borrowing |
There was a problem hiding this comment.
It doesn't: such a reference would be able to implement DerefMut, and it would almost certainly be a good idea to. With that trait implementation, Own<'_, T> would reborrow automatically as a receiver, and would allow manual reborrowing (e.g. in argument position) using the syntax &mut *o (where o is the Own<'_, T>). I am inclined to think that automatically reborrowing in argument position is a bad idea regardless of the syntax used for the reference (basically because reborrowing an owning reference does something significantly different from moving it, and both are plausible uses for an owning reference in argument position), so this syntax actually doesn't have any syntax overhead for reborrowing.
There was a problem hiding this comment.
Maybe this wasn't well formulated here, (re-)borrowing as &mut is certainly possible.
(Re-)borrowing of expressions as &own is what I'm concerned about, i.e., how to replace &own expression.
(I should remove the "re-" part here anyway, since that likely doesn't make much sense for owning references)
There was a problem hiding this comment.
I see the &own operator as more like a move than a borrow: I guess you can think of it as a delayed move (because you are creating an object with the ability to move the thing it points to; from a borrow-checking point of view, &own as an operator can be thought of as moving a value and borrowing the memory that previously contained it). We probably need a good name for the operator that makes it clear what it does, because Rust's current terminology doesn't seem to be up to the task of naming it unambiguously.
Having a good name for the operator would both be important for teaching people how this sort of reference works, and help to avoid misunderstandings like this.
There was a problem hiding this comment.
Yes, that is a good point. "borrow" is a bad name when talking about &own (and I mention this in the RFC). I have argued that &move makes sense when talking about the "borrow" operation, but I prefer &own when talking about the resulting type. However, we cannot really mix&match these I think.
This would also mean that we'd want to rename the borrow operator, which seems a bit... fundamental? "Creating a reference" still works, but it's not as concise.
There was a problem hiding this comment.
I guess the best way to explain it is that giving someone a T is giving them ownership of the value by moving it, whereas giving them a &own T/&move T gives them ownership of the value without immediately moving it (although because they have ownership, they can move it). There is a borrow involved, but it's of the underlying memory rather than the object itself (and that borrow is the reason the &own/&mut reference has a lifetime – it wouldn't otherwise need one).
Perhaps "take ownership" is a good short description – I like "take" because this is consistent with the other operations named "take" in Rust (in that you are changing responsibility for an object), and "take ownership" because it's only the ownership that's moving, the object itself isn't moving along with it.
This reasoning doesn't help much with what to name the operator, though.
There was a problem hiding this comment.
I just think that the &own operator will be easiest to teach if it is primarily framed as a kind of borrow operator because then you can start your mental model with a category of borrow operators which all do the same thing - they all create a reference to a value. Then you build on that foundation to understand their differences between the borrow operators in terms of their ownership semantics.
There was a problem hiding this comment.
FWIW here's a possible teaching tool.
& |
&mut |
&own |
|
|---|---|---|---|
| Reference | ✅ | ✅ | ✅ |
| Exclusive | ❌ | ✅ | ✅ |
| Ownership | ❌ | ❌ | ✅ |
There was a problem hiding this comment.
I do agree that an "owning borrow" is understandable, even if it seems like a bit of an oxymoron. I would not want to change the name of the borrow operation in the entirety of Rust, and I don't tihnk it should have a different name when using a new reference.
There was a problem hiding this comment.
Hmm actually now I'm getting cold feet about "Owning Borrow but it doesn't borrow but it borrows the allocation if you think in those terms". Maybe it is better to call it Owning Reference so that "borrow" is preferred to always mean "borrow a value". "borrow" is the greatest common denominator semantic of the existing operators, but that doesn't hold with &own.
There was a problem hiding this comment.
Note that's it's already possible to "borrow a value forever" and I have never seen anyone complain about that before.
| However, there are a number of downsides to this approach: | ||
|
|
||
| - The `Box` API is too general. | ||
| For example, it would mean that `Own<'a, T>: Clone` if `T: Clone`. However, calling this method would have to panic with the `Noop` allocator. |
There was a problem hiding this comment.
There has been discussion about splitting the concept of an allocator from the concept of a deallocator (and allowing a Box to contain just a deallocator). Although this isn't currently implemented in allocator_api, it is as I understand it a possibility that is intentionally being left open.
I believe that &own is the equivalent of a Box that has a no-op deallocator that is not usable as an allocator. In the world where deallocators and allocators are separate, some of Box's methods/traits would require Box's second type parameter to be both an allocator and deallocator, whereas others would be usable with just a deallocator; and in that world, Box::clone would require a full allocator, so it would not be implemented on Own<'a, T> regardless of whether T were Clone or not.
That said, I think the "implement &own in terms of Box" technique is backwards; Box probably can, and probably should, be implemented in terms of &own instead (although there are likely to be some opsem issues with this, I am hopeful that they can be resolved).
There was a problem hiding this comment.
Interesting, I have not seen such proposals. Certainly, if "Box that doesn't allocate and does nothing when deallocating" fits into the design of Box, then we could do that, although it still feels a bit hacky.
I agree that we could probably implement Box as a struct Box<T, A>(unsafe<'a> &'a own T, A) (or similar), since &own mostly represents what we want Unique to be.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
There was a problem hiding this comment.
Even without deallocator splitting, the claim in the text
it would mean that
Own<'a, T>: CloneifT: Clone. However, calling this method would have to panic with theNoopallocator.
seems to be false. If Noop: Clone did not hold, then neither does Own<'a, T>: Clone, because impl Clone for Box<T, A> requires A: Clone. There would still be the hazard of an always-failing allocator, but one would have to actually take/borrow that allocator from the box to achieve that failure; there is no need to have a broken Clone impl.
There was a problem hiding this comment.
Coming in here a bit late perhaps, I do think there's quite a bit of value to be gained from modeling &own as little more than syntax sugar for Box<T, Noop>; for one, having spent some time poking around at this RFC and related discussions, I don't see much reason why Box and &own are opsemantically too different - both are some variety of "&mut with a Drop tacked on" - and the current ideas for a distinct Deallocator trait very much are intended to model this (in fact, the most common usecase for a custom deallocator is precisely a no-op).
If the semantics are indeed the same, there's perhaps a point to be made about having the ugly-syntax Own<'a, T> form of this as an unstable api in std; that's also a much smaller process to go through for experimentation.
There was a problem hiding this comment.
I removed the note on Clone and just replaced it with the allocator. I assumed that cloning was possible, and it's not too far of since you can apparently do Box::new_in(&*old_box, old_box.allocator()), which changes the type to a Box<T, &Noop<'a>> which is almost like cloning.
But yes, with a Deallocator trait this argument becomes even less relevant, and I mostly agree with nia on this now (after some discussion). I think it may be interesting to model a Own<'a, T> = Box<T, Noop>, if only to show that the lang effort is mostly about the syntax (since this feature shouldn't require any fundamentally new compiler capabilities).
In that case I believe that this RFC still has value, but the scope might change.
|
|
||
| ### Alternative: Add remote drop flags to support pinning | ||
|
|
||
| Unfortunately, `Pin<&own T>` is unsound with regards to the drop guarantee. |
There was a problem hiding this comment.
I think Pin<&'static own T> is sound with regards to the Pin drop guarantee (basically because if the &'static own gets forgotten, there is no way to ever access the memory again).
Pin<&'a own T> with non-static 'a is unsound, but this is in the same way (and for the same reason) that Pin<Box<T, A>> is unsound unless A: StaticAllocator. As such, I think it conceptually has the same solution, "it's unsound to pin through something that owns memory if the memory could be repurposed outside its lifetime, so doing so requires unsafe unless the memory can be proven to live for 'static". To me this is a strong argument against remote drop flags: there isn't really a reason to handle this any differently from how Box handles it.
There was a problem hiding this comment.
Yes, &'static own is safe to pin. I think this was meant in comparison to what the moveit crate does for its owning refs, which support pinning via remote drop
There was a problem hiding this comment.
Yes, a Pin<&'static own T> is sound, which i briefly mentioned in the drawbacks section. And by the Box<T, Noop<'a>> analogy, we can see how that relates to StaticAllocator. And while I agree that it has the same solution, with Box you can make a Box<T, A> sound if you're careful around creating it (by using a proper allocator), whereas a Pin<&'a own T> is always unsound (unless 'a: 'static). This to me is a meaningful difference.
While I'm not a fan of adding drop flags, I don't understand how this is a strong argument against them though: Sure, we can just say "its unsound, don't do it", but with drop flags it can be sound, so surely that's an argument for it? Can you elaborate on this maybe?
There was a problem hiding this comment.
The argument is basically that I wouldn't expect pinning an &own to have more functionality on owned references than Box::pin/Box::into_pin do on boxes; it would be much easier to fit a drop flag into a Box than it would be to fit it into a reference, so if such an API were provided, I would expect to see it on Box first.
Fixed formatting and some typos
| The syntax of borrow expressions is extended in the same way. | ||
| Similarly to the owned reference type expression, if `Expression` begins with an the identifier `own`, it must be wrapped in parentheses. | ||
|
|
||
| ```grammar,expressions | ||
| BorrowExpression -> | ||
| (`&`|`&&`) Expression | ||
| | (`&`|`&&`) `mut` Expression | ||
| | (`&`|`&&`) `own` Expression | ||
| | (`&`|`&&`) `raw` `const` Expression | ||
| | (`&`|`&&`) `raw` `mut` Expression | ||
| ``` |
There was a problem hiding this comment.
🤔 So raw can be contextual in &raw const because of the full keywords const and mut which prevents existing code from using the same sequence. But &own standing on its own would really need own to be a full keyword and otherwise we'll get very weird syntax or type errors if people aren't migrating via cargo fix.
Example:
&own[1..]previously means&(own[1..])of type&[T], after this RFC it means&own([1..])of type&own [RangeFrom<i32>].&own()previously means&(own())calling theownfunction and gets a temporary reference, after this RFC it is an owned reference of the unit typeif x != &own { stmt; }previously is a normal conditional statement, after this RFC it comparesxwith the owned reference of the result of{ stmt; }(and ends up with syntax error because theifexpression is missing a block)&own!(expr)previously is a reference of the result of theown!macro, after this RFC this becomes&own (std::ops::Not::not(expr)).
There was a problem hiding this comment.
Oh hm, I thought they were contextual because they appeared after the &... So this still works, but migration would be a bit of a pain if not done automatically...
I guess in some weird edge cases this change could even go unnoticed and open up soundness holes and such.
There was a problem hiding this comment.
Would be interesting to see how often this appears in real code, but probably occasionally, especially with the macro and function, which to comply with naming guidelines
There was a problem hiding this comment.
Actually, making own a full keyword doesn't help with this at all, does it? You would still get all the problems you mentioned, just that we'd do it over an edition, presumably while linting in older editions.
In that sense, a contextual keyword would still significantly reduce the breakage.
There was a problem hiding this comment.
&move I think is likely to produce less breakage; are there any ambiguities other than with &move || {…}? You could resolve that ambiguity using precedence rules (i.e. interpreting the ambiguous case as &(move || {…}) for backwards compatibility, and writing the other possibility as &move (|| {…})).
&own with a full keyword would work better in a new edition, but would frequently have to be written as &k#own in older editions, which is a bit of a mess (and possibly also not 100% backwards compatible, although that particular sequence of tokens is unlikely to appear in a macro argument).
There's also the possibility of decoupling the syntax for the operator and for the type, e.g. Own<'a, T> for the type and .move for the operator.
There was a problem hiding this comment.
I guess regarding not having a keyword: This is like try but worse, so not having a keyword is probably not a good idea. If we want to avoid an additional keyword (and overloading move), we could use something like & ref(own) or maybe &own ref. I do like &ref(own), since it does work with any reference type (without requiring a keyword), possibly even with custom references, as well as in pattern matching. But the additional parentheses are visual clutter. I think I'll drop by the custom-ref zulip and see what their thoughts are.
There was a problem hiding this comment.
I have started a discussion thread on Zulip: #t-lang/custom-refs > Syntax of custom ref ops (borrowing/pattern matching)
There, I'm proposing &ref<CustomRef> expr as the borrowing syntax for custom references, as well as ref<CustomRef> IDENTIFIER for corresponding binding modes.
I'm not sure if we are willing to decide on a syntax for custom references yet, but if own needs to pick a different syntax, I would prefer that it is compatible with custom references.
Of course, making own a keyword is still on the table, if we deem it worthy.
There was a problem hiding this comment.
I'd like to emphasize that &move teaches the wrong semantics of these references, which are:
- about
own-ership, i.e., the right and duty to drop the ownee, - by reference, i.e., with NO
move-ing whatsoever!
Using &move would be quite an oxymoron w.r.t. the latter bullet!
The only case where a move actually may happen with a &own reference is through the imagined deref_move() / * operation, depending on how it's spelled out and what its details are. But given an !Sized ownee, it will very much not be moving.
So whilst I do agree that "the syntactical transition" is an important aspect of designing a language feature affecting syntax, wherein we should strive to make such a transition as smooth as possible, and that re-use of the move pre-existing keyword would help immensely in this regard, we should however very much not lose sight of the ultimate goal in question here, and not over-compromise it just for the sake of transition.
Having an own!(…) macro in the pre-existing editions is quite a manageable transition helper which would allow us to make own a keyword in the newer editions
There was a problem hiding this comment.
I realized recently that &box may also be a possible name (and has no current syntax clashes because box is a reserved keyword with no uses, although it might clash with planned future syntax such as box patterns). I don't really like it, but it does make some sort of intuitive sense (especially because when I was trying to implement &own on my own, I called it BorrowedBox). The concept would be that a &box borrow would be borrowing memory in order to use it like a box, with a value already inside (which could be moved out of it because that's how boxes work).
This would require generalizing the concept of "box" to not necessarily be stored on the heap, but I think that's fine: with custom allocators, boxes can be stored in all sorts of places already (and a Box would store an &box internally, in this model, and just handle deallocation after the value is dropped). In fact, even in current Rust, LLVM can sometimes optimize Box to be stored on the stack rather than the heap. To me, the most important parts of the "box" concept is that it's a piece of memory for which you own the value inside and can mutate and move it, and &box would satisfy that requirement.
There was a problem hiding this comment.
I have thought about this, and it makes some semantic sense, especially if you consider the Box<T> type to be effectively a wrapper around unsafe<'a> &'a box T. It is also nice in the sense that it makes it clear that &own behaves more like a Box than like a &mut, for example, requiring a mutable binding in order to be modified.
However, I feel like it is just in uncanny valley for me: &box T is just too close to a &Box<T> to feel distinct.
| This `Box` is effectively the same as `&own`, except for missing ergonomics. | ||
|
|
||
| > Note that `bumpalo:box:Box` provides a pinning API, which is an known to be unsound. | ||
|
|
There was a problem hiding this comment.
This section is missing prior art from other languages.
Most notably, C++ has the concept of an "xvalue", which is a value that has identity and can be moved out of; an "xvalue reference" would be analogous to the concept discussed in this RFC. (It isn't exactly the same, because moves in C++ work differently to how they work in Rust; in Rust, a moved-from object is unsound to use from safe code, whereas in C++, a moved-from object is re-initialized to a valid but unspecified value. But it seems to be the same if you allow for the difference in how moves work.)
C++ doesn't have xvalue references, but it does have rvalue references, which are like xvalue references with the exception that you cannot observe their address ("rvalues" include xvalues and temporaries, and the "cannot observe their address" restriction appears to be intended to prevent you reusing memory that was used to store a temporary). These use the syntax T&& for the type (C++'s normal reference type is written T& and is the equivalent of Rust's &UnsafeCell<T>), and do not have an explicit operator to create them (rather, they are created automatically when you attempt to move a T into a variable or function argument that expects a T&&). The usual way to intentionally create an rvalue reference is to add a call to std::move, which is the identity function on type T&& (so the argument gets moved into the function as an rvalue reference, and then the rvalue reference is returned).
Rust already provides a way to see the address of a temporary (you can apply &mut to a temporary in Rust), and thus it doesn't need a distinction between xvalues and other types of rvalues (nor a way to prevent observing the address of an rvalue that is not an xvalue). As such, translating the C++ design to Rust would look something like this: an value of type T can be coerced into a value of type &own T, which has a borrow-checker effect similar to moving the value and borrowing the memory containing it (and the resulting &'a own T represents both ownership of the value, and a borrow of the memory for lifetime 'a).
It's notable that C++ went from having one reference type to two once rvalue references were added, which seems like even more of an extreme change than adding a new reference type to Rust. (Rust has two "main" reference types, &T and &mut T, but also a number of more minor reference types like Pin<&mut T> and &Cell<T>, and also Box which is not technically a reference type but acts a lot like one in practice.)
There was a problem hiding this comment.
I did some brief research into this, but my first instinct was that rvalue references are not that similar, but I'm by no means a cpp expert. I can look into this again and see how close they relate, and add a section to prior art.
There was a problem hiding this comment.
Ok, maybe I can wrap this in my own words to see if I understood this correctly:
Moving in C++ are enabled by rvalue references T&&. These are a special kind of reference which allows the user to steal its contents, leaving the original place in a possibly invalid state. This is essential for enabling move semantics in C++. This is evident since std::move operates on rvalue references. Unlike this proposal, rvalue references are always created via coercion, often performed by explicitly using std::move.
Additionally, you cannot observe the address behind an rvalue reference, to avoid writing to a temporary place (citation needed). In Rust, this is not needed due to "temporary lifetime extension", which puts the temporary in a place which indeed possesses an observable address.
There was a problem hiding this comment.
This is almost correct, but (unlike in Rust) moves in C++ don't stop the destructor of the moved-from value running. (I think this is right, at least – I'm not a C++ expert.) So they have to set the moved-from memory to a state that can at least be destroyed correctly, rather than leaving it entirely invalid.
| - It is possible to move out of an `&own T`. | ||
| - Dropping an `&own T` drops the inner `T`. |
There was a problem hiding this comment.
Perhaps here we could already start talking of ?Sized-ness: the first bullet requires Sized; the more fundamental property, as it covers all of ?Sized, is the latter bullet
There was a problem hiding this comment.
In my mind, whether a value can be moved or not is somewhat orthogonal to whether it is (in principle) possible to move out of a &own. Especially if you consider an API of the form fn move_to_mut(source: &own [T], target: &mut [T]) which does kinda move the unsized [T]. You could argue that it actually moves the elements, not the full DST, in which case we are still able to partially move out of T: ?Sized types.
Maybe I'm just being pedantic though, I can be convinced otherwise :)
Also, I consider the "moving out" the fundamental property, and dropping just a consequence of that, since otherwise people might wonder "why would I need a reference that drops".
| This simple approach to owning references cannot support pinning. | ||
| A `Pin<&'a own T>` is unsound, since forgetting the reference violates the drop guarantee (unless `'a: 'static`). | ||
| This may be confusing to users, since they often only consider the immovability guarantee of `Pin`. | ||
| Unfortunately, the [alternative](#Alternative-Add-remote-drop-flags-to-support-pinning) is more complex, and we believe that it is not worth it. |
There was a problem hiding this comment.
Notably, it would not make &own refs as "transparent and zero-cost" as their syntax would let transpire. Own-refs with drop flags can exist and be useful, but they should probably come as a third-party library, and its OwnRef<'_, T> wrapper type, rather than being blessed in the language.
- (I am saying this as the author of a crate which does feature this very thing.)
|
|
||
| ### Does `&'a own T` need to be covariant in `'a`? | ||
|
|
||
| While it is undisputed that it is *possible* to enable this property, there are concerns about its usefulness. |
There was a problem hiding this comment.
there are concerns about its usefulness
We discussed about this on Zulip and came up with several examples where it is useful to be covariant over 'a (and whilst we did not show it, there would be value in covariance over T as well). So I actually deem this statement to be incorrect or a huge understatement.
Perhaps it's a matter of terminology: whilst it would be very useful/ergonomic to be covariant, it is true it would not be fully necessary. And there is also some usefulness in the "initialization prove [proof] for certain in-place-init proposals."
So this section could be a bit rephrased as which one may be deemed more useful in practice?.
-
Perhaps a quick mention to the second, optional, branding lifetime infecting
owncould be warranted:In case
&'r own Twere chosen to be covariant over'r, generativity-based initialization proofs could still be used with these references by considering that&ownreferences could be allowed, in the future, to carry a second lifetime parameter, optional and defaulting to'static, which would be invariant:&'r own<'id> T. If so, most&own-ref APIs could simply be amended to be<'id>-generic (and forwarding). And generativity-based initialization proofs would be able to use it.The other option for generativity-based initialization proofs would be for them to involve their own custom
Own-ref like type, carrying that second/branding lifetime parameter. It would keep&ownsimple and lean, but at the cost of requiring them to duplicate its logic (though they could perhaps avoid this by usingBox<T, NoopDeallocator<'r, 'id>>…).
There was a problem hiding this comment.
One of the reasons I worded it like this was because I was unsure if initialization proofs are the only interesting area. It appears to me now that they are (and I disagree with the use there) so I may change this to stronger wording, like you mentioned here.
There was a problem hiding this comment.
I think the "branded &own" is potentially very useful, but that the branding is not a normal sort of lifetime and shouldn't be written as one. (I posted something very similar on IRLO a while back, using placeholder syntax of @a for "a lifetime that also brands a reference to remain in a particular location".)
I don't think mixing the lifetime of &own with a location invariant works in practice. For example, imagine you have an &'a own to the first element of a linked list. You can destructure that into an &'a own to the value inside the first element and an &'a own to the second element. This is the same 'a as the original list had (even if you are using invariant lifetimes), so you have a reference that is meant to prove that a particular memory location is initialized but it actually proves that a different memory location is initialized, which could easily be a soundness problem.
As such, instead of a lifetime, you need a different sort of generic to make &own work as an initialisation proof. (I would love to see a world in which you have both &'a own T that just controls the lifetime of the borrow of the containing memory, and &@a own T that brands its location. But the two types would have a substantially different API.)
| The final option is to use the "trivial copy" property of `Copy` types, and simply restore the value after the borrow. | ||
|
|
||
| ```rs | ||
| let x = 5u32; | ||
| { | ||
| let temp = x; // Compiler generated | ||
| let owned = &own x; | ||
| *owned += 2; | ||
| x = temp; // Compiler generated | ||
| } | ||
| assert_eq!(x, 5); // Now the assertion passes | ||
| ``` |
There was a problem hiding this comment.
So, the devil is in the details. If we do this, we'd have to have:
let x = 5u32; // let's paper over `x` potentially being deemed immutable in the AM.
{
let r: &mut u32 = &uniq x; // Let's get an exclusive-and-thus-mutable ref to `x`, despite the lack of `mut` on it.
let copy: u32 = *r;
// Let's set up a guard which restores/fixes `x` on drop.
let mut guard = mem::DropGuard::new(r, move |r| *r = copy);
let mut owned: &own u32 = own_ref::assume_owns(&mut *guard);
*owned += 2;
} // <- `guard` is dropped, and restores `*r = copy`.
assert_eq!(x, 5); // Now the assertion passesAnd now the question: what is the lifetime 'r of that resulting &own u32? And the answer is that it is scoped to this sub-scope, as it needs to be shorter than the life-span of our guard, lest guard and owned incorrectly alias with each other.
And if so, then we might as well simplify all this down to:
let x = 5u32;
{
let mut owned: &own u32 = &own { x }; // compiler to set up a "long"-lived copy-of-`x` temporary.
*owned += 2;
}
assert_eq!(x, 5); // The assertion passes as `x` itself was not the borrowee.So I'd rather we did this, except it does mean that, either way, the lifetime of the resulting &own would not be as big as if x had not been Copy…
There was a problem hiding this comment.
Hm true, you end up being limited to the scope anyway... So this is a non-option. I kinda expected as much, but couldn't find a good example. Thanks for that, I'll amend this!
Once again, this shows to me that we should just move out of places even if their type is Copy. If we want the behavior of borrow-without-modify, then borrowing a temporary copy like in your example is the best approach IMO.
There was a problem hiding this comment.
I've decided to remove the "temporary copy" version, sonce it wasn't really thought through and doesn't provide anything really.
Instead the question is now just "mutate or move out", and I believe "move out" is the correct answer here, especially with your MaybeUninit example as well (which I should add).
| However, implementing such a trait is difficult, since it is unclear when the "shell" of a type (e.g., the allocation of `Box`) will be freed. | ||
| Therefore, the design of such a trait is left for a future RFC, especially since it may involve additional missing Rust features (e.g., self-referential types); |
There was a problem hiding this comment.
As of now, this a bit too underdeveloped, so people will read this and may not be convinced or whatnot, and may want to relitigate this. I do completely understand the worry about excessive info dump here, so I would suggest that at least be added a link to the Zulip thread where we delved into this question, for those curious to look more into it, or even wanting to chime in.
There was a problem hiding this comment.
I linked to the zulip thread, though it is much more rambly than I remember. Is this sufficient?
|
|
||
| The "Immobile types and guaranteed destructors" project goal aims to introduce the `Move` and `Forget` traits in order to supersede pinning. | ||
|
|
||
| A type `T: !Move` cannot be moved by value. |
There was a problem hiding this comment.
This is a good example of the &move oxymoron I was hinting at: Rust should very much not let &move (impl !Move) be a thing, when it could instead choose for it to be spelled out as &own (impl !Move)
There was a problem hiding this comment.
You could look at it the other way though, that &move is always movable, regardless of the inner type. I'm not disagreeing with you on own vs move, but the response to "Help, I cannot move my type" could be "use &move to make it movable".
|
|
||
| A type `T: !Forget` must be dropped before its backing allocation may be reused. | ||
| This trivially makes `&own T: !Forget`, since forgetting the reference is equivalent to forgetting the value. | ||
| However, we would additionally require that `&own T: !Leak`, since leaking the reference would allow reusing the backing allocation without ever dropping `T`. |
There was a problem hiding this comment.
Hmm, there is a question of lifetimes, here. Given T : !Forget, and calling 't "the (intersection) lifetime of T", i.e.,
- the maximum
'uso thatT : 'uholds, - the intersection of
'a, 'b, …, 'nwhenTis ultimatelySomeName<'a, 'b, …, 'n>.
Then, &'t own T could very much be Leak, at least (except when "T would be contravariant over 't i.e., over each of 'a, 'b, …, 'n infecting it 🤔).
Ok, to simplify a bit. Given T : !Forget + 'static, &'static own T could be Leaked.
So rather than a blanket &'r own T : !Leak restriction, it should rather be : Leak iff "'r : 'big_enough".
In the "&'r own T does not entail T : 'r" universe, we could state 'big_enough = 'static without loss of generarily. But with T : 'r, and the mentioned contravariance, it's not so clear cut… cc @ais523 WDYT?
There was a problem hiding this comment.
The wording I came up with when discussing this on Zulip (#t-lang/move-trait > Leak, Forget, and &own) was that:
- A
T: !Forgetmust be deconstructed before the backing allocation is reused. - A
T: !Leakmust be deconstructed before any lifetime inTends.
A &'a own T where T: !Forget means that we must deconstruct T, and by extent &own T, before we reclaim the backing allocation of T. We know the backing allocation of T may be reclaimed after 'a expires, which mean we must deconstruct &'a own T before that. which actually just means it is !Leak.
But yes, since 'static never expires, a &'static own T is Leak regardless of T: !Forget. I think of this as "you may leak T: !Leak + 'static types". If T: !Leak, we still have &own T: !Leak (except if everything is 'static.
There was a problem hiding this comment.
I love to see &own refs getting love and proper consideration! Thanks for this RFC, @CheaterCodes, and all the time you've poured into it! I appreciate how extensive / much coverage of edge cases it already has 🙏
I know we should keep comments/noise low for these high-activity issues, but I also think it's a bit sad for only the remarks/nits/pieces of criticism to be the ones allowed to be expressed; it results in too negative a feedback experience. So I hope people won't mind the "noise" of some positive feedback
Co-authored-by: Daniel Henry-Mantilla <daniel.henry.mantilla@gmail.com>
| ]); | ||
| ``` | ||
|
|
||
| Due to thier owning nature, in some cases a `&own T` behaves more like a `Box<T>` than a `&mut T`. |
There was a problem hiding this comment.
This contains a typo (should be "to their").
| Analog to other reference types, we could introduce a new binding mode `own`. | ||
| As an example, this enables an ergonomic way to recursively consume a slice: |
There was a problem hiding this comment.
If this was allowed, it would impose interesting requirements on &own T leaving behind bit patterns that are valid for T, or else such pattern matching is unsound.
The RFC should take an explicit stance on whether code like this is sound:
fn make_bad(x: &own bool) {
unsafe { (x as *mut bool).cast::<u8>().write(2) };
}I would expect such code to be sound, since I would expect &own to be basically an &mut that transfers full ownership of the T into the function -- that's what the RFC says. Logically, this means that make_bad has to give the right to access *x back to the caller, but it does not have to promise the caller anything about the contents of *x -- ownership of the content, including basic validity facts, are fully given up by the caller.
There was a problem hiding this comment.
Hm... thinking about it some more, this seems related to rust-lang/unsafe-code-guidelines#394. What actually is the postcondition of drop glue? In that issue I argued that it should leave behind a bit-valid value, and arguably that would imply that make_bad should also leave behind a bit-valid value.
There was a problem hiding this comment.
If this was allowed, it would impose interesting requirements on &own T leaving behind bit patterns that are valid for T, or else such pattern matching is unsound.
Can you maybe clarify on this? I think pattern matching is a very important feature (even though I left it as a future possibility). If this somehow conflicts with make_bad, I think it would be a strong argument, but I don't currently see it.
As far as I understand, slice patterns shouldn't be a problem, and even with enums I don't know if they can even observe the modification? If you can maybe think of an example, I would be very interested!
There was a problem hiding this comment.
arguably that would imply that make_bad should also leave behind a bit-valid value.
From rust-lang/rust#160436:
Boxand&ownshould have in common that it's fine to put arbitrary garbage data into them before doing a shallow drop.
If we just say dropping can do anything to the backing allocation, that would probably make our lives easier. It means you can't soundly read something that was stored in niches of a dropped value such as an enum discriminant, but it would mean (a) mostly unified semantics for &mut, &own, and Box in the sense that they all "conceptually" own their entire backing allocation; (b) we can have both take and replace_with_own soundly and safely; (c) opens up some idea of box/&own-projection where e.g. a Box with a noop-deallocator can point to arbitrary data, even locals. All that we really lose I think is the ability to read an enum discriminant after the value contained in said enum was already dropped, and even then only if the type of the value contained niches.
There was a problem hiding this comment.
All that we really lose I think is the ability to read an enum discriminant after the value contained in said enum was already dropped, and even then only if the type of the value contained niches.
Is this something Rust does? Does optimization benefit from it? Or is it something people do unsafely? Do we think we might need this capability some time in the future?
I do agree that this issue seems central to Box/own semantics, and maybe something that needs to be figured out sooner rather than later (since it does not only affect &own).
There was a problem hiding this comment.
I think it makes sense that if you have
let mut v: Option<NonZeroU32>and you somehow got a &own NonZeroU32 from that, then you can't afterwards call v.is_none() anymore because v is in a half-dropped state.
But maybe already getting the &own shouldn't be allowed?
There was a problem hiding this comment.
It seems like currently Rust does indeed occasionally read the discriminant after moving out of a place, see rust-lang/rust#91029.
Though it seems like this is mostly considered a compiler bug?
There was a problem hiding this comment.
If this was allowed, it would impose interesting requirements on
&own Tleaving behind bit patterns that are valid forT, or else such pattern matching is unsound.The RFC should take an explicit stance on whether code like this is sound:
fn make_bad(x: &own bool) { unsafe { (x as *mut bool).cast::<u8>().write(2) }; }I would expect such code to be sound, since I would expect
&ownto be basically an&mutthat transfers full ownership of theTinto the function -- that's what the RFC says. Logically, this means thatmake_badhas to give the right to access*xback to the caller, but it does not have to promise the caller anything about the contents of*x-- ownership of the content, including basic validity facts, are fully given up by the caller.
I'd agree. Being able to get an &own T to a value is equivalent to being able to call drop_in_place on the value, which is unsafe for a reason. If you pass off an &own T, you have to expect that the callee may invalidate the memory state.
| pub trait Future { | ||
| type Output; | ||
|
|
||
| // Required method | ||
| fn poll(self: &own Self, cx: &mut Context<'_>) -> Poll<'_, Self>; | ||
| } |
There was a problem hiding this comment.
This doesn't work?
You said this yourself earlier, but &own T is not compatible with pinning.
There was a problem hiding this comment.
Because this does not rely on pinning, but instead on the Forget and Leak auto-traits propagating through the type system. As a result, the &own Self is effectively pinned, since the type system prevents it from being leaked. This is definitely half-baked future talk though.
There was a problem hiding this comment.
But once you add those traits to the language, Future can be simplified to &mut self with Self: !Move + !Forget. We would not need &own.
There was a problem hiding this comment.
The point of &own here is to allow the future to consume itself when it finishes, avoiding the need to add a runtime marker. Similar to the Iterator shown in the motivation section.
| pub enum Poll<'a, F: Future> { | ||
| Ready(F::Output), | ||
| Pending(&'a own F), | ||
| } | ||
|
|
||
| pub trait Future { | ||
| type Output; | ||
|
|
||
| // Required method | ||
| fn poll(self: &own Self, cx: &mut Context<'_>) -> Poll<'_, Self>; | ||
| } |
There was a problem hiding this comment.
Nothing here enforces that this returns an &own to the same future as the self argument. The caller would have to check that the future did not swap it out for some other future.
There was a problem hiding this comment.
Does this matter to the caller though? If the result of poll is that another future should be continued to get polled, I don't see this as a problem.
There was a problem hiding this comment.
Currently Tokio just stores the future by value in its task struct, and takes a new reference to it on each poll. Tokio would need to be changed to store a pointer to the future next to the future so that the pointer can be updated to an unrelated future in that case. Seems pretty weird to me.
There was a problem hiding this comment.
The same would apply to every level of .await in an async fn, since at every level the future could be swapped out for another one. You'll need to store a lot of pointers if the callstack is deep.
There was a problem hiding this comment.
Hm... I do wonder if there's something related to in-place-init with out pointers here? With out pointers, we would go from &uninit to some init proof &init while statically tracking that they refer to the same allocation. Something like futures could do the same, consuming a &own but (conditionally) returning an &init init proof for the same allocation, indicating that it is actually not consumed.
I'm not necessarily saying that this is what futures should look like, but an example for an API that could be done using owning pointers, but isn't possible today.
There was a problem hiding this comment.
You would need init proofs to go that route yes. But if that's the example you want to give, then I'd like to ask this question:
Are you proposing to add &own T because it is useful as its own stand-alone feature, or are you proposing to add it as a step on the way to in-place init?
I ask because I am deeply sceptical that init proofs are the right way to solve in-place init. See e.g. #t-lang/in-place-init > Teaching
There was a problem hiding this comment.
I believe that in-place-init via out pointers is good, but I personally think &own should not be used for that (and I believe I have stated this in the RFC too).
My example was purely about solving the issue of fusing iterators/closures, without regard for existing ecosystem changes. Merely as a "look what we could do", absolutely not as a "should do".
There was a problem hiding this comment.
A few days before this RFC was posted, I was working on a more general type of reference, which included as special cases both &mut, and an &own variant that forgot the targeted value rather than dropping it when the &own was dropped. I think this Future example can't be written with &own because it doesn't pin correctly, but works better with the generalization.
The basic idea was for this general reference type to be Ref<'a, T, U>, representing a reference that for the entire lifetime 'a points to a U, and is currently pointing to a T (for some types T and U for which U: 'a and T can safely transmute to U; there is no requirement that T: 'a). In this model, &'a mut T is Ref<'a, T, T>, and &'a own T is a DropGuarded Ref<'a, T, MaybeUninit<U>> where U is a placeholder type with the same size and alignment as T (but 'static lifetime). Unlike &own, Ref is safely pinnable as long as its two type parameters have the same destructor.
In order to make this sort of "type-shifting Future" work, you could represent futures as an enum (which they already are, pretty much), with the bounding type U being the entire enum (so that it can be dropped properly) and the current type T being a pattern type on U restricted to only the variants that could be polled (i.e. not the panicked or complete variant). You would also want some way to prove that the output reference pointed to the same place as the input pointer (which the "initialization proof" variant of in-place-init also wants).
I don't think it makes sense to propose this more general reference as part of this RFC (as the motivation is fairly light compared to &own which has a lot of uses, and &own running the destructor on drop obviously makes more sense than forgetting a value it could have moved). However, some examples that appear to need &own but don't quite work might work in terms of the more general reference type instead, and it makes sense to do that rather than trying to force them into &own. (And it's good to have a concrete example of something which can be written using the general type but can't be written with &own.)
View all comments
Introduce owning references
&owninto the language, which allows passing ownership of the pointee without moving its value.Owning references can be moved out of (including partial moves), and drop their pointee when dropped.
Important
Since RFCs involve many conversations at once that can be difficult to follow, please use review comment threads on the text changes instead of direct comments on the RFC.
If you don't have a particular section of the RFC to comment on, you can click on the "Comment on this file" button on the top-right corner of the diff, to the right of the "Viewed" checkbox. This will create a separate thread even if others have commented on the file too.
Rendered