Defer blocks and async drop

It seems to be fashionable to drop language design ideas in blog posts at the moment, so I thought I'd join in the fun with a post about defer blocks (don't run off to diss the idea on reddit just yet, there is a reason for this!). The motivation here is an alternative solution to async drop, but there are also some smaller benefits in non-async code. Big disclaimer that this is a very early stage of design and might be a bad idea or have huge problems.

Background

Defer blocks are a language feature in Go for executing code when a function exits (as opposed to where the defer block is defined). They are used to release resources like locks, files, etc. In Rust, we'd use the RAII pattern and destructors (drop) to achieve the same thing. Defer blocks and destructors have pros and cons: destructors let us do the same thing whenever an object is dropped, so you don't need to remember to write a defer block to unlock a mutex. On the other hand, if you want to operate over multiple resources or do some one-off cleanup, then defer blocks let you do so without the boilerplate of writing a struct and Drop impl per function. In other words, destructors are better suited to tidying up individual objects, and defer blocks are better suited to tidying up the state of functions.

An oft-considered extension to Rust destructors is async drop, that is a destructor which is an async function. Whilst this is conceptually simple (and there is plenty of motivation), the details get really nasty (Sabrina Jewson's blog post covers most of the issues in depth). Some of the problems are that await becomes implicit in some places rather than being uniformly explicit, it's unclear which runtime to run async drop on, how it would interact with non-async drop would need to be worked out, it would mean async code executing in many unexpected places (as the result of assignment, for example), the practical implications of running long-running async code during unwinding, and so forth.

My hypothesis is that async drop is not the right abstraction after all. I believe that there is an intuition that drop should be lightweight, relatively quick, and very likely to succeed (which is one reason why the requirement not to panic in drop is not too onerous). Actions which do not fall into this category are better off in an explicit shutdown method or similar. However, anything that you'd want to execute asynchronously is usually not a quick, lightweight action. Therefore, I think most use cases for async drop would be better as explicit async methods (like shutdown). BUT we still want to ensure that they are called under all (or most) circumstances without lots of error-prone code. I think defer blocks are the right solution for this.

Defer blocks in Rust

So, how would it look? Well, the syntax and basic idea are pretty simple: you can write defer { ... } inside function bodies. You can write anything inside the block that you can write in a regular block. A defer block is a statement, not an expression, and the body must have type (). The block is executed when the function exits, after the body of the function completes but before destructors are run (which means you can reference local variables more easily, but more on this later). E.g.,

fn main() {
    defer {
        println!("hello");
    }

    println!("world");
}

When executed, this program prints

world
hello

One key feature of defer blocks is they are only run on normal return (including via ? or macros), but not on panic or abort. E.g.,

fn handle_err<T>(result: Result<T, Err>) -> T {
    defer {
        println!("hello");
    }

    match result {
        Ok(t) -> {
            println!("Not an error");
            t
        }
        Err(_) -> {
            panic!()
        }
    }    
}

If called with an Ok input, this will print

Not an error
hello

If called with an Err input, it will panic without printing anything (other than the usual panic output).

How does this help?

Defer blocks make a distinction between tidying up a specific object (use drop) and tidying up a function (use defer). That means that you would never need to go through the boilerplate of creating an object just to do RAII. Furthermore, they make a distinction between heavyweight cleanup (use defer) and essential cleanup (use drop). So, if we had a web server running as a loop in a function, we might use a defer block to do a graceful shutdown which closes all connections by sending an appropriate response, and a non-graceful shutdown which just releases resources back to the OS and releases any currently held mutexes. This does mean thinking about panics a little differently - recovering from a panic becomes even more of an anti-pattern.

The really nice thing is that async code just works. Drop is never async, there is no such thing as async drop, but when you want that kind of behaviour, you can await within a defer block (if the defer block is in an async function). All the tricky questions with async drop disappear! Note that we don't need async defer blocks, the features are orthogonal and just work together.

There are some downsides: we're adding surface area to the language and we're adding a second way to do something which is already possible. To rebut, this is a very simple addition: it's pretty orthogonal to the rest of the language and isn't complicated to explain. I think it is fairly easy to state when it would be idiomatic to use one syntax or the other. It is also (due to the behaviour on panic) not the same behaviour as RAII, this might be regarded as a good thing or a bad thing.

Details and questions

Of course nothing is ever so simple and there are plenty of details to work out. I think the biggest issues are about the significance of where the block is declared and what local state can be referenced.

One option would be to only allow defer blocks to be declared at the start of functions (perhaps as part of the function header), but this is pretty limiting. In particular, it means there is no way to distinguish between variables with the same name. E.g.,

fn main() {
    defer {
        println!("{:x}");
    }
    let x = 0;
    let x = 42;
}

Which x is printed? What if one were inside the scope of a block?

An alternative is to allow defer blocks anywhere. If the block is inside a scope, should it be executed when the scope ends or when the function returns? If the latter, can it reference variables declared in the scope? Should a defer block be able to refer to variables declared after the block?

I assume that following the usual naming rules around macro hygiene should just work, but I haven't thought about this deeply.

Do we allow multiple defer blocks? (I think we should, for composablity, in particular around macros). What order are they executed in? Presumably reverse order of declaration. Should destructors be interleaved with defer blocks if that is suggested by the order of declaration?

Should variables be moved into defer blocks or passed by reference? I think there is a case for both, since consuming data seems like a reasonable use case for defer blocks, but so does logging (or otherwise examining) data which might be returned or stored (there is no obvious way to pass data out of a defer block). This suggests we might need some rules/analysis like closures or await points (would we need move defer { ... }?). Talking of await, I'm not sure if there are interactions with defer? If a variable is mentioned in a defer block but which would otherwise be destroyed before the end of the function, should that extend the lifetime of the variable or be an error?

Finally, how could defer blocks be implemented? I think they would need to be first class constructs in the compiler. We couldn't desugar to destructors due to the different behaviour around panics and async/await.

Conclusion

Well, that is a lot of open questions, and when I've mentioned this idea before there has been some resistance (I think a combination of the overlap with RAII, an aversion to Go features on principle, and really wanting async drop despite its problems). However, I think it's worth at least considering defer blocks since they work much better with async code.