`juju nuke` promises nuclear bomb but delivers firecracker

[astrojuanlu]

Bug Description

To Reproduce

1. Have some naughty models
2. Run `jhack nuke
3. (No profit)

Environment

ubuntu@data-platform-dev2:~$ jhack jenv
                                 juju info v0.1                                 
┌──────────────┬───────────────────────────────────────────────────────────────┐
│ jhack        │ 0.4.4.0.24                                                    │
│ python       │ 3.11.14 (/home/ubuntu/.local/share/uv/tools/jhack/bin/python) │
│ juju-* snaps │  juju │ 3.6.12 - 33252 (3.6/stable)                           │
│ microk8s     │ MicroK8s v1.32.9 revision 8476                                │
│ lxd          │ 5.21.4 LTS                                                    │
│ multipass    │ Not Installed.                                                │
│ multipassd   │ Not Installed.                                                │
│ os           │ Ubuntu 24.04.3 LTS                                            │
│ kernel       │ Linux 6.8.0-88-generic x86_64                                 │
└──────────────┴───────────────────────────────────────────────────────────────┘

Relevant log output

ubuntu@data-platform-dev2:~$ jhack --loglevel=DEBUG nuke jujubilant
::= Verbose mode (DEBUG). =::
INFO:jhack.nuke:starting jhack nuke
DEBUG:jhack.nuke:Gathering nukeables for 'jujubilant' with _selectors = 'ram'
INFO:jhack.nuke:gathering apps and relations (ram)
INFO:jhack.nuke:gathering apps and relations
ERROR:jhack:ERROR Unhandled Error: error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:34819->127.0.0.1:34724: write tcp4 127.0.0.1:34819->127.0.0.1:34724: write: broken pipe%!(EXTRA string=logger, string=UnhandledError)
ERROR Unhandled Error: error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:36039->127.0.0.1:39804: write tcp4 127.0.0.1:36039->127.0.0.1:39804: write: broken pipe%!(EXTRA string=logger, string=UnhandledError)
ERROR model jujuc-uk8s-40:jujubilant3 not found
ERROR:jhack:juju status -m jujubilant3 --relations produced no output.
ERROR:jhack:This usually means that the model 'jujubilant3' does not exist
ERROR:jhack.nuke:nuke attempted to get the status of jujubilant3 but the model is probably dead already.
INFO:jhack.nuke:gathering models
INFO:jhack.nuke:collected jujubilant0-40 for nukage
INFO:jhack.nuke:collected jujubilant0 for nukage
INFO:jhack.nuke:collected jujubilant1 for nukage
INFO:jhack.nuke:collected jujubilant2 for nukage
DEBUG:jhack.nuke:Gathered: [model 'jujubilant0-40', model 'jujubilant0', model 'jujubilant1', model 'jujubilant2']
INFO:jhack.nuke:collecting for nukage: model 'jujubilant0-40'
INFO:jhack.nuke:collecting for nukage: model 'jujubilant0'
INFO:jhack.nuke:collecting for nukage: model 'jujubilant1'
INFO:jhack.nuke:collecting for nukage: model 'jujubilant2'
Are you sure you want to nuke:
        ⚛ model 'jujubilant0-40'
        ⚛ model 'jujubilant0'
        ⚛ model 'jujubilant1'
        ⚛ model 'jujubilant2'
Please confirm [Y/n]: 
                                                                                                                                       
                                                                    ____                                                               
                                                            __,-~~/~    `---.                                                          
                                                          _/_,---(      ,    )                                                         
                                                      __ /        <    /   )  \___                                                     
                                       - ------===;;;'====------------------===;;;===----- -  -                                        
                                                         \/  ~"~"~"~"~"~\~"~)~"/                                                       
                                                         (_ (   \  (     >    \)                                                       
                                                          \_( _ <         >_>'                                                         
                                                             ~ `-i' ::>|--"                                                            
                                                                 I;|.|.|                                                               
                                                                <|i::|i|`.                                                             
                                                               (` ^'"`-' ")                                                            
                                                                                                                                       
DEBUG:jhack.nuke:firing juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant0-40 ~]==⚛❯ model 'jujubilant0-40'
DEBUG:jhack.nuke:firing juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant0 ~]==⚛❯ model 'jujubilant0'
DEBUG:jhack.nuke:firing juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant1 ~]==⚛❯ model 'jujubilant1'
DEBUG:jhack.nuke:firing juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant2 ~]==⚛❯ model 'jujubilant2'
                                                       ~]==⚛❯  jujubilant0-40  ⚛                                                       
DEBUG:jhack.nuke:nuking model 'jujubilant0-40' with juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant0-40
                                                        ~]==⚛❯  jujubilant2  ⚛                                                         
DEBUG:jhack.nuke:nuking model 'jujubilant2' with juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant2
                                                        ~]==⚛❯  jujubilant0  ⚛                                                         
DEBUG:jhack.nuke:nuking model 'jujubilant0' with juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant0
                                                        ~]==⚛❯  jujubilant1  ⚛                                                         
DEBUG:jhack.nuke:nuking model 'jujubilant1' with juju destroy-model --force --no-wait --destroy-storage --no-prompt jujubilant1
                                             ~]==⚛❯ model 'jujubilant0-40' still in flight                                             
                                              ~]==⚛❯ model 'jujubilant0' still in flight                                               
                                              ~]==⚛❯ model 'jujubilant1' still in flight                                               
                                              ~]==⚛❯ model 'jujubilant2' still in flight                                               
                                                                ✞ RIP ✞

Additional context

Looks like the problem is in juju destroy-model --force --no-wait --destroy-storage --no-prompt <model-name> not delivering. But I somehow expected juju nuke to do more behind the scenes to actually nuke the model.

[PietroPasotti]

If you know of an even more nuky way to go about it, let me know.
I wouldn't go as far as deleting the lxd containers/vms or the kubernetes model without even letting juju know.

[PietroPasotti]

I've had cases in the past where models would hang in destroying state. I don't see a solution to that short of doing juju db surgery to remove knowledge of the models and their applications, but that is a bit too... much.

[astrojuanlu]

I've had cases in the past where models would hang in destroying state.

Yeah, it happens to me all the time...

I wouldn't go as far as deleting the lxd containers/vms or the kubernetes model without even letting juju know.

I do this often. But then the model never leaves the "destroying" state.

Interestingly there's juju unregister for controllers, but nothing similar for models.

[PietroPasotti]

yeah deleting the resources directly is guaranteed to leave the juju model 'destroying' forever, that's why I wouldn't go that way.
So... what do you propose to do?

[PietroPasotti]

on the one hand the juju team always told me 'if you destroy with force you're on your own', but at the same time if I just want something to die quickly, and it hangs forever in 'destroying'... that might be a bug.

[astrojuanlu]

but at the same time if I just want something to die quickly, and it hangs forever in 'destroying'... that might be a bug.

Yes exactly. To me, saving a few seconds or keystrokes with jhack nuke rather than juju destroy-model ... is not a big deal. It just happens to me too often that destroy-model hangs forever, and I still don't understand why.

Maybe I should focus my attention there and report the issue upstream. In the meantime, I don't really have a better suggestion, for now.

[Gu1nness]

Ok so I faced that a few times in situations where my charm was creating itself some k8s resources.
Eg: creates a service, which does not get deleted when destroying the model. Probably I was not smart on the labelling or constraints to bind that model to the model namespace.

It made me wonder if jhack could maybe try to do something a bit smarter to detect resources that where created on top of what juju creates to delete them ?

[PietroPasotti]

That feels risky, but then again the command is called 'nuke'...
What do you have in mind @Gu1nness ? Can we rely on a managed-by resource label or something like that?