Hand Drill caused crash :(

[SPOOKEXE]

• [Y] Verify that this issue is related to Torch and not a Torch plugin or the vanilla game
• [?] Ensure that the issue is reproducible for testing (provide a link to a test world if necessary)
• [N] Is this a suggestion?

Torch Version: 1.3.1.304-master SE Version: 1.205.26

Expected Behavior

No crashing

Observed Behavior

TORCH and SERVER crashed together.

Steps to Reproduce

The issue got triggered on small voxel terrain pieces when you try hand drill them. I'm assuming it was a desync issue where I had aimed over voxel terrain to mine it (using right click), and after I had mine it the server received another event to mine it but it no longer existed. My latency was 290-310ms (AUS -> US) on a private hosted server.

Other Information

   at System.MulticastDelegate.ThrowNullThisInDelegateToInstance()
   at Sandbox.Game.Weapons.MyHandDrill.<>c.<IsDrillingAnObjectChanged>b__158_0(MyCharacter x)
   at VRage.Network.MyEventTable.TryGet[T](Object associatedObject, Func`2 getter, T arg, CallSite& site)
   at VRage.Network.MyReplicationLayerBase.TryGetStaticCallSite[T](Func`2 callSiteGetter, CallSite& site)
   at VRage.Network.MyReplicationLayerBase.GetCallSite[T](Func`2 callSiteGetter, T arg)
   at VRage.Network.MyReplicationLayerBase.RaiseEvent[T1,T2,T3](T1 arg1, T3 arg3, Func`2 action, T2 arg2, EndpointId endpointId, Nullable`1 position)
   at Sandbox.Engine.Multiplayer.MyMultiplayer.RaiseEvent[T1,T2](T1 arg1, Func`2 action, T2 arg2, EndpointId targetEndpoint)
   at Sandbox.Game.Weapons.MyHandDrill.IsDrillingAnObjectChanged(Boolean value)
   at Sandbox.Game.Weapons.MyHandDrill.DrillingPerformedCallback(Boolean success)
   at Sandbox.Game.Weapons.MyDrillBase.OnDrillResults(Dictionary`2 materials, Vector3D hitPosition, Boolean collectOre, Action`1 OnDrillingPerformed)
   at Sandbox.Game.Weapons.MyDrillBase.<>c__DisplayClass109_0.<TryDrillVoxels>b__0(Single x, MyVoxelMaterialDefinition y, Dictionary`2 z)
   at Sandbox.Game.Entities.MyVoxelBase.<>c__DisplayClass137_0.<CutOutShapeWithPropertiesAsync>b__1()
   at ParallelTasks.Parallel.RunCallbacks()
   at Sandbox.Game.World.MySession.Update(MyTimeSpan updateTime)
   at Sandbox.MySandboxGame.Update()
   at Sandbox.Engine.Platform.Game.UpdateInternal()
   at Sandbox.Engine.Platform.Game.RunSingleFrame()
   at Sandbox.Engine.Platform.FixedLoop.<>c__DisplayClass11_0.<Run>b__0()
   at Sandbox.Engine.Platform.GenericLoop.Run(VoidAction tickCallback)
   at Sandbox.Engine.Platform.Game.RunLoop()
   at Sandbox.MySandboxGame.Run(Boolean customRenderLoop, Action disposeSplashScreen)
   at Torch.VRageGame.DoStart() in C:\ProgramData\Jenkins\.jenkins\workspace\Torch_master\Torch\VRageGame.cs:line 326
   at Torch.VRageGame.Run() in C:\ProgramData\Jenkins\.jenkins\workspace\Torch_master\Torch\VRageGame.cs:line 126
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
System.ArgumentException: Delegate to an instance method cannot have null 'this'.
   at System.MulticastDelegate.ThrowNullThisInDelegateToInstance()
   at Sandbox.Game.Weapons.MyHandDrill.<>c.<IsDrillingAnObjectChanged>b__158_0(MyCharacter x)
   at VRage.Network.MyEventTable.TryGet[T](Object associatedObject, Func`2 getter, T arg, CallSite& site)
   at VRage.Network.MyReplicationLayerBase.TryGetStaticCallSite[T](Func`2 callSiteGetter, CallSite& site)
   at VRage.Network.MyReplicationLayerBase.GetCallSite[T](Func`2 callSiteGetter, T arg)
   at VRage.Network.MyReplicationLayerBase.RaiseEvent[T1,T2,T3](T1 arg1, T3 arg3, Func`2 action, T2 arg2, EndpointId endpointId, Nullable`1 position)
   at Sandbox.Engine.Multiplayer.MyMultiplayer.RaiseEvent[T1,T2](T1 arg1, Func`2 action, T2 arg2, EndpointId targetEndpoint)
   at Sandbox.Game.Weapons.MyHandDrill.IsDrillingAnObjectChanged(Boolean value)
   at Sandbox.Game.Weapons.MyHandDrill.DrillingPerformedCallback(Boolean success)
   at Sandbox.Game.Weapons.MyDrillBase.OnDrillResults(Dictionary`2 materials, Vector3D hitPosition, Boolean collectOre, Action`1 OnDrillingPerformed)
   at Sandbox.Game.Weapons.MyDrillBase.<>c__DisplayClass109_0.<TryDrillVoxels>b__0(Single x, MyVoxelMaterialDefinition y, Dictionary`2 z)
   at Sandbox.Game.Entities.MyVoxelBase.<>c__DisplayClass137_0.<CutOutShapeWithPropertiesAsync>b__1()
   at ParallelTasks.Parallel.RunCallbacks()
   at Sandbox.Game.World.MySession.Update(MyTimeSpan updateTime)
   at Sandbox.MySandboxGame.Update()
   at Sandbox.Engine.Platform.Game.UpdateInternal()
   at Sandbox.Engine.Platform.Game.RunSingleFrame()
   at Sandbox.Engine.Platform.FixedLoop.<>c__DisplayClass11_0.<Run>b__0()
   at Sandbox.Engine.Platform.GenericLoop.Run(VoidAction tickCallback)
   at Sandbox.Engine.Platform.Game.RunLoop()
   at Sandbox.MySandboxGame.Run(Boolean customRenderLoop, Action disposeSplashScreen)
   at Torch.VRageGame.DoStart() in C:\ProgramData\Jenkins\.jenkins\workspace\Torch_master\Torch\VRageGame.cs:line 326
   at Torch.VRageGame.Run() in C:\ProgramData\Jenkins\.jenkins\workspace\Torch_master\Torch\VRageGame.cs:line 126
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()```

[SPOOKEXE]

[SPOOKEXE]

message.txt

Also here is another other issue where we ran out of memory through another memory leak - we were unable to get any other logs as torch completely crashed

[Sardaukai]

We also got that Sandbox.Game.Weapons.MyHandDrill.<>c.b__158_0(MyCharacter x) from Feb '24 till May '24. There is also an SE bug entry, but wont't fix due to mod or plugin usage... (https://support.keenswh.com/spaceengineers/pc/topic/41569)

As far as we investigate it only appread at our modded start planet (Terra Remake) and only if people use their hand drills.
Block drills are not effected.
Which mods/plugins do you use? Does it occur on asteroids too?

Maybe it helps to rise "Drill Update Rate (ticks)" (with DePatch or another plugin). Since we use 40 we haven't any crashs fingerscrossed

For memory leak it could help to use Performance Improvements (Plugin via Torch)

Both are not Torch related bugs imo...

[SPOOKEXE]

Here is the mods list: https://steamcommunity.com/sharedfiles/filedetails/?id=3365390013

We ended up moving to the default space engineers server executables and had 0 problems with it.

[SPOOKEXE]

Compare to your mods list and see which are the common

[Sardaukai]

There are two mods we also use which are voxel changing. Smooth Voxel and Seamless.

We deactivated both and error occurred again.

If you switched to regular dedicated server and the error is gone it could be an indication it is a torch error.

It would be interesting to test it with torch and drill tick rate 40 again...

[SPOOKEXE]

Yep, we told them about it and they were saying it wasn't them but the mods 🤷

[MLenterman]

Im also experiencing the same issue. Are you by any chance using the Floating Objects Torch plugin with auto ore pickup enabled? I have that one on my list of suspects due to modifying some things surrounding MyHandDrill (https://github.com/PveTeam/TorchPlugins/blob/master/FloatingObjects/Patches/HandDrillPatch.cs). Currently testing if removing this plugin fixes the issue.

Can't really pinpoint what exactly triggers the crashes. Seems to happen often just before or just after a world save. Also a few times when the server is under heavy load due to combat or something.

[SPOOKEXE]

I don't believe we had that plugin, as from what I can recall, we always had to manually pick up ore so I don't think we had it. We no longer have the files either as we did a complete reinstall of mods on the normal server instead of torch with instant success.

[Sardaukai]

We also used the pickup plugin, but our server is wiped. We started a new one without issues.
Auto Ore pickup is used again without any problems.