qDup helloworld not working (deadlocks)

[zakkak]

I tried following https://github.com/Hyperfoil/qDup/blob/master/docs/tutorial/helloworld.adoc but once I start qDup it prints the following and then gets stuck

> java -jar ~/code/qDup/cli/target/qDup.jar -B . helloworld.yaml
16:48:41.218 Running qDup version 0.10.5-SNAPSHOT @ unknown
16:48:41.218 output path = .
16:48:41.218 shell exit code checks enabled
16:48:41.251 json server listening at blanc:31337

It looks like qDup is getting stuck at

qDup/qDup/src/main/java/io/hyperfoil/tools/qdup/Run.java

Line 524 in 11c8c61

runLatch.await();

while nothing else seems to be happening.

[zakkak]

Similar to #178, switching from zsh to bash resolves the issue.

[zakkak]

There seems to be a race between io.hyperfoil.tools.qdup.stream.SuffixStream#write(byte[], int, int) and io.hyperfoil.tools.qdup.stream.SuffixStream.FoundRunnable#run

write may update writeIndex while (or just before) run is reading it.

The reason run reads writeIndex seems to be to avoid running if a subsequent write has happened. I am not sure why that matters, but the way it's checked is racy so it seems to beat the purpose anyway.

[franz1981]

Hi @zakkak did you keep on investigating on this, are you blocked?

[willr3]

There seems to be a few places to improve qDup here.

1. The SuffixStream.write updates SuffixStream.writeIndex which is used by SuffixStream.FoundRunnable.run.
   The FoundRunnable was added to prevent qDup from inocorrectly detecting the prompt (and thus assuming the previous command finished) whenever the prompt is part of the output of the sh command. For example, sh: echo $PS1 These variables probably need to be volatile and use atomic reference updates to ensure we do not have race conditions for the theads over SuffixStream.writeIndex and SuffixStream.future

2. Your branch with the change to filter out null bytes is brilliant. I've not seen null bytes in the output buffer from a terminal. Was the remote session connecting to a server, primary workstation (laptop), or using LOCAL connection?

[zakkak]

Hi @zakkak did you keep on investigating on this, are you blocked?

I did some more investigation but gave up as I couldn't understand what's generating all the special non-visible bytes in zsh.

The SuffixStream.write updates SuffixStream.writeIndex which is used by SuffixStream.FoundRunnable.run. The FoundRunnable was added to prevent qDup from incorrectly detecting the prompt (and thus assuming the previous command finished) whenever the prompt is part of the output of the sh command. For example, sh: echo $PS1. These variables probably need to be volatile and use atomic reference updates to ensure we do not have race conditions for the theads over SuffixStream.writeIndex and SuffixStream.future.

👍 it will also need some handling of failed atomic updates.

Your branch with the change to filter out null bytes is brilliant. I've not seen null bytes in the output buffer from a terminal. Was the remote session connecting to a server, primary workstation (laptop), or using LOCAL connection?

LOCAL seems to work fine.

user@localhost seems to trigger the issue (when using zsh).

Note that the prompt changes when you ssh in a machine (it adds an globe icon and the hostname).

If you want me to try anything or provide more info please let me know.

[willr3]

I plan to setup zsh in a vm next week (this week is booked) to try and recreate. Can you share what you use for your prompt in zsh?

[zakkak]

Thank you @willr3, although not a high priority it's certainly welcome.

I use the default prompt of https://starship.rs

Reminder: The issue only pops up when connecting through ssh. When using LOCAL it works as expected.