Doesn't appear to trace into multiprocessing.Process()

[rossburton]

I've wrapped a block of code that basically does:

class Parser(multiprocessing.Process):
    ...

for i in range(0, self.num_processes):
    parser = Parser()
    parser.start()

The resulting trace shows multiprocessing fork but doesn't show anything from the children.

[gaogaotiantian]

Did you use the command line interface?

[rossburton]

No, the code is buried inside another fork that viztracer doesn't follow yet as its done manually.

But I might have been doing it wrong... I'll finish debugging tomorrow.

[gaogaotiantian]

viztracer follows raw os.fork automatically. If you could put together a minimal example I can take a look. The key thing is that you need to start the tracing from command line (viztracer xxx). The inline with VizTracer(): won't pick up multi-process.

[rossburton]

Aaah, ok. Is there a way to get it to follow multiprocess explicitly or does it have to be done by the command line monkeypatching early?

[gaogaotiantian]

For now all the heavy lifting is done at the monkey patch from cmdline command. I’ve explored the possibility of doing it automatically but have not gotten anything work yet.

[rossburton]

I've got a non-minimal reproducer... Clone https://git.yoctoproject.org/poky/ then:

$ cd poky
$ . oe-init-build-env
$ viztracer bitbake -p

The output shows the main process doing some work and then forking off a worker, which has a UI thread and a worker thread. This worker uses multiprocessing.Process to parse the files in a process pool, but the viztracer output only shows the worker waiting for the parsers to finish and doesn't show the actual parsers.

I admit that's incredibly non-minimal but maybe it's obvious?