Feature Request: Support Perfetto Protobuf Input

[aradzu10]

Title

Feature Request: Support Perfetto Protobuf Input

Body

Please integrate support for Perfetto Protobuf input. This offers two key advantages:

1. Eliminates RAM Limitations: Protobuf allows streaming data directly to disk as the program runs (e.g., flushing every 1MB). This avoids the memory overhead of storing the entire trace in RAM before writing.
2. Compact File Size: The binary format is significantly smaller than textual output.

[gaogaotiantian]

Steaming is not a feature of protobuf itself. protobuf is a nice format for streaming, that's true. The reason why viztracer chose a pure RAM solution is to minimize run-time overhead. If you write to disk while running your program, you will see a big gap at a random place and it's difficult to tell if it's caused by periodic disk write.

Protobuf is much smaller than json, that's correct. It's also much harder to test and maintain. You'll need extra dependencies. All the test framework needs to be redone.

I'm okay with protobuf as an optional output format as long as viztracer can still run without any 3rd party dependencies, but I don't yet have time to do that.

Protobuf is the first citizen to Perfetto because well .. it's google. But they really made effort to support JSON better just because too many people are using it - it's just much easier to use.

[aradzu10]

I agree with you, but I think the RAM and file size limitation, for production or large scale use are important.

I would implement it as follow:

1. Use a circular buffer with configurable size (as for now).
2. When it gets full, write it to disk - with a "trace event" of "__internal_writing_to_disk".

This should not really impact the code run time:

1. If it happens every 1MB, you will almost don't "feel" the writing time - due to balance between IO and size.
2. This time will be, when combine, much faster that the end time we have now.

I really like this because:

1. There is no RAM limitation, and much Compact file size - which makes everything faster.
2. You can view the trace mid progress, which is really strong.

If you agree with my idea, I might try to implement it.
But, lets talk first :)