Compile project multiple times with changed flags

[niomate]

Hey there,

I was wondering whether it is possible to recompile the same project multiple times during an experiment.

Basically what I want to do is to compile the project, run an analysis, clean the build directory, then change a compiler flag and do it all over again, possibly multiply times.

How would I go about that if it is possible to do ?

Cheers and thanks in advance

[simbuerg]

Hi,

it is possible and there are multiple ways of achieving it.
Let me outline one roughly for you. First, you need to consider that benchbuild will, by default, use a combination of experiment, project and version information to derive a build directory inside your build prefix.
Second, remember that there is a default order of actions that form the build plan for each combination of experiment+project.
This order is defined by your implementation of Experiment.actions and Experiment.actions_for_project method.
By default you will get the following actions:

See benchbuild.experiment.actions:

• Clean project directory (benchbuild.utils.actions.Clean)
• Make project build directory (benchbuild.utils.actions.MakeBuildDir)
• Setup project build environment (benchbuild.utils.actions.ProjectEnvironment)

See benchbuild.experiment.default_runtime_actions:

• Compile the project (benchbuild.utils.actions.Compile)
• Run all workloads (benchbuild.utils.actions.RunWorkloads)
• Clean project directory (benchbuild.utils.actions.Clean)

Simple experiments, e.g., benchbuild.experiments.raw.RawRuntime, will only define the extensions for the compiler and the runtime environment in addition to compilation flags and use the defaults.
However, you can completely customize how you want to order these steps. In your case, I would provide an implementation of Experiment.actions. For each project I would clone the original project using the project's clone() method. This ensures that you have the same properties but a fresh uuid identifying the project. Then I would set the CFLAGS, LDFLAGS and runtime extension as you like.
Now there are multiple ways to continue. If I want all different compiled binaries to be in isolated build directories,
I would return a list of actions containing the steps as outlined above.

Example:

# In your subclass of benchbuild.experiment.Experiment``:
def actions(self) -> Actions:
    actions: Actions = []

    def new_actions(self, p: Project):
        atomic_actions: Actions = [
            actns.Clean(p),
            actns.MakeBuildDir(p),
            actns.Echo(
                message="Selected {0} with version {1}".
                    format(p.name, version_str)
                ),
                actns.ProjectEnvironment(p),
            ]
            atomic_actions.extend(self.actions_for_project(p))
        return [actns.RequireAll(actions=atomic_actions)]


    for prj_cls in self.projects:
        prj_actions: Actions = []
        for revision in self.sample(prj_cls):
            version_str = str(revision)

            p = prj_cls(revision)

            p_1 = p.clone()
            # Configure the "new" project instance as you wish:
            p_1.cflags = <whatever cflags you need>
            p_1.ldflags = <whatever ldflags you need>
            p_1.runtime_extension = <what runtime extension you need>
            p_1.compiler_extension = <what compiler extension you need>

            prj_actions = new_actions(p_1)
            actions.extend(prj_actions)

            p_2 = p.clone()
            # Configure the "new" project instance as you wish:
            p_2.cflags = <whatever cflags you need>
            p_2.ldflags = <whatever ldflags you need>
            p_2.runtime_extension = <what runtime extension you need>
            p_2.compiler_extension = <what compiler extension you need>

            prj_actions = new_actions(p_2)
            actions.extend(prj_actions)

        return actions

This gives you the following properties:

• Each project instance has a custom build directory that is not equivalent to the one benchbuild uses by default. While benchbuild should not have a problem with that, your custom code might. Just be aware of that.
• Each configuration of a project will start with a clean copy of the source code.
• Each build of a project can run in parallel (if there would be an action for that ;-)).
• Each series of project steps will fail as early as possible. However, subsequent configurations can
  continue to run.
• While debugging, if you specify BB_CLEAN=false, you have each build located in a separate directory (named by it's generated UUID).

[niomate]

Thank you so much for the fast reply, worked like a charm ! :)