Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,20 +20,20 @@ Last updated: 2026-09-05
- Replacement source branches, their local worktrees, and the seven superseded
Repo Assist source branches were deleted after merge/closure verification.

## Retained work
## Completed multiplication inlining experiment

- PR 223 (https://github.com/NichUK/FixedPointNano/pull/223) remains
intentionally open as a draft from
`repo-assist/perf-multiply-inline-2026-07-12-d0c46fd9cc0e034f` into `main`.
- Its exact head is `83b673f452ed33e4d0d4653fc6aebb8becf5bc7b`. GitHub reports
it is mergeable but unstable: the bot auto-approval workflow fails because the
GitHub Actions author cannot approve its own PR; this is not a code/test failure.
- It has no Copilot review or activity. Persistent monitoring is unavailable
outside an active task; resume from this record before taking action.
- Do not adopt its multiplication inlining hint until a current `develop`
candidate has controlled baseline-versus-candidate BenchmarkDotNet evidence
for multiplication and representative loops, with numerical equivalence and
generated-code/code-size inspection. The experiment and decision thresholds are
specified in `docs/performance/multiplication-inlining-test-plan.md`. Next action
is to implement that fixture and gather the required evidence or close the draft
deliberately; do not merge its stale `main`-based branch.
- Baseline `c4263974dbab1c3692ee9f01b46fc5176e16b197` and one-line candidate
`48ff2f09673c2c190099d192c7f289060080e62c` passed 1,560 tests and produced
byte-identical deterministic corpus checksums.
- Five ABBA blocks completed all 20 process invocations. Every workload's paired
95% confidence interval crossed parity; none reached the required 8/10 direction
count. The aggregate candidate/baseline ratio was 1.0224 with a 95% interval of
0.9803-1.0640 and the candidate faster in 4/10 pairs.
- The attribute was rejected because it demonstrated no repeatable consumer
benefit. It is absent from the delivery branch. The fixture and concise evidence
are retained under `docs/performance/results/2026-09-05-multiplication-inlining`;
uncommitted raw artifacts were retained outside the repository on the measurement
host.
- PR 223 was closed intentionally with the measured rationale. Its rejected remote
source branch was deleted after verifying that its only useful change was the
tested inlining attribute.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,3 +113,5 @@ are not evidence of a speedup. Rounding rewrites and additional inlining hints
remain deferred until controlled before/after benchmarks demonstrate a benefit.
The acceptance criteria and paired-run protocol for the multiplication hint are
defined in [the multiplication inlining test plan](https://github.com/NichUK/FixedPointNano/blob/main/docs/performance/multiplication-inlining-test-plan.md).
The completed experiment found no repeatable benefit, so the hint was not adopted;
the [full result is retained on GitHub](https://github.com/NichUK/FixedPointNano/blob/main/docs/performance/results/2026-09-05-multiplication-inlining/REPORT.md).
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes;
using Fpn = Seerstone.FixedPointNano;

namespace FixedPointNano.Benchmarks;

[MemoryDiagnoser]
public class FixedPointNanoMultiplyInliningBenchmarks
{
private const int BatchSize = 1024;
private const int PowExponent = 5;
private Fpn[] _amounts = [];
private Fpn[] _dependentFactors = [];
private Fpn[] _left = [];
private Fpn[] _notionalPrices = [];
private Fpn[] _notionalQuantities = [];
private Fpn[] _powValues = [];
private Fpn[] _right = [];
private Fpn[] _squareValues = [];
private Fpn _singleLeft;
private Fpn _singleRight;

[GlobalSetup]
public void Setup()
{
_amounts = new Fpn[BatchSize];
_dependentFactors = new Fpn[BatchSize];
_left = new Fpn[BatchSize];
_notionalPrices = new Fpn[BatchSize];
_notionalQuantities = new Fpn[BatchSize];
_powValues = new Fpn[BatchSize];
_right = new Fpn[BatchSize];
_squareValues = new Fpn[BatchSize];

for (var index = 0; index < BatchSize; index++)
{
var leftSign = (index & 1) == 0 ? 1L : -1L;
var rightSign = index % 3 == 0 ? -1L : 1L;
_left[index] = Fpn.FromRaw(leftSign * (750_000_001L + (index * 1_000_003L)));
_right[index] = Fpn.FromRaw(rightSign * (250_000_003L + (index * 7_919L)));

_dependentFactors[index] = index % 2 == 0
? Fpn.FromRaw(1_000_100_003L)
: Fpn.FromRaw(999_899_997L);

_notionalPrices[index] = Fpn.FromRaw(10_000_000_001L + (index * 10_000_019L));
var quantitySign = index % 5 == 0 ? -1L : 1L;
_notionalQuantities[index] = Fpn.FromRaw(quantitySign * (1_000_000_007L + (index * 100_003L)));

_amounts[index] = Fpn.FromRaw((index % 101) * 10_000_000L);
_squareValues[index] = Fpn.FromRaw(250_000_001L + (index * 10_000_019L));
_powValues[index] = Fpn.FromRaw(900_000_001L + ((index % 201) * 1_000_003L));
}

_left[0] = Fpn.Zero;
_right[0] = Fpn.FromRaw(123_456_789L);
_left[1] = Fpn.FromRaw(1L);
_right[1] = Fpn.FromRaw(499_999_999L);
_left[2] = Fpn.FromRaw(1L);
_right[2] = Fpn.FromRaw(500_000_000L);
_left[3] = Fpn.FromRaw(1L);
_right[3] = Fpn.FromRaw(500_000_001L);
_left[4] = Fpn.FromRaw(3L);
_right[4] = Fpn.FromRaw(500_000_000L);
_left[5] = Fpn.FromRaw(-3L);
_right[5] = Fpn.FromRaw(500_000_000L);
_left[6] = Fpn.FromRaw(2_000_000_000_000L);
_right[6] = Fpn.FromRaw(3_000_000_000L);
_amounts[0] = Fpn.Zero;
_amounts[1] = Fpn.FromRaw(1L);
_amounts[2] = Fpn.FromRaw(499_999_999L);
_amounts[3] = Fpn.FromRaw(500_000_000L);
_amounts[4] = Fpn.FromRaw(500_000_001L);
_amounts[5] = Fpn.FromRaw(999_999_999L);
_amounts[6] = Fpn.One;
_squareValues[0] = Fpn.FromRaw(2_000_000_000_000L);
_squareValues[1] = Fpn.FromRaw(-2_000_000_000_000L);

_singleLeft = _left[17];
_singleRight = _right[17];
}

[Benchmark]
[MethodImpl(MethodImplOptions.NoInlining)]
public Fpn SingleMultiply()
{
return _singleLeft * _singleRight;
}

[Benchmark(OperationsPerInvoke = BatchSize)]
[MethodImpl(MethodImplOptions.NoInlining)]
public long IndependentMultiplyBatch()
{
var checksum = 0L;
for (var index = 0; index < BatchSize; index++)
{
checksum = unchecked(checksum + (_left[index] * _right[index]).RawValue);
}

return checksum;
}

[Benchmark(OperationsPerInvoke = BatchSize)]
[MethodImpl(MethodImplOptions.NoInlining)]
public long DependentMultiplyChain()
{
var current = Fpn.One;
for (var index = 0; index < BatchSize; index++)
{
current *= _dependentFactors[index];
}

return current.RawValue;
}

[Benchmark(OperationsPerInvoke = BatchSize)]
[MethodImpl(MethodImplOptions.NoInlining)]
public long NotionalBatch()
{
var sum = Fpn.Zero;
for (var index = 0; index < BatchSize; index++)
{
sum += _notionalPrices[index] * _notionalQuantities[index];
}

return sum.RawValue;
}

[Benchmark(OperationsPerInvoke = BatchSize)]
[MethodImpl(MethodImplOptions.NoInlining)]
public long LerpBatch()
{
var checksum = 0L;
for (var index = 0; index < BatchSize; index++)
{
checksum = unchecked(checksum + Fpn.Lerp(_left[index], _right[index], _amounts[index]).RawValue);
}

return checksum;
}

[Benchmark(OperationsPerInvoke = BatchSize)]
[MethodImpl(MethodImplOptions.NoInlining)]
public long SquareBatch()
{
var checksum = 0L;
for (var index = 0; index < BatchSize; index++)
{
checksum = unchecked(checksum + Fpn.Square(_squareValues[index]).RawValue);
}

return checksum;
}

[Benchmark(OperationsPerInvoke = BatchSize)]
[MethodImpl(MethodImplOptions.NoInlining)]
public long PowBatch()
{
var checksum = 0L;
for (var index = 0; index < BatchSize; index++)
{
checksum = unchecked(checksum + Fpn.Pow(_powValues[index], PowExponent).RawValue);
}

return checksum;
}

public IReadOnlyList<KeyValuePair<string, long>> CaptureChecksums()
{
IReadOnlyList<KeyValuePair<string, long>> checksums =
[
new(nameof(IndependentMultiplyBatch), IndependentMultiplyBatch()),
new(nameof(DependentMultiplyChain), DependentMultiplyChain()),
new(nameof(NotionalBatch), NotionalBatch()),
new(nameof(LerpBatch), LerpBatch()),
new(nameof(SquareBatch), SquareBatch()),
new(nameof(PowBatch), PowBatch()),
];

foreach (var checksum in checksums)
{
VerifyChecksum(checksum.Key, checksum.Value, GetExpectedChecksum(checksum.Key));
}

return checksums;
}

private static long GetExpectedChecksum(string benchmark)
{
return benchmark switch
{
nameof(IndependentMultiplyBatch) => 6_000_323_985_234L,
nameof(DependentMultiplyChain) => 999_994_879L,
nameof(NotionalBatch) => 9_816_311_662_619L,
nameof(LerpBatch) => 46_212_822_455L,
nameof(SquareBatch) => 8_038_421_873_095_119L,
nameof(PowBatch) => 1_050_735_154_786L,
_ => throw new ArgumentOutOfRangeException(nameof(benchmark), benchmark, "Unknown benchmark."),
};
}

private static void VerifyChecksum(string benchmark, long actual, long expected)
{
if (actual != expected)
{
throw new InvalidOperationException(
$"The {benchmark} corpus checksum was {actual}, but {expected} was expected.");
}
}
}
13 changes: 13 additions & 0 deletions benchmarks/FixedPointNano.Benchmarks/Program.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
using System.Globalization;
using BenchmarkDotNet.Running;

namespace FixedPointNano.Benchmarks;
Expand All @@ -6,6 +7,18 @@ internal static class Program
{
private static void Main(string[] args)
{
if (args.Length == 1 && string.Equals(args[0], "--verify-inlining-corpus", StringComparison.Ordinal))
{
var benchmarks = new FixedPointNanoMultiplyInliningBenchmarks();
benchmarks.Setup();
foreach (var checksum in benchmarks.CaptureChecksums())
{
Console.WriteLine($"{checksum.Key}={checksum.Value.ToString(CultureInfo.InvariantCulture)}");
}

return;
}

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
}
}
9 changes: 9 additions & 0 deletions docs/performance/multiplication-inlining-test-plan.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Multiplication inlining test plan

## Outcome

The 2026-09-05 experiment did not support adding the attribute. Every workload's
paired 95% confidence interval crossed parity, no workload met the required
direction count, and the aggregate candidate-to-baseline point estimate was
`1.0224` (2.24% slower). The stale proposal was closed without adoption. See the
[experiment report](results/2026-09-05-multiplication-inlining/REPORT.md) and its
[machine-readable summary](results/2026-09-05-multiplication-inlining/summary.json).

## Decision

This experiment decides whether to add
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Multiplication inlining experiment

- Baseline: `c4263974dbab1c3692ee9f01b46fc5176e16b197`
- Candidate: `48ff2f09673c2c190099d192c7f289060080e62c`
- Runtime: .NET 10.0.9; SDK 10.0.301; BenchmarkDotNet 0.14.0
- Host: Windows 11 Pro build 26200; Intel Core i7-1265U, 10 cores / 12 logical processors
- Protocol: five ABBA blocks, 20 separate processes, 10 warmups, 15 measurements, 250 ms target iteration

## Recommendation

**Reject the candidate and do not merge the inlining attribute.** The mandatory adoption gates do not pass. The formal timing classification is inconclusive because every 95% paired confidence interval crosses 1.00, while the aggregate point estimate is 2.24% slower and only 4/10 aggregate pairs favor the candidate. A repeat on dedicated lab hardware is the valid next action only if further evidence is desired.

## Results

| Workload | Baseline geometric mean | Candidate geometric mean | Candidate / baseline | Paired bootstrap 95% CI | Candidate faster | Gate |
|---|---:|---:|---:|---:|---:|---|
| IndependentMultiplyBatch | 9.209 ns | 9.779 ns | 1.0619 | 0.9672-1.1750 | 4/10 | Fail |
| DependentMultiplyChain | 5.814 ns | 5.726 ns | 0.9848 | 0.9123-1.0552 | 6/10 | Fail |
| NotionalBatch | 7.955 ns | 8.004 ns | 1.0061 | 0.9153-1.1003 | 5/10 | Fail |
| LerpBatch | 11.025 ns | 11.734 ns | 1.0644 | 0.8919-1.2633 | 5/10 | Fail |
| SquareBatch | 7.376 ns | 7.020 ns | 0.9518 | 0.8217-1.1031 | 6/10 | Fail |
| PowBatch | 42.365 ns | 45.384 ns | 1.0713 | 0.9319-1.2395 | 6/10 | Fail |
| **Aggregate** | - | - | **1.0224** | **0.9803-1.0640** | **4/10** | **Fail** |

All benchmarked workloads allocated 0 B in both variants. Both variants passed all 1,560 tests, the focused 1,356-case math comparison suite, and byte-identical corpus checksum verification.

## Generated code

Under runtime defaults, IndependentMultiplyBatch, NotionalBatch, and PowBatch were instruction-equivalent after address normalization. DependentMultiplyChain, LerpBatch, and SquareBatch differed, but all six hot callers had exactly identical native code sizes between variants (23,287 B aggregate, 0% growth). With tiered compilation disabled, all six hot callers were instruction-equivalent and size-equivalent. The diagnostic SingleMultiply changed and is excluded from the adoption gate.

## Gate decision

- Relevant generated-code difference: pass under defaults for three consumers.
- IndependentMultiplyBatch improves at least 2% with CI below 1.00: fail; point estimate is 6.19% slower.
- DependentMultiplyChain improves at least 2% with CI below 1.00: fail; point estimate improves 1.52%, CI crosses 1.00.
- Representative workload improves at least 1% with CI below 1.00: fail; every CI crosses 1.00.
- At least 8/10 pairs favor the candidate: fail for every workload.
- No representative regression over 1%: fail by point estimate for IndependentMultiplyBatch, LerpBatch, and PowBatch.
- Allocation and code-size budgets: pass.

## Quality and evidence

Fifteen of 20 default-runtime invocations emitted a multimodal-distribution warning, confirming meaningful host noise. No command failed and all raw reports were retained; the wide confidence intervals prevent claiming a speedup.

The exact tested fixture validated its checksums during `GlobalSetup`, invoking
each workload once before BenchmarkDotNet's ten warmup iterations. The retained
fixture performs validation only through `--verify-inlining-corpus` so future
measurements begin with corpus initialization alone.

- Machine/runtime: `dotnet-info.txt`, `os.txt`, `cpu.txt`, `power-plan.txt`
- Correctness: `baseline-tests.txt`, `candidate-tests.txt`, `baseline-checksums.txt`, `candidate-checksums.txt`, `checksum-comparison.txt`
- Assembly: `assembly-comparison.txt`, `disasm-default-*`, `disasm-tieredoff-*`
- Timing: `timing-default/`, `paired-ratios.csv`, `paired-analysis.txt`
- Secondary sensitivity: `tieredoff-sensitivity.txt`
- Machine-readable result: `summary.json`
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Baseline and candidate experiment commands
Baseline: c4263974dbab1c3692ee9f01b46fc5176e16b197
Candidate: 48ff2f09673c2c190099d192c7f289060080e62c
Validation (run once in each exact-commit worktree):
dotnet build FixedPointNano.slnx -c Release --nologo
dotnet test FixedPointNano.slnx -c Release --no-build --nologo
dotnet run --project benchmarks\FixedPointNano.Benchmarks -c Release --no-build -- --verify-inlining-corpus
dotnet test tests\FixedPointNano.Tests\FixedPointNano.Tests.csproj -c Release --no-build --nologo --filter FullyQualifiedName~FixedPointNanoMathComparisonTests

Default disassembly (run once in each exact-commit worktree):
dotnet run --project benchmarks\FixedPointNano.Benchmarks -c Release --no-build -- --filter *FixedPointNanoMultiplyInliningBenchmarks* --disasm --launchCount 1 --warmupCount 10 --iterationCount 15 --iterationTime 250

Tiered-compilation-disabled disassembly/sensitivity (run once in each exact-commit worktree):
$env:DOTNET_TieredCompilation = "0"
dotnet run --project benchmarks\FixedPointNano.Benchmarks -c Release --no-build -- --filter *FixedPointNanoMultiplyInliningBenchmarks* --disasm --launchCount 1 --warmupCount 10 --iterationCount 15 --iterationTime 250
Remove-Item Env:DOTNET_TieredCompilation

Default timing (five ABBA blocks, 20 separate invocations):
dotnet run --project benchmarks\FixedPointNano.Benchmarks -c Release --no-build -- --filter *Batch* *DependentMultiplyChain* --launchCount 1 --warmupCount 10 --iterationCount 15 --iterationTime 250
Loading
Loading