Skip to content

Add benchmark tests for DbfSharp #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
google-labs-jules wants to merge 1 commit into from
Closed

Add benchmark tests for DbfSharp #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions BenchmarkDotNet.Artifacts/DbfSharp.Benchmark.DbfReaderBenchmarks-20250808-130146.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
// Validating benchmarks:
// ***** BenchmarkRunner: Start *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start *****
// start dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:ArtifactsPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/" /p:OutDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:OutputPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:PublishDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/publish/" in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1
// command took 3.67 sec and exited with 0
// start dotnet build -c Release --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:ArtifactsPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/" /p:OutDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:OutputPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:PublishDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/publish/" --output "/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1
// command took 17.33 sec and exited with 0
// ***** Done, took 00:00:21 (21.13 sec) *****
// Found 1 benchmarks:
// DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob

// **************************
// Benchmark: DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet DbfSharp.Benchmark-DefaultJob-1.dll --anonymousPipes 113 114 --benchmarkName DbfSharp.Benchmark.DbfReaderBenchmarks.ReadAllRecords --job Default --benchmarkId 0 in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0
// Failed to set up high priority (Permission denied). In order to run benchmarks with high priority, make sure you have the right permissions.
// BeforeAnythingElse

// Benchmark Process Environment Information:
// BenchmarkDotNet v0.15.2
// Runtime=.NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2
// GC=Concurrent Workstation
// HardwareIntrinsics=AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT VectorSize=256
// Job: DefaultJob

OverheadJitting 1: 1 op, 299667.00 ns, 299.6670 us/op

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> DbfSharp.Core.Exceptions.DbfNotFoundException: DBF file not found: people.dbf
at DbfSharp.Core.DbfReader.Create(String filePath, DbfReaderOptions options) in /app/DbfSharp.Core/DbfReader.cs:line 340
at DbfSharp.Benchmark.DbfReaderBenchmarks.ReadAllRecords() in /app/DbfSharp.Benchmark/DbfReaderBenchmarks.cs:line 14
at BenchmarkDotNet.Autogenerated.Runnable_0.WorkloadActionNoUnroll(Int64 invokeCount) in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/DbfSharp.Benchmark-DefaultJob-1.notcs:line 311
at BenchmarkDotNet.Engines.Engine.Measure(Action`1 action, Int64 invokeCount)
at BenchmarkDotNet.Engines.Engine.RunIteration(IterationData data)
at BenchmarkDotNet.Engines.EngineFactory.Jit(Engine engine, Int32 jitIndex, Int32 invokeCount, Int32 unrollFactor)
at BenchmarkDotNet.Engines.EngineFactory.CreateReadyToRun(EngineParameters engineParameters)
at BenchmarkDotNet.Autogenerated.Runnable_0.Run(IHost host, String benchmarkName) in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/DbfSharp.Benchmark-DefaultJob-1.notcs:line 176
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
--- End of inner exception stack trace ---
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/DbfSharp.Benchmark-DefaultJob-1.notcs:line 57
// AfterAll
No Workload Results were obtained from the run.
// Benchmark Process 4820 has exited with code 255.

// ** Remained 0 (0.0 %) benchmark(s) to run. Estimated finish 2025-08-08 13:02 (0h 0m from now) **
// ***** BenchmarkRunner: Finish *****

// * Export *
BenchmarkDotNet.Artifacts/results/DbfSharp.Benchmark.DbfReaderBenchmarks-report.csv
BenchmarkDotNet.Artifacts/results/DbfSharp.Benchmark.DbfReaderBenchmarks-report-github.md
BenchmarkDotNet.Artifacts/results/DbfSharp.Benchmark.DbfReaderBenchmarks-report.html

// * Detailed results *
DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob
Runtime = .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2; GC = Concurrent Workstation
There are not any results runs

// * Summary *

BenchmarkDotNet v0.15.2, Linux Ubuntu 24.04.2 LTS (Noble Numbat)
Intel Xeon Processor 2.30GHz, 1 CPU, 4 logical and 4 physical cores
.NET SDK 9.0.109
[Host] : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2
DefaultJob : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2


| Method | Mean | Error |
|----------------------------------- |-----:|------:|
| 'Read all records from a DBF file' | NA | NA |

Benchmarks with issues:
DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob

// * Legends *
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
1 ns : 1 Nanosecond (0.000000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
Run time: 00:00:00 (0.4 sec), executed benchmarks: 1

Global total time: 00:00:21 (21.71 sec), executed benchmarks: 1
// * Artifacts cleanup *
Artifacts cleanup is finished
176 changes: 176 additions & 0 deletions BenchmarkDotNet.Artifacts/DbfSharp.Benchmark.DbfReaderBenchmarks-20250808-130308.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
// Validating benchmarks:
// ***** BenchmarkRunner: Start *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start *****
// start dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:ArtifactsPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/" /p:OutDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:OutputPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:PublishDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/publish/" in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1
// command took 6.62 sec and exited with 0
// start dotnet build -c Release --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:ArtifactsPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/" /p:OutDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:OutputPath="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" /p:PublishDir="/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/publish/" --output "/app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0/" in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1
// command took 18.26 sec and exited with 0
// ***** Done, took 00:00:25 (25.05 sec) *****
// Found 1 benchmarks:
// DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob

// **************************
// Benchmark: DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet DbfSharp.Benchmark-DefaultJob-1.dll --anonymousPipes 113 114 --benchmarkName DbfSharp.Benchmark.DbfReaderBenchmarks.ReadAllRecords --job Default --benchmarkId 0 in /app/DbfSharp.Benchmark/bin/Release/net9.0/DbfSharp.Benchmark-DefaultJob-1/bin/Release/net9.0
// Failed to set up high priority (Permission denied). In order to run benchmarks with high priority, make sure you have the right permissions.
// BeforeAnythingElse

// Benchmark Process Environment Information:
// BenchmarkDotNet v0.15.2
// Runtime=.NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2
// GC=Concurrent Workstation
// HardwareIntrinsics=AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT VectorSize=256
// Job: DefaultJob

OverheadJitting 1: 1 op, 288501.00 ns, 288.5010 us/op
WorkloadJitting 1: 1 op, 30140263.00 ns, 30.1403 ms/op

OverheadJitting 2: 16 op, 363121.00 ns, 22.6951 us/op
WorkloadJitting 2: 16 op, 1446368.00 ns, 90.3980 us/op

WorkloadPilot 1: 16 op, 939888.00 ns, 58.7430 us/op
WorkloadPilot 2: 32 op, 1814393.00 ns, 56.6998 us/op
WorkloadPilot 3: 64 op, 3307920.00 ns, 51.6863 us/op
WorkloadPilot 4: 128 op, 6482129.00 ns, 50.6416 us/op
WorkloadPilot 5: 256 op, 13951201.00 ns, 54.4969 us/op
WorkloadPilot 6: 512 op, 22474772.00 ns, 43.8960 us/op
WorkloadPilot 7: 1024 op, 39491888.00 ns, 38.5663 us/op
WorkloadPilot 8: 2048 op, 83103041.00 ns, 40.5777 us/op
WorkloadPilot 9: 4096 op, 179982396.00 ns, 43.9410 us/op
WorkloadPilot 10: 8192 op, 413165656.00 ns, 50.4353 us/op
WorkloadPilot 11: 16384 op, 659961306.00 ns, 40.2808 us/op

OverheadWarmup 1: 16384 op, 53474.00 ns, 3.2638 ns/op
OverheadWarmup 2: 16384 op, 51684.00 ns, 3.1545 ns/op
OverheadWarmup 3: 16384 op, 51425.00 ns, 3.1387 ns/op
OverheadWarmup 4: 16384 op, 58996.00 ns, 3.6008 ns/op
OverheadWarmup 5: 16384 op, 52501.00 ns, 3.2044 ns/op
OverheadWarmup 6: 16384 op, 51480.00 ns, 3.1421 ns/op
OverheadWarmup 7: 16384 op, 51500.00 ns, 3.1433 ns/op
OverheadWarmup 8: 16384 op, 51458.00 ns, 3.1407 ns/op

OverheadActual 1: 16384 op, 52706.00 ns, 3.2169 ns/op
OverheadActual 2: 16384 op, 51699.00 ns, 3.1555 ns/op
OverheadActual 3: 16384 op, 51655.00 ns, 3.1528 ns/op
OverheadActual 4: 16384 op, 51481.00 ns, 3.1422 ns/op
OverheadActual 5: 16384 op, 52581.00 ns, 3.2093 ns/op
OverheadActual 6: 16384 op, 51494.00 ns, 3.1429 ns/op
OverheadActual 7: 16384 op, 51531.00 ns, 3.1452 ns/op
OverheadActual 8: 16384 op, 51481.00 ns, 3.1422 ns/op
OverheadActual 9: 16384 op, 52440.00 ns, 3.2007 ns/op
OverheadActual 10: 16384 op, 51471.00 ns, 3.1415 ns/op
OverheadActual 11: 16384 op, 51420.00 ns, 3.1384 ns/op
OverheadActual 12: 16384 op, 51507.00 ns, 3.1437 ns/op
OverheadActual 13: 16384 op, 52377.00 ns, 3.1968 ns/op
OverheadActual 14: 16384 op, 51606.00 ns, 3.1498 ns/op
OverheadActual 15: 16384 op, 51532.00 ns, 3.1453 ns/op

OverheadWarmup 1: 16384 op, 51718.00 ns, 3.1566 ns/op
OverheadWarmup 2: 16384 op, 52372.00 ns, 3.1965 ns/op
OverheadWarmup 3: 16384 op, 51446.00 ns, 3.1400 ns/op
OverheadWarmup 4: 16384 op, 58767.00 ns, 3.5869 ns/op
OverheadWarmup 5: 16384 op, 51481.00 ns, 3.1422 ns/op
OverheadWarmup 6: 16384 op, 52266.00 ns, 3.1901 ns/op

// BeforeActualRun
WorkloadActual 1: 16384 op, 393475292.00 ns, 24.0158 us/op
WorkloadActual 2: 16384 op, 399702934.00 ns, 24.3959 us/op
WorkloadActual 3: 16384 op, 380332294.00 ns, 23.2136 us/op
WorkloadActual 4: 16384 op, 391050207.00 ns, 23.8678 us/op
WorkloadActual 5: 16384 op, 388226419.00 ns, 23.6955 us/op
WorkloadActual 6: 16384 op, 390641857.00 ns, 23.8429 us/op
WorkloadActual 7: 16384 op, 390583348.00 ns, 23.8393 us/op
WorkloadActual 8: 16384 op, 387881649.00 ns, 23.6744 us/op
WorkloadActual 9: 16384 op, 387307597.00 ns, 23.6394 us/op
WorkloadActual 10: 16384 op, 393756155.00 ns, 24.0330 us/op
WorkloadActual 11: 16384 op, 390015344.00 ns, 23.8046 us/op
WorkloadActual 12: 16384 op, 392858004.00 ns, 23.9781 us/op
WorkloadActual 13: 16384 op, 390921771.00 ns, 23.8600 us/op
WorkloadActual 14: 16384 op, 392079844.00 ns, 23.9307 us/op
WorkloadActual 15: 16384 op, 386203479.00 ns, 23.5720 us/op

// AfterActualRun
WorkloadResult 1: 16384 op, 393423760.00 ns, 24.0127 us/op
WorkloadResult 2: 16384 op, 380280762.00 ns, 23.2105 us/op
WorkloadResult 3: 16384 op, 390998675.00 ns, 23.8647 us/op
WorkloadResult 4: 16384 op, 388174887.00 ns, 23.6923 us/op
WorkloadResult 5: 16384 op, 390590325.00 ns, 23.8397 us/op
WorkloadResult 6: 16384 op, 390531816.00 ns, 23.8362 us/op
WorkloadResult 7: 16384 op, 387830117.00 ns, 23.6713 us/op
WorkloadResult 8: 16384 op, 387256065.00 ns, 23.6362 us/op
WorkloadResult 9: 16384 op, 393704623.00 ns, 24.0298 us/op
WorkloadResult 10: 16384 op, 389963812.00 ns, 23.8015 us/op
WorkloadResult 11: 16384 op, 392806472.00 ns, 23.9750 us/op
WorkloadResult 12: 16384 op, 390870239.00 ns, 23.8568 us/op
WorkloadResult 13: 16384 op, 392028312.00 ns, 23.9275 us/op
WorkloadResult 14: 16384 op, 386151947.00 ns, 23.5688 us/op
// GC: 47 11 0 1129619968 16384
// Threading: 0 0 16384

// AfterAll
// Benchmark Process 5560 has exited with code 0.

Mean = 23.780 μs, StdErr = 0.058 μs (0.24%), N = 14, StdDev = 0.216 μs
Min = 23.210 μs, Q1 = 23.677 μs, Median = 23.838 μs, Q3 = 23.912 μs, Max = 24.030 μs
IQR = 0.235 μs, LowerFence = 23.324 μs, UpperFence = 24.265 μs
ConfidenceInterval = [23.537 μs; 24.024 μs] (CI 99.9%), Margin = 0.243 μs (1.02% of Mean)
Skewness = -1.13, Kurtosis = 3.84, MValue = 2

// ** Remained 0 (0.0 %) benchmark(s) to run. Estimated finish 2025-08-08 13:03 (0h 0m from now) **
// ***** BenchmarkRunner: Finish *****

// * Export *
BenchmarkDotNet.Artifacts/results/DbfSharp.Benchmark.DbfReaderBenchmarks-report.csv
BenchmarkDotNet.Artifacts/results/DbfSharp.Benchmark.DbfReaderBenchmarks-report-github.md
BenchmarkDotNet.Artifacts/results/DbfSharp.Benchmark.DbfReaderBenchmarks-report.html

// * Detailed results *
DbfReaderBenchmarks.'Read all records from a DBF file': DefaultJob
Runtime = .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2; GC = Concurrent Workstation
Mean = 23.780 μs, StdErr = 0.058 μs (0.24%), N = 14, StdDev = 0.216 μs
Min = 23.210 μs, Q1 = 23.677 μs, Median = 23.838 μs, Q3 = 23.912 μs, Max = 24.030 μs
IQR = 0.235 μs, LowerFence = 23.324 μs, UpperFence = 24.265 μs
ConfidenceInterval = [23.537 μs; 24.024 μs] (CI 99.9%), Margin = 0.243 μs (1.02% of Mean)
Skewness = -1.13, Kurtosis = 3.84, MValue = 2
-------------------- Histogram --------------------
[23.093 μs ; 24.147 μs) | @@@@@@@@@@@@@@
---------------------------------------------------

// * Summary *

BenchmarkDotNet v0.15.2, Linux Ubuntu 24.04.2 LTS (Noble Numbat)
Intel Xeon Processor 2.30GHz, 1 CPU, 4 logical and 4 physical cores
.NET SDK 9.0.109
[Host] : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2
DefaultJob : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX2


| Method | Mean | Error | StdDev | Gen0 | Gen1 | Allocated |
|----------------------------------- |---------:|---------:|---------:|-------:|-------:|----------:|
| 'Read all records from a DBF file' | 23.78 μs | 0.243 μs | 0.216 μs | 2.8687 | 0.6714 | 67.33 KB |

// * Hints *
Outliers
DbfReaderBenchmarks.'Read all records from a DBF file': Default -> 1 outlier was removed, 2 outliers were detected (23.21 μs, 24.40 μs)

// * Legends *
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Gen0 : GC Generation 0 collects per 1000 operations
Gen1 : GC Generation 1 collects per 1000 operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
1 μs : 1 Microsecond (0.000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
Run time: 00:00:08 (8.2 sec), executed benchmarks: 1

Global total time: 00:00:33 (33.42 sec), executed benchmarks: 1
// * Artifacts cleanup *
Artifacts cleanup is finished
Loading
Loading