-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathChangeLog
More file actions
2275 lines (1960 loc) · 100 KB
/
Copy pathChangeLog
File metadata and controls
2275 lines (1960 loc) · 100 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
* Mon Jul 27 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- Linux builds and runs again (Qt 6, qmake, GCC 13, Ubuntu 24.04). It is still
NOT a supported platform -- no instrument hardware has been exercised and the
GUI is only smoke-tested offscreen -- but the tree compiles warning-clean,
loads all 43 driver modules and runs the embedded Python engine. Most of the
work was in the .pro files: kame.pri had no eigen3 include path off
macOS/Windows; the Ruby branch still pointed at /usr/lib/ruby/1.8/i386-linux/
and now asks the interpreter via RbConfig; and there was NO pybind11 branch
for Linux at all, so USE_PYBIND11 was never defined -- meaning no Python
scripting, no Jupyter, no MCP server, and .kam files silently falling back to
the legacy Ruby loader. See INSTALL.linux for the recipe and the remaining
gaps.
- GPIB: the usermode NI USB-GPIB driver now builds on Linux too, so
`Device = GPIB` works there without a kernel module when linux-gpib is absent
and libusb is present. osx_compat.h keeps its historical name but is plain
POSIX; the only thing that had blocked Linux was its kernel-style min/max
FUNCTION macros colliding with <limits> in C++ (libstdc++ strict where libc++
tolerated it), now guarded. Also fixes a macOS-side build bug: HAS_LIBUSB held
the string "false" when MacPorts libusb was missing, which is non-empty and
would read as "yes".
- Python: a KAME script with both a function and a script-level variable failed
with NameError on every platform. loadSequence() ran user scripts with a bare
exec() inside a function, which hands the script that function's locals, so a
script-level `x = ...` landed where the script's own top-level defs could not
see it. Tracebacks now also name the file instead of "<string>".
- Python: a host merely missing numpy reported "no IPython", which silently
disabled the Jupyter menu, the notebook and the MCP server -- and then told
you to install ipykernel, which was already there. numpy is a convenience
import for user scripts and is referenced nowhere in KAME itself; it no longer
takes IPython support down with it.
- Ruby: a startup exception used to leave nothing on the terminal but
"exception(s) occurred" -- printErrorInfo() wrote through rb_p to Ruby's
$stdout, which xrubysupport.rb has already redirected into the GUI pane.
Class, message and backtrace now also go to stderr. A SIGTERM/SIGINT
shutdown is no longer reported as a Ruby error (SignalException and
SystemExit derive from Exception, not StandardError, so the support script's
own rescue could not catch them).
- New tools/iftest_serial_gpib.py: hardware-free smoke test driving
XCharInterface's own Device/Port/Control/Query nodes -- serial and Prologix
over a real pty (including a *IDN? round trip and the full adapter-init
sequence on the wire), plus the usermode GPIB path's libusb enumeration.
* Sat Jul 25 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- kamepoolalloc (Jul 25-27): realtime support, as a written contract rather than
a claim -- see kamepoolalloc/design/RT_READINESS.md and the README's "The
realtime contract" (preconditions -> guarantees -> exclusions, with the
exclusions stated plainly, including that there is no numeric WCET).
* Per-thread realtime levels. KAME_RT_DEFER stops a thread's free() from
entering the kernel -- chunk page-reclaim is skipped (the chunk stays
immediately recyclable, its pages just stay warm) and a large-tier munmap is
parked for kame_pool_rt_drain(). All of it is on cold release paths, so it
measured within noise (-2.8 % median, individual reps positive), and on the
band the recycle cache cannot absorb (> 256 MiB) it moves free latency from
a 20.5 us median / 678 us max to 128 ns / 792 ns. KAME enables this
process-wide via the new kame_pool_set_realtime_default().
* KAME_RT_STRICT additionally bounds the cross-thread batch's amortization
spike (p99.9 96 ns vs 1,792 ns) but costs ~48 % of cross-thread small-free
throughput, so it is per-thread only and KAME deliberately does not enable
it -- KAME's deadlines are instrument I/O at millisecond scale.
* kame_pool_prewarm() allocates, PAGE-TOUCHES and frees given size classes;
the previously documented allocate/free idiom left pages mapped but
unfaulted, so the first realtime write still took a fault.
kame_pool_reserve_regions(), kame_pool_mlock_regions() (pins the pool's own
regions -- surgical where mlockall(MCL_FUTURE) is blunt), and
kame_pool_set_thp_policy() (Linux transparent hugepages: first touch inside
a 2 MiB range can make the kernel zero a whole hugepage, and khugepaged's
compaction can stall an unrelated fault) round it out.
* Observability: kame_pool_rt_violations() counts times a realtime thread
entered the kernel for a new mapping -- a test can assert it stayed zero.
The deferral backlog is capped (kame_pool_set_rt_pending_cap, default 1
GiB); an early version was unbounded and parked 12.6 GB of VA for 40 frees.
* Verified with a new per-op WCET tail harness (tests/bench/bench_rt_wcet.cpp)
reporting max and percentiles rather than means -- means are exactly the
statistic that hides a syscall -- on both same-thread and cross-thread
paths.
* Thu Jul 16 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- Python: creating a driver from a Python cell / MCP / an asyncio task could
abort the process. Node construction may build Qt widgets, which must happen
on the GUI thread; createByTypename now dispatches through kame_mainthread()
for lists that report they are not thread-safe during creation, mirroring
what the .kam loader already did -- so every Python caller is protected, not
just the .kam path.
- NMR frequency spectrum: don't kill the tuning-finished listener from
rearrangeInstrum while the auto-tuner is running.
* Mon Jul 13 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- Fix the startup crash on Intel Macs (x86-64), which turned out not to be our
bug: older Apple clang's `preserve_most` calling convention clobbers the
RETURN register on x86-64, so kame_page_cold() handed back garbage. The
attribute is removed from value-returning functions. The defensive null-clamp
added while chasing it is also removed -- experiment showed the TLV-null it
guarded against was a phantom.
- Fix a rare abort at startup (__cxa_pure_virtual): the scripting worker thread
was started from the base constructor, so it could dispatch through a vtable
that still pointed at the pure-virtual placeholder. Qt 6.8's earlier
startup-thread scheduling began winning that race on some macOS setups.
- kamepoolalloc: publish the fast-TSD slot offset only after planting the slot
(scan race), and make kame_page() total.
* Sun Jul 12 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- STM snapshot isolation, second pass: NMR T1 double-counted and ODMR halved
their data because a Payload's pointee was mutated in place while other
snapshots still held it. Those sites now clone-on-write, and the remaining
latent cases were converted to pointer-to-const so the compiler refuses the
mutation instead of it being a silent isolation violation.
- digitalcamera: publish the frame buffer together with its geometry, so a
reader can never pair a new buffer with stale dimensions.
* Sat Jul 11 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- Crash audit of the 2026-07 reports, and the rules it produced. Fixed: Python
bindings entering STM negotiation while holding the GIL (deadlock, ending in
the negotiation watchdog abort) and reading Payloads before the first record;
the recorder holding file mutexes across STM operations (deadlock); MonteCarlo
freeing FFTW buffers inside an iterate_commit closure, which re-runs on every
CAS retry (double free); libusb entry points called with a null handle, which
asserts and aborts the process instead of returning an error; acquisition poll
loops spinning without backoff after an interface error; sibling node-name
collisions that made the later node unreachable from Python/.kam/NodeBrowser;
and listeners doing Qt UI work inline on whichever thread committed.
- Five of those rules are now mechanically checked by tools/audit/run_audits.sh,
wired into a pre-commit hook and CI, with pre-existing findings grandfathered
under a ratchet. The rules themselves are written up in CLAUDE.md for anyone
adding a driver.
* Fri Jul 10 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- graph: fix a system-wide STM stall (ending in the negotiation HANG watchdog's
abort) when the on-screen-object paint path held its mutex across a Snapshot
while a driver thread took the same mutex from inside an in-flight
transaction. The paint path now copies what it needs under a short lock and
releases it before snapshotting; a racing writer only costs one stale frame.
- New modules/relay: relay / digital-output driver (LCUS-1 family, and an
XRelayViaSTM variant driving relays through other drivers' STM nodes).
- Laser driver refactor; lasermoduleform aligned to the house UI style, and the
.ui conventions that survey produced are documented in CLAUDE.md.
- ui: add or fix tab stops across the driver forms.
* Sun Jul 05 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- STM: close a preempt window in the negotiation tag teardown. Clearing a
linkage's tag used a plain store after checking it was ours, so an older
transaction's stamp landing between the check and the store was erased,
dropping its priority claim for a round. It is now a mine-only
compare-and-swap, mirroring what the global-mode release already did after the
same bug class caused an observed hang. Livelock-freedom was never at risk;
what this restores is the "oldest completes in finitely many CASes" bound,
which the machine-checked model had been assuming.
- verification (Jul 2-6): pre-submission TLA+ fidelity work -- a second-model
re-check of every previously examined item, a new hard-link nested-external
model, and a correspondence pass over the 16 slide decks and the READMEs that
corrected mechanism and verification wording in several places.
* Wed Jul 01 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- Fix NMR T1 / phase-inversion-cycling timing regression (introduced 8.1,
commit 5185cd09). In XPrimaryDriver::finishWritingRaw the `skipped` latch was
hoisted out of the iterate_commit closure and captured by reference, so it was
never reset between CAS retries; a first-attempt XSkippedRecordError then
suppressed record() on a subsequent successful retry, silently dropping a valid
primary (DSO/pulser) record. That stalled the XNMRPulseAnalyzer average and
hence the completed-average onRecord cascade that drives the PIC invertPhase
toggle and the T1 setNextP1 pulser reprogram. Reset skipped/time_recorded/err
at the top of the closure to restore the pre-refactor per-retry semantics; the
secondary-driver path re-declares these inside its for(;;) loop and was
unaffected.
* Tue Jun 30 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- graph: fix crash when a XWaveNGraph column is read before it is filled.
getColumn()/precision() dereferenced a null m_cols slot (a column exists
from setCols()/setColCount() but stays null until setColumn()); e.g.
probing an NMR spectrum from Python/MCP before a scan completes now
returns an empty buffer and the fallback precision instead of segfaulting.
* Wed Jun 24 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- Thamway NMR (USB): fix process abort when the device is closed (USB link
lost / interface stopped) concurrently with a bulk transfer. asyncBulkWrite
/ asyncBulkRead now throw a catchable XInterfaceError on a null device
handle instead of calling libusb_submit_transfer(), which hit an internal
assertion and abort()ed the whole process (seen from the XThamwayPROT
status-poll thread).
- Thamway NMR: back off 1 s in the status-poll loop after an interface error
so a closed device is not hammered on repeated retries.
* Sat Jun 20 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- STM: harden snapshot isolation by migrating large/variable-size Payload
heap members (camera/ODMR images, math-tool masks, X2DImagePlot image,
DSO reference/FIR waveforms, NMR FT buffers, XComboNode strings) to
shared_ptr<const T>, so in-place mutation is a compile error and older
snapshots keep an immutable view; make FFT/FIR exec() const (per-call
scratch, new-array fftw_execute on a shared immutable plan) so the
transform objects can be shared across snapshots and threads.
- optics: prevent SIGABRT on quit from eGrabber static destruction — give
the s_gentl / s_discovery singletons a no-throw deleter that swallows
GenTL close exceptions, so teardown at exit() no longer aborts via
std::terminate.
* Fri Jun 19 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- AI/MCP safety guidance: document instrument-protection rules across the
manual, kame_python_api.md, and MCP server instructions so an agent
observes them by default — NMR RF-power duty limits (PW1/PW2/CombPW <=
min(Tau*0.3, 15) us, RT >= 15 ms) and Thamway PROT Level guard; cryogenic
TargetTemp above ~295 K requires confirmation and can overshoot >=10%;
motor moves may be irreversible and Position is often open-loop, and
GoHomeMotor runs until a home sensor triggers (runaway if none installed).
- manual: add per-driver scripting node names for all controllable drivers
(DC source, DMM, DSO, lock-in, magnet PS, SG, network analyzer, temp
controller, flow controller, motor, turbo pump, laser, NMR pulser, the
NMR analysis chain, auto LC tuner, camera, and spectrometer), plus
operational recipes for Motor (Target starts the move; poll Ready) and
Auto LC Tuner (Target starts tuning; poll Tuning; do not drive STM motors
mid-tune).
- drivers: fix four node-name bugs affecting name-based access (Python,
.kam load, Node Browser): magnetps "OutpuField" -> "OutputField";
pumpcontroller runtime node "RotationSpeed" (collision) -> "Runtime";
digitalcamera EM-gain "CameraGain" (collision) -> "EMGain";
opticalspectrometer "TimeTorStrobeSignal" -> "TimeToStrobeSignal".
- optics: guard camera Payload.rawCounts()/darkCounts() against uncaptured
frames — they dereferenced an empty count buffer and aborted the whole
process from the Python thread; now throw a clean Python exception.
- kame: optionally emit an aggregate STM commit-count footer
("# stm_total_tx_commits:") when saving a .kam file, for measuring live
commit throughput; gated on the --logging debug-log mode so production
saves stay bit-for-bit identical with zero per-node cost when off.
* Thu Jun 18 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- manual: flesh out the Function Generator section with node names, units,
combo choices, and the three 0-sentinel conventions (BurstCycles 0=INFinity,
PulseWidth 0=use Duty, PulsePeriod 0=follow Freq) so kame_manual is
actionable for AI control.
- kamepoolalloc (Linux LD_PRELOAD robustness): intercept the aligned-allocation
family (memalign/posix_memalign/aligned_alloc) and reallocarray, and
co-interpose malloc_usable_size, routing them through the pool and resolving
real-libc entries via dlsym(RTLD_NEXT); fixes musl/Alpine hang (real-libc
bypass had only a glibc path, so overrides tail-recursed) and musl SIGSEGVs
(rptest/rocksdb aligned allocs, teardown owner force-walk poke), redis-server
SEGV from malloc_usable_size on pool pointers, and Fedora GCC/libstdc++ 14+
build errors from missing <cstdlib>/<cstdint> includes.
* Mon Jun 15 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- MCP/Python: fix the MCP server launcher to pick the Jupyter shebang
interpreter (the one that actually has mcp/jupyter_client), probe candidates
with the server's real imports, and wait for the port to LISTEN before
declaring success (reaping the child on failure so it cannot linger as a
zombie). Reuse the previous port+token from ~/.kame_mcp_url so .mcp.json is
stable across restarts, killing our own verified stale server to reclaim the
port; KAME_MCP_PORT / KAME_MCP_TOKEN override. Windows parity via
tasklist/taskkill.
- Python: sleep() no longer crashes in execute_code_async worker threads —
resolve the scripting-thread TLS via getattr and fall back to a plain
interruptible wait off scripting threads. MCP execute_code returns compact
PNG figures (dpi=100, tight bbox).
- IPython tab: clickable "Quick launch" links (Jupyter notebook / Claude Code
terminal / Claude app); kame: links route through XPython::handleLink,
others open via QDesktopServices (also revives the dead log/notebook links).
Notebook launch prompts for the workspace folder rather than the app-binary
cwd. Kernel startup Status changed "No connection" -> "idle".
- build: link CMAKE_DL_LIBS in the in-tree kamepoolalloc/kamestm test targets
(4 places) so the Linux malloc_usable_size co-interpose's dlsym resolves on
glibc < 2.34; fixes undefined-reference-to-dlsym link failure on Ohtaka.
* Fri Jun 12 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- kamestm (Jun 9-30): memoize the Snapshot/Transaction payload subscript with a
per-lookup memo, tuned across several revisions (widen to fully-associative
slots, MRU tiering, hot/cold layout split) and finally simplified to a single
self-validating Payload* kept plain non-atomic since the lookup path is
single-threaded; skip the final-level weak->strong promotion on the hint walk
and shrink Packet by making m_subpackets an intrusive local_shared_ptr
(-8 B/Packet). Behavior-neutral throughput work.
- kamestm negotiation (Jun 9-30): move the negotiate sleep/wake off
std::condition_variable to XWaitCell (__ulock on macOS), with env knobs
KAME_NEG_SLEEP_US_PER_MS (sleep-chunk granularity) and
KAME_STM_CAS_BOUNDED_RETRY (bounded WEAK retry on spurious failure); a
sole-runner no longer voluntarily yields in the low-contention shortcut.
- atomic_smart_ptr (Jun 9-30): add opt-in biased reference counting, per-type
opt-in and default OFF (no change to existing types), plus a
force_incomplete_ref opt-out for circular/incomplete template-id members.
- kamepoolalloc (Jun 9-30): hot/cold split of operator new/delete for a
frameless freelist fast path (41% fewer hot-path instructions), a
size->bucket lookup table extended over the full bucketed range, and
word-cache made default ON; all gated/tuning changes are default-behavior
neutral.
- verification (Jun 9-30): extensive ongoing TLA+/GenMC work -- pre-submission
fidelity dossier (spec<->C++ action correspondence ranked by confidence),
parameterized-cutoff liveness, hard-link non-atomic re-verification, and
memory-ordering (cds) tests.
* Thu Jun 11 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- arbfunc (LXI 3390): make burst cycle count settable from the UI via a
new BurstCycles field (BURS:NCYC; 0 = INFinity), read back on connect.
- arbfunc (LXI 3390): enable the PulseWidth and PulsePeriod fields with a
0-sentinel convention resolving the width/duty exclusivity — width > 0
drives FUNC:PULSE:WIDTH (skips DCYC), period > 0 overrides 1/Freq; 0
keeps the legacy duty- and Freq-driven behavior.
- scripting: show the executing notebook cell in the XScriptingThread
Status outside sleep() via IPython pre_run_cell/post_run_cell hooks
("run Cell In[N]: ..." / "idle (Cell In[N] done|ERROR)"), clear stale
Actions on cell start, fix the In[N] off-by-one, and make isRunning() a
prefix match so suffixed labels still count as running.
* Wed Jun 10 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- MCP server: add Jupyter notebook cell editing (notebook_status /
notebook_read / notebook_edit) via the Jupyter contents REST API; a second
ZMQ connection watches iopub broadcasts to report the currently executing
cell even while the kernel is busy, and edits refuse to touch the running
cell.
- MCP server: make execute_code_async jobs interruptible — add stop_job
tool and an mcp_checkpoint(progress) helper that publishes progress and
cooperatively stops the job (get_result now reports progress and a
"stopped" status).
- MCP server: add the user's manual as Markdown (doc/manual/kame-8-en.md,
converted from the docx) served section-wise by a new kame_manual tool.
- docs/MCP: steer AI image analysis to the raw-count Python functor / math
tools; X2DImagePlot to_png() is display-only, legitimate for viewing and
binary segmentation / mask generation but not for quantitative pixel
reads. Fix pytestdriver.py NumPySum functor to the 7-arg masked signature.
* Tue Jun 09 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- graph: fix drag-zoom on an axis flashing and instantly reverting on macOS
trackpads set to two-finger secondary click; Qt 6.8 synthesizes an extra
right press/release on finger-lift whose near-zero movement hit the
single-click autoscale branch. Debounce the synthetic SelFinish (within
200ms/5px of a committed drag-zoom) and accept() mouse press/move/release
events.
* Mon Jun 8 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- kamepoolalloc: the atomic_shared_ptr orphan-chunk reclaim chain is now the
default chunk-recycle mechanism; the §36 raw Treiber stack is retired.
A chunk left non-empty by an exited owner thread is pushed onto a per-template
lock-free atomic_shared_ptr orphan chain (orphan_chain_push); an allocating
thread later adopts it (orphan_chain_pop + re-own, reusing its free slots) and
a sweep pass (orphan_chain_scrub) reclaims it once fully drained. This replaces
the §36 ABA-tagged Treiber stack, which leaked drained orphans (never freed
while stacked). The chain is built on kamepoolalloc/atomic_smart_ptr.h — the
same lock-free reference-counted smart pointer as the STM, relocated there
(§36b) as the shared home for both subsystems.
Adopt safety (TLA+): a re-owned chunk carries an owner-ref — a self-referential
local_shared_ptr the chunk holds to itself (m_owner_self_ref) — so the owner's
free becomes refcount-mediated (reset the self-ref → disposer) instead of a
direct deallocate_chunk that ignored outstanding refs. OrphanChain_adopt.tla
pinned down the gap this closes (Inv_NoBadOwnerFree: the owner could free a
chunk a concurrent sweeper still load_shared-pinned — a UAF that 21.3M-op
ASan/TSan stress could NOT reproduce; only the model catches it). Kept as a
standing regression guard: tests/tlaplus/run_orphan_chain.sh (10 OrphanChain_*
model/cfg checks; the BIT_OWNED disposer gate is proven load-bearing).
Also: §74 — allocate_chunk<ALLOC>() de-duplicated onto the shared claim_chunk()
(the single region-walk + mmap + bitmap-claim site, shared with the dedicated
large path). Width-aware RADIX_VA_LIMIT (32-bit clean). Verified on Linux
(g++ 13.3): TSAN/ASan race- and UAF-free, alloc_thread_churn reserved plateau,
full ctest; macOS build + ctest + alloc_stress (residual 0) + 64 B hot-path
bench unchanged (the orphan machinery is entirely cold-path). kame.pro /
tests.pri now reference the relocated atomic_*.h from kamepoolalloc/
(atomic_smart_ptr.h added — the inline-compiled allocator includes it
unconditionally now that the chain is the default).
* Sat May 3 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- TLA+ formal models: three fixes and ohtaka verification results.
(1) BundleUnbundle_2level_LLfree_dynamic.tla — BundlePhase1 deadlock
fix for release_superfine config. When a child is released mid-collection
(ActiveChildren shrinks), Thread A's first disjunct
(∃c ∈ ActiveChildren : subwrappers[c] = Null) goes FALSE with all
remaining children already collected, leaving no enabled action.
Fix: second disjunct (∀c ∈ ActiveChildren : subwrappers[c] ≠ Null)
drops stale released-child entries and advances to bundle_phase2; Phase 2
CAS fails (parent updated by release) → BundleRetryPC → ReadParent
picks up the new parent wrapper. C++-faithful: same retry path as C++
(prestamp CAS → collection loop → Phase 2 CAS fail → snap_read retry).
Verified: release_superfine_live 2-thread exhaustive run on ohtaka —
413,884,516 distinct states / depth 320 / 7h 13min / counter 15–55 /
5,972 terminal emits / queue 0 (exhaustion) — Safety + liveness PASS.
(2) BundleUnbundle_3level_LLfree.tla — InnerPhase2 restart fix.
QuiescentCheck violated at depth 61 on ohtaka (spec 8fb19385, 3L
superfine confA, 2 leaf + 1 root). Root cause: InnerPhase2 (Phase 2
CAS for inner child = Parent) used UNCHANGED local on failure, leaving
subwrappers[Parent] non-Null. BundlePhase1 on restart saw
subwrappers[Parent] ≠ Null and skipped re-collecting Parent, proceeding
to BundlePhase2 with stale subpackets (pre-CommitChild payload value) →
lost increment. Fix: same pattern as InnerPhase3/InnerPhase4 — clear
outer bundle state (wrapper, subwrappers, subpackets) on failure, eagerly
tag bundleNode (Grand) in addition to inner child (Parent). C++-faithful:
bundle_subpacket returning DISTURBED from inner bundle Phase 2 causes
the outer Phase 1 child_retry to continue without updating
subwrappers_org[i], re-reading Parent fresh — equivalent to clearing and
re-collecting. All three inner-phase failure paths (InnerPhase2/3/4) now
consistently clear outer bundle state on DISTURBED. Sanity checks:
1-thread PASS (47 states); 2-thread coarse PASS (1,497,098 states /
depth 98 / 1:35 + liveness). 3L superfine confA/B re-submitted to ohtaka.
(3) Ohtaka results recorded:
3L 3thr superfine confC live: 640,894,951 states / depth 88 / 15:25:00
/ counter 4–15 / 1,140 terminal emits — Safety + liveness PASS.
First formal proof of 3-level 3-thread superfine liveness.
2L-dyn 3thr-A live (Ins={1},Root={2},Leaf={3}): 53,397 states / depth 68
/ 7 s / counter 10–15 / 42 terminals — PASS + liveness.
2L-dyn 3thr-B live (Ins={1},Root={2,3},Leaf={}): 149,137 states / depth
82 / 14 s / counter 8–15 / 22 terminals — PASS + liveness.
2L-dyn release superfine 2t live: 413,884,516 states / depth 320 / 7:13
/ counter 15–55 / 5,972 terminals — PASS + liveness (see item 1).
Dynamic spec 2-thread verification now complete: coarse/superfine ×
no-release/release, all Safety + liveness PASS.
See tests/tlaplus/doc/verification_log.md for full results table.
* Thu Apr 30 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- TLA+ formal models (tests/tlaplus/BundleUnbundle_3level_LLfree.tla,
BundleUnbundle_2level_LLfree.tla): C++-fidelity fixes that close the
gap between TLC verification and the C++ STM implementation. After
these, every cfg passes both safety (TerminalPayloadCheck and friends)
and liveness (EventuallyAllDone), and the generated C11 stress test
no longer hangs.
(1) BundlePhase1 coarse path inner-bundle: drop the
linkage[n].hasPriority filter on grandchild CAS. C++ Phase 3 of an
inner recursive bundle (transaction_impl.h:2487-2511) ALWAYS CASes
each child to a fresh bundled_ref wrapper regardless of prior state;
the filter caused TLA+ to leave already-bundled grandchildren
untouched, allowing a peer's stale snapshotForUnbundle pointer to
CAS-succeed and produce a lost-increment race that does not occur
in C++.
(2) GenSerial: replaced ad-hoc globalSerial-based uniqueness with
C++-faithful TID-encoded base-B Lamport (mirrors transaction.h:
547-576 SerialGenerator::gen). serial = counter * SerialBase + TID;
GenSerial uses TLS counter (serial[t]) plus the wrapper's
m_bundle_serial passed in, advances counter past last_serial, then
increments and re-encodes with TID. Two threads with the same
counter produce different serial values via different TID lower
digits, mirroring C++ pointer distinctness. The globalSerial
variable is removed; SerialBound and DebugSerialBound are neutered
to TRUE for cfg back-compat.
(3) BundlePhase3 disturbed restart (3-level coarse + fine):
regenerate bundleSer via GenSerial when looping back to
bundle_phase1. C++ bundle() retry loop allocates new
PacketWrappers with fresh bundle_serial per iteration; without
regen the retried Phase 3 emits structurally identical wrappers
and a peer's stale pointer compares equal.
(4) UnbundleWalk casTargets: always root-first regardless of
UnbundleWalkAtomic. C++ walkUpChainImpl is recursive — the deepest
call (root) emplace_back's into cas_infos first. Leaf-first in
fine mode let a peer CommitGrand interject between t1's Parent CAS
and t1's Grand CAS, never seeing Grand changed → stale-snapshot
CAS succeeds → lost increment.
(5) InnerPhase3 / outer BundlePhase3 fine success path: update
innerSubWs[gc] / subwrappers[c] to the new wrapper. TLA+-specific:
C++ pointer identity automatically invalidates the failure-branch
guard after a successful CAS, but TLA+ value equality lets the
failure branch falsely re-fire on the just-CAS'd entry, creating a
single-thread state-space explosion under Lamport.
- TLA+ verification status (all PASS, laptop -Xmx14g):
3-level 1-thread fine : 46 distinct, depth 29, < 1 s, counter 7.
3-level coarse 2t : 1.5 M distinct, depth 98, 1:53, counter 6–22.
3-level superfine 2t : 12.1 M distinct, depth 140, 18:39, counter 22–26.
2-level micro (fine) : 804 K distinct, depth 89, 54 s, counter 6–18.
2-level superfine : 2.5 M distinct, depth 129, 3:06, counter 6–23.
2-level phase0only : 927 K distinct, depth 87, 52 s, counter 6–18.
2-level phase3only : 2.4 M distinct, depth 129, 2:31, counter 6–24.
3-level off (Priv=FALSE): diverges (killed at 118M states, §4–§5).
Lamport counter = serial ÷ (1 + |Threads|). See doc/verification_log.md.
* Mon Apr 27 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- STM core: K=0 NUMA scaling (8.5 M → 314 M c/s on ohtaka 128c, ×37).
Two structural cacheline-contention bottlenecks removed:
(a) s_running global atomic counter — replaced with per-thread
PaddedCounter held in TLS shared_ptr; global registry is an
atomic_shared_ptr<vector<weak_ptr>> COW-published on first use,
pruning expired entries on each registration. numThreadsRunning()
sums across registered threads (called only from negotiate paths).
(b) KAME pool allocator — per-thread chunk pinning. Slow-path
compare_exchange-claims an unclaimed chunk via m_thread_pinned_count
(0→1) and caches it in TLS s_my_chunk; subsequent allocate() calls
bypass the bit0-lock CAS on s_chunks_of_type[]. Pin counters are
registered with a namespace-scope thread_local cleanup object
(tls_alloc_pin_cleanup) so chunks are returned to the global pool
on thread exit. allocate_pooled's non-atomic oldv==0 fast-write
removed (always-CAS) so the chunk bitmap stays thread-safe even
when multiple threads share a chunk transiently. release_allocator
gated on m_thread_pinned_count == 0.
- STM core: KAME_CACHE_LINE macro (ABI-driven: x86_64/Cortex 64,
Apple Silicon/POWER 128, A64FX 256). Applied to RunnerCounterEntry,
s_privileged_tidstamp, NegotiateSleepSlot, s_max_c_obs — replacing
the previous fixed alignas(64) which under-padded on Apple Silicon
(128-byte cacheline) and POWER.
- STM core: fair_mode_blocks_me() compares only the TID half of the
packed tid+stamp slot, not the full stamp. The previous full-stamp
inequality self-deadlocked when a nested Tx (e.g. an outer Tx's
retry path triggering ~Node()→releaseAll() with its own
iterate_commit_if) ran on the privilege-holding thread: the inner
Tx carried a different started_time and was wrongly fair-mode
blocked by its own slot. Confirmed via lldb backtrace of a
transaction_dynamic_node_test hang (frame #15-16: Snapshot ctor
→ fast_vector destructor → ~Node → releaseAll → nested Tx waiting
in negotiate_sleep). Diagnosis was previously masked at PRIV_AGE
≥20 ms because the race window was rarely entered; surfaced when
the threshold was lowered.
- STM core: KAME_STM_PRIV_AGE_NORMAL_US single-knob configuration
(verdict gate and preempt floor unified). With the
fair_mode_blocks_me TID-fix in place the threshold can be set in
the sub-millisecond range without reintroducing the 1ms
test_dyn churn-deadlock. OS-aware default: 750 µs on POSIX
(Mac/Linux/ohtaka — sweep winner on Mac M3 Air for K=10 N=128:
1.25 M → 2.48 M c/s), 10 ms on Windows (default scheduler quantum
≈15.6 ms forces a higher floor). Override at build time with
-DKAME_STM_PRIV_AGE_NORMAL_US=N. min_privilege_age_us(LOWEST) =
30 ms, (UI_DEFERRABLE) = 50 ms unchanged.
- KAME allocator: ALLOC_PAGE_SIZE arch-aware default (Apple arm64
16 KiB, POWER 64 KiB, else 4 KiB). The previous fixed 4 KiB caused
silent mprotect() failure on macOS arm64 once chunk-size growth
produced a non-16 KiB-aligned size (500 KiB at the 4th GROW step),
manifesting as SIGBUS on the next placement-new write. mprotect
failures now abort with diagnostic (errno + page-alignment of addr
and size) instead of being lost to assert(no-op under NDEBUG).
- STM core: low-contention sleep shortcut. When the process-wide
numThreadsRunning() ≤ 2 the privileged-TID escape cannot help —
its age-difference threshold needs ≥3-thread spread to develop —
and the standard 1 ms CV sleep chunk becomes the K=1 N=2
throughput ceiling, limiting per-thread rate to ~1 kHz for
sub-µs commits. Replaced with std::this_thread::yield() when
numThreadsRunning() ≤ 2 && ms ≤ 1 so Greedy CM (older Tx wins)
drives a tight CAS-retry alternation. Gated on the global
runner count to match the privilege slot's process-wide scope.
- STM core: C_obs lower bound (KAME_STM_C_OBS_MIN, default 2).
At C_obs=1 the √C lottery threshold collapses to 1.0
(always-fire), causing notify_n_contenders() to be called on
every iteration even though only one or two threads are
actually involved. The wake-broadcast overhead dominates K=1
N=2-style anti-symmetric workloads. Floor C_obs at 2 in the
lottery formula so the threshold becomes 0.5 (= the natural
rate for 2-thread alternation) without inflating contender
counts elsewhere. On Mac M3 Air, K=1 N=2 (3-level) jumps from
1.3 M → 5.7 M c/s (×4.5); on iMac Pro from 597 K → 1614 K
c/s (×2.7). K=1 N=8/32/128 also improve +17–50%; K=10 / K=0
cells unaffected; test_dyn × 20 hang-free.
- KAME allocator: kame.app startup-crash fix on macOS Apple Silicon.
ICU/Foundation allocate via libsystem malloc during early process
startup (before main() reaches activateAllocator()), then free
those pointers after the pool has been activated. Two defensive
changes prevent the pool deallocate path from dereferencing
uninitialized state on those non-pool pointers:
(a) deallocate_pooled_or_free() now mirrors new_redirected()'s
g_sys_image_loaded gate — pre-activation deletes go straight to
std::free, since the pool's mmap'd regions don't exist yet.
(b) deallocate_<>() validates s_chunks[cidx] before the virtual
call: ((uintptr_t)palloc <= 1) returns false (== "not our pointer"),
catching both the nullptr (chunk released) and the (PoolAllocatorBase*)1u
in-creation sentinel that allocate_chunk's CAS sets between slot
claim and final pointer assignment. Crash trace was
icu::Locale::init → operator delete[] → deallocate_<0,262144>
bad-vtable-access, surfaced after per-thread chunk pinning made
chunk creation/release more frequent.
- tests/transaction_payload_integrity_3level_mixed_test: KAME_FIRSTTOUCH
env-var opt-in. With KAME_FIRSTTOUCH=1, leaf nodes are allocated
inside the worker thread (first-touch on its NUMA node) instead of
on the main thread; default 0 preserves the original main-thread
pattern used by paper figures. Allocator-pinning largely subsumes
this on ohtaka, but the knob is retained for ablation studies.
* Sun Apr 26 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- STM core: livelock-free fair-mode privilege escape. Globally
registered "privileged TID+stamp" (s_privileged_tidstamp): when the
livelock probe fires verdict=LIVELOCK on the Greedy CM winner (oldest
Tx, all linkage tags claimed) it CAS-claims the slot. Other threads
see fair_mode_blocks_me() and yield via negotiate(). Age-preempt: a
challenger Tx older than the holder by ≥ min_privilege_age_us(prio)
CAS-overrides the slot. release_privileged_tidstamp() is CAS-based
(only clears if our stamp still resides) so a preempted holder's
release does not erase the preemptor's claim.
- STM core: ScopedNegotiateLinkage RAII helper centralises the
negotiate-before-CAS / tag-on-disturb / commit pattern. Applied at
every CAS-retry / spin loop site: Node::snapshot, Node::bundle outer
+ Phase 1 child_retry, Node::commit, Node::insert(online_after_insertion),
Node::eraseSerials, Node::unbundle (cas_infos + final sublinkage).
TagMode::OnEntry (eager tag at iter top) / OnExit (lazy on dtor
unless commit() called); retry==-1 sentinel for unconditional tag
mode. commit_after_cas(started_time) shorthand combines
tags_successful_cas + commit at sites where the scope's linkage is
the CAS target.
- STM core: retry==0 yield. negotiate_after_retry_pause self-gates at
retry==0 — fast-return unless fair_mode_blocks_me reports another
Tx holds the privileged slot, in which case we run retry_pause +
negotiate to yield. Required for the livelock-free guarantee
(otherwise retry==0 hammers race the privileged Tx's commit CAS).
- STM core: tag_as_contender uses Option B (relaxed store + acquire
verify). Eliminates the younger-overrides-older race that produced
multi-Tx 0-commits stalls in the 3-level mixed test under heavy
contention; if our store is overwritten between the store and
verify, we skip the m_tagged_linkages push (the surviving stamp's
owner is responsible for its own drop).
- STM core: KAME_STM_ASSERT_PRIVILEGE invariant check (default 0;
enable with -DKAME_STM_ASSERT_PRIVILEGE=1). The
ScopedNegotiateLinkage dtor asserts when the scope held privilege
on entry AND still holds it on exit without commit() — i.e. the
privileged Tx failed a CAS / loop iteration without making forward
progress, breaking the livelock-free guarantee. Catches any future
regression in the negotiate / tag_as_contender coverage.
- STM core: cache-line align (alignas(64)) on the global atomics
s_privileged_tidstamp / s_running / s_max_c_obs to avoid
false-sharing under N=128 contention.
- Removed KAME_STM_TAG_ON_DISTURB ablation knob: the disturb-path
tag_as_contender is now unconditional (required for the age-preempt
visibility invariant on every contended linkage).
- kame.pri: bump CONFIG from c++11 → c++17. The STM core extensively
uses C++17 inline static class members and inline thread_local
variables (already accepted as extensions by clang under -std=c++11
but warning-free under c++17).
- kame app + modules: use std::allocator instead of the legacy
lock-free pool. `DEFINES += USE_STD_ALLOCATOR` is added to
`kame.pri` (modules inherit via `include(../kame.pri)`). The pool
predates the current STM design; std is preferred for portability
and correctness with the new fair-mode / RAII-negotiate machinery.
Tests retain the pool allocator (via `tests/CMakeLists.txt`'s
`USE_KAME_ALLOCATOR=ON` default on macOS/Linux) so the pool path
stays exercised. Override per-build with
`cmake .. -DUSE_KAME_ALLOCATOR=OFF`.
* Thu Apr 17 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- TLA+ BundleUnbundle (Layer 2): systematic comparison of fine-grained model
against C++ transaction_impl.h. Identified 9 differences; 6 implemented as
"superfine" mode (C++-faithful failure handling), 1 in fine (#3 no-rollback),
2 documented as fidelity notes (#7 serial reuse, #8 serial source).
superfine covers: pre-bundle serial CAS (#1), inner bundle phases (#2),
Phase3 serial/parent DISTURBED check (#4), casTargets root-first order (#5),
Phase1 retry-same-child (#6). These are C++ performance optimizations that
don't affect protocol correctness (verified by fine-mode model checking).
- TLA+ BundleUnbundle: Phase3 fine failure no longer rolls back children (#3).
Matches C++ behavior: re-collection re-adopts bundled children via
CollectSubpacket's bundledBy==node path. State space unchanged (1.2M).
- TLA+ BundleUnbundle: MaxSerial cannot be replaced by natural-number serials.
Commit retry loops (iterBudget not consumed) create unbounded serial growth
without modular wrap-around; state space becomes an infinite DAG.
* Wed Apr 16 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- TLA+ BundleUnbundle (Layer 2): fix BundlePhase3 fine-grained allDone check.
Previous check (~hasPriority) incorrectly counted children bundled by OTHER
threads as done. Now verifies children match BundledRefWrapper(node, ser)
for THIS bundle's serial. Found via 2-level fine model checking.
- TLA+ BundleUnbundle: increase MaxSerial from 128 to 1024 (3-level) and
512 (2-level). Fine-grained mode generates ~64 serials per commit cycle
due to CAS retries; MaxSerial/2 must exceed worst-case consumption.
- TLA+ atomic_shared_ptr (Layer 0): add iterBudget for deterministic
termination. Per-thread operation counter guards new operations but not
Reset (must release references). TerminalCheck invariant verifies final
state: installed object has refcount=1, all others freed.
* Mon Apr 14 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (8.1)
- Fix XDriver documentation: timeAwared() is acquisition start time, not
operator-visible time. Corrected in README.md, CLAUDE.md, kame_python_api.md.
- License changed to GPL2+ from LGPL2+, in most of codes.
* Wed Apr 09 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- TLA+ formal verification: apply Gemini-identified model fixes.
atomic_shared_ptr: fix overflow guard deadlock, multiset thr_holds for
correct repeated-scan refcounting, raise MaxGlobalRC in step3 cfg.
stm_commit: fix CommitSerializes invariant (MAX→SUM).
BundleUnbundle: drop modular serial arithmetic, use monotonic increment
with StateConstraint (consistent with 2-level model).
- TLA+ atomic_shared_ptr: simplify reserve_scan_ two-step read into
single atomic step (matches C++ and GenMC models).
* Tue Apr 08 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (8.1)
- Arbitrary mask contour lines: non-highlighted state draws left/right
boundary lines with median-based xmin/xmax per band (~128 bands),
closed at top/bottom. Lines break at empty bands.
Highlighted state: filled texture overlay (unchanged).
- Picking for arbitrary mask uses quad approximation (no texture),
fixing broken text textures caused by GL texture misuse in pick pass.
- Label placed at mask top-center for arbitrary masks.
- Rect BB suppressed for arbitrary mask (contour replaces it).
- Math tool API: rename Begin/End to First/Last (inclusive endpoint naming).
1D tools: first()/last(), 2D tools: firstX()/firstY()/lastX()/lastY().
.kam backward compatibility via _KamNode aliases in xpythonsupport.py.
- Add XGraph2DMathTool::setArbitraryMask() — atomically sets MaskType to
Arbitrary and writes the mask bitmap in one iterate_commit.
Exposed to Python via pybind11 on the registered concrete types.
- Register C++ concrete 2D math tool types (XGraph2DMathToolSum,
XGraph2DMathToolAverage) so Python dynamic_cast() resolves properly.
- Add imageWidth()/imageHeight() to X2DImagePlot Python bindings.
- Fix OSO ghost artifacts on tool release: OnScreenObject::invalidate()
flag checked by isValid(); clearOnScreenObjects() invalidates before clear.
- Fix Snapshot-based update() in XGraph1DMathToolX/XGraph2DMathToolX:
read functor and mask from Snapshot(*this) instead of tr[*this] to
prevent driver transaction CoW from overwriting user-set mask/MaskType.
- Arbitrary mask highlight: render as GL alpha texture on single quad
(eliminates overlapping-quad butterfly artifacts). Y-axis inversion
handled via screen-space comparison. Depth test disabled during draw.
- Fix +1 dimension mismatch between OSO display and computation ROI
in updateAdditionalOnScreenObjects.
- Fix libpng sRGB warning: strip color space from grayscale QImage in to_png().
- MCP server: streamline to 6 tools (remove read_node, set_node, read_scalar,
list_children, list_scalars); replace list_children with recursive tree tool;
compact kame_status output.
- Fix xpythonsupport.py: MCP venv path detection (cross-platform, search up
from KAME_ResourceDir); fix _sys -> sys typo.
* Mon Apr 07 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- 2D math tools: add configurable mask shape (Rectangle / Ellipse / Arbitrary).
Each XGraph2DMathTool now has an XComboNode maskType() selecting the
mask applied within the rectangular selection. Ellipse inscribes an
ellipse; only pixels inside it contribute to Sum/Average/etc.
Arbitrary lets Python set the mask via Payload.setMask()/mask().
Mask is stored in Payload::m_mask; pixels() counts from the stored mask
for all shapes. regenerateMask(tr) called on coord/type changes.
Connector popup adds a "Mask Shape" submenu per tool.
OnScreenRectObject::EllipseTool draws the inscribed ellipse (48 segs).
Python 2D functors now receive a 7th arg (mask: uint8 numpy array).
- 2D math tool highlight: OnPlotMaskObject renders mask-based highlight
overlay using horizontal row-spans, correctly visualising all shapes.
- Fix: OnPlotObject/OnAxisObject z-offset was never applied (for-loop
iterated by value instead of reference).
- Fix: .kam loading hang when creating Graph2DMathTool — MaskType combo
items were populated via the outer measurement transaction, which could
not see the non-transactionally created child node.
* Mon Apr 06 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (8.0)
- Smart window placement: new driver/graph windows cascade automatically on
first show via FrmKameMain::eventFilter. QForm template auto-installs the
filter for all top-level windows. showNormal()+raise() still used for
restoring minimized windows on driver list click.
- NI USB-GPIB: fix missing mutex_unlock on early ENOMEM return in ni_usb_write;
return -EIO instead of 0 on short bulk write.
- NI USB-GPIB: prevent duplicate interrupt URB polling thread in win_compat.h.
- Version bump to 8.0.
- RawStreamRecordReader: auto-rewind at EOF on forward playback so FastForward
works without pressing First; add NULL check for m_pGFD in execute thread.
- MCP server for AI-assisted experiment automation (kame_mcp_server.py).
Connects to the embedded IPython kernel via jupyter_client, exposing
10 tools: kame_api, execute_code, execute_code_async, get_result,
read_node (bulk comma-separated paths), set_node, read_scalar,
list_children, list_scalars, kame_status.
execute_code returns matplotlib plots as MCP ImageContent (displayed
inline in Claude Code CLI/desktop/web); %matplotlib inline is set
automatically on first kernel connection.
execute_code_async runs long experiments (sweeps, scans) in a background
thread; results stored in kernel globals, polled via get_result.
Kernel connection is reused across tool calls (was reconnecting each time).
Tool-generated code uses IPython expression results (not print) to avoid
KAME's HTML stdout redirect; HTML object reprs are filtered out.
Includes kame_python_api.md reference with correct Snapshot/node patterns,
Root children table, and XScalarEntry["Value"] access.
When launching a Jupyter notebook, KAME auto-generates .mcp.json in the
notebook workspace with auto-detected Python path for zero-configuration
Claude Code integration. No side effects when MCP is not used.
Graceful error message when mcp/jupyter_client packages are missing.
* Wed Apr 02 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (8.0-beta3)
- XCalibratedEntry: add FLAG_AVOID_DUP to onSelectionChanged listener so that
duplicate deferred events from Source and Curve onValueChanged are coalesced.
Fixes proxy being released and recreated when both fire in quick succession
(e.g. during .kam loading), which cleared the axis selector's resolved value.
- XPointerItemNode: use fresh list snapshot for pending-label resolution too
(not just re-broadcast), so entries from nested insertions are visible.
- Comprehensive bug audit: 20 fixes across 12 files.
- xnodeconnector: fix bitwise OR (|) that should be logical OR (||) in popup status check.
- xnodeconnector: fix uninitialized m_pWindow when constructor receives nullptr.
- xnodeconnector: prevent QColorDialog signal from being connected multiple times.
- xnodeconnector: NULL -> nullptr in disconnect() call.
- tds: fix array underflow yin[ch_cnt-1] when ch_cnt==0; reorder NR_P parsing before YMU/YOF.
- tds: increase snprintf format buffer from 9 to 16 bytes.
- xpythonsupport: scope GIL release to blocking wait only in kame_mainthread, fixing Python
object operations (status.is(), PyErr_SetString) performed without GIL.
- xpythonsupport: initialize *ret = py::none() before try block in mainthread_callback to
prevent returning uninitialized py::object on exception.
- xpythonsupport.py: fix thread ID comparison — use native_id instead of str(Thread).
- xpythonsupport.py: skip comment lines during .kam Ruby-to-Python translation.
- pythondriver: re-enable GIL acquire in Payload::local() (was commented out).
- pythondriver: null-check shared_ptrs before pushing to garbage queue in Payload destructor.
- secondarydriverinterface: bounds-check std::find() result before dereference in onItemChanged.
- allocator_prv: add virtual destructor to PoolAllocatorBase.
- analyzer: null-check m_entries.lock() in XValGraph (onAxisChanged, onVisualization, onStoreChanged)
and XCalibratedEntry (onSelectionChanged).
- analyzer: capture m_entry before retry loop in onSourceDriverRecord to avoid race condition.
- cyfxusb_libusb: safe bounds check for USB string descriptor buffer.
- lecroy: cast count to size_t in buffer size calculation to prevent integer overflow.
- xpythonmodule: catch pybind11::cast_error in payload down-caster loop.
- tests/CMakeLists: fix test name typo (negotioation -> negotiation).
* Tue Apr 01 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- XPointerItemNode: lsnOnListChanged now takes a fresh list snapshot before forwarding the
onListChanged event, so that entries created by nested signal handlers (e.g. CalibratedEntry
proxy inserted during pending-label resolution) are visible to FLAG_AVOID_DUP listeners
such as axis combo boxes.
- Event: added isStrictlyNewerThan() and operator== on Snapshot for more precise FLAG_AVOID_DUP
deduplication (prefer new event when serials are equal).
* Tue Mar 31 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.9.1)
- (8.0-beta2)
- STM: SerialGenerator serial layout changed to 48-bit counter (upper) + 16-bit thread ID (lower);
gen(last_serial) advances the thread-local counter past the last committed serial of the node
being read, so snapshot serials carry temporal information about the data they observe.
Snapshot::operator< and Snapshot::serial() added for temporal ordering.
Event::serial() exposes the snapshot serial for signal ordering (SFINAE fallback to 0 for
non-Snapshot talkers).
- .kam loading: fixed crash when createByTypename() returns None for unrecognised driver types
(version skew); silently falls back to _KamFakeNode so loading continues.
- kame.pro: commented out for 4res in mac.
* Sun Mar 29 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.9)
- (8.0-beta1)
- CalibratedEntry: new XCalibratedEntry node derives a scalar entry from another entry via a calibration curve; proxy entry is inserted into XScalarEntryList and works with TextWriter and graph XYZ selectors.
- CalibrationTable: cspline calibration table is now fully editable in the UI; log-scale axes for XResistanceThermometer; XGenericCalibration added.
- XPointerItemNode: deferred label resolution — autoselects item after list is populated (e.g. after .kam loading).
- Python: fixed GIL freeze on startup (M4 / fast machines); Python thread releases GIL while waiting for driver modules to load before running xpythonsupport.py.
- Python: IPython kernel initialisation is faster again; deferred scripts (pydrivers.py etc.) run on first kame_pybind_one_iteration() tick after the kernel is up.
- Scripting: fixed potential infinite lock in Ruby/Python scripting threads under heavy STM collision.
- Stability: fixed crash during .kam loading under heavy STM collision — stale QToolButton pointer in XWaveNGraph now guarded with QPointer.
- Stability: replaced try/catch(NodeNotFoundError) containment checks with shot.isUpperOf() to avoid masking outer exception handlers.
- Graph: X/Y/Z axis selectors (combo boxes showing scalar entries) are now embedded directly in each graph window.
- CalibratedEntry: new UI panel to manage source/curve selection for calibrated entries.
- UI: driver list column widths resize to contents.
- .kam loading: Python-based loader replaces Ruby when Python is available; significantly faster due to lower-latency main-thread dispatch via kame_mainthread() vs. Qt event queue.
* Thu Mar 26 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.8.1-1)
- usermode-linux-gpib: Fix for read with eos and serial poll. Now working well.
- jupyter: Stop button now works; routes SIGINT to the KAME process instead of the dummy subprocess.
- jupyter: Restart button no longer breaks the connection to the embedded kernel (async override + re-patch).
- usermode-linux-gpib: Fix for USB bulk transfer stalls on macOS; 40ms sleep after interrupt poll timeout reduces event-lock contention.
- DSO: MathTool support added.
- Graph: thicker data trace lines, neutral night-mode background, improved colour palette, grid alpha hierarchy (major/minor), wider font size range.
- 2D image display: gamma correction now applied on screen matching dump output; gamma slider works via Metal/sRGB-aware pre-encoding.
- UI: tab order corrected in dsoform, networkanalyzerform, nmrpulseform; QGroupBox stylesheet; English grammar fixes in several UI strings.
* Wed Mar 25 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.8.0-1)
- gpib: In OSX, NI USB-GPIB-HS(+) can work without kernel driver. usermode linux-gpib was embedded.
- gpib: many improvements in Prologix USB-GPIB.
- MathTools: storing/loading in .kam is possible.
- small bug fixes by claude code.
* Wed Mar 18 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.7.2-1)
- python: fix deadlock using local() in Payload.
- Lakeshore 340: fix for a power range combbox.
* Thu Mar 12 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.7.1-1)
- python: fix bool() and float() conversions from XBoolNode, XDoubleNode.
- QCheckBox: suppress foolish click at inactive window.
- Ocean Optics/Insight spectrometers: HR2000 works.
- Scalar entries from MathTools: the same timestamp as those from driver's entries.
* Fri Jan 30 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.7.0-1)
- tcp: Proper timeout on connection, reading, writing., sharing the same EOS setting with serialport.
- tcp: Faster and stable reading.
- interfacelistconnector: suggests serial ports in popup menu.
- cyfxusb: Discovers newly connected USB devices when "Device" combobox is clicked.
- cyfxusb: Detects disconnection and reconnection.
- cyfxusb_win32: fallbacks to libusb if no device found with ezusb.sys/cyusb3.sys.
- cyfxusb_libusb: fix abort behaviour during concurrent async read to the same ep.
- cyfxusb_libusb: several work-a-round for WinUSB device, but not working yet.
- xitemnodebase, xqcomboxconnector: onItemRefreshRequested().
- usertempcontrol: SI930X, worked with TCP/IP.
- opengl: GL_TRIANGLE_FAN instead of GL_QUADS
* Fri Jan 16 2026 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.6.0-1)
- Graph: popup and highlighting on math tools.
- Ocean Optics/Insight spectrometers: works with >0.5s exposure. USB2000 works.
* Mon Dec 22 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.5.0-1)
- Graph: fix for cheap intel integrated graphics. GLselect no longer used. Changing to color picking technique.
* Sun Dec 14 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.4.0-1)
- Win32: default compiler is now changed to clang++, with llvm-mingw64 build of Qt.
- win32-clang-g++: use dllexport instead of export-all-symbols.
- Fujikin MFC: suppressing communication errors for recent MFC.
- SI930* temp. monitor.: experimental yet.
- Ruby: fix for possible crash after touch().
- ODMR: many improvements and 2D imaging.
- Prologix: putting waits for better stability.
* Wed Oct 15 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.3.1-1)
- Fix for thread safety regarding math tools.
- Lakeshore 218 temp monitor
- Many improvements on ODMR
* Tue Jul 29 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.3.0-1)
- Displays fitted curves for math tools.
- Fix for compilation issues with some versions of pybind11 and ruby.
- Many improvements on digital camera and ODMR
* Tue May 27 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.2.5-1)
- Improved terminating procedure/close menu behavior.
- Fix for Prologix GPIB for drivers without serial poll.
- Scanner variants for Lakeshore 370/372
- better channel configs in Lakeshore 350/370/372
- Fix for status update in PROT.
* Wed Apr 2 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.2.4-1)
- 7.2.3 may crashe in race condition.
* Tue Jan 28 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.2.3-1)
- 7.2.2 crashes in win32.
* Sat Jan 25 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.2.2-1)
- hyperlinks in IPython messages.
- Better icons.
* Mon Jan 20 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.2.1-1)
- Redirecting stdout/err to display area in notebook.
* Sun Jan 19 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.2.0-1)
- The first MDI child window is now for IPython.
* Sat Jan 18 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.8-1)
- Binding for Jupyter notebook.
* Sat Jan 18 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.7-1)
- Fix for stdio behvior of python line interpreter.
* Wed Jan 15 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.6-1)
- Support for IPython kernel and popen for Jupyter console/qtconsole.
* Sat Jan 11 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.5-1)
- Fix for 4-terminal method.
- Python driver: 4-terminal method with multi-setting.
* Tue Jan 7 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.4-1)
- Fix for mathtool and multi-entry support
- pdb can be used under the terminal
- Python driver: 4-terminal method.
* Thu Jan 2 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.3-1)
- Ruby 3.1 works well with mac(in debug mode) and mingw64.
* Wed Jan 1 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.2-1)
- Fix for init. issue in ruby >= 3., fine with 3.3 in Mac.
- Ruby 3.1 works with mingw64 when launched from Qt creator.
- Selectable scripting shell.
* Wed Jan 1 2025 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.1.0-1)
- Compiles with ruby 3.3. Works with 3.2 in Mac.
- Python scripting for several drivers, 1D/2D Math tools.
* Sat Nov 23 2024 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.0.2-1)
- Fix for terrible bug.
* Fri Nov 22 2024 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (7.0.0-1)
- Python scripting is now possible.
- Prologix GPIB-USB controler, set default for GPIB when NI488.2 is not found.
- Fix for bugs in JAI.
- Experimental: Lakeshore M81
* Thu Oct 17 2024 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (6.1.1-1)
- Euresys: fix for startup
* Sun Oct 6 2024 Kentaro Kitagawa <kitag@issp.u-tokyo.ac.jp>
- (6.1.0-1)
- CopperMountain: fix for byte order issue for newer revisions.
- Oriental Motor CVD2B/5B STM drivers
- Filter wheel driver and color imager
- Fix stability/usablitiy for math tools
- Fix for digital camera, text collapse issue, aspect ratio issue
- Ruby pressure scales for optical specrometer
- Ocean insight/optics USB4000
- Sigma optics PAMC104 piezo motor driver
- (experimental) LDC-37**, LDX-32**, Euresys eGrabber, Optotune ICC4C2000
* Tue Dec 19 2023 Kentaro Kitagawa <kitagawa@phys.s.u-tokyo.ac.jp>
- (6.0.0-1)
- Runs with mingw64.
- Fix for message window in win32.
- Firewire digitalcameras in maxosx.
- ODMR staffs.
- Math tools on graphs. ex. center of gravity.
- NMRPulse: displays areas for calculations
- SG: more modulation features for agilent E-series SGs.
* Tue Jun 27 2023 Kentaro Kitagawa <kitagawa@phys.s.u-tokyo.ac.jp>
- (5.8.6-1)
- foures: bug fix for multi channel simultaneous measurement
* Sun Jun 25 2023 Kentaro Kitagawa <kitagawa@phys.s.u-tokyo.ac.jp>
- (5.8.5-1)