From a441df904c88575b7d0f2648f5cfc3575d0fb9d1 Mon Sep 17 00:00:00 2001 From: DaveTempleman Date: Mon, 3 Aug 2026 10:02:54 +0100 Subject: [PATCH 1/9] Changed active low signals to standard active high. Made the header length shorter. reformatted to make all ..= to .. as was both types in document --- .../Datastreaming_udp_packet_formats.md | 74 +++++++++---------- 1 file changed, 33 insertions(+), 41 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 11153c723..74d2df792 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -25,41 +25,41 @@ Always `0xFFFFFFFF`. ### Word 2: Header Information -- **Bit 0..=7**: Header Flags - - **Bit 0**: End of run header marker (active low) - - **Bit 1**: Veto frame packet header marker (active low) - - **Bit 2**: Pause frame packet header marker (active low) - - **Bit 3**: No frame sync (active low) (not implemented) - - **Bit 4..=15**: Reserved for future use -- **Bits 16..=31**: PCB Board Number, encoded as an integer. e.g. for `PC1234M1S`, the board identifier would be `1234`. Ignore any `PC` prefix and any suffix such as `M1S`. +- **Bit 0..7**: Header Flags + - **Bit 0**: End of run header marker + - **Bit 1**: Veto frame packet header marker + - **Bit 2**: Pause frame packet header marker + - **Bit 3**: No frame sync (not implemented) + - **Bit 4..15**: Reserved for future use +- **Bits 16..31**: PCB Board Number, encoded as an integer. e.g. for `PC1234M1S`, the board identifier would be `1234`. Ignore any `PC` prefix and any suffix such as `M1S`. ### Word 3: GPS timestamp -- **Bits 0..=3**: seconds (most significant bits; combine with least significant bits from word 5) -- **Bits 4..=9**: minutes -- **Bits 10..=14**: hours -- **Bits 15..=23**: days -- **Bits 24..=31**: years (as offset from year 2000) +- **Bits 0..3**: seconds (most significant bits; combine with least significant bits from word 5) +- **Bits 4..9**: minutes +- **Bits 10..14**: hours +- **Bits 15..23**: days +- **Bits 24..31**: years (as offset from year 2000) ### Word 4: GPS timestamp -- **Bits 0..=9**: nanoseconds -- **Bits 10..=19**: microseconds -- **Bits 20..=29**: milliseconds -- **Bits 30..=31**: seconds (least significant bits; combine with most significant bits from word 4) +- **Bits 0..9**: nanoseconds +- **Bits 10..19**: microseconds +- **Bits 20..29**: milliseconds +- **Bits 30..31**: seconds (least significant bits; combine with most significant bits from word 4) ### Word 5: Frame number -- **Bits 0..=31**: frame number as u32 +- **Bits 0..31**: frame number as u32 ### Word 6: period number -- **Bits 0..=15**: Period number -- **Bits 16..=31**: Frame repeat number +- **Bits 0..15**: Period number +- **Bits 16..31**: Frame repeat number ### Word 7: events in frame -- **Bits 0..=31**: number of neutron events in this frame. +- **Bits 0..31**: number of neutron events in this frame. :::{note} This is not necessarily the same as the number of events in this UDP message, as the events may be split between @@ -70,15 +70,15 @@ See header word 8 if looking for length of *this* message. ### Word 8: packet length & protons-per-pulse -- **Bits 0..=7**: protons-per-pulse in this ISIS frame. +- **Bits 0..7**: protons-per-pulse in this ISIS frame. - **Bits 16..27**: number of 32-bit words from the beginning of this header to the start of the next header. -- **Bits 28..=31**: unused +- **Bits 28..31**: unused To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math}`1.738{\times}10^{-6}`. {#ds_veto_bit_definitions} ### Word 9: vetoes -- **Bits 0..=15**: Common vetoes +- **Bits 0..15**: Common vetoes - **Bit 0**: Data Overflow veto - **Bit 1**: SMP veto - **Bit 2**: TS2 pulse veto @@ -90,31 +90,23 @@ To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math - **Bit 8**: Dwell Period veto - **Bit 9**: Status Packet CRC fault veto - **Bits 10..=15**: Reserved for future use -- **Bits 16..=31**: Instrument Specific Vetoes - - **Bits 16..=19**: External vetoes - - **Bits 20..=23**: Fast chopper vetoes (Fermi) - - **Bits 24..=31**: Reserved for future use +- **Bits 16..31**: Instrument Specific Vetoes + - **Bits 16..19**: External vetoes + - **Bits 20..23**: Fast chopper vetoes (Fermi) + - **Bits 24..31**: Reserved for future use ### Word 10: next frame address -- **Bits 0..=31**: Address of the next frame, in bytes. +- **Bits 0..31**: Address of the next frame, in bytes. ### Word 11: streamed frame number -- **Bits 0..=31**: streamed frame number. +- **Bits 0..31**: streamed frame number. -### Word 12: checksum +### Word 12: unused -- **Bits 0..=31**: Pre-DDR checksum +- **Bits 0..31**: Reserved for future use -### Word 13: unused +### Word 13: checksum -- **Bits 0..=31**: Reserved for future use - -### Word 14: unused - -- **Bits 0..=31**: Unused - -### Word 15: unused - -- **Bits 0..=31**: Unused +- **Bits 0..31**: Pre-DDR checksum From dd198eec92cad202752c0c727768d8c5812c931e Mon Sep 17 00:00:00 2001 From: DaveTempleman Date: Mon, 3 Aug 2026 16:36:08 +0100 Subject: [PATCH 2/9] Revise UDP packet format for board-specific parameters Updated Word 12 and Word 13 to include board-specific parameters and removed unused sections. --- .../Datastreaming_udp_packet_formats.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 74d2df792..b84d833dd 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -103,10 +103,21 @@ To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math - **Bits 0..31**: streamed frame number. -### Word 12: unused +### Word 12: board-specific parameters 0 -- **Bits 0..31**: Reserved for future use +- **Bits 0..7**: pos_bits_per_ch - how many positional bits needed for each channel (only need 5 bits for 32 bits) +- **Bits 15..8**: diag_bits_per_ch - how many bits needed diagnostic data for each channel (only need 5 bits for 32 bits) +- **Bits 16..23**: channel_bits - how bits for channel (make this the most significant bits) +- **Bits 24..31**: board_address - which board in the system -### Word 13: checksum +### Word 13: board-specific parameters 1 + +- **Bits 0..31**: detector_id_offset + +### Word 14: unused + +- **Bits 0..31**: unused + +### Word 15: checksum - **Bits 0..31**: Pre-DDR checksum From 267a440f9c416c43728d7eb262a933a4d6403334 Mon Sep 17 00:00:00 2001 From: DaveTempleman Date: Tue, 4 Aug 2026 11:21:56 +0100 Subject: [PATCH 3/9] Add the 'time channel delay' into the header --- .../Datastreaming_udp_packet_formats.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index b84d833dd..6d77e8f00 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -10,7 +10,8 @@ trailing zero-padding bytes, but these may be added in future and should be igno ## Header format -The header is 16 words (64 bytes). +The header has an adjustable length with words 0 to 12 have fixed functionality and the final packet will be the DDR checksum. +Words in between these will be board specific and their functionality defined in the header type. ### Word 0: marker word @@ -103,21 +104,21 @@ To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math - **Bits 0..31**: streamed frame number. -### Word 12: board-specific parameters 0 +### Word 12: Frame sync delay + +- **Bits 0..31**: Delay of the frame sync from Time Of Flight pulse. + +### Word 13: board-specific parameters 0 - **Bits 0..7**: pos_bits_per_ch - how many positional bits needed for each channel (only need 5 bits for 32 bits) - **Bits 15..8**: diag_bits_per_ch - how many bits needed diagnostic data for each channel (only need 5 bits for 32 bits) - **Bits 16..23**: channel_bits - how bits for channel (make this the most significant bits) - **Bits 24..31**: board_address - which board in the system -### Word 13: board-specific parameters 1 +### Word 14: board-specific parameters 1 - **Bits 0..31**: detector_id_offset -### Word 14: unused - -- **Bits 0..31**: unused - ### Word 15: checksum - **Bits 0..31**: Pre-DDR checksum From 2e00227bc0fcddc03219f9157c2b0b62a5b793b6 Mon Sep 17 00:00:00 2001 From: DaveTempleman Date: Tue, 11 Aug 2026 13:56:17 +0100 Subject: [PATCH 4/9] Update header flags and vetoes descriptions Clarified the description of Bit 1 in the header flags and added a note about veto values. --- .../datastreaming/Datastreaming_udp_packet_formats.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 6d77e8f00..a9fcb3b3f 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -28,7 +28,7 @@ Always `0xFFFFFFFF`. - **Bit 0..7**: Header Flags - **Bit 0**: End of run header marker - - **Bit 1**: Veto frame packet header marker + - **Bit 1**: Veto frame packet header marker - This show that the frame has been hard vetoed. If a mask is set to cover this corresponding but this won't be set. - **Bit 2**: Pause frame packet header marker - **Bit 3**: No frame sync (not implemented) - **Bit 4..15**: Reserved for future use @@ -79,6 +79,7 @@ To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math {#ds_veto_bit_definitions} ### Word 9: vetoes +These values be even if there is a veto mark that corresponds to them - **Bits 0..15**: Common vetoes - **Bit 0**: Data Overflow veto - **Bit 1**: SMP veto From 1953cb397f4d66ce8e66af27406f90e38bed08cf Mon Sep 17 00:00:00 2001 From: DaveTempleman Date: Fri, 14 Aug 2026 09:45:14 +0100 Subject: [PATCH 5/9] Refine UDP packet format documentation Removed unnecessary comments about maximum values and clarified wording in the UDP packet format documentation. --- .../Datastreaming_udp_packet_formats.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index a9fcb3b3f..c91930730 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -20,15 +20,15 @@ Always `0xFFFFFFFF`. ### Word 1: header information -- **Bits 0..7**: Length of header, in 32-bit words (max 255 *too many?*) -- **Bits 8..23**: Header Type (max 65535 *too many?*) -- **Bits 24..31**: Marker - Always `0xFF` *(think it will be useful to keep extra Fs to show start of header but not sure if necessary?)* +- **Bits 0..7**: Length of header, in 32-bit words +- **Bits 8..23**: Header Type +- **Bits 24..31**: Marker - Always `0xFF` *(This will be useful to keep extra Fs to show start of header but not sure if necessary?)* ### Word 2: Header Information - **Bit 0..7**: Header Flags - **Bit 0**: End of run header marker - - **Bit 1**: Veto frame packet header marker - This show that the frame has been hard vetoed. If a mask is set to cover this corresponding but this won't be set. + - **Bit 1**: Veto frame packet header marker - This show that the frame has been hard vetoed. If a mask is set to cover this corresponding bit this won't be set. - **Bit 2**: Pause frame packet header marker - **Bit 3**: No frame sync (not implemented) - **Bit 4..15**: Reserved for future use @@ -72,7 +72,7 @@ See header word 8 if looking for length of *this* message. ### Word 8: packet length & protons-per-pulse - **Bits 0..7**: protons-per-pulse in this ISIS frame. -- **Bits 16..27**: number of 32-bit words from the beginning of this header to the start of the next header. +- **Bits 16..27**: number of 32-bit words from the beginning of this header to the start of the next header or till the end of this UDP packet. - **Bits 28..31**: unused To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math}`1.738{\times}10^{-6}`. @@ -91,7 +91,7 @@ These values be even if there is a veto mark that corresponds to them - **Bit 7**: Period overflow veto pause veto - **Bit 8**: Dwell Period veto - **Bit 9**: Status Packet CRC fault veto - - **Bits 10..=15**: Reserved for future use + - **Bits 10..15**: Reserved for future use - **Bits 16..31**: Instrument Specific Vetoes - **Bits 16..19**: External vetoes - **Bits 20..23**: Fast chopper vetoes (Fermi) From c62c05c49be3f4f2cf892a3556a04fb4798ac624 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Tue, 18 Aug 2026 11:54:41 +0100 Subject: [PATCH 6/9] Update doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md --- .../datastreaming/Datastreaming_udp_packet_formats.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index c91930730..7963aca0c 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -112,7 +112,7 @@ These values be even if there is a veto mark that corresponds to them ### Word 13: board-specific parameters 0 - **Bits 0..7**: pos_bits_per_ch - how many positional bits needed for each channel (only need 5 bits for 32 bits) -- **Bits 15..8**: diag_bits_per_ch - how many bits needed diagnostic data for each channel (only need 5 bits for 32 bits) +- **Bits 8..15**: diag_bits_per_ch - how many bits needed diagnostic data for each channel (only need 5 bits for 32 bits) - **Bits 16..23**: channel_bits - how bits for channel (make this the most significant bits) - **Bits 24..31**: board_address - which board in the system From dc3183b38e787214a71b28624d35a71d74241d52 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Tue, 18 Aug 2026 13:17:25 +0100 Subject: [PATCH 7/9] Add example --- .../Datastreaming_udp_packet_formats.md | 47 ++++++++++++++++--- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 7963aca0c..dfec68bdf 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -109,17 +109,52 @@ These values be even if there is a veto mark that corresponds to them - **Bits 0..31**: Delay of the frame sync from Time Of Flight pulse. -### Word 13: board-specific parameters 0 +### Variable number of board-specific parameters + +The interpretation of these words depends on the board number. There may be zero or more of these. The number of words of board-specific parameters is "length of header" (from word 1) minus 14. + +See {ref}`ds_board_specific_header_parameters` for interpretation of these words for different board types. + +### Last Word: checksum + +- **Bits 0..31**: Pre-DDR checksum + +--- + +{#ds_board_specific_header_parameters} +## Board-specific parameters + +### `pc3544` / `pc3634` / `pc3877` + +The scheme used by these boards is for 64-bit neutron events to look like: + +``` +1110000T TTTTTTTT TTTTTTTT TTTTTTTT +CCCxxxxx xxxxxxxx xxDDDDDD DDPPPPPP +``` + +Where: +- `1` is a bit that is *always* set to `1` +- `0` is a bit that is *always* set to `0` +- `T` is a bit that forms part of the raw timestamp +- `C` is a bit that forms part of the channel index +- `D` is a bit that forms part of 'diagnostic data' (for example pulse height) +- `P` is a bit that forms part of the 'position' (pixel) that this event corresponds to +- `x` is an unused bit + +The example above has `pos_bits_per_ch = 6`, `diag_bits_per_ch = 8`, `channel_bits = 3`. + +The detector ID sent to Kafka then needs to be `detector_id_offset + P + (C * (2^channel_bits))`. + +#### Word 13: board-specific parameters 0 - **Bits 0..7**: pos_bits_per_ch - how many positional bits needed for each channel (only need 5 bits for 32 bits) -- **Bits 8..15**: diag_bits_per_ch - how many bits needed diagnostic data for each channel (only need 5 bits for 32 bits) +- **Bits 8..15**: diag_bits_per_ch - how many bits needed diagnostic data for each channel (only need 5 bits for 32 bits) - **Bits 16..23**: channel_bits - how bits for channel (make this the most significant bits) - **Bits 24..31**: board_address - which board in the system -### Word 14: board-specific parameters 1 +#### Word 14: board-specific parameters 1 - **Bits 0..31**: detector_id_offset -### Word 15: checksum - -- **Bits 0..31**: Pre-DDR checksum +--- From 4be7d69c1687049255e1ab8b09ad39e88305786e Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Tue, 18 Aug 2026 13:19:30 +0100 Subject: [PATCH 8/9] remove final delimiter --- .../datastreaming/Datastreaming_udp_packet_formats.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index dfec68bdf..883f3a588 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -156,5 +156,3 @@ The detector ID sent to Kafka then needs to be `detector_id_offset + P + (C * (2 #### Word 14: board-specific parameters 1 - **Bits 0..31**: detector_id_offset - ---- From 43d8b7302949c750d064d8c06c205c5ea85bd3bb Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Tue, 18 Aug 2026 14:26:31 +0100 Subject: [PATCH 9/9] correct equation --- .../datastreaming/Datastreaming_udp_packet_formats.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 883f3a588..e9f732773 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -144,7 +144,7 @@ Where: The example above has `pos_bits_per_ch = 6`, `diag_bits_per_ch = 8`, `channel_bits = 3`. -The detector ID sent to Kafka then needs to be `detector_id_offset + P + (C * (2^channel_bits))`. +The detector ID sent to Kafka then needs to be `detector_id_offset + P + (C * (2^pos_bits_per_ch))`. #### Word 13: board-specific parameters 0