Skip to content

Support UTF-8 SMS submission using UCS-2 - #21

Open
dormancygrace wants to merge 1 commit into
obsy:masterfrom
dormancygrace:agent/sms-submit-ucs2
Open

Support UTF-8 SMS submission using UCS-2#21
dormancygrace wants to merge 1 commit into
obsy:masterfrom
dormancygrace:agent/sms-submit-ucs2

Conversation

@dormancygrace

Copy link
Copy Markdown

Summary

  • strip a leading + before semi-octet phone-number encoding and validate all destination digits
  • automatically encode non-ASCII UTF-8 messages as UCS-2 with the single-message 70-character limit
  • reject malformed UTF-8, unsupported non-BMP characters, invalid numbers, and oversized messages
  • propagate PDU and modem submission errors instead of continuing with invalid state
  • add exact GSM-7 and UCS-2 SMS-SUBMIT test vectors

This addresses the missing send-side support discussed in #12 and #14. ASCII messages keep the existing GSM-7 path.

Validation

  • make -C pdu_lib test
  • PDU encoder tests under ASan and UBSan
  • cross-compiled for OpenWrt MIPS32r2/musl
  • installed and exercised on a physical Fibocom NL678 modem; an international-number Cyrillic message was accepted by the modem as +CMGS: 8

The hardware test used the same Cyrillic text as the committed UCS-2 test vector.

Encode non-ASCII UTF-8 input as UCS-2.

Normalize leading plus signs in international numbers.

Reject malformed input instead of submitting a broken PDU.

Add deterministic ASCII and UCS-2 submit test vectors.

Make the send path propagate encoding and modem errors.
@dormancygrace

Copy link
Copy Markdown
Author

Hardware delivery follow-up: both UTF-8/UCS-2 test messages were confirmed received, not merely accepted by the modem.

Tested on a physical Fibocom NL678 with two international destinations (+7... and +972...). The modem returned +CMGS: 8 and +CMGS: 9; both recipients received the exact Cyrillic text.

@obsy

obsy commented Aug 10, 2026

Copy link
Copy Markdown
Owner

I will test it with our (Polish) diacritics and let you know how it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants