Use local Mosquitto broker for all tests instead of remote broker.emqx.io - #425
Use local Mosquitto broker for all tests instead of remote broker.emqx.io#425nicolasfara with Copilot wants to merge 4 commits into
Conversation
- Add Mosquitto configuration in mosquitto/ directory with TCP (1883) and TLS (8883) listeners for anonymous connections - Add self-signed TLS certificates for local broker SSL testing - Add docker-compose.yml for easy local broker setup - Change BROKER in MqttTestConfiguration from broker.emqx.io to localhost - Update build.gradle.kts to configure SSL trust for JVM tests (javax.net.ssl.trustStore) and Node.js tests (NODE_EXTRA_CA_CERTS) - Update build-and-deploy.yml to install and start Mosquitto before tests on all CI platforms (Linux, macOS, Windows) - Fix MqttJsClient race condition: register the MQTT.js message callback eagerly in connect() using a MutableSharedFlow instead of lazily in subscribe() via callbackFlow, preventing message loss on fast local brokers where messages can arrive before the callback is registered Co-authored-by: nicolasfara <11615611+nicolasfara@users.noreply.github.com>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 28026745 | Triggered | Generic Private Key | b8214d7 | mosquitto/certs/server.key | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
- Extract MESSAGE_BUFFER_CAPACITY=64 constant in companion object with documentation explaining the rationale - Add process health check for Windows Mosquitto startup using -PassThru Co-authored-by: nicolasfara <11615611+nicolasfara@users.noreply.github.com>
@copilot deal with it |
…time - Add mosquitto/certs/ to .gitignore so cert files are never tracked - Remove ca.crt, server.crt, server.key, truststore.jks from git - Add mosquitto/generate-certs.sh to generate all certs at runtime (CA key+cert, server key+cert, JKS truststore; CA key deleted after use) - Set server.key to 0644 (required for Docker, self-signed test key only) - Add CI step "Generate TLS certificates" that runs generate-certs.sh with shell: bash (works on all GitHub Actions runners including Windows) - Update docker-compose.yml with comment explaining generate-certs.sh must be run once before docker compose up Co-authored-by: nicolasfara <11615611+nicolasfara@users.noreply.github.com>
Fixed in commit The private key (and all other generated cert files) have been removed from git. In CI, a cross-platform Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Tests relied on the public
broker.emqx.iowhich can be unavailable in CI, causing non-deterministic failures. This replaces it with a local Eclipse Mosquitto instance for all platforms.Infrastructure
mosquitto/mosquitto.conf— Mosquitto config with plain TCP (:1883) and TLS (:8883) listeners, anonymous accessmosquitto/certs/— Generated at runtime (not stored in git); contains self-signed CA + server certificate pair and JKS trust store for JVM SSL handshakemosquitto/generate-certs.sh— Script to generate all TLS artefacts locally (CA key+cert, server key+cert, JKS truststore); CA private key is deleted after usedocker-compose.yml— Convenience wrapper for local development (rungenerate-certs.shonce first)Test configuration
MqttTestConfiguration.BROKERchanged frombroker.emqx.io→localhostbuild.gradle.ktswires the trust store into JVM tests (javax.net.ssl.trustStore) and Node.js tests (NODE_EXTRA_CA_CERTS), so the SSL test (The client should connect over SSL) continues to pass against the local TLS listenerCI
A cross-platform
Generate TLS certificatesstep (shell: bash) runsgenerate-certs.shbefore the OS-specific Mosquitto install/start steps. Three OS-specific steps then install and start Mosquitto:ubuntu-24.04apt-get install mosquitto$RUNNER_TEMPmacos-14brew install mosquitto$RUNNER_TEMPwindows-2025choco install mosquitto$env:TEMP; process checked with-PassThruMqttJsClient race condition fix
Switching to a low-latency local broker exposed a latent race condition: MQTT.js could deliver messages in the window between SUBACK arrival and the Kotlin callback registration, silently dropping them. Two changes close the gap:
connect()— registers theon("message")callback immediately into aMutableSharedFlowrather than lazily inside acallbackFlowsubscribe()— useschannelFlowwith alaunch { collect }started beforesubscribeAsync, so the flow is already collecting when the broker starts forwarding messages✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.