Skip to content

Use local Mosquitto broker for all tests instead of remote broker.emqx.io - #425

Draft
nicolasfara with Copilot wants to merge 4 commits into
masterfrom
copilot/rely-on-local-mqtt-broker
Draft

Use local Mosquitto broker for all tests instead of remote broker.emqx.io#425
nicolasfara with Copilot wants to merge 4 commits into
masterfrom
copilot/rely-on-local-mqtt-broker

Conversation

Copilot AI commented Mar 8, 2026

Copy link
Copy Markdown
Contributor

Tests relied on the public broker.emqx.io which can be unavailable in CI, causing non-deterministic failures. This replaces it with a local Eclipse Mosquitto instance for all platforms.

Infrastructure

  • mosquitto/mosquitto.conf — Mosquitto config with plain TCP (:1883) and TLS (:8883) listeners, anonymous access
  • mosquitto/certs/ — Generated at runtime (not stored in git); contains self-signed CA + server certificate pair and JKS trust store for JVM SSL handshake
  • mosquitto/generate-certs.sh — Script to generate all TLS artefacts locally (CA key+cert, server key+cert, JKS truststore); CA private key is deleted after use
  • docker-compose.yml — Convenience wrapper for local development (run generate-certs.sh once first)

Test configuration

  • MqttTestConfiguration.BROKER changed from broker.emqx.iolocalhost
  • build.gradle.kts wires the trust store into JVM tests (javax.net.ssl.trustStore) and Node.js tests (NODE_EXTRA_CA_CERTS), so the SSL test (The client should connect over SSL) continues to pass against the local TLS listener

CI

A cross-platform Generate TLS certificates step (shell: bash) runs generate-certs.sh before the OS-specific Mosquitto install/start steps. Three OS-specific steps then install and start Mosquitto:

Runner Install Config
ubuntu-24.04 apt-get install mosquitto Written to $RUNNER_TEMP
macos-14 brew install mosquitto Written to $RUNNER_TEMP
windows-2025 choco install mosquitto Written to $env:TEMP; process checked with -PassThru

MqttJsClient race condition fix

Switching to a low-latency local broker exposed a latent race condition: MQTT.js could deliver messages in the window between SUBACK arrival and the Kotlin callback registration, silently dropping them. Two changes close the gap:

  1. connect() — registers the on("message") callback immediately into a MutableSharedFlow rather than lazily inside a callbackFlow
  2. subscribe() — uses channelFlow with a launch { collect } started before subscribeAsync, so the flow is already collecting when the broker starts forwarding messages
override fun subscribe(topic: String, qos: MqttQoS) = subscribedTopics.getOrPut(topic) {
    channelFlow {
        // Collecting starts here — before the SUBSCRIBE is sent
        val job = launch {
            messageSharedFlow.filter { matchesTopicFilter(it.topic, topic) }.collect { send(it) }
        }
        client.subscribeAsync(topic).await()
        awaitClose { job.cancel() }
    }.cancellable().flowOn(dispatcher)
}

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

- Add Mosquitto configuration in mosquitto/ directory with TCP (1883)
  and TLS (8883) listeners for anonymous connections
- Add self-signed TLS certificates for local broker SSL testing
- Add docker-compose.yml for easy local broker setup
- Change BROKER in MqttTestConfiguration from broker.emqx.io to localhost
- Update build.gradle.kts to configure SSL trust for JVM tests
  (javax.net.ssl.trustStore) and Node.js tests (NODE_EXTRA_CA_CERTS)
- Update build-and-deploy.yml to install and start Mosquitto before
  tests on all CI platforms (Linux, macOS, Windows)
- Fix MqttJsClient race condition: register the MQTT.js message callback
  eagerly in connect() using a MutableSharedFlow instead of lazily in
  subscribe() via callbackFlow, preventing message loss on fast local
  brokers where messages can arrive before the callback is registered

Co-authored-by: nicolasfara <11615611+nicolasfara@users.noreply.github.com>
@gitguardian

gitguardian Bot commented Mar 8, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
28026745 Triggered Generic Private Key b8214d7 mosquitto/certs/server.key View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

- Extract MESSAGE_BUFFER_CAPACITY=64 constant in companion object with
  documentation explaining the rationale
- Add process health check for Windows Mosquitto startup using -PassThru

Co-authored-by: nicolasfara <11615611+nicolasfara@users.noreply.github.com>
Copilot AI changed the title [WIP] Use local MQTT broker for CI testing Use local Mosquitto broker for all tests instead of remote broker.emqx.io Mar 8, 2026
@nicolasfara

Copy link
Copy Markdown
Owner

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id GitGuardian status Secret Commit Filename
28026745 Triggered Generic Private Key b8214d7 mosquitto/certs/server.key View secret
🛠 Guidelines to remediate hardcoded secrets

  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@copilot deal with it

…time

- Add mosquitto/certs/ to .gitignore so cert files are never tracked
- Remove ca.crt, server.crt, server.key, truststore.jks from git
- Add mosquitto/generate-certs.sh to generate all certs at runtime
  (CA key+cert, server key+cert, JKS truststore; CA key deleted after use)
- Set server.key to 0644 (required for Docker, self-signed test key only)
- Add CI step "Generate TLS certificates" that runs generate-certs.sh
  with shell: bash (works on all GitHub Actions runners including Windows)
- Update docker-compose.yml with comment explaining generate-certs.sh
  must be run once before docker compose up

Co-authored-by: nicolasfara <11615611+nicolasfara@users.noreply.github.com>

Copilot AI commented Mar 8, 2026

Copy link
Copy Markdown
Contributor Author

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to comp...

Fixed in commit 92a2a4e.

The private key (and all other generated cert files) have been removed from git. mosquitto/certs/ is now listed in .gitignore. A new mosquitto/generate-certs.sh script generates the full certificate chain at runtime — the CA private key is deleted immediately after signing the server cert, so only the public artefacts (ca.crt, server.crt, server.key, truststore.jks) remain on disk.

In CI, a cross-platform bash step runs the script before Mosquitto is started on every OS. For local development, users run the script once before docker compose up or ./gradlew test.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • invalid.broker.com
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/mktt/mktt/build/tmp/jvmTest/work -javaagent:/home/REDACTED/work/mktt/mktt/build/tmp/.cache/expanded/zip_27729b840c216bdc69a9315e04b27ab1/jacocoagent.jar=destfile=build/jacoco/jvmTest.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath14925039421337538748txt -Xmx512m -Dfile.encoding=UTF-8 -Djavax.net.ssl.trustStore=/home/REDACTED/work/mktt/mktt/mosquitto/certs/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain &#39;Gradle Test Executor 1&#39; -j /home/REDACTED/.cargo/bin/git (dns block)
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/mktt/mktt/build/tmp/jvmTest/work -javaagent:/home/REDACTED/work/mktt/mktt/build/tmp/.cache/expanded/zip_27729b840c216bdc69a9315e04b27ab1/jacocoagent.jar=destfile=build/jacoco/jvmTest.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath13192016213541936894txt -Xmx512m -Dfile.encoding=UTF-8 -Djavax.net.ssl.trustStore=/home/REDACTED/work/mktt/mktt/mosquitto/certs/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain &#39;Gradle Test Executor 1&#39; git 4/lib/jspawnhelper -1 --format=%h nfig/composer/ve--oneline 4/lib/jspawnhelp-3 (dns block)
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/mktt/mktt/build/tmp/jvmTest/work -javaagent:/home/REDACTED/work/mktt/mktt/build/tmp/.cache/expanded/zip_27729b840c216bdc69a9315e04b27ab1/jacocoagent.jar=destfile=build/jacoco/jvmTest.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath9229891409777635792txt -Xmx512m -Dfile.encoding=UTF-8 -Djavax.net.ssl.trustStore=/home/REDACTED/work/mktt/mktt/mosquitto/certs/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain &#39;Gradle Test Executor 1&#39; --format=%h 4/lib/jspawnhelper --count 2.0.2..HEAD it 4/lib/jspawnhelp-3 (dns block)
  • scans-in.gradle.com
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED -XX:MaxMetaspaceSize=2g -Xmx2g -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants