Add (semi)automated integration testing#47
Conversation
- version 0.26.1 defaults to skipping both firefox and chrome - search paths for firefox profiles are hardcoded relative to the running user's home directory
Currently tests are executed on: - Ubuntu 24.04 - Debian 13 targeting only Firefox.
Versions of the tool differ between Ubuntu 24.04 and Debian 13 with slight modifications to the interface, necessitating conditional checks to determine how to correctly call it. Additionally, it proved useless for Chrome because of a hard-coded profile path which doesn't meet the reality of where Chrome actually stores its profile. In the end, it made sense to use modutil to register the cac module across all identified profiles, to include Chrome and both versions of Firefox.
|
I completely agree. I really appreciate your work on this! I'll start by saying that I'm happy to integrate this. I looked at using Vagrant in the past. The only thing that dissuaded me was that most user issues stem from some quirk of a service or browser, which has alway required interactive troubleshooting since you need a CAC. I've had multiple people successfully configure their machine with the script, but 1 out of 10 will have some issue where the browser doesn't properly query the cert DB or request the pin as expected. To remediate, we would run specific steps manually to rebuild where I thought the issue occurred. A CI test would indicate a successful configuration, but in those specific cases, the process failed. That being said, some testing is always better than none, so thank you again.
I will review and merge unless I have a question. Please be patient with me as I have a newborn. I'll try to get it merged as soon as I have free time. |
|
Congrats! I completely understand and it's no rush at all :) There is just too much variation in desktop configuration to definitively account for all edge cases. Perhaps for another weekend, I was thinking of adding options for dry run and specifying profile paths to modify explicitly if desirable. Also I'm curious if there would be any benefit in maintaining a deb package for the CA certs (e.g., following the example in |
CI testing would be ideal but I found most approaches to be cumbersome given the direct involvement of
systemd. The Vagrant approach in this PR accomplishes more-or-less automated testing by running the script on Ubuntu 24.04/Debian 13 instances and calling BATS tests to determine thatThere is probably a better way to isolate re-running the script on a provisioned system's clean state in the
Vagrantfilebut I don't yet know enough ruby/vagrant to pull that off so a typical workflow is:In case of a failure in a test, vagrant process quits but it's possible to
because provisioned boxes continue running and determine exactly what happened but once fixed the entire procedure has to repeat to ensure a clean state.
On a separate note, during testing I found that
pkcs11-registeris really only useful for apt-installed Firefox (and even then there are annoying interface differences between versions in upstream Ubuntu and Debian repos) and decided to switch entirely tomodutilfor module registration (which also proved cleaner as it was already being used for snapped FF).