feat(shared/statsd-exporter): Bump to 0.30.0

[NickLarsenNZ]

Part of #1479

Description

Bump airflow and superset to use statsd-exporter 0.30.0 (as agreed in slack-1).

Mirror statsd_exporter, and use Patchable (as discussed in slack-2) to pull the source instead of from Nexus.

We want to avoid mirroring source in Nexus, and instead use a git mirror and Patchable to build the software. Since statsd-exporter doesn't have an upload script (well, now it does, but this will be removed), it is a good time to fix that.

Note

I was going to leave images on 0.28.0 to ensure the change in build procedure is fine, but since #1522 was already merged, we'll skip that additional verification.
I have tested this with airflow on 0.28.0, and have merged upstream and made a few necessary changes like deleting a new file.

Test results

Airflow smoke tests rely on metrics passed through statsd.

--- PASS: kuttl (187.71s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.6,oci.stackable.tech_sdp_airflow_3.0.6-stackable0.0.0-dev_openshift-false_executor-celery (185.10s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.6,oci.stackable.tech_sdp_airflow_3.0.6-stackable0.0.0-dev_openshift-false_executor-kubernetes (187.70s)
PASS

Superset smoke tests rely on metrics passed through statsd.

--- PASS: kuttl (104.69s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_superset-6.0.0,oci.stackable.tech_sdp_superset_6.0.0-stackable0.0.0-dev_openshift-false (104.67s)
PASS

[sbernauer]

Do we still need this? Or can we use the tag/commit/ whatever we are on that is generated by patchable?
Especially important as soon as we actually start patching stuff

[NickLarsenNZ]

I didn't quite understand. This tar happens after the patchable checkout (so after any would-be patches are applied).

Did you mean something else?

FWIW, I kept the same order as OPA:

docker-images/opa/Dockerfile

Lines 76 to 84 in 1150cac

	cd "$(/stackable/patchable --images-repo-root=src checkout opa ${PRODUCT_VERSION})"
	ORIGINAL_VERSION=${PRODUCT_VERSION}
	NEW_VERSION="${PRODUCT_VERSION}-stackable${RELEASE_VERSION}"
	sed -i 's/var Version = "'${ORIGINAL_VERSION}'"/var Version = "'${NEW_VERSION}'"/g' v1/version/version.go
	# Create snapshot of the source code including custom patches
	tar -czf /stackable/opa-${NEW_VERSION}-src.tar.gz .

I just removed the SDP version bits of code in between (since this is an intermediary dependency, and I'm not sure we want to have another place to remember to update SDP versions (ie: the boil config for statsd).

[sbernauer]

I was referring to

Unfortunately, we need to create a dummy Git repository to allow cyclonedx-gomod to determine the version of statsd_exporter

That was back in the day where we downloaded and extracted a .tar.gz. But now we actually have a git repo (don't we?), so we should be able to remove this "let's init a dummy git repo" workaround.
Didn't test it though

[NickLarsenNZ]

I didn't test it, since opa does the same thing (it probably needs its thing removed too then).

IMO, this is ready to go, and that improvement can be done across the board. I'm happy to raise it straight after.

What do you think?

[sbernauer]

Sure thing, happy to merge if you promise to have a look ;)

[NickLarsenNZ]

I tried the naiive way, to no avail.

diff --git i/shared/statsd-exporter/Dockerfile w/shared/statsd-exporter/Dockerfile
index 5b0210b..66da2b8 100644
--- i/shared/statsd-exporter/Dockerfile
+++ w/shared/statsd-exporter/Dockerfile
@@ -49,13 +49,6 @@ go install "github.com/CycloneDX/cyclonedx-gomod/cmd/cyclonedx-gomod@v${CYCLONED
   # Create snapshot of the source code including custom patches
   tar -czf /statsd_exporter/statsd-exporter-${PRODUCT_VERSION}-src.tar.gz .
 
-  # Unfortunately, we need to create a dummy Git repository to allow cyclonedx-gomod to determine the version of statsd_exporter
-  rm .git
-  git init
-  git config user.email "fake.commiter@stackable.tech"
-  git config user.name "Fake commiter"
-  git commit --allow-empty --message "Fake commit, so that we can create a tag"
-  git tag "v${PRODUCT_VERSION}"
   go build -o /statsd_exporter/statsd_exporter
   $GOPATH/bin/cyclonedx-gomod app -json -output-version 1.5 -output /statsd_exporter/statsd_exporter-${PRODUCT_VERSION}.cdx.json -packages -files
 )

14.46 2:00PM ERR error="failed to determine version of main module: reference not found"

Maybe we still need a tag, which then needs all that extra git stuff. Not sure it's worth it right now :/