Most images don't load using 0.13.8

[makinori]

Background information

• Dendrite version or git SHA: 0.13.8+763c79f
• SQLite3 or Postgres?: Postgres (15-alpine)
• Running in Docker?: Yes
• go version: sh: go: not found
• Client used (if applicable): Element on Web, Element X, Cinny

Description

• What is the problem: Requested images return no data, but status 200 OK
• Who is affected: Single user
• How is this bug manifesting: Images rarely or never load in when I'm using the above clients
• When did this first appear: Since the 0.13.8 update

I've tried resetting all my clients too. Example below returns Content-Length: 144 with Content-Type: application/json without data. Couldn't find any relevant logs either.

Steps to reproduce

• Setup Dendrite monolith using Docker
• Use any of the above clients

See images below

[makinori]

Seems like images might be coming through now on multiple devices. Maybe it's related to some sort of cache in Dendrite

[makinori]

This is still happening. Every time I restart any client, all images are lost but over time they start to appear. Very strange

[makinori]

#3424 might have helped but still not fully fixed

[makinori]

Seems like the fix is to just refresh element a few times and the image requests slowly turn from content-type json to jpeg

[S7evinK]

dendrite/mediaapi/routing/routing.go

Line 204 in 6cd1285

w.Header().Set("Content-Type", "application/json")

Is the only place where we actively set the Content-Type to application/json. I couldn't find a reason why we would return HTTP 200 but still have application/json set, yet.

[pat-s]

Since some weeks I also have this issue. Not exactly sure when it started, but reverting to 0.13.7 didn't help.

I see the following in the logs:

 time="2024-10-06T11:04:39.793349630Z" level=error msg="failed to open file" MediaID=VCJpgjatIIsIuTzgLKwjBiYx Origin=obermui.de error="os.Open: open /data/media_store/-/Q/YSg8KmsDhugLEHnnAShX7wSr2kgmuS │
│ 8BYCE7qJLWo/file: no such file or directory" req.id=1e7gVaofwHzQ req.method=OPTIONS req.path=/_matrix/media/v3/thumbnail/obermui.de/VCJpgjatIIsIuTzgLKwjBiYx 

seems like dendrite is failing to query images from other servers? Did something change for synapse-based installations so that the path is now different?

For the above, the queried path doesn't seem to exist on the server:

https://obermui.de/_matrix/media/v3/thumbnail/obermui.de/VCJpgjatIIsIuTzgLKwjBiYx

EDIT: In my case it was likely a different issue and solved by #2954 (comment).

[sleroq]

Seems like the fix is to just refresh element a few times and the image requests slowly turn from content-type json to jpeg

Seems like there is a limit on the number of thumbnails server able to generate at the same time. For me it's 5. When you refresh, browser won't download cached ones and eventually you load everything.

On the server it looks like this:

Details

monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | time="2024-10-17T08:08:23.240293449Z" level=info msg="Signalling other goroutines waiting for this goroutine to generate the thumbnail." Height=480 MediaID=a1039ef6a83a883e5579c65a2503482b6e10b878ccca5c189260a3fb76c60fbf Origin=sleroq.link ResizeMethod=scale Width=640 req.id=hobHFli7AMuA req.method=GET req.path=/_matrix/media/v3/thumbnail/sleroq.link/a1039ef6a83a883e5579c65a2503482b6e10b878ccca5c189260a3fb76c60fbf
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | time="2024-10-17T08:08:23.255353988Z" level=info msg="Signalling other goroutines waiting for this goroutine to generate the thumbnail." Height=480 MediaID=da2498ff0cd0c84686161db92cdbba24880e8a638a157b129b1b4ba5ddf4c9de Origin=sleroq.link ResizeMethod=scale Width=640 req.id=o7taqB4UG4ve req.method=GET req.path=/_matrix/media/v3/thumbnail/sleroq.link/da2498ff0cd0c84686161db92cdbba24880e8a638a157b129b1b4ba5ddf4c9de
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)