Skip to content

feat: implement etag on fetch entities actions #182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tomrndom wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: implement etag on fetch entities actions #182

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6e2a585
feat: implement etag on fetch entities actions
tomrndom Oct 9, 2025
25cf3bf
fix: change etagCache structure, adjust functions, clear cache on rea…
tomrndom Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 66 additions & 72 deletions src/actions/fetch-entities-actions.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,58 @@ import SummitAPIRequest from "../utils/build-json/SummitAPIRequest";
import EventAPIRequest from "../utils/build-json/EventsAPIRequest";
import SpeakersAPIRequest from "../utils/build-json/SpeakersAPIRequest";

@smarcet smarcet Mar 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

etagCache grows unbounded with no eviction strategy. In long,running sessions (conference attendees keeping tabs open for hours), this could consume significant memory.

Consider using a Map with a size limit:

const etagCache = new Map();
const MAX_CACHE_SIZE = 100;

// Before inserting:
if (etagCache.size >= MAX_CACHE_SIZE) {
  const oldest = etagCache.keys().next().value;
  etagCache.delete(oldest);
}


const etagCache = new Map();
const MAX_CACHE_SIZE = 100;

export const clearEtagCacheForUrl = (urlPattern) => {
for (const key of etagCache.keys()) {
if (key.includes(urlPattern)) {
etagCache.delete(key);
}
}
};

@smarcet smarcet Mar 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ETag cache may conflict with the real,time updates system. When Ably pushes an update and the sync strategies call these fetch functions to get fresh data, the cached ETag could cause a 304 response, returning stale data instead of the updated entity.

Scenario:

  1. User loads summit → cached with ETag
  2. Backend updates summit → Ably message triggers synchEntityData
  3. fetchSummitById sends If,None,Match with old ETag
  4. Server responds 304 → stale cached data returned

Consider exporting a clearEtagCache function to invalidate entries when real,time updates are received:

export const clearEtagCacheForUrl = (urlPattern) => {
  for (const key of Object.keys(etagCache)) {
    if (key.includes(urlPattern)) {
      delete etagCache[key];
    }
  }
};

Then call it from sync strategies before re,fetching.

const fetchWithEtag = async (url) => {
const headers = {};

if (etagCache.has(url)) {
const { etag } = etagCache.get(url);
if (etag) {
headers['If-None-Match'] = etag;
}
}

const res = await fetch(url, {
method: 'GET',
cache: "no-store",
headers,
});

if (res.status === 304 && etagCache.has(url)) {
const { body } = etagCache.get(url);
return body;
}

if (res.status === 200) {
const data = await res.json();
const responseETAG = res.headers.get('etag');
if (responseETAG) {
if (etagCache.size >= MAX_CACHE_SIZE) {
const oldest = etagCache.keys().next().value;
etagCache.delete(oldest);
}
etagCache.set(url, { etag: responseETAG, body: data });
}
return data;

@smarcet smarcet Mar 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non,200/304 status codes are silently swallowed here. If the server returns 401, 403, 404, or 500, the function just returns null with no indication of what went wrong.

Consider adding error logging:

if (!res.ok) {
  console.error(`fetchWithEtag failed (${res.status}):`, url);
}

return null;

}

if (!res.ok) {
console.error(`fetchWithEtag failed (${res.status}):`, url);
}

return null;
};

/**
* @param summitId
* @param eventId
Expand All @@ -18,16 +70,7 @@ export const fetchEventById = async (summitId, eventId, accessToken = null) => {
}

const apiUrlWithParams = EventAPIRequest.build(apiUrl);

return fetch(apiUrlWithParams, {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
return fetchWithEtag(apiUrlWithParams);
}

/**
Expand All @@ -39,15 +82,8 @@ export const fetchEventById = async (summitId, eventId, accessToken = null) => {
export const fetchStreamingInfoByEventId = async (summitId, eventId, accessToken) => {
const apiUrl = URI(`${process.env.GATSBY_SUMMIT_API_BASE_URL}/api/v1/summits/${summitId}/events/${eventId}/published/streaming-info`);
apiUrl.addQuery('access_token', accessToken);
return fetch(apiUrl.toString(), {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
const url = apiUrl.toString();
return fetchWithEtag(url);
}

/**
Expand All @@ -64,19 +100,11 @@ export const fetchEventTypeById = async (summitId, eventTypeId, accessToken = nu
apiUrl.addQuery('access_token', accessToken);
}

return fetch(apiUrl.toString(), {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
const url = apiUrl.toString();
return fetchWithEtag(url);
}

/**
*
* @param summitId
* @param locationId
* @param expand
Expand All @@ -94,19 +122,11 @@ export const fetchLocationById = async (summitId, locationId, expand, accessToke
if (expand)
apiUrl.addQuery('expand', expand);

return fetch(apiUrl.toString(), {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
const url = apiUrl.toString();
return fetchWithEtag(url);
}

/**
*
* @param summitId
* @param speakerId
* @param accessToken
Expand All @@ -122,20 +142,10 @@ export const fetchSpeakerById = async (summitId, speakerId, accessToken = null)
}

const apiUrlWithParams = SpeakersAPIRequest.build(apiUrl);

return fetch(apiUrlWithParams, {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
return fetchWithEtag(apiUrlWithParams);
}

/**
*
* @param summitId
* @param accessToken
* @returns {Promise<Response>}
Expand All @@ -149,16 +159,7 @@ export const fetchSummitById = async (summitId, accessToken = null) => {
}

const apiUrlWithParams = SummitAPIRequest.build(apiUrl);

return fetch(apiUrlWithParams, {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
return fetchWithEtag(apiUrlWithParams);
}

/**
Expand All @@ -171,24 +172,17 @@ export const fetchTrackById = async (summitId, trackId, accessToken = null) => {
let apiUrl = URI(`${process.env.GATSBY_SUMMIT_API_BASE_URL}/api/public/v1/summits/${summitId}/tracks/${trackId}`);

const fields = [
"id", "name", "code", "order", "parent_id", "color","text_color",
"id", "name", "code", "order", "parent_id", "color", "text_color",
"subtracks.id", "subtracks.name", "subtracks.code", "subtracks.order",
"subtracks.parent_id", "subtracks.color", "subtracks.text_color",
];
const relations = ['subtracks','subtracks.none'];
const relations = ['subtracks', 'subtracks.none'];
const expand = ['subtracks']

apiUrl.addQuery('fields', fields.join(','));
apiUrl.addQuery('relations', relations.join(','));
apiUrl.addQuery('expand', expand.join(','));

return fetch(apiUrl.toString(), {
method: 'GET',
cache: "no-store",
}).then(async (response) => {
if (response.status === 200) {
return await response.json();
}
return null;
});
}
const url = apiUrl.toString();
return fetchWithEtag(url);
}
4 changes: 3 additions & 1 deletion src/workers/sync_strategies/activity_synch_strategy.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import AbstractSynchStrategy from "./abstract_synch_strategy";
import {fetchEventById, fetchStreamingInfoByEventId} from "../../actions/fetch-entities-actions";
import {clearEtagCacheForUrl, fetchEventById, fetchStreamingInfoByEventId} from "../../actions/fetch-entities-actions";
import {insertSorted, intCheck, rebuildIndex} from "../../utils/arrayUtils";
import {
BUCKET_EVENTS_DATA_KEY,
Expand Down Expand Up @@ -161,8 +161,10 @@ class ActivitySynchStrategy extends AbstractSynchStrategy{
switch (entity_operator) {
case 'INSERT':
case 'UPDATE':{
clearEtagCacheForUrl(`/v1/summits/${this.summit.id}/events/${entity_id}/published`);
let entity = await fetchEventById(this.summit.id, entity_id, this.accessToken);
if(this.accessToken && this._shouldFetchStreamingInfo(this.currentLocation)) {
clearEtagCacheForUrl(`/v1/summits/${this.summit.id}/events/${entity_id}/published/streaming-info`)
const streaming_info = await fetchStreamingInfoByEventId(this.summit.id, entity_id, this.accessToken);
if(streaming_info) entity = {...entity, ...streaming_info};
}
Expand Down
4 changes: 3 additions & 1 deletion src/workers/sync_strategies/activity_type_synch_strategy.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import AbstractSynchStrategy from "./abstract_synch_strategy";
import {fetchEventTypeById} from "../../actions/fetch-entities-actions";
import {clearEtagCacheForUrl, fetchEventTypeById} from "../../actions/fetch-entities-actions";
import {
BUCKET_EVENTS_DATA_KEY,
BUCKET_SUMMIT_DATA_KEY,
Expand All @@ -17,6 +17,8 @@ class ActivityTypeSynchStrategy extends AbstractSynchStrategy{

const {entity_operator, entity_id} = payload;

clearEtagCacheForUrl(`/v1/summits/${this.summit.id}/event-types/${entity_id}`);

const entity = await fetchEventTypeById(this.summit.id, entity_id, this.accessToken);

if (entity_operator === 'UPDATE') {
Expand Down
3 changes: 2 additions & 1 deletion src/workers/sync_strategies/speaker_synch_strategy.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import AbstractSynchStrategy from "./abstract_synch_strategy";
import {fetchSpeakerById} from "../../actions/fetch-entities-actions";
import {clearEtagCacheForUrl, fetchSpeakerById} from "../../actions/fetch-entities-actions";
import {
BUCKET_SUMMIT_DATA_KEY,
BUCKET_EVENTS_DATA_KEY,
Expand All @@ -23,6 +23,7 @@ class SpeakerSynchStrategy extends AbstractSynchStrategy {
const {entity_operator, entity_id} = payload;
if (entity_operator === 'UPDATE') {

clearEtagCacheForUrl(`/v1/summits/${this.summit.id}/speakers/${entity_id}`);

const entity = await fetchSpeakerById(this.summit.id, entity_id, this.accessToken);

Expand Down
4 changes: 3 additions & 1 deletion src/workers/sync_strategies/summit_synch_strategy.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import AbstractSynchStrategy from "./abstract_synch_strategy";
import {fetchSummitById} from "../../actions/fetch-entities-actions";
import {clearEtagCacheForUrl, fetchSummitById} from "../../actions/fetch-entities-actions";
import {
BUCKET_SUMMIT_DATA_KEY,
saveFile
Expand All @@ -17,6 +17,8 @@ class SummitSynchStrategy extends AbstractSynchStrategy {

const {entity_operator} = payload;

clearEtagCacheForUrl(`/v1/summits/${this.summit.id}?`);

let entity = await fetchSummitById(this.summit.id, this.accessToken);

let eventsData = [...this.allEvents];
Expand Down
3 changes: 2 additions & 1 deletion src/workers/sync_strategies/track_synch_strategy.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import AbstractSynchStrategy from "./abstract_synch_strategy";
import {fetchTrackById} from "../../actions/fetch-entities-actions";
import {clearEtagCacheForUrl, fetchTrackById} from "../../actions/fetch-entities-actions";
import {
BUCKET_EVENTS_DATA_KEY,
BUCKET_EVENTS_IDX_DATA_KEY,
Expand Down Expand Up @@ -159,6 +159,7 @@ class TrackSynchStrategy extends AbstractSynchStrategy {
switch (entity_operator) {
case 'INSERT':
case 'UPDATE':{
clearEtagCacheForUrl(`/v1/summits/${this.summit.id}/tracks/${entity_id}`);
const entity = await fetchTrackById(this.summit.id, entity_id, this.accessToken);
if (!entity) return Promise.reject('TrackSynchStrategy::process entity not found.');
return this._handleUpsert(entity, payload);
Expand Down
4 changes: 3 additions & 1 deletion src/workers/sync_strategies/venue_room_synch_strategy.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import AbstractSynchStrategy from "./abstract_synch_strategy";
import { fetchLocationById } from "../../actions/fetch-entities-actions";
import { clearEtagCacheForUrl, fetchLocationById } from "../../actions/fetch-entities-actions";
import {
BUCKET_EVENTS_DATA_KEY,
BUCKET_EVENTS_IDX_DATA_KEY,
Expand All @@ -20,6 +20,8 @@ class VenueRoomSynchStrategy extends AbstractSynchStrategy{

const {entity_operator, entity_id} = payload;

clearEtagCacheForUrl(`/v1/summits/${this.summit.id}/locations/${entity_id}`);

const entity = await fetchLocationById(this.summit.id, entity_id, 'floor,venue' , this.accessToken);

let eventsData = [...this.allEvents];
Expand Down