Skip to content

fix(repos): repair moved GitHub slugs and preserve partial repository-stat cache #1107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
castrojo wants to merge 1 commit into
base: v4
Choose a base branch
from
+41 −10
Open

fix(repos): repair moved GitHub slugs and preserve partial repository-stat cache #1107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions docs/donations/projects.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,13 +221,11 @@ These are the [CNCF](https://www.cncf.io/) and [OpenSSF](https://openssf.org/) p
description="Simple backup utility for GNOME"
sponsorUrl="https://liberapay.com/DejaDup"
icon="https://github.com/deja-dup.png"
githubRepo="deja-dup/deja-dup"
/>
<ProjectCard
name="Mission Center"
description="Monitor your CPU, Memory, Disk, Network and GPU usage"
icon="https://github.com/Flavius42.png"
githubRepo="Flavius42/mission-center"
icon="https://github.com/mission-center-devs.png"
/>
<ProjectCard
name="Impression"
Expand All @@ -251,8 +249,8 @@ These are the [CNCF](https://www.cncf.io/) and [OpenSSF](https://openssf.org/) p
<ProjectCard
name="Refine"
description="GNOME tweaking tool"
icon="https://github.com/tesk-g.png"
githubRepo="tesk-g/refine"
icon="https://github.com/TheEvilSkeleton.png"
githubRepo="TheEvilSkeleton/Refine"
/>
<ProjectCard
name="Smile"
Expand Down
43 changes: 38 additions & 5 deletions scripts/fetch-github-repos.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,14 @@ const GITHUB_REPOS = [
"flattool/ignition",
"tchx84/Flatseal",
"mjakeman/extension-manager",
"deja-dup/deja-dup",
"Flavius42/mission-center",
// deja-dup/deja-dup and Flavius42/mission-center have no canonical GitHub
// repo: both projects moved development to GNOME GitLab and no longer
// publish an equivalent GitHub repository, so they are intentionally
// omitted here (see docs/donations/projects.mdx).
"adhami3310/impression",
"PintaProject/Pinta",
"GNOME/Showtime",
"tesk-g/refine",
"TheEvilSkeleton/Refine", // formerly tesk-g/refine; account renamed
"mijorus/smile",

// Homebrew CLI Tools (bluefin-cli)
Expand Down Expand Up @@ -140,6 +142,18 @@ async function fetchAllRepos() {

console.log(`Fetching ${GITHUB_REPOS.length} GitHub repos...`);

// Load whatever cache already exists on disk so a failed fetch for one
// repo (rate limiting, transient network error, etc.) doesn't erase stats
// that were previously fetched successfully for that same repo.
let existingCache = {};
if (fs.existsSync(OUTPUT_FILE)) {
try {
existingCache = JSON.parse(fs.readFileSync(OUTPUT_FILE, "utf-8"));
} catch (error) {
console.warn(`⚠️ Could not parse existing cache: ${error.message}`);
}
}

const resultsMap = await sequentialFetchWithDelay(
GITHUB_REPOS,
async (repoPath) => {
Expand All @@ -148,11 +162,30 @@ async function fetchAllRepos() {
},
);

const repos = Object.fromEntries(resultsMap);
const freshRepos = Object.fromEntries(resultsMap);

// Merge: start from cached entries for repos still tracked (dropping any
// that were removed from GITHUB_REPOS), then overlay this run's
// successes. A repo that failed this run keeps its last-known-good stats
// instead of disappearing entirely.
const repos = {};
for (const repoPath of GITHUB_REPOS) {
if (existingCache[repoPath]) {
repos[repoPath] = existingCache[repoPath];
}
}
Object.assign(repos, freshRepos);

const failedCount = GITHUB_REPOS.length - Object.keys(freshRepos).length;
console.log(
`\nSuccessfully fetched ${Object.keys(repos).length}/${GITHUB_REPOS.length} repos`,
`\nSuccessfully fetched ${Object.keys(freshRepos).length}/${GITHUB_REPOS.length} repos`,
);
if (failedCount > 0) {
const recoveredFromCache = Object.keys(repos).length - Object.keys(freshRepos).length;
console.log(
` ${recoveredFromCache} of the ${failedCount} failures were preserved from the existing cache.`,
);
}

// Don't fail build if no repos fetched - the component will just not show stats
if (Object.keys(repos).length === 0) {
Expand Down