Add which entry command#21790
Conversation
MikeMcQuaid
left a comment
There was a problem hiding this comment.
Nice work @ooye-sanket! Looks good so far. Please add a very basic integration test and args check for which-entry. Also, please inline functions that are only used once as there's a fair bit of indirection right now.
| rescue JSON::ParserError | ||
| [] |
There was a problem hiding this comment.
Probably worth a opoo warning of the exception in this case?
|
Thanks @MikeMcQuaid! I've pushed the changes:
Let me know if the level of inlining looks right or if you'd like it structured differently! |
MikeMcQuaid
left a comment
There was a problem hiding this comment.
Thanks! Please ensure CI syntax job passing before asking for re-review.
| if db_path | ||
| existing = db_path.exist? ? db_path.readlines(chomp: true).reject(&:empty?) : [] | ||
| lines = existing.reject { |l| l.start_with?("#{formula.full_name}(") } | ||
| lines << line if line |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
9e3e0a6 to
00d9e78
Compare
MikeMcQuaid
left a comment
There was a problem hiding this comment.
Thanks, looking good, a few small things!
|
|
||
| sig { override.void } | ||
| def run | ||
| db_path = Pathname(T.unsafe(args).output_db) if T.unsafe(args).output_db |
There was a problem hiding this comment.
This shouldn't be necessary, you may need to run brew typecheck --update to avoid the need for T.unsafe.
| sig { params(db_path: Pathname, name: String, line: T.nilable(String)).void } | ||
| def write_db(db_path, name, line) | ||
| lines = db_path.readlines(chomp: true).compact_blank if db_path.exist? | ||
| lines = (lines || []).reject { |l| l.start_with?("#{name}(") } |
There was a problem hiding this comment.
| lines = (lines || []).reject { |l| l.start_with?("#{name}(") } | |
| lines = Array(lines).reject { |l| l.start_with?("#{name}(") } |
There was a problem hiding this comment.
Also, there's an inconsistency here, right? We reject lines that start with name( but we don't actually write the version in parentheses anymore.
There was a problem hiding this comment.
Fixed as now uses Array(lines) and the reject pattern is corrected to "#{name}:" to match the new format without pkg_version.
There was a problem hiding this comment.
You need to support both the old and new formats
There was a problem hiding this comment.
Sorry for the review delay here. I agree with Mike's feedback, and also have one other thought.
More generally, I'm not sure whether this actually needs to be a brew command at all. I will leave a comment on the other PR with more details
Edit: oh, one more thing, we should remove the existing command-not-found-db-update.yml workflow
|
hey @Rylan12 The reason for keeping it as a brew command is that it already handles formula loading, bottle manifest caching, disabled/deprecated checks, and DB file management in a tested, typed Ruby context. Doing all that reliably in bash would be fragile. But happy to discuss if there's a simpler scope you have in mind. |
|
Let's keep it as Ruby for now and we can see how it's looking when the other PR is ready. |
|
I don't mind keeping it in Ruby for now, so this is not a blocker.
I think doing it in bash would actually let us bypass these complexities altogether, resulting in a simpler system overall. There would be no need to load formulae at all, and the deprecated/disabled info is available via the same bottle manifest that we already need to use. Fair enough on the DB management, although I think a reasonably well-crafted I haven't attempted to write it out, so I could be wrong |
|
Hey @MikeMcQuaid I've made all the changes (dropped pkg_version, fixed the reject pattern, removed T.unsafe) but I'm unable to run brew typecheck --update locally to regenerate the RBI file. Could you help run it or point me to how to get past this? Everything else should be good to go! |
|
@ooye-sanket what happens when you run it locally? |
|
@ooye-sanket nudge on above question! |
MikeMcQuaid
left a comment
There was a problem hiding this comment.
Waiting on author answering question.
91f72e6 to
64df19c
Compare
|
Hey @MikeMcQuaid, sorry for the delay! I ran brew typecheck --update locally and it worked successfully it generated the new sorbet/rbi/dsl/homebrew/dev_cmd/which_entry.rbi RBI file which I've now committed. I've also rebased onto latest upstream/main. Ready for re-review Note: the brew typecheck CI failure is in extend/os/mac/keg_relocate.rb and extend/os/mac/keg.rb these are pre-existing errors unrelated to this PR. CC @Rylan12 |
That's not true: these are passing on |
|
Hey @MikeMcQuaid, CI is now green! , The earlier typecheck failure was caused by our update accidentally removing mach_o_shim.rbi I've now restored it. Ready for re-review! 🙏 |
|
|
||
| sig { override.void } | ||
| def run | ||
| db_path = Pathname(T.unsafe(args).output_db) if T.unsafe(args).output_db |
5680029 to
50f706c
Compare
|
Hey @MikeMcQuaid, sorry for the noise! All 3 issues fixed. |
|
|
||
| sig { override.void } | ||
| def run | ||
| db_path = Pathname(T.must(args.output_db)) if args.output_db |
There was a problem hiding this comment.
IMO we should just make this argument required and raise a usage error if it's not passed. Or accept it as a named arg
There was a problem hiding this comment.
Fixed --output-db is now required and raises a UsageError if not passed.
| sig { params(db_path: Pathname, name: String, line: T.nilable(String)).void } | ||
| def write_db(db_path, name, line) | ||
| lines = db_path.readlines(chomp: true).compact_blank if db_path.exist? | ||
| lines = (lines || []).reject { |l| l.start_with?("#{name}(") } |
| sig { params(db_path: Pathname, name: String, line: T.nilable(String)).void } | ||
| def write_db(db_path, name, line) | ||
| lines = db_path.readlines(chomp: true).compact_blank if db_path.exist? | ||
| lines = (lines || []).reject { |l| l.start_with?("#{name}(") } |
There was a problem hiding this comment.
Also, there's an inconsistency here, right? We reject lines that start with name( but we don't actually write the version in parentheses anymore.
|
@ooye-sanket, some of the issues Mike pointed out don't actually seem to have been fixed yet. When you fix those, can you please run the commands locally on your machine and comment the output here to help verify that this works as expected? It would be helpful to see a sample |
MikeMcQuaid
left a comment
There was a problem hiding this comment.
Comments not addressed. Do not mark as resolved or request re review until 1) all comments past and present resolved and 2) all @Rylan12's comments and questions answered.
If I'm requested for review again and this still isn't ready: I'm just likely to close this out, sorry. Please raise the bar for yourself here. Thank you ❤️
…fix write_db, remove old workflow
|
Hey @MikeMcQuaid and @Rylan12, all comments are now addressed in the latest commits.
Here's proof the command works locally:
Entries are sorted, idempotent, and correctly formatted as name:executables.
NOTE : The only failing CI check is Docker / docker (x86_64 Ubuntu 22.04) which is a GitHub infrastructure issue apt-get update failing with permission errors, unrelated to our code. All 32 other checks are passing |
| sig { params(db_path: Pathname, name: String, line: T.nilable(String)).void } | ||
| def write_db(db_path, name, line) | ||
| lines = db_path.readlines(chomp: true).compact_blank if db_path.exist? | ||
| lines = (lines || []).reject { |l| l.start_with?("#{name}(") } |
There was a problem hiding this comment.
You need to support both the old and new formats
|
Hey @Rylan12, all feedback addressed! please have a look |
There was a problem hiding this comment.
@ooye-sanket, this implementation does not work correctly for any formulae that have multiple executables.
For future pushes, please include proof that you've tested the changes and that they work correctly for the standard case and also edge cases. Since this will run on every formula update, it's imperative that the code is sound and can handle all of the possible situations that may arise.
To help with this, unit tests would be useful for all changes (aside from a full integration tests, which are okay to skip because they're too slow).
| manifest_path = manifest_resource.cached_download | ||
| return [] unless manifest_path.exist? |
There was a problem hiding this comment.
I think .fetch is probably good enough here. It's called by default with verify_download_integrity: true, so I think we'll get an error before this point if we weren't able to download the manifest. I'd say we can just get rid of these lines since I'm not convinced they add additional safety. It seems appropriate to me for this to fail if the formula is bottled but unable to download the bottle manifest. That does seem error-worthy
| manifest_path = manifest_resource.cached_download | |
| return [] unless manifest_path.exist? |
| rescue JSON::ParserError => e | ||
| opoo "Failed to parse bottle manifest for #{formula.name}: #{e.message}" | ||
| [] |
There was a problem hiding this comment.
Let's get rid of this. BottleResource#manifest_annotations raises a nice error if the JSON doesn't parse, and I think we want a hard error here. Also it's a bit weird to rescue a JSON parse error when we don't actually call JSON.parse directly in this method.
| rescue JSON::ParserError => e | |
| opoo "Failed to parse bottle manifest for #{formula.name}: #{e.message}" | |
| [] |
| def path_exec_files | ||
| exec_files = manifest_annotations["sh.brew.path_exec_files"] | ||
| return [] if exec_files.blank? | ||
|
|
||
| exec_files.split.map { |f| File.basename(f) }.sort | ||
| end |
There was a problem hiding this comment.
This does not work correctly for annotations that have multiple entries. The entries are comma-separated, so split doesn't work.
Additionally, the File.basename doesn't belong in this method. This method should return the data directly, and leave it up to the caller how to process it.
| def path_exec_files | |
| exec_files = manifest_annotations["sh.brew.path_exec_files"] | |
| return [] if exec_files.blank? | |
| exec_files.split.map { |f| File.basename(f) }.sort | |
| end | |
| def path_exec_files | |
| manifest_annotations.fetch("sh.brew.path_exec_files", "").split(",") | |
| end |
MikeMcQuaid
left a comment
There was a problem hiding this comment.
@ooye-sanket please stop requesting my review here, thanks. I'll review when @Rylan12 is ✅.



Adds
brew which-entry <formula>a developer command that generatesa single
executables.txtentry for a formula by readingsh.brew.path_exec_filesfrom its bottle manifest.This is intended to replace the batch
brew which-updatejob with aper-formula update triggered by homebrew/core CI on each merge.
What it does
sh.brew.path_exec_files--output-db=<file>: upserts the formula's line in the DB fileand removes it if the formula is disabled, deprecated, or deleted
--output-db=: prints the DB line to stdoutRelationship to other PRs
The companion homebrew/core PR (Homebrew/homebrew-core#272976) slims
the existing workflow down to just call
brew which-entryper formuladetected by
brew test-bot --only-formulae-detect, replacing thecurrent daily batch job entirely.
@MikeMcQuaid please have a look!
brew lgtm(style, typechecking and tests) with your changes locally?