Skip to content

Fix keyword weights being flattened to 1 - #32

Open
Ppantaleo wants to merge 1 commit into
lepidus:mainfrom
Ppantaleo:fix/keyword-frequency-count
Open

Fix keyword weights being flattened to 1#32
Ppantaleo wants to merge 1 commit into
lepidus:mainfrom
Ppantaleo:fix/keyword-frequency-count

Conversation

@Ppantaleo

Copy link
Copy Markdown

The problem

array_unique() is applied to the journal-wide list of keyword names before array_count_values() counts them:

$uniqueKeywords = array_unique(array_map('strtolower', array_column($keywords, 'name')));
$countKeywords = array_count_values($uniqueKeywords);

By the time the count runs every value is distinct, so each keyword gets a count of 1. Two consequences:

  • Every word ends up the same size. fontSize.domain() in block.tpl gets [1, 1], a degenerate domain, so d3 maps every keyword to the same font size and the cloud carries no information.
  • array_slice() keeps an arbitrary 50 keywords rather than the 50 most frequent, since arsort() has nothing to sort by.

The fix

Deduplicate per publication instead of across the whole journal, so a word's weight is the number of articles that use it. A keyword repeated inside a single publication still counts once.

Also passes preserve_keys: true to array_slice(): a purely numeric keyword ("2020") had its key reindexed and reached the template as "0".

Testing

Verified on OJS 3.5.0.5 against a journal with 879 published publications. The block now returns covid-19: 22, argentina: 17, editorial: 15, ..., matching the counts obtained directly via SQL. Before the change every entry was size: 1.

Note on #31

#31 touches the same lines but addresses a different problem — the block coming out empty when getBySymbolic() returns a flat array. It keeps the array_unique() before the count, so it does not change the weights. Happy to rebase if #31 lands first.

array_unique() was applied to the journal-wide list of keyword names before
array_count_values() counted them. Since every value was distinct by then,
each keyword got a count of 1: the cloud rendered every word at the same
font size and the top-50 slice kept an arbitrary set instead of the most
frequent ones.

Deduplicate per publication instead, so a word's weight is the number of
articles that use it. Also preserve keys in the array_slice() call: a purely
numeric keyword had its key reindexed and reached the template as "0".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant