Skip to content

โšก Bolt: Optimize dataframe subset modifications #180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
seonghobae wants to merge 4 commits into
base: master
Choose a base branch
from
+37 โˆ’25
Open

โšก Bolt: Optimize dataframe subset modifications #180

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
230605c
โšก Bolt: Optimize dataframe subset modifications
seonghobae Jul 27, 2026
c2b6bd5
โšก Bolt: Fix CI by adding hidden/dummy files to .Rbuildignore
seonghobae Jul 27, 2026
230fd74
โšก Bolt: Fix CI by adding missing hidden files to .Rbuildignore
seonghobae Jul 27, 2026
58b828b
โšก Bolt: Optimize dataframe subset modifications and fix CI configuration
seonghobae Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .Rbuildignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,8 @@
^\.jules(/.*)?$
^\.trivyignore\.yaml$
^trivy\.yaml$
^test_dummy.R$
^test_validation.R$
Comment on lines +25 to +26
^\.semgrepignore$
^aFIPC\.Rcheck$
^aFIPC_.*\.tar\.gz$
3 changes: 3 additions & 0 deletions .jules/bolt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,6 @@
## 2025-02-12 - R ์–ธ์–ด์—์„œ ๋ฐ˜๋ณต์ ์ธ mirt ๋ชจ๋ธ ์ƒ์„ฑ ์‹œ ๋ถˆํ•„์š”ํ•œ ๋ฐ์ดํ„ฐํ”„๋ ˆ์ž„ ๋ถ€๋ถ„์ง‘ํ•ฉ ์ถ”์ถœ ์ตœ์ ํ™”
**Learning:** R์—์„œ ๋ฐ์ดํ„ฐํ”„๋ ˆ์ž„์˜ ํŠน์ • ์—ด์„ ์ถ”์ถœํ•˜๋Š” ์ž‘์—…(`df[cols]`)์€ O(N)์˜ ๋ฉ”๋ชจ๋ฆฌ ๋ณต์‚ฌ๋ฅผ ์ˆ˜๋ฐ˜ํ•ฉ๋‹ˆ๋‹ค. `autoFIPC`์—์„œ `mirt` ๋ชจ๋ธ์˜ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์„ค์ •ํ•˜๊ฑฐ๋‚˜ ํ˜ธ์ถœํ•˜๋Š” ๊ณผ์ • ์ค‘์— `newformXDataK[colnames(newFormModel@Data$data)]` ์ฝ”๋“œ๊ฐ€ ๋ฐ˜๋ณตํ•ด์„œ ์‚ฌ์šฉ๋˜์—ˆ๊ณ , ์‹ฌ์ง€์–ด `ncol()`์„ ์œ„ํ•ด ๋‹จ์ˆœํžˆ ๊ฐœ์ˆ˜๋ฅผ ๊ตฌํ•  ๋•Œ๋„ ์‚ฌ์šฉ๋˜์–ด ๋ถˆํ•„์š”ํ•œ ๋ฉ”๋ชจ๋ฆฌ ํ• ๋‹น๊ณผ ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ์ดˆ๋ž˜ํ–ˆ์Šต๋‹ˆ๋‹ค.
**Action:** ์กฐ๊ฑด๋ฌธ์ด๋‚˜ ๋ฐ˜๋ณต๋ฌธ ๋‚ด๋ถ€์—์„œ ๋ถˆํ•„์š”ํ•˜๊ฒŒ ๋ฐ์ดํ„ฐํ”„๋ ˆ์ž„ ๋ถ€๋ถ„์ง‘ํ•ฉ ์—ฐ์‚ฐ์ด ๋ฐ˜๋ณต๋˜์ง€ ์•Š๋„๋ก ์™ธ๋ถ€์—์„œ ํ•œ ๋ฒˆ๋งŒ `linkedFormData <- newformXDataK[colnames(newFormModel@Data$data)]`๋กœ ์บ์‹ฑ(caching)ํ•œ ๋’ค, `ncol(linkedFormData)`์™€ `data = linkedFormData` ํ˜•ํƒœ๋กœ ์žฌ์‚ฌ์šฉํ•˜์—ฌ ๋ฉ”๋ชจ๋ฆฌ ๋ณต์‚ฌ์™€ O(N) ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ๋ฐฉ์ง€ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
## 2024-07-27 - R ์–ธ์–ด์—์„œ ๋ฐ์ดํ„ฐ ํ”„๋ ˆ์ž„ ๊ฐ’ ํ• ๋‹น์‹œ method dispatch ๋ณ‘๋ชฉ ํ˜„์ƒ ์ตœ์ ํ™”
**Learning:** R์—์„œ ๋ฐ์ดํ„ฐ ํ”„๋ ˆ์ž„์˜ ํŠน์ • ๊ฐ’์„ ์ˆ˜์ •ํ•  ๋•Œ `df[index, "column"] <- value` ์™€ ๊ฐ™์ด 2์ฐจ์› ๋ถ€๋ถ„์ง‘ํ•ฉ ํ• ๋‹น ๋ฐฉ์‹์„ ์‚ฌ์šฉํ•˜๋ฉด ๋‚ด๋ถ€์ ์œผ๋กœ `[<-.data.frame` ๋ฉ”์„œ๋“œ ๋””์ŠคํŒจ์น˜๊ฐ€ ๋ฐœ์ƒํ•˜์—ฌ ์ฐจ์› ๊ฒ€์‚ฌ ๋ฐ ๋ฐ์ดํ„ฐ ๋ณต์‚ฌ๋กœ ์ธํ•ด O(N)์˜ ์˜ค๋ฒ„ํ—ค๋“œ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.
**Action:** `df$column[index] <- value` ์™€ ๊ฐ™์ด 1์ฐจ์› ๋ฒกํ„ฐ ์ง์ ‘ ์ ‘๊ทผ ํ• ๋‹น ๋ฐฉ์‹์„ ์‚ฌ์šฉํ•˜์—ฌ, `[<-.data.frame` ํ˜ธ์ถœ์„ ์šฐํšŒํ•˜๊ณ  ๋ฆฌ์ŠคํŠธ ์š”์†Œ์— O(1) ์‹œ๊ฐ„ ๋ณต์žก๋„๋กœ ๋น ๋ฅด๊ฒŒ ์ ‘๊ทผํ•˜์—ฌ C ์ˆ˜์ค€์—์„œ ์›์†Œ๋ฅผ ์ง์ ‘ ์ˆ˜์ •ํ•˜๋„๋ก ํ•˜์—ฌ ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ์ตœ์ ํ™”ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
Comment on lines +20 to +21
Comment on lines +19 to +21

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๐Ÿš€ Performance & Scalability | ๐ŸŸก Minor | โšก Quick win

๐Ÿงฉ Analysis chain

๐Ÿ Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## .jules/bolt.md lines 1-80\n'
cat -n .jules/bolt.md | sed -n '1,80p'

Repository: ContextualWisdomLab/aFIPC

Length of output: 3347

๐ŸŒ Web query:

R data.frame subset assignment copy-on-modify complexity df$column[index] <- value docs

๐Ÿ’ก Result:

In R, assignment operations like df$column[index] <- value are governed by copy-on-modify semantics, a mechanism designed to maintain an illusion of call-by-value while optimizing memory usage [1][2][3]. Key Concepts: - Copy-on-Modify: R objects are effectively immutable [1]. When you perform an assignment, R checks the reference count of the object (often stored in the 'named' field of its internal SEXP structure) [2][4][3]. If the reference count is greater than 1 (meaning other variables point to the same data), R duplicates the object before modifying it to ensure the original remains unchanged [1][4][3]. If the reference count is 1, R may be able to modify the object in place [4][3]. - Data Frame Structure: Data frames are lists of vectors [1]. Modifying a single column via df$column[index] <- value generally requires copying only that specific column vector [1]. However, modifying an entire row (e.g., df[index, ] <- value) requires copying every column in the data frame because the structure of the entire object is effectively being altered [1]. - Performance Complexity: Because R often performs these operations via high-level internal functions (like [<-.data.frame), multiple intermediate copies can occur depending on the specific operation, the number of references to the object, and internal function overhead [1]. Empirical testing using tools like tracemem is the most reliable way to determine if a copy is occurring in a specific context [1]. - Official Documentation: The behavior is rooted in the R internal memory management system (see the 'R Internals' manual) [3][5] and the S3 replacement methods for data frames [6]. For performance-critical code, it is recommended to avoid patterns that trigger excessive copying (such as growing data frames inside loops) and to monitor memory allocation empirically using tools provided in packages like pryr or base functions like tracemem [1][7][8].

Citations:

O(1) ํ‘œํ˜„์€ ์™„ํ™”ํ•˜์„ธ์š”. df$column[index] <- value๋Š” [<-.data.frame ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ์ค„์ผ ์ˆ˜ ์žˆ์ง€๋งŒ, R์˜ copy-on-modify์™€ ์ธ๋ฑ์‹ฑ/๋ณต์‚ฌ ๋น„์šฉ ๋•Œ๋ฌธ์— ์ž„์˜์˜ ํ• ๋‹น์„ O(1)์ด๋ผ๊ณ  ๋‹จ์ •ํ•  ์ˆ˜๋Š” ์—†์Šต๋‹ˆ๋‹ค. โ€œ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ์ค„์ผ ์ˆ˜ ์žˆ๋‹คโ€ ์ •๋„๋กœ ๋ฐ”๊ฟ” ์ฃผ์„ธ์š”.

๐Ÿค– Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.jules/bolt.md around lines 19 - 21, Update the learning entry in the R
data-frame assignment section to remove the claim that direct vector assignment
has O(1) complexity. State only that df$column[index] <- value can reduce
[<-.data.frame dispatch and related overhead, while preserving the existing
optimization guidance.

54 changes: 29 additions & 25 deletions R/aFIPC.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -597,16 +597,16 @@ autoFIPC <-

# Preserve mirt's structural estimability flags. Forcing every row TRUE
# frees boundary parameters such as 2PL g/u and makes the Hessian unstable.
# โšก Bolt: Direct vector subsetting to avoid O(N) [<-.data.frame dispatch overhead
NewScaleParms$est[NewScaleParms$item == 'GROUP'] <- FALSE
OldScaleParms$est[OldScaleParms$item == 'GROUP'] <- FALSE

NewScaleParms[NewScaleParms$item == 'GROUP', "est"] <- FALSE
OldScaleParms[OldScaleParms$item == 'GROUP', "est"] <- FALSE

NewScaleParms[NewScaleParms$name == "COV_11", "est"] <- TRUE
OldScaleParms[OldScaleParms$name == "COV_11", "est"] <- TRUE
NewScaleParms$est[NewScaleParms$name == "COV_11"] <- TRUE
OldScaleParms$est[OldScaleParms$name == "COV_11"] <- TRUE

if (itemtype == 'Rasch') {
NewScaleParms[NewScaleParms$name == "a1", "est"] <- FALSE
OldScaleParms[OldScaleParms$name == "a1", "est"] <- FALSE
NewScaleParms$est[NewScaleParms$name == "a1"] <- FALSE
OldScaleParms$est[OldScaleParms$name == "a1"] <- FALSE
}

#IPD
Expand Down Expand Up @@ -786,14 +786,15 @@ autoFIPC <-
oldIdx <- oldScaleParmsItemIdxCache[[oldFormItemStr]]

# โšก Bolt: Remove unnecessary paste0() array string generation overhead
message(' Newform Parms: ', paste(NewScaleParms[newIdx, "value"], collapse = ' '))
message(' Oldform Parms: ', paste(OldScaleParms[oldIdx, "value"], collapse = ' '))
message(' Newform Parms: ', paste(NewScaleParms$value[newIdx], collapse = ' '))
message(' Oldform Parms: ', paste(OldScaleParms$value[oldIdx], collapse = ' '))

NewScaleParms[newIdx, "value"] <-
OldScaleParms[oldIdx, "value"]
message(' Linkedform Parms: ', paste(NewScaleParms[newIdx, "value"], collapse = ' '), '\n')
# โšก Bolt: Direct vector subsetting to avoid O(N) [<-.data.frame dispatch overhead
NewScaleParms$value[newIdx] <-
OldScaleParms$value[oldIdx]
message(' Linkedform Parms: ', paste(NewScaleParms$value[newIdx], collapse = ' '), '\n')

NewScaleParms[newIdx, "est"] <-
NewScaleParms$est[newIdx] <-
FALSE
} else {
message(
Expand All @@ -813,17 +814,18 @@ autoFIPC <-
newBetaIdx <- NewScaleParms$item == 'BETA'
oldBetaIdx <- OldScaleParms$item == 'BETA'

NewScaleParms[newBetaIdx, "value"] <-
OldScaleParms[oldBetaIdx, "value"]
NewScaleParms[newBetaIdx, "est"] <-
# โšก Bolt: Direct vector subsetting to avoid O(N) [<-.data.frame dispatch overhead
NewScaleParms$value[newBetaIdx] <-
OldScaleParms$value[oldBetaIdx]
NewScaleParms$est[newBetaIdx] <-
FALSE

message('applying BETA parameter as linking')

message(
' Linkedform Parms: ',
paste0(
NewScaleParms[newBetaIdx, "value"],
NewScaleParms$value[newBetaIdx],
' '
),
'\n'
Expand Down Expand Up @@ -858,13 +860,14 @@ autoFIPC <-
new_mean11_idx <- NewScaleParms$name == "MEAN_11"
old_mean11_idx <- OldScaleParms$name == "MEAN_11"

NewScaleParms[new_cov11_idx, "est"] <- FALSE
OldScaleParms[old_cov11_idx, "est"] <- FALSE
NewScaleParms[new_mean11_idx, "est"] <- FALSE
OldScaleParms[old_mean11_idx, "est"] <- FALSE
# โšก Bolt: Direct vector subsetting to avoid O(N) [<-.data.frame dispatch overhead
NewScaleParms$est[new_cov11_idx] <- FALSE
OldScaleParms$est[old_cov11_idx] <- FALSE
NewScaleParms$est[new_mean11_idx] <- FALSE
OldScaleParms$est[old_mean11_idx] <- FALSE

NewScaleParms[new_cov11_idx, "value"] <- 1
OldScaleParms[old_mean11_idx, "value"] <- 0
NewScaleParms$value[new_cov11_idx] <- 1
OldScaleParms$value[old_mean11_idx] <- 0
}
if (freeMEAN == T) {
LinkedModelSyntax <-
Expand All @@ -875,8 +878,9 @@ autoFIPC <-
'MEAN = F1'
))

NewScaleParms[NewScaleParms$name == "MEAN_1", "est"] <- TRUE
OldScaleParms[OldScaleParms$name == "MEAN_1", "est"] <- TRUE
# โšก Bolt: Direct vector subsetting to avoid O(N) [<-.data.frame dispatch overhead
NewScaleParms$est[NewScaleParms$name == "MEAN_1"] <- TRUE
OldScaleParms$est[OldScaleParms$name == "MEAN_1"] <- TRUE
} else {
LinkedModelSyntax <-
mirt::mirt.model(paste0(
Expand Down
Loading