Feature/custom r functions and secrets - #2
Merged
Conversation
Adds a new custom_r_path column to survey_runs storing a file path to user-defined R functions. The file content is injected into every OpenCPU evaluation and knitr rendering call for the run, so custom functions are available in showif, value, feedback, relative_to, branch conditions, external URLs, email body, and all other R evaluation contexts. Key design decisions: - File-based storage (same pattern as custom_css / custom_js) - Instance-level caching on Run model (file read at most once per request) - No function signature changes — opencpu_* functions auto-inject via a small opencpu_custom_r() helper that retrieves from the active RunSession - No caller modifications needed in RunUnit subclasses or SpreadsheetRenderer Changes: - sql/patches/057_custom_r_path.sql: new patch - application/Model/Run.php: property, caching getter, saveSettings, export - templates/admin/run/settings.php: R Functions tab with Ace editor - application/Functions.php: opencpu_custom_r() helper + inject in all opencpu_evaluate and knitr functions - application/Api/V1/RunResource.php: API read/write exposure
…e error
The opencpu_evaluate() function sends R code via the /base/R/identity
endpoint which parses the parameter value as an R expression. Defining
functions inside the IIFE's (function() { ... })() body caused an
'Unparsable argument' error from OpenCPU. Moving function definitions
to the outer { } compound expression (outside the IIFE) resolves this.
The IIFE's child scope inherits from the outer scope, so custom
functions remain callable from user code inside the IIFE. Functions
that need run data should accept it as parameters, since injected
data variables (survey results, etc.) are defined inside the IIFE.
Also added documentation in the admin UI about the scoping behavior.
…ction definitions OpenCPU's /base/R/identity parses POST parameter values as R expressions. Complex multi-line expressions containing function() definitions caused an 'Unparsable argument' error from OpenCPU's R parser. Wrap custom R code in eval(parse(text = json_encode(...))) so the parser only sees a string literal. The code is evaluated at runtime inside the IIFE, defining functions in the same scope as injected data variables and user code. This also keeps custom functions in the same environment as (data frames defined by getRunData()), so functions can reference run data directly rather than requiring parameters.
Two-part fix for SQLSTATE[22001] 'Data too long for column result_log': 1. Schema: bump result_log from TEXT (65KB) to MEDIUMTEXT (16MB) - sql/patches/058_result_log_mediumtext.sql - Consistent with state_log (already LONGTEXT) in the same table - Eliminates the crash ceiling for error dumps containing long R code 2. Code: add truncation guards at all three write paths - UnitSession::end() before the UPDATE - UnitSession::logResult() before the UPDATE - EmailQueue::logResult() before the UPDATE - Truncates at 60000 bytes (safe margin below original TEXT limit) - Uses mb_strcut() to avoid splitting multi-byte UTF-8 characters - Belt-and-suspenders: keeps result_log reasonable regardless of column capacity
Design: Instead of adding new DB columns, tables, encryption, or files, leverage the existing custom R code injection mechanism. Users define secrets as R string literals with a secret_ prefix (e.g. secret_api_key <- "sk-...") directly in the R Functions editor. These are already injected via eval(parse(text = ...)) into every OpenCPU evaluation — zero new code on the participant hot path. Redaction: Known secret values (extracted via regex from the custom R file) are replaced with [SECRET REDACTED] at every output boundary where they could leak: - opencpu_log() — exception messages and traces - opencpu_debug() — R code display, error responses, console, stdout, session info - notify_user_error() — error bodies shown to admin/test sessions - Run::export() — custom_r in export JSON - RunResource::getRun() — custom_r in API responses Security trade-offs: - No encryption at rest — encryption defends against DB-only reads at complexity cost; the highest-probability threats (log/output leakage) are only covered by redaction regardless. - Minimum 6-char filter prevents false positives on short strings that appear commonly in output. - Values must be string literals (no expressions) — documented in the UI. - The study admin owns both the secrets and the R code, so the R evaluation context naturally has access to them (same trust model as .formr). Also: Renamed tab from "R Functions" to "R Functions & Secrets" and updated the help text and placeholder to document the secret_ convention.
…n UI - New survey_run_secrets table with FK cascade (patch 059) - RunSecret model: encrypt at rest via Crypto::encrypt(), decrypt on read - Admin UI: separate R Functions and Secrets tabs with auto-save AJAX - Secrets accessible in R as .formr$secret_<name> in all evaluation paths - Redaction sourced from DB (not custom_r parsing); removed 6-char minimum - Redact OpenCPU errors in result_log (DB) and study admin email notifications - Removed custom_r defense-in-depth regex (separate tab makes it unnecessary) - Export includes secret names only (values cannot leave instance encryption) - All 144 unit tests pass
…blic blurb; increase for custom CSS, JS, and R functions
… the API token injection
… usage guidelines
…dy error export
Adds a server-side R syntax check via OpenCPU's base::parse() that runs
every time the admin saves custom R code in the run settings. The check
is non-blocking — saving always succeeds even when the code is broken,
keeping formr a power-tool that trusts the user to save work-in-progress.
New endpoint:
- AdminAjaxController::ajaxValidateRCode() — accepts r_code POST param,
delegates to opencpu_validate_r_code(), returns {valid, message} JSON.
New helper:
- opencpu_validate_r_code(string ): array in Functions.php — calls
OpenCPU's /base/R/parse with json_encode()d code (so the POST value
arrives as an R string literal, not as raw R code), strips the
'R Error: ' prefix from parse failures, returns valid=true/false/null
for success/parse-error/server-down.
UI changes in the R Functions settings tab:
- Merged separate Save + Check R Code buttons into one always-clickable
'Save & Test R Syntax' button (no .save_settings class, so
run_settings.js leaves it alone — stays enabled regardless of dirty
state).
- Sequential flow: save the form to disk first, then validate syntax.
- Result rendered in a <pre> with white-space: pre-wrap so R's
multi-line error format (line number, source line, caret pointer) is
preserved exactly as printed.
- On parse failure, a 'Copy for LLM' button appears below the error
that copies a structured block:
TASK: Debug this R code syntax error.
CODE:
<user's code>
ERROR:
<full R error message>
- Progress indicator cycles Saving… → Checking syntax… → result.
- Results display above the editor (always in view, not scrolled off).
Fixes to the secrets management UI (incidental):
- Enter key in secret-name or secret-value fields triggers Add.
- Focus returns to the name field after adding a secret.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.