refactor: Convert RESTful endpoints to encrypted Protobuf and fix issues - #4
Merged
Conversation
- Add GoFrame CLI generated code headers to main account API file - Rename all account service methods to use 'Gf' prefix for GoFrame compatibility (GfListAccounts, GfCreateAccount, etc.) - Replace original request/response types with GoFrame wrapper types that include g.Meta annotations - Generate GoFrame-compatible request wrappers with proper HTTP method mappings and metadata - Maintain backward compatibility while adding GoFrame framework support
refactor(balance): update test coverage for balance logic
…nt POST methods - Update all API endpoint paths to include /v1 version prefix - Change HTTP methods from various (GET, PUT, DELETE) to consistent POST for all endpoints - Remove pointer embedding from request structs for consistency - Standardize URL path naming to use action-based patterns (e.g., /list-accounts, /create-account) instead of resource-based patterns - Apply changes across all API modules (account, auth, config, dashboard, data, health, task, transaction, user) fix(internal): update database migration file reference - Replace test data SQL file reference with proper initialization script refactor(internal): remove hello controller and update bootstrapping - Remove unused hello controller module - Update command initialization to exclude hello service binding
…/response handling, including nonce and session key management with Redis integration.
- Move Redis initialization logic to centralized boot/redis.go - Replace direct Redis client creation with unified Redis adapter - Add Redis type constants for sync_lock and ale purposes - Remove os package import from ALE and sync modules - Change ALE middleware logging from Debugf to Infof for decrypted content
fix(account): update account logic to use DAO column references feat(account): implement proper balance handling for account operations fix(account): add user ownership verification for account updates refactor(account): improve account creation with default currency feat(transaction): pass transaction context to service methods fix(seeding): correct table name in database migration check refactor(dashboard): use DAO column references for dashboard queries refactor(task): use DAO column references for task queries refactor(balance): update balance updates to use DAO column references
…sponse Add new count_without_equity field to GetAccountTransactionCountRes to track transaction counts excluding equity transactions. BREAKING CHANGE: GetAccountTransactionCount now returns two values (count, countWithoutEquity) instead of one. feat(user): add main currency field to user profile update Add main_currency field to UserInput and UpdateUserInput models to allow users to set their preferred main currency. refactor(account): improve account deletion logic with transaction safety Replace manual account deletion logic with transaction-safe approach using new SoftDelete helper function. Add proper error handling and rollback mechanisms for account deletion operations. refactor(utils): add generic auth utility functions for resource verification Introduce FieldAccessor pattern and GetAndVerify function to centralize resource access control and user ownership verification logic across different entities. refactor(transaction): update CreateTransaction to use TX interface directly Change CreateTransaction signature to accept gdb.TX interface directly instead of pointer to transaction, improving consistency and reducing pointer-related complexity. chore(generated): remove timestamps from auto-generated files Remove creation timestamps from generated protobuf and DAO files to prevent unnecessary changes during regeneration. style(auth): clean up spacing in auth wrapper definitions Add consistent blank lines between GoFrame wrapper type definitions for improved code readability.
- Replace Task model with TaskOutput[P, R] generic type
- Introduce TaskPayload and TaskResult type constraints
- Update all Task references to use TaskOutput[any, any]
- Add proper time type imports and replace interface{} with *gtime.Time
- Update TaskCreateInput to use generic parameter for payload type
BREAKING CHANGE: Task model now uses generic types requiring explicit type parameters
- Register separate Redis instance for caching (DB1) in boot/redis.go - Implement caching for account operations with automatic cache invalidation - Add caching for dashboard summaries and monthly stats with short TTL - Implement caching for task operations with cache invalidation on state changes - Add caching for transaction operations with cache invalidation on updates - Implement caching for user profiles with cache invalidation on updates - Extend account migration logic to use proper double-entry bookkeeping - Add cache invalidation for account and transaction operations - Introduce cache utility functions for get-or-load and invalidate patterns The caching implementation provides graceful degradation when Redis is unavailable and maintains data consistency through strategic cache invalidation. fix: restrict manual balance updates for equity accounts during migrations - Prevent manual balance modifications for EXPENSE, INCOME, and EQUITY account types - Only allow balance changes through transaction-based operations - Enhance account migration to preserve accounting equation using equity accounts
- Add functionality to persist dashboard snapshots to the database, allowing for recovery after Redis restarts. - Introduce a new table `dashboard_snapshots` to store snapshot data with unique constraints on user ID, snapshot type, and snapshot key. - Implement methods for persisting, loading, and restoring snapshots from the database. - Create a worker to handle dashboard refresh events and trigger snapshot rebuilds based on user actions. - Set up a periodic flush mechanism to persist all snapshots from Redis to the database at configurable intervals. - Update cache configuration to include a snapshot flush interval, defaulting to 24 hours. - Modify transaction and task processing logic to trigger dashboard snapshot rebuilds after relevant operations.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements a comprehensive refactoring of the API architecture to use Application Layer Encryption (ALE) combined with Protobuf serialization. The changes convert all RESTful endpoints to encrypted Protobuf format, add Redis-backed dashboard snapshot caching, and introduce a message queue worker for asynchronous dashboard updates.
Changes:
- Migrated all API endpoints from RESTful to ALE+Protobuf format with new
Gfprefixed controller methods - Implemented Redis-backed dashboard snapshot system with periodic persistence to database
- Added HMAC signature verification and session key management for enhanced security
- Removed debug endpoints and hello world examples
- Updated database schema with dashboard_snapshots table and modified account/transaction columns for Money type
Reviewed changes
Copilot reviewed 143 out of 224 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/logic/dashboard/worker.go | Adds dashboard worker for debounced snapshot rebuilding via RabbitMQ |
| internal/logic/dashboard/snapshot_persist.go | Implements Redis-to-DB snapshot persistence with restore functionality |
| internal/crypto/aes_gcm.go | Adds HMAC signing, session key generation, and hex key support |
| internal/ale/session_key.go | Manages ALE session keys with Redis storage and in-memory fallback |
| internal/boot/boot.go | Initializes ALE system and starts dashboard workers |
| internal/controller//.go | Converts controllers to use ALE+Protobuf with helper conversion functions |
| api/**/v1/*.go | Updates API definitions to use Protobuf-generated types with GoFrame wrappers |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…action.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
gin-melodic
added a commit
that referenced
this pull request
Aug 24, 2026
* feat: implement user authentication with login, registration, 2FA, JWT, and Turnstile, along with initial test data. * CI & Template Implementation (#1) * feat: Implement pull request template and GitHub Actions CI workflow for Go projects. * feat: Add loading indicator for assistant messages and refine message streaming logic * feat: add support for new data processing pipeline and update related configurations * feat: improve account delete logic. (#2) * feat: improve account delete logic. Delete account directly if has not transport records. * fix:account unit test * Feature/import export (#3) * feat:import & export function. * feat:import & export function. * refactor: Convert RESTful endpoints to encrypted Protobuf and fix issues (#4) * feat(account): update API interface with GoFrame compatible wrappers - Add GoFrame CLI generated code headers to main account API file - Rename all account service methods to use 'Gf' prefix for GoFrame compatibility (GfListAccounts, GfCreateAccount, etc.) - Replace original request/response types with GoFrame wrapper types that include g.Meta annotations - Generate GoFrame-compatible request wrappers with proper HTTP method mappings and metadata - Maintain backward compatibility while adding GoFrame framework support * test: refactor test suite to support new money representation model * refactor(go.mod): remove unused gogf grpcx and file registry dependencies * feat(ale): implement Application Layer Encryption system refactor(balance): update test coverage for balance logic * refactor(utils): remove unused helper functions and simplify balance logic * refactor(api): standardize API endpoints with versioning and consistent POST methods - Update all API endpoint paths to include /v1 version prefix - Change HTTP methods from various (GET, PUT, DELETE) to consistent POST for all endpoints - Remove pointer embedding from request structs for consistency - Standardize URL path naming to use action-based patterns (e.g., /list-accounts, /create-account) instead of resource-based patterns - Apply changes across all API modules (account, auth, config, dashboard, data, health, task, transaction, user) fix(internal): update database migration file reference - Replace test data SQL file reference with proper initialization script refactor(internal): remove hello controller and update bootstrapping - Remove unused hello controller module - Update command initialization to exclude hello service binding * feat: Introduce Application Layer Encryption (ALE) for secure request/response handling, including nonce and session key management with Redis integration. * chore(deps): update Go dependencies and refactor Redis initialization - Move Redis initialization logic to centralized boot/redis.go - Replace direct Redis client creation with unified Redis adapter - Add Redis type constants for sync_lock and ale purposes - Remove os package import from ALE and sync modules - Change ALE middleware logging from Debugf to Infof for decrypted content * feat(auth): make JWT token expiry configurable via env vars and config fix(account): update account logic to use DAO column references feat(account): implement proper balance handling for account operations fix(account): add user ownership verification for account updates refactor(account): improve account creation with default currency feat(transaction): pass transaction context to service methods fix(seeding): correct table name in database migration check refactor(dashboard): use DAO column references for dashboard queries refactor(task): use DAO column references for task queries refactor(balance): update balance updates to use DAO column references * feat(account): add count_without_equity field to transaction count response Add new count_without_equity field to GetAccountTransactionCountRes to track transaction counts excluding equity transactions. BREAKING CHANGE: GetAccountTransactionCount now returns two values (count, countWithoutEquity) instead of one. feat(user): add main currency field to user profile update Add main_currency field to UserInput and UpdateUserInput models to allow users to set their preferred main currency. refactor(account): improve account deletion logic with transaction safety Replace manual account deletion logic with transaction-safe approach using new SoftDelete helper function. Add proper error handling and rollback mechanisms for account deletion operations. refactor(utils): add generic auth utility functions for resource verification Introduce FieldAccessor pattern and GetAndVerify function to centralize resource access control and user ownership verification logic across different entities. refactor(transaction): update CreateTransaction to use TX interface directly Change CreateTransaction signature to accept gdb.TX interface directly instead of pointer to transaction, improving consistency and reducing pointer-related complexity. chore(generated): remove timestamps from auto-generated files Remove creation timestamps from generated protobuf and DAO files to prevent unnecessary changes during regeneration. style(auth): clean up spacing in auth wrapper definitions Add consistent blank lines between GoFrame wrapper type definitions for improved code readability. * refactor(task): update Task model to use generic types with constraints - Replace Task model with TaskOutput[P, R] generic type - Introduce TaskPayload and TaskResult type constraints - Update all Task references to use TaskOutput[any, any] - Add proper time type imports and replace interface{} with *gtime.Time - Update TaskCreateInput to use generic parameter for payload type BREAKING CHANGE: Task model now uses generic types requiring explicit type parameters * feat: add Redis caching layer and implement cache invalidation strategy - Register separate Redis instance for caching (DB1) in boot/redis.go - Implement caching for account operations with automatic cache invalidation - Add caching for dashboard summaries and monthly stats with short TTL - Implement caching for task operations with cache invalidation on state changes - Add caching for transaction operations with cache invalidation on updates - Implement caching for user profiles with cache invalidation on updates - Extend account migration logic to use proper double-entry bookkeeping - Add cache invalidation for account and transaction operations - Introduce cache utility functions for get-or-load and invalidate patterns The caching implementation provides graceful degradation when Redis is unavailable and maintains data consistency through strategic cache invalidation. fix: restrict manual balance updates for equity accounts during migrations - Prevent manual balance modifications for EXPENSE, INCOME, and EQUITY account types - Only allow balance changes through transaction-based operations - Enhance account migration to preserve accounting equation using equity accounts * feat: Implement dashboard snapshot persistence and management - Add functionality to persist dashboard snapshots to the database, allowing for recovery after Redis restarts. - Introduce a new table `dashboard_snapshots` to store snapshot data with unique constraints on user ID, snapshot type, and snapshot key. - Implement methods for persisting, loading, and restoring snapshots from the database. - Create a worker to handle dashboard refresh events and trigger snapshot rebuilds based on user actions. - Set up a periodic flush mechanism to persist all snapshots from Redis to the database at configurable intervals. - Update cache configuration to include a snapshot flush interval, defaulting to 24 hours. - Modify transaction and task processing logic to trigger dashboard snapshot rebuilds after relevant operations. * feat(auth): add default user plan during registration * fix: update go.mod and go.sum after go mod tidy * Update internal/controller/auth/auth_v1_gf_logout.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/controller/transaction/transaction_v1_gf_update_transaction.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/crypto/aes_gcm.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/boot/redis.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * feat: add main currency field to registration process and validate currency input * feat(auth): add password update functionality and improve logging Add UpdatePassword method to auth service with validation and transaction support Move auth routes to authenticated group and skip auth for public routes Improve proto logging with key-value formatting and request/response details Update documentation for cache usage in various service methods * fix(middleware): reorder auth middleware and improve ale validation Ensure AuthMiddleware runs before ALEMiddleware for proper session validation. Add debug logging and enhance ALE mode handling by: - Checking for auth header in session mode - Falling back to bootstrap mode when needed - Adding detailed debug logs for troubleshooting * feat: Implement initial database schema, account management service, and related user and task components. * feat: implement account management logic and JWT-based authentication middleware * feat: implement application bootstrapper, authentication logic, and development build script * refactor(dashboard): improve money summation logic in dashboard Replace index-based initialization with nil check for clearer intent and more robust handling of empty account lists * build:support air for macOS. * feat: implement ALE middleware for encrypted request handling and integrate data export task logic * fix: harden beta accounting and session flows * feat: add read-only accounting reconciliation gate * fix: close beta money and input boundaries * feat: automate demo data and extend balance trends --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Related Issue
Description of Changes
This PR merges the
feature/feature-pbbranch intodevelopfor the gaap-api backend project. The changes implement a significant refactor of the API architecture:Convert RESTful interfaces to ALE+PB combination: All existing RESTful API endpoints have been migrated to use Application Layer Encryption (ALE) combined with Protobuf serialization for enhanced security and type safety. This aligns with the project's security-first approach and Protobuf-first design pattern.
Fix all interface issues and tests: Comprehensive fixes have been applied to resolve interface inconsistencies, error handling, and validation problems. Unit tests have been updated and expanded to cover the new ALE+PB implementation, including property-based tests for financial data integrity and boundary conditions.
The changes include updates to controllers, logic layers, and middleware to support the new encrypted Protobuf communication. Frontend protobuf types have been regenerated, and all related services have been updated to handle the new API format.
Type of Change
Checklist
Special Notes
This is a breaking change that requires frontend updates to use the new ALE+PB endpoints. The migration maintains backward compatibility during the transition period through feature flags. Property-based tests ensure mathematical correctness for all financial operations. Reviewed for compliance with GAAP's "Boundary Verification Strategy" and "AI-Native Development" philosophy. Database schema remains unchanged.