refactor: gRPC/Armeria migration with full module boundary enforcement

[yasithdev]

Summary

Complete modernization of Airavata's service architecture — replaces Thrift with gRPC/Armeria, consolidates entities (-107K lines net), enforces module boundaries, adds server-side user storage API, and provides a transport-agnostic Python SDK facade.

1,681 files changed | 88,340 insertions | 193,478 deletions

What Changed

Service Communication: Thrift → gRPC/Armeria

• Unified server on port 9090: gRPC + REST (HTTP/JSON transcoding) via Armeria
• 23 proto service definitions, 249 RPC methods, all with REST annotations
• Centralized GrpcStatusMapper for consistent error codes across all handlers
• GrpcAuthInterceptor for Bearer token authentication
• DocService at /docs

Entity Consolidation

• Remove 55 dead entity/PK classes across 5 modules
• Consolidate status/error/IO entities with discriminator columns
• Convert child tables to JSON columns
• Flyway V1 (baseline) + V2 (consolidation) covering all 76 entity tables

Module Boundary Enforcement

~45 entities/repositories relocated to correct bounded-context modules. All cross-module access via shared interfaces.

Move	From → To
App catalog + parsers	compute → research
Agent deployment info	agent → compute
Community users	credential → iam
Usage reporting commands	iam → compute
Tags, resources, resource stars	research → sharing
User storage preferences	storage → compute
Resource scheduling entities	orchestration → compute

Server-Side User Storage API (NEW)

New UserStorageService gRPC API (13 RPCs) for managing user files on any registered storage resource:

RPC	REST Path	Description
UploadFile	POST /api/v1/user-storage/{id}/files/{path}	Upload file to storage
DownloadFile	GET /api/v1/user-storage/{id}/files/{path}	Download file content
FileExists	GET /api/v1/user-storage/{id}/files/{path}:exists	Check file existence
DirExists	GET /api/v1/user-storage/{id}/dirs/{path}:exists	Check directory existence
ListDir	GET /api/v1/user-storage/{id}/dirs/{path}	List directory contents
DeleteFile	DELETE /api/v1/user-storage/{id}/files/{path}	Delete file
DeleteDir	DELETE /api/v1/user-storage/{id}/dirs/{path}	Delete directory
MoveFile	POST /api/v1/user-storage/{id}/files/{path}:move	Move/rename file
CreateDir	POST /api/v1/user-storage/{id}/dirs/{path}	Create directory
CreateSymlink	POST /api/v1/user-storage/{id}/files/{path}:symlink	Create symlink
GetFileMetadata	GET /api/v1/user-storage/{id}/files/{path}:metadata	Get file metadata
ListExperimentDir	GET /api/v1/user-storage/experiments/{id}/dirs/{path}	List experiment dir
GetDefaultStorageResourceId	GET /api/v1/user-storage/default-storage-resource	Get default storage

Eliminates client-side storage backends — portal just calls the SDK.

Python SDK: Transport-Agnostic Facade

from airavata_sdk import AiravataClient

client = AiravataClient(host, port, token, gateway_id)
client.research.create_experiment(...)
client.storage.upload_file(path, content, name)
client.sharing.share_resource_with_users(...)

Sub-client	Methods	Wraps
client.compute	68	ResourceService, Gateway/Group/User ResourceProfile
client.storage	28	Storage resources, data movement, user storage operations
client.credential	10	CredentialService
client.research	107	Experiment, Project, AppCatalog, Parser, DataProduct, Notification + 5 research services
client.iam	30	GatewayService, IamAdmin, UserProfile
client.sharing	81	SharingService, GroupManager
client.agent	24	AgentInteraction, Plan

Infrastructure

• Dockerfile health check: port 9090, /internal/actuator/health
• persistence.xml regenerated (76 entities match code)
• Explicit @Table annotations on all entities
• All cross-module repo instantiation replaced with interface injection

Modules After Refactor

Module	Responsibility
agent-service	Agent sidecar lifecycle, bidirectional gRPC, execution state
compute-service	HPC resource catalog, profiles, scheduling, agent deployment
credential-service	SSH keys, passwords, credential store
storage-service	Storage resources, data movement, user storage file API
research-service	Research catalog, app catalog, parsing, data products, replicas
iam-service	Identity, access management, gateways, user profiles
sharing-service	Permissions, groups, tags, resource discovery
orchestration-service	Process/task/job execution, workflow orchestration

Test Plan

• mvn clean compile -T4 — BUILD SUCCESS
• mvn test -T4 — 23/23 tests pass
• Python SDK imports verified (7 sub-clients, 348 methods)
• Architect review: no circular deps, no orphaned files, no stale references
• LaunchExperiment flow verified — server handles storage setup, no client pre-processing needed
• tilt up — verify health at http://localhost:9090/internal/actuator/health
• Integration tests: mvn test -pl airavata-api -Dgroups=runtime

🤖 Generated with Claude Code