Skip to content

[Feature] Notifications/Transfer Event Support #6

Description

@therangerangler

Summary

Add support for publishing transfer events to AWS SNS and/or SQS, allowing File Mover Express to integrate with other services and workflows when transfers complete, fail, or encounter errors.

Background

File Mover Express already has an internal event bus (src/cli/events/bus.go) that fires well-defined events throughout the transfer lifecycle — job created, job complete, job error, task complete, and more. Today these events are consumed internally by the GUI and logged. They are not exposed externally.

Adding SNS/SQS as an optional event destination would allow users to:

  • Trigger downstream workflows when a transfer job completes (e.g. kick off a transcoding pipeline)
  • Alert on transfer failures via email, SMS, or PagerDuty through SNS subscriptions
  • Feed transfer events into other services via SQS consumers
  • Build audit trails of transfer activity without polling

Existing events available to publish

The following events are already defined and carry useful data:

  • JobCompleteEvent — job ID, name, completion time, whether any tasks errored or were skipped
  • JobErrorEvent — job ID, name, error time, error message
  • TaskCompleteEvent — task ID, direction (upload/download), destination path
  • JobCreateEvent — fired when a new job is queued
  • JobProgressEvent — periodic progress updates

Proposed approach

Add an optional notification configuration block to the config file:

notifications:
  sns:
    enabled: true
    topic_arn: "arn:aws:sns:us-east-1:123456789:my-topic"
    events:
      - job_complete
      - job_error
  sqs:
    enabled: false
    queue_url: "https://sqs.us-east-1.amazonaws.com/123456789/my-queue"
    events:
      - job_complete
      - job_error
      - task_complete

On startup, if notifications are configured, register a new listener on the existing event bus. When a matching event fires, publish it to the configured SNS topic or SQS queue as a JSON payload.

The existing AWS credentials (profile/region from the transfer profile) should be reused for SNS/SQS access.

Implementation notes

  • The event bus already supports RegisterListener with event type filters — a notification publisher would just be another listener
  • Each event type already implements String() and carries structured data, making JSON serialisation straightforward
  • SNS and SQS both use the AWS SDK v2 which is already a dependency
  • IAM permissions for sns:Publish or sqs:SendMessage would need to be added to the setup documentation

Relevant files

  • src/cli/events/bus.go — event bus, RegisterListener is the integration point
  • src/cli/types/eventtypes/ — all event type definitions
  • src/cli/types/configtypes/config.go — where the new notifications config block would be added
  • docs/Configuration.md — would need updating with the new config options
  • docs/Setup.md — would need the additional IAM permissions documented

Acceptance criteria

  • SNS and/or SQS publishing can be enabled via configuration
  • Users can choose which event types trigger a notification
  • Notifications are non-blocking — a failure to publish should log a warning but not affect the transfer
  • Credentials use the existing AWS profile configuration
  • Documentation updated for configuration and required IAM permissions

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions