Skip to content

Feature/update cloud example#345

Merged
LorenzzoQM merged 2 commits into
developfrom
feature/update_cloud_example
Jul 18, 2026
Merged

Feature/update cloud example#345
LorenzzoQM merged 2 commits into
developfrom
feature/update_cloud_example

Conversation

@LorenzzoQM

Copy link
Copy Markdown
Contributor

Description

Closes #342

Updates the cloud with reimaging example script to incorporate recent changes and new actor-critic network architecture.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How should this pull request be reviewed?

  • By commit
  • All changes at once

Future Work

Update with the related publication information once available.

Checklist

  • I have performed a self-review of my code
  • I have commented my code in hard-to-understand areas
  • I have made corresponding changes to the documentation and release notes
  • My changes generate no new warnings
  • [] I have added tests that prove my fix is effective or that my feature works

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the “Cloud Environment with Re-imaging” example notebook to reflect the updated observation/action space and demonstrates new greedy heuristics plus a custom token/attention-based actor-critic architecture for RLlib PPO. Also documents the update in the v1.3.0 release notes.

Changes:

  • Refresh the environment configuration (observations/actions/lookahead) and reward logic in cloud_environment_with_reimaging.ipynb.
  • Add greedy heuristic baselines (re-imaging and single-picture) and a custom RLlib RLModule with attention-based actor/critic definitions.
  • Add a release note entry indicating the example has been updated.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
examples/cloud_environment_with_reimaging.ipynb Updates the cloud re-imaging tutorial with new observation/action definitions, heuristics, and custom actor-critic/RLModule code.
docs/source/release_notes.rst Adds a release note entry referencing the updated example notebook.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
@LorenzzoQM
LorenzzoQM force-pushed the feature/update_cloud_example branch 2 times, most recently from 0fcd701 to a41a458 Compare July 13, 2026 22:11
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb
Comment thread examples/cloud_environment_with_reimaging.ipynb Outdated
Comment thread examples/cloud_environment_with_reimaging.ipynb
@LorenzzoQM
LorenzzoQM force-pushed the feature/update_cloud_example branch from a41a458 to 9da53ac Compare July 17, 2026 20:07
@LorenzzoQM
LorenzzoQM force-pushed the feature/update_cloud_example branch from 9da53ac to baf361b Compare July 17, 2026 20:12

@Yume27 Yume27 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great example and show the new structure! Maybe good to think about splitting the code into two later, but good for now. And passed the tests and ran correctly.

@LorenzzoQM
LorenzzoQM merged commit 06d9454 into develop Jul 18, 2026
13 of 14 checks passed
@LorenzzoQM
LorenzzoQM deleted the feature/update_cloud_example branch July 18, 2026 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update cloud example with re-imaging

3 participants