[Suggestion- Extending Results] FAI as alternative uses-cases to MCTS

[0bserver07]

• One of the big results in "Learning to Plan Chemical Syntheses" or "Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies" is the MCTS with NNs. Perhaps if there is a way to create a separate agent (place-holder) for other models, we can extend the use to further applications, specifically synthesis models through RL bases realm.

[sergio-hcsoft]

Totally doable, especially having the LJ potential as a reward. We already tried it as an optimization problem, you have the results in the GAS paper in arxiv https://arxiv.org/abs/1705.08691

[Guillemdb]

• Short answer: Just change MCTS for FMC. It will probably improve, because FMC tends to perform well in those kinds of energy landscapes.

I will take a look at the papers to give a more specific answer.

On Monday, I will add in the technical report the easiest ways to combine our techniques with NNs, and discuss in the architecture section a more efficient way to implement the algorithm.

Please, do not try to use this code in a real world application, because it wont scale well. It is meant to be used in my talks. If you are willing to spend some time trying this on chemistry, it would be nice to know the specific problem you want to try, because there are several approaches that we think that may work in chemistry.

Do you know any library for chemistry in python which allows to do something similar than gym does for Atari games? Do you already have a model you can sample? If you do, we can discuss how to adapt it to work with FMC.

[Guillemdb]

If you are gonna take an AlphaZero like approach where you assume you have a good model, stationarity, ergodicity, and so on maybe this is better than FMC.

Swarm wave code. Swarm wave code example notebook. This single core implementation can generate on MsPacman about 30k high quality samples per minute. So properly scaled you should be able to feed NNs like a boss.

FMC is meant to be applied when the model we sample is super noisy, or to something difficult like robots. For something like Atari games is an overkill. When you have a perfect model a Swarm Wave performs about 100 times faster. Here and here you can find visual representations of what it does.