[WIP] Adding MultiAgent Utilities by AdityaKapoor74 · Pull Request #323 · SforAiDl/genrl

AdityaKapoor74 · 2020-09-03T05:38:21Z

No description provided.

lgtm-com · 2020-09-03T06:03:03Z

This pull request introduces 1 alert when merging 0927001 into c3ef4d9 - view on LGTM.com

new alerts:

1 for Variable defined multiple times

Sharad24 · 2020-09-03T06:03:57Z

Looks pretty good.

lgtm-com · 2020-09-03T07:37:43Z

This pull request introduces 2 alerts when merging daa8b2a into c3ef4d9 - view on LGTM.com

new alerts:

1 for Syntax error
1 for Variable defined multiple times

lgtm-com · 2020-09-03T08:37:37Z

This pull request introduces 2 alerts when merging 44db72e into c3ef4d9 - view on LGTM.com

new alerts:

1 for Syntax error
1 for Variable defined multiple times

lgtm-com · 2020-09-03T09:22:44Z

This pull request introduces 2 alerts when merging d8cf1a9 into c3ef4d9 - view on LGTM.com

new alerts:

1 for Syntax error
1 for Variable defined multiple times

lgtm-com · 2020-09-03T10:14:30Z

This pull request introduces 5 alerts when merging 1365585 into c3ef4d9 - view on LGTM.com

new alerts:

2 for Syntax error
2 for First argument to super() is not enclosing class
1 for Variable defined multiple times

Sharad24 · 2020-09-03T11:18:37Z

        self.env.close()
+
+
+class MultiGymWrapper(gym.Wrapper):


Whats the difference between this and a normal Wrapper? Cant you use a TorchWrapper instead of creating this?

This is just a template, I haven't added anything yet

Sharad24

Can you install pre-commit? You can see the instructions in Contributing.md

lgtm-com · 2020-09-03T13:01:20Z

This pull request introduces 5 alerts when merging 6f0563e into c3ef4d9 - view on LGTM.com

new alerts:

2 for Syntax error
2 for First argument to super() is not enclosing class
1 for Variable defined multiple times

lgtm-com · 2020-09-04T06:19:20Z

This pull request introduces 7 alerts when merging 5061abe into 0fe4180 - view on LGTM.com

new alerts:

2 for Mismatch between signature and use of an overriding method
2 for Wrong number of arguments in a class instantiation
1 for First argument to super() is not enclosing class
1 for Variable defined multiple times
1 for Wrong number of arguments in a call

codecov · 2020-09-04T07:10:11Z

Codecov Report

Merging #323 into master will decrease coverage by 3.46%.
The diff coverage is 36.07%.

@@            Coverage Diff             @@
##           master     #323      +/-   ##
==========================================
- Coverage   90.77%   87.31%   -3.47%     
==========================================
  Files          89       89              
  Lines        3772     4027     +255     
==========================================
+ Hits         3424     3516      +92     
- Misses        348      511     +163

Impacted Files	Coverage Δ
genrl/agents/deep/a2c/a2c.py	`93.58% <ø> (ø)`
genrl/agents/deep/ddpg/ddpg.py	`83.05% <0.00%> (-4.45%)`	⬇️
genrl/core/rollout_storage.py	`56.04% <9.21%> (-33.58%)`	⬇️
genrl/environments/suite.py	`73.68% <11.11%> (-19.42%)`	⬇️
genrl/core/buffers.py	`63.72% <12.82%> (-31.52%)`	⬇️
genrl/environments/gym_wrapper.py	`54.21% <27.50%> (-24.86%)`	⬇️
genrl/agents/deep/ppo1/ppo1.py	`97.61% <33.33%> (-2.39%)`	⬇️
genrl/utils/utils.py	`86.62% <78.82%> (-7.63%)`	⬇️

Sharad24 · 2020-09-04T10:49:13Z


  - repo: https://github.com/timothycrosley/isort
-    rev: 4.3.2
+    rev: 5.5.0


Its very risky to change the versions here because of compatibility. Can you revert these?

When I change that I get some errors. Any other way to rectify it?

Have you changed the isort.cfg file?

Sharad24 · 2020-09-04T11:20:43Z

+class Critic(MlpValue):
+    def __init__(
+        self,
+        state_dim: spaces.Space,
+        action_dim: spaces.Space,
+        fc_layers: Tuple = (32, 32),
+        val_type: str = "V",
+        **kwargs,
+    ):
+        super(Critic, self).__init__(
+            state_dim, action_dim, fc_layers, val_type, **kwargs
+        )
+
+        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+    def forward(self, state):
+
+        state = self.model(state)
+
+        return state
+
+    def get_value(self, state):
+        # state = torch.FloatTensor(state).to(self.device)
+        value = self.forward(state)
+        return value


Whats different in this Critic? You could just use the MlpValue directly?

I'll add in the changes on the fly. I am just creating a template so that it can be used readily

… shared

Shared single ACs

…enrl into multiagentutils

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

hades-rp2010 and others added 7 commits September 1, 2020 15:49

Single actor critic shared params

1d49049

Shared layers for multi ACs

ef4a179

Merge branch 'master' of https://github.com/SforAiDl/genrl

2ecd086

Fix lint errors (1)

53450a8

Fixed tests

274aff9

Changes to dicstrings and classes

38f95f0

adding MultiAgentBuffer

0927001

shared mlp

daa8b2a

adding changes

44db72e

AdityaKapoor74 added 2 commits September 3, 2020 14:14

new mlp for maddpg

4ef8f48

adding environment loader

d8cf1a9

AdityaKapoor74 added 2 commits September 3, 2020 15:10

Adding Actor and Critic classes

8d2cf06

adding new functionalities

1365585

Sharad24 reviewed Sep 3, 2020

View reviewed changes

Comment thread genrl/core/actor_critic.py Outdated

Sharad24 reviewed Sep 3, 2020

View reviewed changes

Comment thread genrl/utils/utils.py Outdated

Sharad24 reviewed Sep 3, 2020

View reviewed changes

AdityaKapoor74 added 2 commits September 3, 2020 18:00

minor changes

5067e42

added return statement to mlp_

6f0563e

rectifying

5061abe

rectifying 2

e6a378c

Sharad24 reviewed Sep 4, 2020

View reviewed changes

hades-rp2010 and others added 6 commits September 4, 2020 22:55

Renaming Multi -> Two and comments

835819e

changing names

793c045

changing names

2635fd5

Merge branch 'master' of https://github.com/AdityaKapoor74/genrl into…

cd87506

… shared

Shared params for single ACs

65b6520

Merge branch 'multiagentutils' into shared

3d01b85

veds12 reviewed Oct 2, 2020

View reviewed changes

Comment thread genrl/utils/utils.py Outdated

veds12 reviewed Oct 2, 2020

View reviewed changes

Comment thread genrl/core/actor_critic.py Outdated

AdityaKapoor74 added 2 commits October 4, 2020 09:35

Merge pull request #1 from hades-rp2010/shared

a62c100

Shared single ACs

Merge branch 'master' into multiagentutils

841ff66

veds12 reviewed Oct 4, 2020

View reviewed changes

Comment thread genrl/agents/deep/ppo1/ppo1.py Outdated

veds12 reviewed Oct 4, 2020

View reviewed changes

Comment thread genrl/core/actor_critic.py

veds12 requested changes Oct 4, 2020

View reviewed changes

Comment thread genrl/core/actor_critic.py

Comment thread genrl/core/actor_critic.py

Comment thread genrl/core/actor_critic.py Outdated

AdityaKapoor74 and others added 16 commits October 4, 2020 11:39

rollout buffer for MA

2be8df5

Merge branch 'multiagentutils' of https://github.com/AdityaKapoor74/g…

ac9b5a8

…enrl into multiagentutils

Update genrl/utils/utils.py

10282f0

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

Update genrl/agents/deep/ppo1/ppo1.py

79b531b

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

Update genrl/core/actor_critic.py

a3885a0

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

Update genrl/core/actor_critic.py

e3dc677

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

Update genrl/core/actor_critic.py

4c2ad51

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

Update genrl/core/actor_critic.py

43554e4

Co-authored-by: Vedant Shah <vedantshah2012@gmail.com>

removing SharedAC class

6828e93

removing SharedAC class

eac920c

rectify

194065f

rectify

c0198bc

rectify

fe40835

rectifying

a50204a

removing unecessary code

4a3cd74

removing unecessary code

602a7b5

Uh oh!

Conversation

AdityaKapoor74 commented Sep 3, 2020

Uh oh!

lgtm-com Bot commented Sep 3, 2020

Uh oh!

Sharad24 commented Sep 3, 2020

Uh oh!

lgtm-com Bot commented Sep 3, 2020

Uh oh!

lgtm-com Bot commented Sep 3, 2020

Uh oh!

lgtm-com Bot commented Sep 3, 2020

Uh oh!

lgtm-com Bot commented Sep 3, 2020

Uh oh!

Uh oh!

Uh oh!

Sharad24 Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

AdityaKapoor74 Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

Sharad24 left a comment

Choose a reason for hiding this comment

Uh oh!

lgtm-com Bot commented Sep 3, 2020

Uh oh!

lgtm-com Bot commented Sep 4, 2020

Uh oh!

codecov Bot commented Sep 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Sharad24 Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

AdityaKapoor74 Sep 5, 2020

Choose a reason for hiding this comment

Uh oh!

sampreet-arthi Sep 5, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Sharad24 Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

AdityaKapoor74 Sep 5, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Sep 4, 2020 •

edited

Loading