Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enlarge #2

Open
wants to merge 148 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
148 commits
Select commit Hold shift + click to select a range
2829001
customization to train with safetygym
CaesarAndylaw Feb 28, 2023
ea36361
headless video rendering implementation
CaesarAndylaw Feb 28, 2023
87c3f93
commented
CaesarAndylaw Feb 28, 2023
558698f
new logger for EpCost, new train safetygym that includes cpo, working…
CaesarAndylaw Mar 2, 2023
84301dd
add qp solution and line search
ruichen-v Mar 2, 2023
bd63cc6
add extra python path
ruichen-v Mar 2, 2023
5e24a61
run time logger implementation, CPO replay buffer, CPO training main …
CaesarAndylaw Mar 2, 2023
6a089a0
Merge remote-tracking branch 'origin/ruic_dev' into main
CaesarAndylaw Mar 2, 2023
d2dba34
fix the minor implementation bugs with logger, network, cpo, and main…
CaesarAndylaw Mar 3, 2023
6995617
initial implementation for CPO, and initial experiments
CaesarAndylaw Mar 5, 2023
193a2b7
successfully train the vanilla policy gradient for reward
CaesarAndylaw Mar 6, 2023
6c8d0cf
switch to an existing pytorch implementation of vanilla policy gradie…
CaesarAndylaw Mar 7, 2023
b7b0053
implementation of ppo, vpg with gpu training loop
CaesarAndylaw Mar 10, 2023
173d78a
trpo implementation completed
CaesarAndylaw Mar 12, 2023
a66e8f6
before meerge
CaesarAndylaw Mar 13, 2023
dc99116
add buf for cpo
ruichen-v Mar 13, 2023
5fb42e1
Core implementation for CPO, the update function
CaesarAndylaw Mar 13, 2023
0e4e655
fix extra char
CaesarAndylaw Mar 13, 2023
2a6a4d2
Merge remote-tracking branch 'origin/ruic_dev' into trpo_test_reward
CaesarAndylaw Mar 13, 2023
cc72d87
merged cpo buf
CaesarAndylaw Mar 13, 2023
7052624
complete version of spinup implementation of constrained policy optim…
CaesarAndylaw Mar 14, 2023
3bfbf4c
remove results
CaesarAndylaw Mar 14, 2023
182fc6e
remove results
CaesarAndylaw Mar 14, 2023
d1354b9
add safety-gym-arm
YIFANSUN98 Mar 16, 2023
b857651
update the plot function to consider the cost rate
CaesarAndylaw Mar 16, 2023
05b1058
update safety gym arm environment. include safety gym arm environment…
CaesarAndylaw Mar 16, 2023
a5a5627
policy model visualization is constructed for PPO
CaesarAndylaw Mar 16, 2023
d6193e7
add arm model with 3 joints
YIFANSUN98 Mar 16, 2023
3b0d613
cpo and trpo video recorder
CaesarAndylaw Mar 17, 2023
e2ee555
select maximum epoch for trpo video recorder
CaesarAndylaw Mar 17, 2023
61f9fbb
update vpg training to include arm environment. cancel the limit of t…
CaesarAndylaw Mar 17, 2023
f01a442
update config
CaesarAndylaw Mar 17, 2023
66c6f07
add walker ant swimmer model
YIFANSUN98 Mar 17, 2023
fe54d27
add swimmer ant walker
YIFANSUN98 Mar 17, 2023
5d1d967
update the cg computation to make hessian computation much accurate
CaesarAndylaw Mar 17, 2023
ad6fee6
add config of new tasks
YIFANSUN98 Mar 17, 2023
37248dd
add config
YIFANSUN98 Mar 17, 2023
6b90a41
add config for new tasks
YIFANSUN98 Mar 17, 2023
c954558
correct implementation of KL computation, change the name of 6dof
CaesarAndylaw Mar 17, 2023
cb85bd6
Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_ben…
CaesarAndylaw Mar 17, 2023
ecbfc26
implement the correct KL divergence computation for CPO
CaesarAndylaw Mar 17, 2023
1c212dc
update walker3d xml
YIFANSUN98 Mar 19, 2023
96ff7ba
stable version of CPO and TRPO
CaesarAndylaw Mar 20, 2023
70bc827
addressed the incoming change of humanoid xml
CaesarAndylaw Mar 20, 2023
17f3843
delete all the original saferl-kit files, they are no longer in use
CaesarAndylaw Mar 20, 2023
ca85623
Adds safety_gym
Vinnie6167 Mar 21, 2023
ceacea9
add ant_tiny
YIFANSUN98 Mar 21, 2023
9e87739
Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_ben…
YIFANSUN98 Mar 21, 2023
6d27bd4
update cpo and robot configuration
CaesarAndylaw Mar 21, 2023
9e4f3ae
update the cpo to enable tuning for cost reduction and make env robus…
CaesarAndylaw Mar 21, 2023
6e394a2
add hopper3d
YIFANSUN98 Mar 21, 2023
aa620c9
Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_ben…
YIFANSUN98 Mar 21, 2023
e5f2e71
create the folder of TRPOLAG, minor update for the safetygym configur…
CaesarAndylaw Mar 22, 2023
f70be42
add 3d hazards for arm3
YIFANSUN98 Mar 22, 2023
48414a3
Merge branch 'tmp' into main
YIFANSUN98 Mar 22, 2023
68febe6
implementation of TRPO-Lagrangian method
CaesarAndylaw Mar 22, 2023
891ccf8
Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_ben…
CaesarAndylaw Mar 22, 2023
56794e7
remove the nested loss cost call
CaesarAndylaw Mar 22, 2023
b3dfe6c
Merge branch 'trpolag' into main
CaesarAndylaw Mar 22, 2023
3b05b5a
stable version of the lagrangian implementation
CaesarAndylaw Mar 25, 2023
61dea2a
implementation of the TRPO feasible actor critic
CaesarAndylaw Mar 27, 2023
8d698e3
stable version of trpofac, and new implementation of SCPO
CaesarAndylaw Mar 27, 2023
4acdb27
add ghost tasks
YIFANSUN98 Mar 27, 2023
b3a3123
add ghost config
YIFANSUN98 Mar 27, 2023
dc2e15e
merge to main
YIFANSUN98 Mar 27, 2023
5f13473
implementation of SCPO progress
CaesarAndylaw Mar 27, 2023
3ccbe64
update the stable and default configurations of safe RL algorithms
CaesarAndylaw Mar 27, 2023
1e4cd39
initial complete implementation of SCPO, and update for the trpofac v…
CaesarAndylaw Mar 28, 2023
6315b2d
modify chase reward
YIFANSUN98 Mar 28, 2023
9c2ea6d
modify config for ghost tasks
YIFANSUN98 Mar 28, 2023
645ead2
merge to main
YIFANSUN98 Mar 28, 2023
3acf09d
update hopper3d
YIFANSUN98 Mar 28, 2023
3c5e2e0
more proper safetygym configuration
CaesarAndylaw Mar 28, 2023
8872a37
adjust hopper3d gear
YIFANSUN98 Mar 28, 2023
28a14e2
merge to main
YIFANSUN98 Mar 28, 2023
57b0d62
bug fixed for scpo, minor update for trpo and safetygym config
CaesarAndylaw Mar 29, 2023
b89ebac
initial implementation of eaiser learnt value function
CaesarAndylaw Apr 2, 2023
c9099de
start implementation of the cost sum value learn implementation
CaesarAndylaw Apr 3, 2023
a3f1207
new PCPO branch
Wei-TianHao Apr 3, 2023
4bc1577
scpo pnorm implementation and no continue experiments
CaesarAndylaw Apr 4, 2023
fe17187
a successful implementation of scpo, which bounds the maximum state w…
CaesarAndylaw Apr 5, 2023
39a82e8
found the key to the good performance, is to balance the data
CaesarAndylaw Apr 5, 2023
2704e9b
add drone
YIFANSUN98 Apr 6, 2023
328d892
stable implementation of scpo downsample, which works pretty well
CaesarAndylaw Apr 6, 2023
c9e28d8
merge to main
YIFANSUN98 Apr 6, 2023
2c2c994
scpo weight up tune
CaesarAndylaw Apr 7, 2023
a8f755d
add robot keepout range
YIFANSUN98 Apr 7, 2023
dc88de6
Merge branch 'main' into tmp
YIFANSUN98 Apr 7, 2023
5f34d05
adjust drone damping
YIFANSUN98 Apr 7, 2023
5eabaea
minor update to scpo
CaesarAndylaw Apr 10, 2023
7e2e039
Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_ben…
CaesarAndylaw Apr 10, 2023
98826a6
update to main
YIFANSUN98 Apr 10, 2023
802c3d0
fix scpo downsample for the continuous environment
CaesarAndylaw Apr 10, 2023
5485986
remove robot keepout range
YIFANSUN98 Apr 11, 2023
8743d5c
preserve the stable verison of the scpo, downsampling and augment
CaesarAndylaw Apr 11, 2023
4fa636f
fix goal_dist bug
YIFANSUN98 Apr 11, 2023
f90c249
Merge branch 'main' into tmp1
YIFANSUN98 Apr 11, 2023
8217431
add walker3d_tiny
YIFANSUN98 Apr 11, 2023
fec21b3
add swimmer_tiny
YIFANSUN98 Apr 11, 2023
331585c
more nonconti environment for the experiments for SCPO, and stable im…
CaesarAndylaw Apr 12, 2023
b449762
stable implementation of scpo adatable scale, which works amazingly well
CaesarAndylaw Apr 12, 2023
152f45b
initial implementation of safe layer
CaesarAndylaw Apr 13, 2023
6c1001f
working version of safelayer
CaesarAndylaw Apr 13, 2023
b7e7540
more tuning version of safelayer, apply the downsampling trick
CaesarAndylaw Apr 13, 2023
c7773a9
add usl
YIFANSUN98 Apr 13, 2023
a5bb87c
ipo
ruichen-v Apr 13, 2023
b6fbda7
Merge branch 'ipo' into main
CaesarAndylaw Apr 13, 2023
7241388
finished PCPO
Wei-TianHao Apr 14, 2023
f6dd791
Merge branch 'main' of https://github.com/intelligent-control-lab/saf…
Wei-TianHao Apr 14, 2023
c3eb1cf
Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_ben…
CaesarAndylaw Apr 14, 2023
848c11b
ipo debug
CaesarAndylaw Apr 14, 2023
4f8d3fd
Merge branch 'usl' of github.com:intelligent-control-lab/safe_rl_benc…
CaesarAndylaw Apr 14, 2023
d9b1a07
stable implementation of usl and safelayer
CaesarAndylaw Apr 17, 2023
7ae3b95
start implementing usl with Monte Carlo estimation of Q function
CaesarAndylaw Apr 18, 2023
6047413
stable version of safelayer and usl and ipo
CaesarAndylaw Apr 18, 2023
cf57b42
update engine
YIFANSUN98 Apr 22, 2023
3e67ee7
update engine
YIFANSUN98 Apr 22, 2023
185a247
start commit for pdo
waynekyrie Apr 25, 2023
6a6cb5c
pdo psuedo
waynekyrie Apr 26, 2023
12ff5c8
update
waynekyrie Apr 27, 2023
a423402
pdo running
waynekyrie Apr 28, 2023
77591c4
update
waynekyrie Apr 30, 2023
8afb3ee
lpg
waynekyrie May 1, 2023
29cfc57
the final implementation of SCPO NIPS
CaesarAndylaw May 2, 2023
8f25f5f
stable version of SCPO, little fix of the hopper xml
CaesarAndylaw May 2, 2023
452881d
update xml files
YIFANSUN98 May 2, 2023
1274e0c
merge to main with new xml
YIFANSUN98 May 2, 2023
3585f95
Merge branch 'main' into pdo
waynekyrie May 2, 2023
5f85421
pdo
waynekyrie May 3, 2023
e43ece0
Merge branch 'pdo' of https://github.com/intelligent-control-lab/safe…
waynekyrie May 3, 2023
a1b86be
pdo
waynekyrie May 3, 2023
8f5d371
stable implementation of lpg and pdo
CaesarAndylaw May 11, 2023
19e8587
.gitignore is now working
CaesarAndylaw May 11, 2023
7be79f8
.gitignore is now working
CaesarAndylaw May 11, 2023
c50fefb
update xmls and configurations for benchmark
YIFANSUN98 May 11, 2023
878fd15
merge to main
YIFANSUN98 May 11, 2023
df3240d
update lpg
waynekyrie May 12, 2023
f5b59a8
update lpg
waynekyrie May 12, 2023
2af63f5
before release, code completed
CaesarAndylaw Jun 12, 2023
aeb9701
guard_paper_version
CaesarAndylaw Jun 12, 2023
07456c2
Add APO; Update TRPO
Oct 19, 2023
50249c7
Add A2C and PPO
Oct 19, 2023
ec69e17
Update: readme
Oct 19, 2023
f25c0ae
Update: readme
Oct 19, 2023
8d114e8
Add ISSA and fix some bugs
Oct 19, 2023
b289d32
Update: about issa video generation
Oct 19, 2023
843b012
Add: ISSA
YIFANSUN98 Oct 19, 2023
ea49a14
Update: readme
YIFANSUN98 Oct 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
21 changes: 20 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -135,4 +135,23 @@ all_logs/
.idea/
all_models/

*.csv
*.csv

results/

comparison_group/

comparison_yifan/

*plot_yifan_maxC.py

*plot_yifan.py

*plot_ds.py

*plot_benchmark.py

*safetygym_config_scpo.py

# video generation for the policy visualization
video/
3 changes: 0 additions & 3 deletions .idea/.gitignore

This file was deleted.

15 changes: 0 additions & 15 deletions .idea/icmlws2022_zq.iml

This file was deleted.

14 changes: 0 additions & 14 deletions .idea/inspectionProfiles/Project_Default.xml

This file was deleted.

6 changes: 0 additions & 6 deletions .idea/inspectionProfiles/profiles_settings.xml

This file was deleted.

4 changes: 0 additions & 4 deletions .idea/misc.xml

This file was deleted.

8 changes: 0 additions & 8 deletions .idea/modules.xml

This file was deleted.

8 changes: 0 additions & 8 deletions .idea/saferl_kit.iml

This file was deleted.

6 changes: 0 additions & 6 deletions .idea/vcs.xml

This file was deleted.

6 changes: 6 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"python.analysis.extraPaths": [
"./saferl_envs"
],
"ros.distro": "noetic"
}
Binary file removed imgs/framework.png
Binary file not shown.
160 changes: 0 additions & 160 deletions plot.py

This file was deleted.

3 changes: 0 additions & 3 deletions plot.sh

This file was deleted.

Loading