Code

Enlarge#2

Code

Open

LiangZhisama wants to merge 148 commits intozlr20:mainzlr20/saferl_kit:mainfrom intelligent-control-lab:enlargeintelligent-control-lab/safe_rl_bench:enlargeCopy head branch name to clipboard

+21,701-66,011

Commits

Commits on Feb 28, 2023

• customization to train with safetygym

  CaesarAndylaw

  committed
  2829001

  Copy full SHA for 2829001

• headless video rendering implementation

  CaesarAndylaw

  committed
  ea36361

  Copy full SHA for ea36361

• commented

  CaesarAndylaw

  committed
  87c3f93

  Copy full SHA for 87c3f93

Commits on Mar 2, 2023

• new logger for EpCost, new train safetygym that includes cpo, working with cpo

  CaesarAndylaw

  committed
  558698f

  Copy full SHA for 558698f

• add qp solution and line search

  ruichen-v

  committed
  84301dd

  Copy full SHA for 84301dd

• add extra python path

  ruichen-v

  committed
  bd63cc6

  Copy full SHA for bd63cc6

• run time logger implementation, CPO replay buffer, CPO training main loop, CPO networks, and CPO policy update

  CaesarAndylaw

  committed
  5e24a61

  Copy full SHA for 5e24a61

• Merge remote-tracking branch 'origin/ruic_dev' into main

  CaesarAndylaw

  committed
  6a089a0

  Copy full SHA for 6a089a0

Commits on Mar 3, 2023

• fix the minor implementation bugs with logger, network, cpo, and main training loop

  CaesarAndylaw

  committed
  d2dba34

  Copy full SHA for d2dba34

Commits on Mar 5, 2023

• initial implementation for CPO, and initial experiments

  CaesarAndylaw

  committed
  6995617

  Copy full SHA for 6995617

Commits on Mar 6, 2023

• successfully train the vanilla policy gradient for reward

  CaesarAndylaw

  committed
  193a2b7

  Copy full SHA for 193a2b7

Commits on Mar 7, 2023

• switch to an existing pytorch implementation of vanilla policy gradient, the core utility function is copiedfrom spinngup implementation

  CaesarAndylaw

  committed
  6c8d0cf

  Copy full SHA for 6c8d0cf

Commits on Mar 10, 2023

• implementation of ppo, vpg with gpu training loop

  CaesarAndylaw

  committed
  b7b0053

  Copy full SHA for b7b0053

Commits on Mar 12, 2023

• trpo implementation completed

  CaesarAndylaw

  committed
  173d78a

  Copy full SHA for 173d78a

Commits on Mar 13, 2023

• before meerge

  CaesarAndylaw

  committed
  a66e8f6

  Copy full SHA for a66e8f6

• add buf for cpo

  ruichen-v

  committed
  dc99116

  Copy full SHA for dc99116

• Core implementation for CPO, the update function

  CaesarAndylaw

  committed
  5fb42e1

  Copy full SHA for 5fb42e1

• fix extra char

  CaesarAndylaw

  committed
  0e4e655

  Copy full SHA for 0e4e655

• Merge remote-tracking branch 'origin/ruic_dev' into trpo_test_reward

  CaesarAndylaw

  committed
  2a6a4d2

  Copy full SHA for 2a6a4d2

• merged cpo buf

  CaesarAndylaw

  committed
  cc72d87

  Copy full SHA for cc72d87

Commits on Mar 14, 2023

• complete version of spinup implementation of constrained policy optimization, trust region policy optimization

  CaesarAndylaw

  committed
  7052624

  Copy full SHA for 7052624

• remove results

  CaesarAndylaw

  committed
  3bfbf4c

  Copy full SHA for 3bfbf4c

• remove results

  CaesarAndylaw

  committed
  182fc6e

  Copy full SHA for 182fc6e

Commits on Mar 16, 2023

• add safety-gym-arm

  YIFANSUN98

  committed
  d1354b9

  Copy full SHA for d1354b9

• update the plot function to consider the cost rate

  CaesarAndylaw

  committed
  b857651

  Copy full SHA for b857651

• update safety gym arm environment. include safety gym arm environment into PPO and TRPO training

  CaesarAndylaw

  committed
  05b1058

  Copy full SHA for 05b1058

• policy model visualization is constructed for PPO

  CaesarAndylaw

  committed
  a5a5627

  Copy full SHA for a5a5627

• add arm model with 3 joints

  YIFANSUN98

  committed
  d6193e7

  Copy full SHA for d6193e7

Commits on Mar 17, 2023

• cpo and trpo video recorder

  CaesarAndylaw

  committed
  3b0d613

  Copy full SHA for 3b0d613

• select maximum epoch for trpo video recorder

  CaesarAndylaw

  committed
  e2ee555

  Copy full SHA for e2ee555

• update vpg training to include arm environment. cancel the limit of the links

  CaesarAndylaw

  committed
  61f9fbb

  Copy full SHA for 61f9fbb

• update config

  CaesarAndylaw

  committed
  f01a442

  Copy full SHA for f01a442

• add walker ant swimmer model

  YIFANSUN98

  committed
  66c6f07

  Copy full SHA for 66c6f07

• add swimmer ant walker

  YIFANSUN98

  committed
  fe54d27

  Copy full SHA for fe54d27

• update the cg computation to make hessian computation much accurate

  CaesarAndylaw

  committed
  5d1d967

  Copy full SHA for 5d1d967

• add config of new tasks

  YIFANSUN98

  committed
  ad6fee6

  Copy full SHA for ad6fee6

• add config

  YIFANSUN98

  committed
  37248dd

  Copy full SHA for 37248dd

• add config for new tasks

  YIFANSUN98

  committed
  6b90a41

  Copy full SHA for 6b90a41

• correct implementation of KL computation, change the name of 6dof

  CaesarAndylaw

  committed
  c954558

  Copy full SHA for c954558

• Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_bench into main

  CaesarAndylaw

  committed
  cb85bd6

  Copy full SHA for cb85bd6

• implement the correct KL divergence computation for CPO

  CaesarAndylaw

  committed
  ecbfc26

  Copy full SHA for ecbfc26

Commits on Mar 19, 2023

• update walker3d xml

  YIFANSUN98

  committed
  1c212dc

  Copy full SHA for 1c212dc

Commits on Mar 20, 2023

• stable version of CPO and TRPO

  CaesarAndylaw

  committed
  96ff7ba

  Copy full SHA for 96ff7ba

• addressed the incoming change of humanoid xml

  CaesarAndylaw

  committed
  70bc827

  Copy full SHA for 70bc827

• delete all the original saferl-kit files, they are no longer in use

  CaesarAndylaw

  committed
  17f3843

  Copy full SHA for 17f3843

Commits on Mar 21, 2023

• Adds safety_gym

  Vinnie6167

  committed
  ca85623

  Copy full SHA for ca85623

• add ant_tiny

  YIFANSUN98

  committed
  ceacea9

  Copy full SHA for ceacea9

• Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_bench into main

  YIFANSUN98

  committed
  9e87739

  Copy full SHA for 9e87739

• update cpo and robot configuration

  CaesarAndylaw

  committed
  6d27bd4

  Copy full SHA for 6d27bd4

• update the cpo to enable tuning for cost reduction and make env robust to simulation failure

  CaesarAndylaw

  committed
  9e4f3ae

  Copy full SHA for 9e4f3ae

• add hopper3d

  YIFANSUN98

  committed
  6e394a2

  Copy full SHA for 6e394a2

• Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_bench into main

  YIFANSUN98

  committed
  aa620c9

  Copy full SHA for aa620c9

Commits on Mar 22, 2023

• create the folder of TRPOLAG, minor update for the safetygym configuration

  CaesarAndylaw

  committed
  e5f2e71

  Copy full SHA for e5f2e71

• add 3d hazards for arm3

  YIFANSUN98

  committed
  f70be42

  Copy full SHA for f70be42

• Merge branch 'tmp' into main

  YIFANSUN98

  committed
  48414a3

  Copy full SHA for 48414a3

• implementation of TRPO-Lagrangian method

  CaesarAndylaw

  committed
  68febe6

  Copy full SHA for 68febe6

• Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_bench into main

  CaesarAndylaw

  committed
  891ccf8

  Copy full SHA for 891ccf8

• remove the nested loss cost call

  CaesarAndylaw

  committed
  56794e7

  Copy full SHA for 56794e7

• Merge branch 'trpolag' into main

  CaesarAndylaw

  committed
  b3dfe6c

  Copy full SHA for b3dfe6c

Commits on Mar 25, 2023

• stable version of the lagrangian implementation

  CaesarAndylaw

  committed
  3b05b5a

  Copy full SHA for 3b05b5a

Commits on Mar 27, 2023

• implementation of the TRPO feasible actor critic

  CaesarAndylaw

  committed
  61dea2a

  Copy full SHA for 61dea2a

• stable version of trpofac, and new implementation of SCPO

  CaesarAndylaw

  committed
  8d698e3

  Copy full SHA for 8d698e3

• add ghost tasks

  YIFANSUN98

  committed
  4acdb27

  Copy full SHA for 4acdb27

• add ghost config

  YIFANSUN98

  committed
  b3a3123

  Copy full SHA for b3a3123

• merge to main

  YIFANSUN98

  committed
  dc2e15e

  Copy full SHA for dc2e15e

• implementation of SCPO progress

  CaesarAndylaw

  committed
  5f13473

  Copy full SHA for 5f13473

• update the stable and default configurations of safe RL algorithms

  CaesarAndylaw

  committed
  3ccbe64

  Copy full SHA for 3ccbe64

Commits on Mar 28, 2023

• initial complete implementation of SCPO, and update for the trpofac video

  CaesarAndylaw

  committed
  1e4cd39

  Copy full SHA for 1e4cd39

• modify chase reward

  YIFANSUN98

  committed
  6315b2d

  Copy full SHA for 6315b2d

• modify config for ghost tasks

  YIFANSUN98

  committed
  9c2ea6d

  Copy full SHA for 9c2ea6d

• merge to main

  YIFANSUN98

  committed
  645ead2

  Copy full SHA for 645ead2

• update hopper3d

  YIFANSUN98

  committed
  3acf09d

  Copy full SHA for 3acf09d

• more proper safetygym configuration

  CaesarAndylaw

  committed
  3c5e2e0

  Copy full SHA for 3c5e2e0

• adjust hopper3d gear

  YIFANSUN98

  committed
  8872a37

  Copy full SHA for 8872a37

• merge to main

  YIFANSUN98

  committed
  28a14e2

  Copy full SHA for 28a14e2

Commits on Mar 29, 2023

• bug fixed for scpo, minor update for trpo and safetygym config

  CaesarAndylaw

  committed
  57b0d62

  Copy full SHA for 57b0d62

Commits on Apr 2, 2023

• initial implementation of eaiser learnt value function

  CaesarAndylaw

  committed
  b89ebac

  Copy full SHA for b89ebac

Commits on Apr 3, 2023

• start implementation of the cost sum value learn implementation

  CaesarAndylaw

  committed
  c9099de

  Copy full SHA for c9099de

• new PCPO branch

  Wei-TianHao

  committed
  a3f1207

  Copy full SHA for a3f1207

Commits on Apr 4, 2023

• scpo pnorm implementation and no continue experiments

  CaesarAndylaw

  committed
  4bc1577

  Copy full SHA for 4bc1577

Commits on Apr 5, 2023

• a successful implementation of scpo, which bounds the maximum state wise cost with a very good performance

  CaesarAndylaw

  committed
  fe17187

  Copy full SHA for fe17187

• found the key to the good performance, is to balance the data

  CaesarAndylaw

  committed
  39a82e8

  Copy full SHA for 39a82e8

Commits on Apr 6, 2023

• add drone

  YIFANSUN98

  committed
  2704e9b

  Copy full SHA for 2704e9b

• stable implementation of scpo downsample, which works pretty well

  CaesarAndylaw

  committed
  328d892

  Copy full SHA for 328d892

• merge to main

  YIFANSUN98

  committed
  c9e28d8

  Copy full SHA for c9e28d8

Commits on Apr 7, 2023

• scpo weight up tune

  CaesarAndylaw

  committed
  2c2c994

  Copy full SHA for 2c2c994

• add robot keepout range

  YIFANSUN98

  committed
  a8f755d

  Copy full SHA for a8f755d

• Merge branch 'main' into tmp

  YIFANSUN98

  committed
  dc88de6

  Copy full SHA for dc88de6

• adjust drone damping

  YIFANSUN98

  committed
  5f34d05

  Copy full SHA for 5f34d05

Commits on Apr 10, 2023

• minor update to scpo

  CaesarAndylaw

  committed
  5eabaea

  Copy full SHA for 5eabaea

• Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_bench into main

  CaesarAndylaw

  committed
  7e2e039

  Copy full SHA for 7e2e039

• update to main

  YIFANSUN98

  committed
  98826a6

  Copy full SHA for 98826a6

• fix scpo downsample for the continuous environment

  CaesarAndylaw

  committed
  802c3d0

  Copy full SHA for 802c3d0

Commits on Apr 11, 2023

• remove robot keepout range

  YIFANSUN98

  committed
  5485986

  Copy full SHA for 5485986

• preserve the stable verison of the scpo, downsampling and augment

  CaesarAndylaw

  committed
  8743d5c

  Copy full SHA for 8743d5c

• fix goal_dist bug

  YIFANSUN98

  committed
  4fa636f

  Copy full SHA for 4fa636f

• Merge branch 'main' into tmp1

  YIFANSUN98

  committed
  f90c249

  Copy full SHA for f90c249

• add walker3d_tiny

  YIFANSUN98

  committed
  8217431

  Copy full SHA for 8217431

• add swimmer_tiny

  YIFANSUN98

  committed
  fec21b3

  Copy full SHA for fec21b3

Commits on Apr 12, 2023

• more nonconti environment for the experiments for SCPO, and stable implementation of adaptable target cost

  CaesarAndylaw

  committed
  331585c

  Copy full SHA for 331585c

• stable implementation of scpo adatable scale, which works amazingly well

  CaesarAndylaw

  committed
  b449762

  Copy full SHA for b449762

Commits on Apr 13, 2023

• initial implementation of safe layer

  CaesarAndylaw

  committed
  152f45b

  Copy full SHA for 152f45b

• working version of safelayer

  CaesarAndylaw

  committed
  6c1001f

  Copy full SHA for 6c1001f

• more tuning version of safelayer, apply the downsampling trick

  CaesarAndylaw

  committed
  b7e7540

  Copy full SHA for b7e7540

• add usl

  YIFANSUN98

  committed
  c7773a9

  Copy full SHA for c7773a9

• ipo

  ruichen-v

  committed
  a5bb87c

  Copy full SHA for a5bb87c

• Merge branch 'ipo' into main

  CaesarAndylaw

  committed
  b6fbda7

  Copy full SHA for b6fbda7

Commits on Apr 14, 2023

• finished PCPO

  Wei-TianHao

  committed
  7241388

  Copy full SHA for 7241388

• Merge branch 'main' of https://github.com/intelligent-control-lab/safe_rl_bench into main

  Wei-TianHao

  committed
  f6dd791

  Copy full SHA for f6dd791

• Merge branch 'main' of github.com:intelligent-control-lab/safe_rl_bench into main

  CaesarAndylaw

  committed
  c3eb1cf

  Copy full SHA for c3eb1cf

• ipo debug

  CaesarAndylaw

  committed
  848c11b

  Copy full SHA for 848c11b

• Merge branch 'usl' of github.com:intelligent-control-lab/safe_rl_bench into usl

  CaesarAndylaw

  committed
  4f8d3fd

  Copy full SHA for 4f8d3fd

Commits on Apr 17, 2023

• stable implementation of usl and safelayer

  CaesarAndylaw

  committed
  d9b1a07

  Copy full SHA for d9b1a07

Commits on Apr 18, 2023

• start implementing usl with Monte Carlo estimation of Q function

  CaesarAndylaw

  committed
  7ae3b95

  Copy full SHA for 7ae3b95

• stable version of safelayer and usl and ipo

  CaesarAndylaw

  committed
  6047413

  Copy full SHA for 6047413

Commits on Apr 22, 2023

• update engine

  YIFANSUN98

  committed
  cf57b42

  Copy full SHA for cf57b42

• update engine

  YIFANSUN98

  committed
  3e67ee7

  Copy full SHA for 3e67ee7

Commits on Apr 25, 2023

• start commit for pdo

  waynekyrie

  committed
  185a247

  Copy full SHA for 185a247

Commits on Apr 26, 2023

• pdo psuedo

  waynekyrie

  committed
  6a6cb5c

  Copy full SHA for 6a6cb5c

Commits on Apr 27, 2023

• update

  waynekyrie

  committed
  12ff5c8

  Copy full SHA for 12ff5c8

Commits on Apr 28, 2023

• pdo running

  waynekyrie

  committed
  a423402

  Copy full SHA for a423402

Commits on Apr 30, 2023

• update

  waynekyrie

  committed
  77591c4

  Copy full SHA for 77591c4

Commits on May 1, 2023

• lpg

  waynekyrie

  committed
  8afb3ee

  Copy full SHA for 8afb3ee

Commits on May 2, 2023

• the final implementation of SCPO NIPS

  CaesarAndylaw

  committed
  29cfc57

  Copy full SHA for 29cfc57

• stable version of SCPO, little fix of the hopper xml

  CaesarAndylaw

  committed
  8f25f5f

  Copy full SHA for 8f25f5f

• update xml files

  YIFANSUN98

  committed
  452881d

  Copy full SHA for 452881d

• merge to main with new xml

  YIFANSUN98

  committed
  1274e0c

  Copy full SHA for 1274e0c

• Merge branch 'main' into pdo

  waynekyrie

  committed
  3585f95

  Copy full SHA for 3585f95

Commits on May 3, 2023

• pdo

  waynekyrie

  committed
  5f85421

  Copy full SHA for 5f85421

• Merge branch 'pdo' of https://github.com/intelligent-control-lab/safe_rl_bench into pdo

  waynekyrie

  committed
  e43ece0

  Copy full SHA for e43ece0

• pdo

  waynekyrie

  committed
  a1b86be

  Copy full SHA for a1b86be

Commits on May 11, 2023

• stable implementation of lpg and pdo

  CaesarAndylaw

  committed
  8f5d371

  Copy full SHA for 8f5d371

• .gitignore is now working

  CaesarAndylaw

  committed
  19e8587

  Copy full SHA for 19e8587

• .gitignore is now working

  CaesarAndylaw

  committed
  7be79f8

  Copy full SHA for 7be79f8

• update xmls and configurations for benchmark

  YIFANSUN98

  committed
  c50fefb

  Copy full SHA for c50fefb

• merge to main

  YIFANSUN98

  committed
  878fd15

  Copy full SHA for 878fd15

Commits on May 12, 2023

• update lpg

  waynekyrie

  committed
  df3240d

  Copy full SHA for df3240d

• update lpg

  waynekyrie

  committed
  f5b59a8

  Copy full SHA for f5b59a8

Commits on Jun 12, 2023

• before release, code completed

  CaesarAndylaw

  committed
  2af63f5

  Copy full SHA for 2af63f5

• guard_paper_version

  CaesarAndylaw

  committed
  aeb9701

  Copy full SHA for aeb9701

Commits on Oct 19, 2023

• Add APO; Update TRPO

  

  李飞翰

  committed
  07456c2

  Copy full SHA for 07456c2

• Add A2C and PPO

  

  李飞翰

  committed
  50249c7

  Copy full SHA for 50249c7

• Update: readme

  

  李飞翰

  committed
  ec69e17

  Copy full SHA for ec69e17

• Update: readme

  

  李飞翰

  committed
  f25c0ae

  Copy full SHA for f25c0ae

• Add ISSA and fix some bugs

  

  李飞翰

  committed
  8d114e8

  Copy full SHA for 8d114e8

• Update: about issa video generation

  

  李飞翰

  committed
  b289d32

  Copy full SHA for b289d32

• Add: ISSA

  YIFANSUN98

  committed
  843b012

  Copy full SHA for 843b012

• Update: readme

  YIFANSUN98

  committed
  ea49a14

  Copy full SHA for ea49a14