Deep RL
Deep Reinforcement Learning is a type of machine learning where an agent Learns by doing actions in an enviornment and recive feedback.
Unit 1
Agent/Entity/Model --> Action --> Environment --> Result
What is Reinforcement Learning?(intutition)
Imagine taking a quiz without prep(same set of questions), review the quiz after 1st attempt, Identify correct and incorrect answers, Another attempt. Repeat, In a cyclic fashion until the quiz is cleared.
- Environment(quiz)
- Agent(Human)
- Guessing Answers(Trial and error)
- Reward(Correct and Incorrect answers).
Idea behind RL is an agent learns from the environment by making actions(trial and error) and recieveing rewars(positive or negative) for those actions.
RL is a framework to solve decision or control problems.
Reinforcement Learning Framework
- RL outputs a sequence of State, Action, Reward, NextState
- Central idea of RL is
reward hypothesis- All goals(tasks) can be viewed as maximizing cumulative reward called as expected return. - That’s why in RL to achieve best behaviour, learn to take actions that maximize expected cumulrative reward.
- In Papers RL is called Markov Decsions Process. MDP states, we only need the current state to decide the action and not history of all states.
- Observations - Entire state of environment is described or available. State - Partial State of environment. Chess Game and Mario game respectivley.
- Actions: Discrete(finite moves) - Tetris Games, Continuous - Driving cars.
- Reward with Discount(gamma) - For every actions, we calculate reward with discount exponent(step). Discount determines what the agent cares for long-term reward(larger gamma) and shot-term reward(smaller gamma). Discount depends on risk of achieveing a desired state with action. Then this is cumulativley summed up to get final reward - expected return.
Tasks
- Episodic - Has starting point and ending point, This creates an episode: list of states, actions, rewards, next states.
- Continuing - Predicting stock market, learning to choose best actions and interacting with environmen is simulatneous.
Exploration/Exploitation Trade-off
- Exploration - Unkown random actions to find more about environment.
- Exploitation - Repeating known actions to maximize reward.
Two main approaches for solving RL Problems
- Reward function - actions to maximize expected cumulative reward. This is what needs to be learned?
- Policy is how we learn this. Policy is a function to be learned. Optimal policy maximizes expected return when agent actions according to it.
- Policy - Agent’s brain, function - to take actions that maximizes expected reward.
- Two way’s to train to find optimal policy. Policy-Based and Value-Based.
- Policy-Based: Learn function directly which action to take. Two ways:
- Deterministic: returns one action.
-
Stochastic: returns probabilites for actions.(Given state, conditional probability) - P(A S) where A is set of Actions S is state.
- Value-Based: Value: Action-Value function, most valuable action to take given a state and action.
Lunar Lander
Required Packages:
- gymnasium[box2d]: LunarLander-v2 environment.
- stable-baselines[extra]: The deep reinforcment learning library.
- huggingface_sb3: Download and Upload models.
- Virtual screen to generate a replay video. packages:
- python3-opngl
- ffmpeg
- xvfb
- pyvirtualdisplay
!apt install swig cmake
!pip install -r https://raw.githubusercontent.com/huggingface/deep-rl-class/main/notebooks/unit1/requirements-unit1.txt -q
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
cmake is already the newest version (3.22.1-1ubuntu1.22.04.2).
Suggested packages:
swig-doc swig-examples swig4.0-examples swig4.0-doc
The following NEW packages will be installed:
swig swig4.0
0 upgraded, 2 newly installed, 0 to remove and 35 not upgraded.
Need to get 1,116 kB of archives.
After this operation, 5,542 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 swig4.0 amd64 4.0.2-1ubuntu1 [1,110 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 swig all 4.0.2-1ubuntu1 [5,632 B]
Fetched 1,116 kB in 0s (7,695 kB/s)
Selecting previously unselected package swig4.0.
(Reading database ... 126308 files and directories currently installed.)
Preparing to unpack .../swig4.0_4.0.2-1ubuntu1_amd64.deb ...
Unpacking swig4.0 (4.0.2-1ubuntu1) ...
Selecting previously unselected package swig.
Preparing to unpack .../swig_4.0.2-1ubuntu1_all.deb ...
Unpacking swig (4.0.2-1ubuntu1) ...
Setting up swig4.0 (4.0.2-1ubuntu1) ...
Setting up swig (4.0.2-1ubuntu1) ...
Processing triggers for man-db (2.10.2-1) ...
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m374.4/374.4 kB[0m [31m15.7 MB/s[0m eta [36m0:00:00[0m
[?25h Preparing metadata (setup.py) ... [?25l[?25hdone
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m177.5/177.5 kB[0m [31m15.7 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m925.5/925.5 kB[0m [31m23.1 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.9/1.9 MB[0m [31m66.6 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m13.7/13.7 MB[0m [31m50.9 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m363.4/363.4 MB[0m [31m5.0 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m13.8/13.8 MB[0m [31m106.3 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m24.6/24.6 MB[0m [31m92.2 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m883.7/883.7 kB[0m [31m50.9 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m664.8/664.8 MB[0m [31m1.3 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m211.5/211.5 MB[0m [31m5.7 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m56.3/56.3 MB[0m [31m12.9 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m127.9/127.9 MB[0m [31m7.4 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m207.5/207.5 MB[0m [31m6.4 MB/s[0m eta [36m0:00:00[0m
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m21.1/21.1 MB[0m [31m47.5 MB/s[0m eta [36m0:00:00[0m
[?25h Building wheel for box2d-py (setup.py) ... [?25l[?25hdone
[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
dopamine-rl 4.1.2 requires gymnasium>=1.0.0, but you have gymnasium 0.28.1 which is incompatible.[0m[31m
[0m
# Virtual display packages
!sudo apt-get update
!apt install python3-opengl
!apt install ffmpeg
!apt install xvfb
!pip3 install pyvirtualdisplay -q
Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
Get:3 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease [3,632 B]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
Get:5 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease [1,581 B]
Hit:6 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease
Get:7 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease [24.3 kB]
Hit:8 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3,401 kB]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,566 kB]
Get:11 https://r2u.stat.illinois.edu/ubuntu jammy InRelease [6,555 B]
Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [56.4 kB]
Get:13 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 Packages [1,804 kB]
Get:14 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy/main amd64 Packages [51.0 kB]
Get:15 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages [2,749 kB]
Get:16 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Get:17 https://r2u.stat.illinois.edu/ubuntu jammy/main all Packages [9,065 kB]
Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [48.5 kB]
Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1,262 kB]
Get:20 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [3,089 kB]
Fetched 23.5 MB in 9s (2,507 kB/s)
Reading package lists... Done
W: Skipping acquire of configured file 'main/source/Sources' as repository 'https://r2u.stat.illinois.edu/ubuntu jammy InRelease' does not seem to provide it (sources.list entry misspelt?)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
freeglut3 libglu1-mesa
Suggested packages:
libgle3 python3-numpy
The following NEW packages will be installed:
freeglut3 libglu1-mesa python3-opengl
0 upgraded, 3 newly installed, 0 to remove and 35 not upgraded.
Need to get 824 kB of archives.
After this operation, 8,092 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 freeglut3 amd64 2.8.1-6 [74.0 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libglu1-mesa amd64 9.0.2-1 [145 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-opengl all 3.1.5+dfsg-1 [605 kB]
Fetched 824 kB in 0s (7,147 kB/s)
Selecting previously unselected package freeglut3:amd64.
(Reading database ... 127061 files and directories currently installed.)
Preparing to unpack .../freeglut3_2.8.1-6_amd64.deb ...
Unpacking freeglut3:amd64 (2.8.1-6) ...
Selecting previously unselected package libglu1-mesa:amd64.
Preparing to unpack .../libglu1-mesa_9.0.2-1_amd64.deb ...
Unpacking libglu1-mesa:amd64 (9.0.2-1) ...
Selecting previously unselected package python3-opengl.
Preparing to unpack .../python3-opengl_3.1.5+dfsg-1_all.deb ...
Unpacking python3-opengl (3.1.5+dfsg-1) ...
Setting up freeglut3:amd64 (2.8.1-6) ...
Setting up libglu1-mesa:amd64 (9.0.2-1) ...
Setting up python3-opengl (3.1.5+dfsg-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.8) ...
/sbin/ldconfig.real: /usr/local/lib/libtcm_debug.so.1 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libhwloc.so.15 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtcm.so.1 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero.so.0 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libur_adapter_opencl.so.0 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libumf.so.0 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libur_loader.so.0 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero_v2.so.0 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ffmpeg is already the newest version (7:4.4.2-0ubuntu0.22.04.1).
0 upgraded, 0 newly installed, 0 to remove and 35 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
xvfb is already the newest version (2:21.1.4-2ubuntu1.7~22.04.15).
0 upgraded, 0 newly installed, 0 to remove and 35 not upgraded.
# Restart runtime to reflect above installation chnages
import os
os.kill(os.getpid(), 9)
# Check virtual display
from pyvirtualdisplay import Display
virtual_display = Display(visible=0, size=(1400, 900))
virtual_display.start()
<pyvirtualdisplay.display.Display at 0x7c06ac441f90>
import gymnasium
from huggingface_sb3 import load_from_hub, package_to_hub
from huggingface_hub import (
notebook_login,
) # To log to our Hugging Face account to be able to upload models to the Hub.
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from stable_baselines3.common.evaluation import evaluate_policy
from stable_baselines3.common.monitor import Monitor
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3.common.callbacks import BaseCallback
import imageio
import numpy as np
gymnasium is the library with implementations of common reinforcement learning environments. It provides an API to do this. Env a imperfect reconsturction of Markov Decision Process is the core. This class provies users ability to generate initial state, make actions, move to new states and visuaize the environment. Main Components are:
make()- Create environemet. ReturnsEnv.Env.reset()- Reset’s environment.Env.step()- Make an action.Env.render()- Visualize environment.
RL Loop has four steps: S0 -> A0 -> S1 -> R1
- S0 - Current state
- A0 - Action
- S1 - New State
- R1 - Reward
Let’s create an environment in gymnaisum to understand the code.
# create environment
env = gymnasium.make("LunarLander-v2")
# Checkout env attributes
vars(env) # Docs: https://gymnasium.farama.org/api/env/
{'_saved_kwargs': {'max_episode_steps': 1000},
'env': <OrderEnforcing<PassiveEnvChecker<LunarLander<LunarLander-v2>>>>,
'_action_space': None,
'_observation_space': None,
'_reward_range': None,
'_metadata': None,
'_cached_spec': None,
'_max_episode_steps': 1000,
'_elapsed_steps': None}
methods = [method for method in dir(env) if "__" not in method] # ignore dunder methods
methods
['_action_space',
'_cached_spec',
'_elapsed_steps',
'_is_protocol',
'_max_episode_steps',
'_metadata',
'_np_random',
'_observation_space',
'_reward_range',
'_saved_kwargs',
'action_space',
'class_name',
'close',
'env',
'metadata',
'np_random',
'observation_space',
'render',
'render_mode',
'reset',
'reward_range',
'spec',
'step',
'unwrapped',
'wrapper_spec']
Main methods, we’re gonna use are reset, step, render, close
# Reset environment to get initial state
observation, info = env.reset() # Returns first agent observation for an episode and information
observation, info
(array([ 0.00559216, 1.4049889 , 0.56639385, -0.26362154, -0.006473 ,
-0.1282967 , 0. , 0. ], dtype=float32),
{})
# Let's make random actions for n steps, check if evironemtn is terminated(environment is ended) or truncated(time step is complete.)
n = 20
rewards = []
for step in range(n):
# all valid actions with space are available in action_space
# Sample a random action
action = env.action_space.sample()
print(f"Action at step-{step}: {action}")
# Execute the action, to do this pass action to env.step()
# step() returns observation, reward, terminated, truncated, info
observation, reward, terminated, truncated, info = env.step(action)
print(f"Observation at step-{step}: {observation}")
print(f"Reward at step-{step}: {reward}")
print(f"Termination at step-{step}?: {terminated}")
print(f"Truncation at step-{step}?: {truncated}")
print(f"Info at step-{step}: {info}")
rewards.append(reward)
# Check if action is terminated or truncated, if yes reset env
if terminated or truncated:
print("Reset Environment")
observation, info = env.reset()
print(f"Reset observation: {observation}")
print(f"Reset info: {info}")
print("\n")
print("-"*100)
print("\n")
env.close() # Close environment after timesteps/truncation
Action at step-0: 0
Observation at step-0: [ 0.01118431 1.398481 0.56562865 -0.28927326 -0.01281276 -0.126807
0. 0. ]
Reward at step-0: -1.0433792455491755
Termination at step-0?: False
Truncation at step-0?: False
Info at step-0: {}
----------------------------------------------------------------------------------------------------
Action at step-1: 1
Observation at step-1: [ 0.01669359 1.3913764 0.55522263 -0.31580722 -0.01705731 -0.084899
0. 0. ]
Reward at step-1: -0.09422056187912517
Termination at step-1?: False
Truncation at step-1?: False
Info at step-1: {}
----------------------------------------------------------------------------------------------------
Action at step-2: 2
Observation at step-2: [ 0.0221446 1.3844972 0.54972064 -0.3058023 -0.02162708 -0.09140395
0. 0. ]
Reward at step-2: 0.8933066844165409
Termination at step-2?: False
Truncation at step-2?: False
Info at step-2: {}
----------------------------------------------------------------------------------------------------
Action at step-3: 2
Observation at step-3: [ 0.02755957 1.378548 0.5463973 -0.2644841 -0.02647345 -0.09693653
0. 0. ]
Reward at step-3: 2.0014075931534707
Termination at step-3?: False
Truncation at step-3?: False
Info at step-3: {}
----------------------------------------------------------------------------------------------------
Action at step-4: 0
Observation at step-4: [ 0.03297472 1.371999 0.5464114 -0.2911633 -0.03131893 -0.09691862
0. 0. ]
Reward at step-4: -1.0519407357500654
Termination at step-4?: False
Truncation at step-4?: False
Info at step-4: {}
----------------------------------------------------------------------------------------------------
Action at step-5: 3
Observation at step-5: [ 0.03846941 1.3648512 0.55639017 -0.31783614 -0.0381609 -0.13685174
0. 0. ]
Reward at step-5: -2.1767224073833504
Termination at step-5?: False
Truncation at step-5?: False
Info at step-5: {}
----------------------------------------------------------------------------------------------------
Action at step-6: 0
Observation at step-6: [ 0.04396439 1.3571042 0.5564114 -0.3445079 -0.0450009 -0.13681279
0. 0. ]
Reward at step-6: -1.292053658727923
Termination at step-6?: False
Truncation at step-6?: False
Info at step-6: {}
----------------------------------------------------------------------------------------------------
Action at step-7: 3
Observation at step-7: [ 0.04955311 1.3487597 0.5681429 -0.37117407 -0.05418586 -0.18371637
0. 0. ]
Reward at step-7: -2.5550997363640975
Termination at step-7?: False
Truncation at step-7?: False
Info at step-7: {}
----------------------------------------------------------------------------------------------------
Action at step-8: 3
Observation at step-8: [ 0.05522938 1.3398143 0.5791156 -0.39802116 -0.06556273 -0.22755782
0. 0. ]
Reward at step-8: -2.7022308773431107
Termination at step-8?: False
Truncation at step-8?: False
Info at step-8: {}
----------------------------------------------------------------------------------------------------
Action at step-9: 1
Observation at step-9: [ 0.06081714 1.3302908 0.56798315 -0.4236946 -0.07467889 -0.18234
0. 0. ]
Reward at step-9: -0.604394479675334
Termination at step-9?: False
Truncation at step-9?: False
Info at step-9: {}
----------------------------------------------------------------------------------------------------
Action at step-10: 1
Observation at step-10: [ 0.06634589 1.3201723 0.56056315 -0.4501134 -0.08229978 -0.15243202
0. 0. ]
Reward at step-10: -0.8384302284899536
Termination at step-10?: False
Truncation at step-10?: False
Info at step-10: {}
----------------------------------------------------------------------------------------------------
Action at step-11: 0
Observation at step-11: [ 0.0718749 1.3094544 0.5605837 -0.47678715 -0.08991963 -0.1524113
0. 0. ]
Reward at step-11: -1.4217237921483843
Termination at step-11?: False
Truncation at step-11?: False
Info at step-11: {}
----------------------------------------------------------------------------------------------------
Action at step-12: 3
Observation at step-12: [ 0.0774949 1.298121 0.5719835 -0.5043436 -0.09983942 -0.19841382
0. 0. ]
Reward at step-12: -2.5884612117968104
Termination at step-12?: False
Truncation at step-12?: False
Info at step-12: {}
----------------------------------------------------------------------------------------------------
Action at step-13: 1
Observation at step-13: [ 0.08304949 1.2862089 0.5637341 -0.52999055 -0.10806701 -0.16456637
0. 0. ]
Reward at step-13: -0.8151434968061426
Termination at step-13?: False
Truncation at step-13?: False
Info at step-13: {}
----------------------------------------------------------------------------------------------------
Action at step-14: 2
Observation at step-14: [ 0.08857632 1.274884 0.56149304 -0.50398964 -0.1168257 -0.17519006
0. 0. ]
Reward at step-14: 1.8412985694742872
Termination at step-14?: False
Truncation at step-14?: False
Info at step-14: {}
----------------------------------------------------------------------------------------------------
Action at step-15: 1
Observation at step-15: [ 0.09404173 1.2629805 0.5537366 -0.5296212 -0.12398729 -0.14324474
0. 0. ]
Reward at step-15: -0.7714011772645211
Termination at step-15?: False
Truncation at step-15?: False
Info at step-15: {}
----------------------------------------------------------------------------------------------------
Action at step-16: 2
Observation at step-16: [ 0.09970455 1.2517269 0.5730341 -0.5007305 -0.1307137 -0.13454041
0. 0. ]
Reward at step-16: 0.6312647373818379
Termination at step-16?: False
Truncation at step-16?: False
Info at step-16: {}
----------------------------------------------------------------------------------------------------
Action at step-17: 3
Observation at step-17: [ 0.10542927 1.23987 0.58076584 -0.52772003 -0.13898689 -0.16547817
0. 0. ]
Reward at step-17: -2.095518689238331
Termination at step-17?: False
Truncation at step-17?: False
Info at step-17: {}
----------------------------------------------------------------------------------------------------
Action at step-18: 0
Observation at step-18: [ 0.11115436 1.2274139 0.58079004 -0.55439115 -0.14725798 -0.16543677
0. 0. ]
Reward at step-18: -1.45599925233617
Termination at step-18?: False
Truncation at step-18?: False
Info at step-18: {}
----------------------------------------------------------------------------------------------------
Action at step-19: 1
Observation at step-19: [ 0.11681537 1.2143726 0.57273 -0.580278 -0.15388255 -0.13250318
0. 0. ]
Reward at step-19: -0.687193068824796
Termination at step-19?: False
Truncation at step-19?: False
Info at step-19: {}
----------------------------------------------------------------------------------------------------
observation.shape
(8,)
Each observation is a vector of shape 8. Why?. Excerpt from source:
` Observation Space: The state is an 8-dimensional vector: the coordinates of the lander in x & y, its linear velocities in x & y, its angle, its angular velocity, and two booleans that represent whether each leg is in contact with the ground or not. `
# Let's look at observation space for this environment.
env.observation_space.shape, env.observation_space.sample()
((8,),
array([ 5.3476025e+01, -1.3053141e+01, -6.4954311e-01, -1.4571518e+00,
2.4859619e+00, -2.2165984e-01, 4.0980566e-02, 9.2513007e-01],
dtype=float32))
Action space has four actions: Excerpt from source:
There are four discrete actions available:
- 0: do nothing
- 1: fire left orientation engine
- 2: fire main engine
- 3: fire right orientation engine
env.action_space.n,
(np.int64(4),)
Reward:
- Is increased/decreased the closer/further the lander is to the landing pad.
- Is increased/decreased the slower/faster the lander is moving.
- Is decreased the more the lander is tilted (angle not horizontal).
- Is increased by 10 points for each leg that is in contact with the ground.
- Is decreased by 0.03 points each frame a side engine is firing.
- Is decreased by 0.3 points each frame the main engine is firing.
- Additional reward of -100 or +100 for crashing or landing.
- Episode is considered a solution if it scores at least 200 points.
rewards_detch = [reward.item() if not isinstance(reward, int) else reward for reward in rewards]
rewards_detch
[-1.0433792455491755,
-0.09422056187912517,
0.8933066844165409,
2.0014075931534707,
-1.0519407357500654,
-2.1767224073833504,
-1.292053658727923,
-2.5550997363640975,
-2.7022308773431107,
-0.604394479675334,
-0.8384302284899536,
-1.4217237921483843,
-2.5884612117968104,
-0.8151434968061426,
1.8412985694742872,
-0.7714011772645211,
0.6312647373818379,
-2.095518689238331,
-1.45599925233617,
-0.687193068824796]
import matplotlib.pyplot as plt
plt.hist(rewards_detch)
(array([3., 2., 3., 3., 4., 1., 0., 2., 0., 2.]),
array([-2.70223088, -2.23186703, -1.76150318, -1.29113934, -0.82077549,
-0.35041164, 0.1199522 , 0.59031605, 1.0606799 , 1.53104375,
2.00140759]),
<BarContainer object of 10 artists>)
Now, We’ve an environment(from gymnasium). Next we need an RL algorthim, eval etc to learn a policy with a model/agent.
We’ll use PPO reliable implementation from stable-baselines3. We’ll dive into PPO later!
We’ll use make_vec_env() from stable-baelines3 to create n environments. n individual environments are stacked into a single environment. This allows us to train an agent in n environments per step. Now actions, observations, rewards have an additional dimension n.
PPO is the algorithm, it has two policies MlpPolicy(vector inputs) and CnnPolicy(frame inputs).
Let’s to a training run with PPO.
env = gymnasium.make("LunarLander-v2", render_mode="rgb_array")
model = PPO(
policy="MlpPolicy",
env=env,
verbose=1,
)
Using cuda device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
model.learn(total_timesteps=int(100)) # test run
---------------------------------
| rollout/ | |
| ep_len_mean | 95.1 |
| ep_rew_mean | -170 |
| time/ | |
| fps | 512 |
| iterations | 1 |
| time_elapsed | 3 |
| total_timesteps | 2048 |
---------------------------------
<stable_baselines3.ppo.ppo.PPO at 0x7c053babf2d0>
# Let's add PPO parameters to the model
from stable_baselines3.ppo import PPO
model = PPO(
policy="MlpPolicy",
env=env,
n_steps=2048,
batch_size=64,
n_epochs=4,
gamma=0.999, # discount hyperparameter
gae_lambda=0.98, # Controls the tradeoff between bias and variance in the advantage estimate. Controls the tradeoff between bias and variance in the advantage estimate.
ent_coef=0.01, # Exploitation/Exploration tradeoff, lower value exploitation, higher value exploration.
verbose=1,
)
Using cuda device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
- Collect 1024 Agent interaction samples(env.step()) from environment, convert them to batchses of size 64.
- 1024 / 64 - 16 batches per epoch.
- 16 * 4(epochs) - 64 gradient updates.
- Repeat until 1M total interactions with environment.
# Let's create a custom callback to store 10 video replay across total_timesteps
from stable_baselines3.common.callbacks import BaseCallback
import imageio
import os
import numpy as np
class TrainingMonitorCallaback(BaseCallback):
def __init__(
self,
model_class,
env_id,
total_timesteps,
n_recordings=10,
save_path="videos",
verbose=0,
):
super().__init__(verbose)
self.model_class = model_class
self.env_id = env_id
self.total_timesteps = total_timesteps
self.n_recordings = n_recordings
self.save_path = save_path
self.verbose = verbose
self.current_step = 0
self.episode_count = 0 # An episode ends when it's complete or terminated(failed)
self.record_points = set(
np.linspace(0, total_timesteps, n_recordings, endpoint=False, dtype=int)
) # Points to record the videos
self.saved_steps = set()
def _on_training_start(self):
os.makedirs(self.save_path, exist_ok=True)
if self.verbose:
print(f"Callback will save videos at steps: {sorted(self.record_points)}")
def record_episode(self, model, env):
obs, info = env.reset() # Initial state
frames = []
terminated = False
truncated = False
while not terminated and not truncated:
frames.append(env.render())
action, _states = model.predict(obs)
obs, rewards, terminated, truncated, info = env.step(action)
return frames
def _on_step(self):
self.current_step += self.training_env.num_envs # Parallel execution on num_envs
# Count completed episodes
self.episode_count += self.locals["dones"]
# Save video recording
if self.current_step in self.record_points and self.current_step not in self.saved_steps:
video_path = os.path.join(self.save_path, f"step_{self.current_step:}.mp4")
eval_env = gymnasium.make(self.env_id, render_mode="rgb_array") # Env for evaluation
frames = self.record_episode(model=self.model, env=eval_env)
imageio.mimsave(video_path, frames, fps=30)
self.saved_steps.add(self.current_step)
if self.verbose:
print(f"[Callback] Saved video frame at step {self.current_step} to {video_path}")
return True # Continue training
def _on_training_end(self):
"""
Called once at the end of training.
"""
print(f"[Callback] Total episodes completed: {self.episode_count}")
!rm -rf videos
total_timestpes = 2_000_000
callback = TrainingMonitorCallaback(
total_timesteps=total_timestpes,
verbose=1,
model_class="PPO", # Not needed
env_id="LunarLander-v2",
n_recordings=100
)
model.learn(total_timesteps=total_timestpes, callback=callback)
Callback will save videos at steps: [np.int64(0), np.int64(20000), np.int64(40000), np.int64(60000), np.int64(80000), np.int64(100000), np.int64(120000), np.int64(140000), np.int64(160000), np.int64(180000), np.int64(200000), np.int64(220000), np.int64(240000), np.int64(260000), np.int64(280000), np.int64(300000), np.int64(320000), np.int64(340000), np.int64(360000), np.int64(380000), np.int64(400000), np.int64(420000), np.int64(440000), np.int64(460000), np.int64(480000), np.int64(500000), np.int64(520000), np.int64(540000), np.int64(560000), np.int64(580000), np.int64(600000), np.int64(620000), np.int64(640000), np.int64(660000), np.int64(680000), np.int64(700000), np.int64(720000), np.int64(740000), np.int64(760000), np.int64(780000), np.int64(800000), np.int64(820000), np.int64(840000), np.int64(860000), np.int64(880000), np.int64(900000), np.int64(920000), np.int64(940000), np.int64(960000), np.int64(980000), np.int64(1000000), np.int64(1020000), np.int64(1040000), np.int64(1060000), np.int64(1080000), np.int64(1100000), np.int64(1120000), np.int64(1140000), np.int64(1160000), np.int64(1180000), np.int64(1200000), np.int64(1220000), np.int64(1240000), np.int64(1260000), np.int64(1280000), np.int64(1300000), np.int64(1320000), np.int64(1340000), np.int64(1360000), np.int64(1380000), np.int64(1400000), np.int64(1420000), np.int64(1440000), np.int64(1460000), np.int64(1480000), np.int64(1500000), np.int64(1520000), np.int64(1540000), np.int64(1560000), np.int64(1580000), np.int64(1600000), np.int64(1620000), np.int64(1640000), np.int64(1660000), np.int64(1680000), np.int64(1700000), np.int64(1720000), np.int64(1740000), np.int64(1760000), np.int64(1780000), np.int64(1800000), np.int64(1820000), np.int64(1840000), np.int64(1860000), np.int64(1880000), np.int64(1900000), np.int64(1920000), np.int64(1940000), np.int64(1960000), np.int64(1980000)]
---------------------------------
| rollout/ | |
| ep_len_mean | 91.5 |
| ep_rew_mean | -189 |
| time/ | |
| fps | 506 |
| iterations | 1 |
| time_elapsed | 4 |
| total_timesteps | 2048 |
---------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 89.2 |
| ep_rew_mean | -174 |
| time/ | |
| fps | 508 |
| iterations | 2 |
| time_elapsed | 8 |
| total_timesteps | 4096 |
| train/ | |
| approx_kl | 0.0051549105 |
| clip_fraction | 0.0184 |
| clip_range | 0.2 |
| entropy_loss | -1.38 |
| explained_variance | 0.00078880787 |
| learning_rate | 0.0003 |
| loss | 3.56e+03 |
| n_updates | 4 |
| policy_gradient_loss | -0.00556 |
| value_loss | 6.75e+03 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 94 |
| ep_rew_mean | -180 |
| time/ | |
| fps | 522 |
| iterations | 3 |
| time_elapsed | 11 |
| total_timesteps | 6144 |
| train/ | |
| approx_kl | 0.0025083504 |
| clip_fraction | 0.000732 |
| clip_range | 0.2 |
| entropy_loss | -1.38 |
| explained_variance | -0.0053813457 |
| learning_rate | 0.0003 |
| loss | 1.81e+03 |
| n_updates | 8 |
| policy_gradient_loss | -0.00443 |
| value_loss | 5.98e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 96 |
| ep_rew_mean | -174 |
| time/ | |
| fps | 528 |
| iterations | 4 |
| time_elapsed | 15 |
| total_timesteps | 8192 |
| train/ | |
| approx_kl | 0.0028335815 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.37 |
| explained_variance | -0.008877039 |
| learning_rate | 0.0003 |
| loss | 2.68e+03 |
| n_updates | 12 |
| policy_gradient_loss | -0.00331 |
| value_loss | 5.4e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 96.7 |
| ep_rew_mean | -166 |
| time/ | |
| fps | 513 |
| iterations | 5 |
| time_elapsed | 19 |
| total_timesteps | 10240 |
| train/ | |
| approx_kl | 0.001997076 |
| clip_fraction | 0.0011 |
| clip_range | 0.2 |
| entropy_loss | -1.36 |
| explained_variance | -0.010858655 |
| learning_rate | 0.0003 |
| loss | 1.51e+03 |
| n_updates | 16 |
| policy_gradient_loss | -0.00305 |
| value_loss | 3.82e+03 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 96.8 |
| ep_rew_mean | -163 |
| time/ | |
| fps | 520 |
| iterations | 6 |
| time_elapsed | 23 |
| total_timesteps | 12288 |
| train/ | |
| approx_kl | 0.0068211444 |
| clip_fraction | 0.0133 |
| clip_range | 0.2 |
| entropy_loss | -1.36 |
| explained_variance | 0.00023049116 |
| learning_rate | 0.0003 |
| loss | 1.37e+03 |
| n_updates | 20 |
| policy_gradient_loss | -0.00535 |
| value_loss | 3.1e+03 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 98.2 |
| ep_rew_mean | -154 |
| time/ | |
| fps | 525 |
| iterations | 7 |
| time_elapsed | 27 |
| total_timesteps | 14336 |
| train/ | |
| approx_kl | 0.009350769 |
| clip_fraction | 0.0542 |
| clip_range | 0.2 |
| entropy_loss | -1.35 |
| explained_variance | -0.0015380383 |
| learning_rate | 0.0003 |
| loss | 1.62e+03 |
| n_updates | 24 |
| policy_gradient_loss | -0.0109 |
| value_loss | 3.25e+03 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 96.1 |
| ep_rew_mean | -136 |
| time/ | |
| fps | 515 |
| iterations | 8 |
| time_elapsed | 31 |
| total_timesteps | 16384 |
| train/ | |
| approx_kl | 0.00814908 |
| clip_fraction | 0.00281 |
| clip_range | 0.2 |
| entropy_loss | -1.36 |
| explained_variance | -0.0022851229 |
| learning_rate | 0.0003 |
| loss | 722 |
| n_updates | 28 |
| policy_gradient_loss | -0.00624 |
| value_loss | 1.91e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 95.4 |
| ep_rew_mean | -131 |
| time/ | |
| fps | 518 |
| iterations | 9 |
| time_elapsed | 35 |
| total_timesteps | 18432 |
| train/ | |
| approx_kl | 0.010069275 |
| clip_fraction | 0.0275 |
| clip_range | 0.2 |
| entropy_loss | -1.34 |
| explained_variance | -0.004091859 |
| learning_rate | 0.0003 |
| loss | 800 |
| n_updates | 32 |
| policy_gradient_loss | -0.00476 |
| value_loss | 1.75e+03 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 20000 to videos/step_20000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 98 |
| ep_rew_mean | -126 |
| time/ | |
| fps | 502 |
| iterations | 10 |
| time_elapsed | 40 |
| total_timesteps | 20480 |
| train/ | |
| approx_kl | 0.0011890483 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.32 |
| explained_variance | -0.0043804646 |
| learning_rate | 0.0003 |
| loss | 1.03e+03 |
| n_updates | 36 |
| policy_gradient_loss | -0.000994 |
| value_loss | 1.99e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 100 |
| ep_rew_mean | -124 |
| time/ | |
| fps | 494 |
| iterations | 11 |
| time_elapsed | 45 |
| total_timesteps | 22528 |
| train/ | |
| approx_kl | 0.0035590017 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.32 |
| explained_variance | 0.0075088143 |
| learning_rate | 0.0003 |
| loss | 931 |
| n_updates | 40 |
| policy_gradient_loss | -0.00246 |
| value_loss | 1.83e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 111 |
| ep_rew_mean | -123 |
| time/ | |
| fps | 490 |
| iterations | 12 |
| time_elapsed | 50 |
| total_timesteps | 24576 |
| train/ | |
| approx_kl | 0.00667898 |
| clip_fraction | 0.0304 |
| clip_range | 0.2 |
| entropy_loss | -1.32 |
| explained_variance | 0.0014611483 |
| learning_rate | 0.0003 |
| loss | 1.36e+03 |
| n_updates | 44 |
| policy_gradient_loss | -0.00549 |
| value_loss | 2.58e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 114 |
| ep_rew_mean | -119 |
| time/ | |
| fps | 493 |
| iterations | 13 |
| time_elapsed | 53 |
| total_timesteps | 26624 |
| train/ | |
| approx_kl | 0.0072960295 |
| clip_fraction | 0.0343 |
| clip_range | 0.2 |
| entropy_loss | -1.32 |
| explained_variance | 0.023665786 |
| learning_rate | 0.0003 |
| loss | 545 |
| n_updates | 48 |
| policy_gradient_loss | -0.00704 |
| value_loss | 1.21e+03 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 118 |
| ep_rew_mean | -116 |
| time/ | |
| fps | 487 |
| iterations | 14 |
| time_elapsed | 58 |
| total_timesteps | 28672 |
| train/ | |
| approx_kl | 0.011461751 |
| clip_fraction | 0.056 |
| clip_range | 0.2 |
| entropy_loss | -1.27 |
| explained_variance | -0.0016262531 |
| learning_rate | 0.0003 |
| loss | 464 |
| n_updates | 52 |
| policy_gradient_loss | -0.00753 |
| value_loss | 1.32e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 119 |
| ep_rew_mean | -115 |
| time/ | |
| fps | 490 |
| iterations | 15 |
| time_elapsed | 62 |
| total_timesteps | 30720 |
| train/ | |
| approx_kl | 0.008486927 |
| clip_fraction | 0.0385 |
| clip_range | 0.2 |
| entropy_loss | -1.26 |
| explained_variance | -0.014918804 |
| learning_rate | 0.0003 |
| loss | 597 |
| n_updates | 56 |
| policy_gradient_loss | -0.00529 |
| value_loss | 1.09e+03 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 133 |
| ep_rew_mean | -116 |
| time/ | |
| fps | 487 |
| iterations | 16 |
| time_elapsed | 67 |
| total_timesteps | 32768 |
| train/ | |
| approx_kl | 0.008016018 |
| clip_fraction | 0.0564 |
| clip_range | 0.2 |
| entropy_loss | -1.27 |
| explained_variance | -0.0005719662 |
| learning_rate | 0.0003 |
| loss | 1.15e+03 |
| n_updates | 60 |
| policy_gradient_loss | -0.00703 |
| value_loss | 1.37e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 138 |
| ep_rew_mean | -103 |
| time/ | |
| fps | 484 |
| iterations | 17 |
| time_elapsed | 71 |
| total_timesteps | 34816 |
| train/ | |
| approx_kl | 0.0022676764 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -1.29 |
| explained_variance | 0.014931381 |
| learning_rate | 0.0003 |
| loss | 562 |
| n_updates | 64 |
| policy_gradient_loss | -0.00179 |
| value_loss | 1.25e+03 |
------------------------------------------
--------------------------------------------
| rollout/ | |
| ep_len_mean | 151 |
| ep_rew_mean | -100 |
| time/ | |
| fps | 482 |
| iterations | 18 |
| time_elapsed | 76 |
| total_timesteps | 36864 |
| train/ | |
| approx_kl | 0.017201278 |
| clip_fraction | 0.14 |
| clip_range | 0.2 |
| entropy_loss | -1.22 |
| explained_variance | -0.00034058094 |
| learning_rate | 0.0003 |
| loss | 218 |
| n_updates | 68 |
| policy_gradient_loss | -0.00577 |
| value_loss | 506 |
--------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 147 |
| ep_rew_mean | -95.3 |
| time/ | |
| fps | 484 |
| iterations | 19 |
| time_elapsed | 80 |
| total_timesteps | 38912 |
| train/ | |
| approx_kl | 0.001284529 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.26 |
| explained_variance | 0.03527212 |
| learning_rate | 0.0003 |
| loss | 225 |
| n_updates | 72 |
| policy_gradient_loss | -0.00185 |
| value_loss | 638 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 40000 to videos/step_40000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 155 |
| ep_rew_mean | -94 |
| time/ | |
| fps | 470 |
| iterations | 20 |
| time_elapsed | 87 |
| total_timesteps | 40960 |
| train/ | |
| approx_kl | 0.0067412006 |
| clip_fraction | 0.0199 |
| clip_range | 0.2 |
| entropy_loss | -1.23 |
| explained_variance | -0.058317423 |
| learning_rate | 0.0003 |
| loss | 261 |
| n_updates | 76 |
| policy_gradient_loss | -0.00208 |
| value_loss | 643 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 164 |
| ep_rew_mean | -95 |
| time/ | |
| fps | 472 |
| iterations | 21 |
| time_elapsed | 91 |
| total_timesteps | 43008 |
| train/ | |
| approx_kl | 0.0070644394 |
| clip_fraction | 0.0234 |
| clip_range | 0.2 |
| entropy_loss | -1.15 |
| explained_variance | -0.072965026 |
| learning_rate | 0.0003 |
| loss | 214 |
| n_updates | 80 |
| policy_gradient_loss | -0.00389 |
| value_loss | 794 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 168 |
| ep_rew_mean | -92.2 |
| time/ | |
| fps | 472 |
| iterations | 22 |
| time_elapsed | 95 |
| total_timesteps | 45056 |
| train/ | |
| approx_kl | 0.0019402597 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | -0.0005172491 |
| learning_rate | 0.0003 |
| loss | 474 |
| n_updates | 84 |
| policy_gradient_loss | -0.00317 |
| value_loss | 900 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 176 |
| ep_rew_mean | -95 |
| time/ | |
| fps | 472 |
| iterations | 23 |
| time_elapsed | 99 |
| total_timesteps | 47104 |
| train/ | |
| approx_kl | 0.0065907203 |
| clip_fraction | 0.0265 |
| clip_range | 0.2 |
| entropy_loss | -1.16 |
| explained_variance | -6.842613e-05 |
| learning_rate | 0.0003 |
| loss | 284 |
| n_updates | 88 |
| policy_gradient_loss | -0.00371 |
| value_loss | 734 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 169 |
| ep_rew_mean | -101 |
| time/ | |
| fps | 474 |
| iterations | 24 |
| time_elapsed | 103 |
| total_timesteps | 49152 |
| train/ | |
| approx_kl | 0.0044139326 |
| clip_fraction | 0.0131 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | 5.4240227e-06 |
| learning_rate | 0.0003 |
| loss | 509 |
| n_updates | 92 |
| policy_gradient_loss | -0.00223 |
| value_loss | 1.29e+03 |
-------------------------------------------
--------------------------------------------
| rollout/ | |
| ep_len_mean | 170 |
| ep_rew_mean | -103 |
| time/ | |
| fps | 475 |
| iterations | 25 |
| time_elapsed | 107 |
| total_timesteps | 51200 |
| train/ | |
| approx_kl | 0.0023994276 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.19 |
| explained_variance | -0.00068461895 |
| learning_rate | 0.0003 |
| loss | 430 |
| n_updates | 96 |
| policy_gradient_loss | -0.0029 |
| value_loss | 737 |
--------------------------------------------
--------------------------------------------
| rollout/ | |
| ep_len_mean | 165 |
| ep_rew_mean | -105 |
| time/ | |
| fps | 474 |
| iterations | 26 |
| time_elapsed | 112 |
| total_timesteps | 53248 |
| train/ | |
| approx_kl | 0.008543575 |
| clip_fraction | 0.106 |
| clip_range | 0.2 |
| entropy_loss | -1.19 |
| explained_variance | -0.00055742264 |
| learning_rate | 0.0003 |
| loss | 189 |
| n_updates | 100 |
| policy_gradient_loss | -0.00441 |
| value_loss | 379 |
--------------------------------------------
--------------------------------------------
| rollout/ | |
| ep_len_mean | 164 |
| ep_rew_mean | -99 |
| time/ | |
| fps | 475 |
| iterations | 27 |
| time_elapsed | 116 |
| total_timesteps | 55296 |
| train/ | |
| approx_kl | 0.009569153 |
| clip_fraction | 0.0864 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | -0.00027024746 |
| learning_rate | 0.0003 |
| loss | 228 |
| n_updates | 104 |
| policy_gradient_loss | -0.00692 |
| value_loss | 520 |
--------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 168 |
| ep_rew_mean | -93.2 |
| time/ | |
| fps | 472 |
| iterations | 28 |
| time_elapsed | 121 |
| total_timesteps | 57344 |
| train/ | |
| approx_kl | 0.0026342047 |
| clip_fraction | 0.0243 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | -9.357929e-05 |
| learning_rate | 0.0003 |
| loss | 281 |
| n_updates | 108 |
| policy_gradient_loss | 0.00251 |
| value_loss | 543 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 178 |
| ep_rew_mean | -86.1 |
| time/ | |
| fps | 469 |
| iterations | 29 |
| time_elapsed | 126 |
| total_timesteps | 59392 |
| train/ | |
| approx_kl | 0.0074533443 |
| clip_fraction | 0.0458 |
| clip_range | 0.2 |
| entropy_loss | -1.21 |
| explained_variance | 8.124113e-05 |
| learning_rate | 0.0003 |
| loss | 137 |
| n_updates | 112 |
| policy_gradient_loss | -0.00503 |
| value_loss | 364 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 60000 to videos/step_60000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 180 |
| ep_rew_mean | -77.9 |
| time/ | |
| fps | 465 |
| iterations | 30 |
| time_elapsed | 131 |
| total_timesteps | 61440 |
| train/ | |
| approx_kl | 0.002352362 |
| clip_fraction | 0.000244 |
| clip_range | 0.2 |
| entropy_loss | -1.23 |
| explained_variance | 0.003699541 |
| learning_rate | 0.0003 |
| loss | 170 |
| n_updates | 116 |
| policy_gradient_loss | -0.00223 |
| value_loss | 581 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 188 |
| ep_rew_mean | -74.9 |
| time/ | |
| fps | 462 |
| iterations | 31 |
| time_elapsed | 137 |
| total_timesteps | 63488 |
| train/ | |
| approx_kl | 0.0014001757 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.19 |
| explained_variance | -0.0037305355 |
| learning_rate | 0.0003 |
| loss | 236 |
| n_updates | 120 |
| policy_gradient_loss | -0.000483 |
| value_loss | 545 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 183 |
| ep_rew_mean | -61.7 |
| time/ | |
| fps | 464 |
| iterations | 32 |
| time_elapsed | 141 |
| total_timesteps | 65536 |
| train/ | |
| approx_kl | 0.0013357033 |
| clip_fraction | 0.00122 |
| clip_range | 0.2 |
| entropy_loss | -1.15 |
| explained_variance | 0.016273439 |
| learning_rate | 0.0003 |
| loss | 130 |
| n_updates | 124 |
| policy_gradient_loss | -0.00132 |
| value_loss | 420 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 185 |
| ep_rew_mean | -47.5 |
| time/ | |
| fps | 465 |
| iterations | 33 |
| time_elapsed | 145 |
| total_timesteps | 67584 |
| train/ | |
| approx_kl | 0.0016057943 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.17 |
| explained_variance | -0.0005495548 |
| learning_rate | 0.0003 |
| loss | 199 |
| n_updates | 128 |
| policy_gradient_loss | -0.00154 |
| value_loss | 427 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 196 |
| ep_rew_mean | -43.3 |
| time/ | |
| fps | 461 |
| iterations | 34 |
| time_elapsed | 150 |
| total_timesteps | 69632 |
| train/ | |
| approx_kl | 0.0008653614 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.16 |
| explained_variance | -0.0010558367 |
| learning_rate | 0.0003 |
| loss | 222 |
| n_updates | 132 |
| policy_gradient_loss | -0.00109 |
| value_loss | 526 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 199 |
| ep_rew_mean | -34.2 |
| time/ | |
| fps | 462 |
| iterations | 35 |
| time_elapsed | 154 |
| total_timesteps | 71680 |
| train/ | |
| approx_kl | 0.00045612612 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.21 |
| explained_variance | 0.006841004 |
| learning_rate | 0.0003 |
| loss | 146 |
| n_updates | 136 |
| policy_gradient_loss | -0.000522 |
| value_loss | 650 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 209 |
| ep_rew_mean | -31.7 |
| time/ | |
| fps | 460 |
| iterations | 36 |
| time_elapsed | 160 |
| total_timesteps | 73728 |
| train/ | |
| approx_kl | 0.00041795854 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.15 |
| explained_variance | -0.007164836 |
| learning_rate | 0.0003 |
| loss | 242 |
| n_updates | 140 |
| policy_gradient_loss | -0.000415 |
| value_loss | 445 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 218 |
| ep_rew_mean | -29.6 |
| time/ | |
| fps | 457 |
| iterations | 37 |
| time_elapsed | 165 |
| total_timesteps | 75776 |
| train/ | |
| approx_kl | 0.003422582 |
| clip_fraction | 0.00684 |
| clip_range | 0.2 |
| entropy_loss | -1.23 |
| explained_variance | 0.039322734 |
| learning_rate | 0.0003 |
| loss | 133 |
| n_updates | 144 |
| policy_gradient_loss | -0.00233 |
| value_loss | 314 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 227 |
| ep_rew_mean | -25.6 |
| time/ | |
| fps | 454 |
| iterations | 38 |
| time_elapsed | 171 |
| total_timesteps | 77824 |
| train/ | |
| approx_kl | 0.0038304387 |
| clip_fraction | 0.0144 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | 0.0015069246 |
| learning_rate | 0.0003 |
| loss | 182 |
| n_updates | 148 |
| policy_gradient_loss | -0.00231 |
| value_loss | 435 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 230 |
| ep_rew_mean | -25.1 |
| time/ | |
| fps | 453 |
| iterations | 39 |
| time_elapsed | 176 |
| total_timesteps | 79872 |
| train/ | |
| approx_kl | 0.0005214426 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.22 |
| explained_variance | 0.077490985 |
| learning_rate | 0.0003 |
| loss | 140 |
| n_updates | 152 |
| policy_gradient_loss | -0.000288 |
| value_loss | 335 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 80000 to videos/step_80000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 222 |
| ep_rew_mean | -19.7 |
| time/ | |
| fps | 450 |
| iterations | 40 |
| time_elapsed | 181 |
| total_timesteps | 81920 |
| train/ | |
| approx_kl | 0.00095533347 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.14 |
| explained_variance | -0.009914517 |
| learning_rate | 0.0003 |
| loss | 169 |
| n_updates | 156 |
| policy_gradient_loss | -0.00105 |
| value_loss | 533 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 223 |
| ep_rew_mean | -16.3 |
| time/ | |
| fps | 451 |
| iterations | 41 |
| time_elapsed | 185 |
| total_timesteps | 83968 |
| train/ | |
| approx_kl | 0.0014807701 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.13 |
| explained_variance | 0.08277339 |
| learning_rate | 0.0003 |
| loss | 232 |
| n_updates | 160 |
| policy_gradient_loss | -0.00177 |
| value_loss | 478 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 224 |
| ep_rew_mean | -16 |
| time/ | |
| fps | 447 |
| iterations | 42 |
| time_elapsed | 192 |
| total_timesteps | 86016 |
| train/ | |
| approx_kl | 0.006245519 |
| clip_fraction | 0.00818 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.108308315 |
| learning_rate | 0.0003 |
| loss | 211 |
| n_updates | 164 |
| policy_gradient_loss | -0.00459 |
| value_loss | 561 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 240 |
| ep_rew_mean | -13.4 |
| time/ | |
| fps | 445 |
| iterations | 43 |
| time_elapsed | 197 |
| total_timesteps | 88064 |
| train/ | |
| approx_kl | 0.0046423045 |
| clip_fraction | 0.0155 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | 0.21044558 |
| learning_rate | 0.0003 |
| loss | 138 |
| n_updates | 168 |
| policy_gradient_loss | -0.00298 |
| value_loss | 238 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 251 |
| ep_rew_mean | -10.9 |
| time/ | |
| fps | 443 |
| iterations | 44 |
| time_elapsed | 203 |
| total_timesteps | 90112 |
| train/ | |
| approx_kl | 0.006927562 |
| clip_fraction | 0.0223 |
| clip_range | 0.2 |
| entropy_loss | -1.27 |
| explained_variance | 0.25908267 |
| learning_rate | 0.0003 |
| loss | 110 |
| n_updates | 172 |
| policy_gradient_loss | -0.0041 |
| value_loss | 198 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | -9.83 |
| time/ | |
| fps | 441 |
| iterations | 45 |
| time_elapsed | 208 |
| total_timesteps | 92160 |
| train/ | |
| approx_kl | 0.0035460428 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.17 |
| explained_variance | 0.2303493 |
| learning_rate | 0.0003 |
| loss | 238 |
| n_updates | 176 |
| policy_gradient_loss | -0.00207 |
| value_loss | 429 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | -7.52 |
| time/ | |
| fps | 438 |
| iterations | 46 |
| time_elapsed | 214 |
| total_timesteps | 94208 |
| train/ | |
| approx_kl | 0.0029151263 |
| clip_fraction | 0.00757 |
| clip_range | 0.2 |
| entropy_loss | -1.23 |
| explained_variance | 0.3778093 |
| learning_rate | 0.0003 |
| loss | 138 |
| n_updates | 180 |
| policy_gradient_loss | -0.00141 |
| value_loss | 427 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | -8.56 |
| time/ | |
| fps | 438 |
| iterations | 47 |
| time_elapsed | 219 |
| total_timesteps | 96256 |
| train/ | |
| approx_kl | 0.0044478094 |
| clip_fraction | 0.01 |
| clip_range | 0.2 |
| entropy_loss | -1.23 |
| explained_variance | 0.612099 |
| learning_rate | 0.0003 |
| loss | 126 |
| n_updates | 184 |
| policy_gradient_loss | -0.0032 |
| value_loss | 216 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 310 |
| ep_rew_mean | -7.9 |
| time/ | |
| fps | 438 |
| iterations | 48 |
| time_elapsed | 224 |
| total_timesteps | 98304 |
| train/ | |
| approx_kl | 0.0029132944 |
| clip_fraction | 0.00122 |
| clip_range | 0.2 |
| entropy_loss | -1.11 |
| explained_variance | 0.6035398 |
| learning_rate | 0.0003 |
| loss | 160 |
| n_updates | 188 |
| policy_gradient_loss | -0.00253 |
| value_loss | 390 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 100000 to videos/step_100000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | -7.86 |
| time/ | |
| fps | 422 |
| iterations | 49 |
| time_elapsed | 237 |
| total_timesteps | 100352 |
| train/ | |
| approx_kl | 0.0007590678 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.15 |
| explained_variance | 0.4904328 |
| learning_rate | 0.0003 |
| loss | 276 |
| n_updates | 192 |
| policy_gradient_loss | -0.00136 |
| value_loss | 620 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 322 |
| ep_rew_mean | -5.49 |
| time/ | |
| fps | 420 |
| iterations | 50 |
| time_elapsed | 243 |
| total_timesteps | 102400 |
| train/ | |
| approx_kl | 0.004991444 |
| clip_fraction | 0.0111 |
| clip_range | 0.2 |
| entropy_loss | -1.15 |
| explained_variance | 0.74268717 |
| learning_rate | 0.0003 |
| loss | 73.5 |
| n_updates | 196 |
| policy_gradient_loss | -0.00393 |
| value_loss | 184 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | -5.54 |
| time/ | |
| fps | 418 |
| iterations | 51 |
| time_elapsed | 249 |
| total_timesteps | 104448 |
| train/ | |
| approx_kl | 0.0076586446 |
| clip_fraction | 0.0638 |
| clip_range | 0.2 |
| entropy_loss | -1.19 |
| explained_variance | 0.7165688 |
| learning_rate | 0.0003 |
| loss | 99.7 |
| n_updates | 200 |
| policy_gradient_loss | -0.00356 |
| value_loss | 194 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 351 |
| ep_rew_mean | -4.43 |
| time/ | |
| fps | 417 |
| iterations | 52 |
| time_elapsed | 255 |
| total_timesteps | 106496 |
| train/ | |
| approx_kl | 0.0042811967 |
| clip_fraction | 0.00525 |
| clip_range | 0.2 |
| entropy_loss | -1.22 |
| explained_variance | 0.7040092 |
| learning_rate | 0.0003 |
| loss | 115 |
| n_updates | 204 |
| policy_gradient_loss | -0.00236 |
| value_loss | 211 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 368 |
| ep_rew_mean | -1.85 |
| time/ | |
| fps | 416 |
| iterations | 53 |
| time_elapsed | 260 |
| total_timesteps | 108544 |
| train/ | |
| approx_kl | 0.0037643583 |
| clip_fraction | 0.00427 |
| clip_range | 0.2 |
| entropy_loss | -1.13 |
| explained_variance | 0.6857211 |
| learning_rate | 0.0003 |
| loss | 124 |
| n_updates | 208 |
| policy_gradient_loss | -0.00308 |
| value_loss | 289 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 370 |
| ep_rew_mean | -1.05 |
| time/ | |
| fps | 414 |
| iterations | 54 |
| time_elapsed | 266 |
| total_timesteps | 110592 |
| train/ | |
| approx_kl | 0.0039367327 |
| clip_fraction | 0.00342 |
| clip_range | 0.2 |
| entropy_loss | -1.19 |
| explained_variance | 0.83010334 |
| learning_rate | 0.0003 |
| loss | 50.3 |
| n_updates | 212 |
| policy_gradient_loss | -0.00201 |
| value_loss | 116 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 378 |
| ep_rew_mean | 2.3 |
| time/ | |
| fps | 411 |
| iterations | 55 |
| time_elapsed | 273 |
| total_timesteps | 112640 |
| train/ | |
| approx_kl | 0.004520635 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -1.12 |
| explained_variance | 0.7304286 |
| learning_rate | 0.0003 |
| loss | 138 |
| n_updates | 216 |
| policy_gradient_loss | -0.00452 |
| value_loss | 247 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 395 |
| ep_rew_mean | 3.58 |
| time/ | |
| fps | 411 |
| iterations | 56 |
| time_elapsed | 278 |
| total_timesteps | 114688 |
| train/ | |
| approx_kl | 0.0062670065 |
| clip_fraction | 0.0281 |
| clip_range | 0.2 |
| entropy_loss | -1.24 |
| explained_variance | 0.8617148 |
| learning_rate | 0.0003 |
| loss | 32.4 |
| n_updates | 220 |
| policy_gradient_loss | -0.00514 |
| value_loss | 64.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 412 |
| ep_rew_mean | 5.06 |
| time/ | |
| fps | 410 |
| iterations | 57 |
| time_elapsed | 284 |
| total_timesteps | 116736 |
| train/ | |
| approx_kl | 0.0042978907 |
| clip_fraction | 0.00989 |
| clip_range | 0.2 |
| entropy_loss | -1.14 |
| explained_variance | 0.77382654 |
| learning_rate | 0.0003 |
| loss | 97.3 |
| n_updates | 224 |
| policy_gradient_loss | -0.00155 |
| value_loss | 191 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 408 |
| ep_rew_mean | -0.207 |
| time/ | |
| fps | 410 |
| iterations | 58 |
| time_elapsed | 289 |
| total_timesteps | 118784 |
| train/ | |
| approx_kl | 0.0032121122 |
| clip_fraction | 0.00916 |
| clip_range | 0.2 |
| entropy_loss | -1.22 |
| explained_variance | 0.8189401 |
| learning_rate | 0.0003 |
| loss | 27.9 |
| n_updates | 228 |
| policy_gradient_loss | -0.00242 |
| value_loss | 85.7 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 120000 to videos/step_120000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 418 |
| ep_rew_mean | 0.57 |
| time/ | |
| fps | 399 |
| iterations | 59 |
| time_elapsed | 302 |
| total_timesteps | 120832 |
| train/ | |
| approx_kl | 0.00055425736 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.67028505 |
| learning_rate | 0.0003 |
| loss | 280 |
| n_updates | 232 |
| policy_gradient_loss | -7.78e-05 |
| value_loss | 522 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 436 |
| ep_rew_mean | 4.28 |
| time/ | |
| fps | 397 |
| iterations | 60 |
| time_elapsed | 309 |
| total_timesteps | 122880 |
| train/ | |
| approx_kl | 0.008951222 |
| clip_fraction | 0.0914 |
| clip_range | 0.2 |
| entropy_loss | -1.14 |
| explained_variance | 0.72118926 |
| learning_rate | 0.0003 |
| loss | 48.5 |
| n_updates | 236 |
| policy_gradient_loss | -0.0058 |
| value_loss | 214 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 446 |
| ep_rew_mean | 5.85 |
| time/ | |
| fps | 397 |
| iterations | 61 |
| time_elapsed | 314 |
| total_timesteps | 124928 |
| train/ | |
| approx_kl | 0.011050964 |
| clip_fraction | 0.0958 |
| clip_range | 0.2 |
| entropy_loss | -1.18 |
| explained_variance | 0.8881838 |
| learning_rate | 0.0003 |
| loss | 23.4 |
| n_updates | 240 |
| policy_gradient_loss | -0.00354 |
| value_loss | 76.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 464 |
| ep_rew_mean | 8.71 |
| time/ | |
| fps | 396 |
| iterations | 62 |
| time_elapsed | 320 |
| total_timesteps | 126976 |
| train/ | |
| approx_kl | 0.0038933342 |
| clip_fraction | 0.00159 |
| clip_range | 0.2 |
| entropy_loss | -1.15 |
| explained_variance | 0.7888298 |
| learning_rate | 0.0003 |
| loss | 81.4 |
| n_updates | 244 |
| policy_gradient_loss | -0.00242 |
| value_loss | 180 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 476 |
| ep_rew_mean | 9.46 |
| time/ | |
| fps | 396 |
| iterations | 63 |
| time_elapsed | 325 |
| total_timesteps | 129024 |
| train/ | |
| approx_kl | 0.002124371 |
| clip_fraction | 0.01 |
| clip_range | 0.2 |
| entropy_loss | -1.17 |
| explained_variance | 0.8479593 |
| learning_rate | 0.0003 |
| loss | 41.7 |
| n_updates | 248 |
| policy_gradient_loss | -0.00138 |
| value_loss | 155 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 493 |
| ep_rew_mean | 11.8 |
| time/ | |
| fps | 396 |
| iterations | 64 |
| time_elapsed | 330 |
| total_timesteps | 131072 |
| train/ | |
| approx_kl | 0.007902085 |
| clip_fraction | 0.0198 |
| clip_range | 0.2 |
| entropy_loss | -1.14 |
| explained_variance | 0.860183 |
| learning_rate | 0.0003 |
| loss | 62.4 |
| n_updates | 252 |
| policy_gradient_loss | -0.00233 |
| value_loss | 113 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 509 |
| ep_rew_mean | 14.7 |
| time/ | |
| fps | 395 |
| iterations | 65 |
| time_elapsed | 336 |
| total_timesteps | 133120 |
| train/ | |
| approx_kl | 0.004301928 |
| clip_fraction | 0.0123 |
| clip_range | 0.2 |
| entropy_loss | -1.13 |
| explained_variance | 0.8848971 |
| learning_rate | 0.0003 |
| loss | 37.4 |
| n_updates | 256 |
| policy_gradient_loss | -0.0028 |
| value_loss | 86.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 527 |
| ep_rew_mean | 16.9 |
| time/ | |
| fps | 395 |
| iterations | 66 |
| time_elapsed | 341 |
| total_timesteps | 135168 |
| train/ | |
| approx_kl | 0.0063031963 |
| clip_fraction | 0.0264 |
| clip_range | 0.2 |
| entropy_loss | -1.17 |
| explained_variance | 0.91382587 |
| learning_rate | 0.0003 |
| loss | 36.1 |
| n_updates | 260 |
| policy_gradient_loss | -0.0012 |
| value_loss | 69 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 543 |
| ep_rew_mean | 19.5 |
| time/ | |
| fps | 393 |
| iterations | 67 |
| time_elapsed | 348 |
| total_timesteps | 137216 |
| train/ | |
| approx_kl | 0.0055164285 |
| clip_fraction | 0.0399 |
| clip_range | 0.2 |
| entropy_loss | -1.11 |
| explained_variance | 0.92202497 |
| learning_rate | 0.0003 |
| loss | 43.9 |
| n_updates | 264 |
| policy_gradient_loss | -0.0048 |
| value_loss | 65.3 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 560 |
| ep_rew_mean | 20.8 |
| time/ | |
| fps | 393 |
| iterations | 68 |
| time_elapsed | 353 |
| total_timesteps | 139264 |
| train/ | |
| approx_kl | 0.00199862 |
| clip_fraction | 0.00317 |
| clip_range | 0.2 |
| entropy_loss | -1.02 |
| explained_variance | 0.6737658 |
| learning_rate | 0.0003 |
| loss | 192 |
| n_updates | 268 |
| policy_gradient_loss | -0.00163 |
| value_loss | 270 |
----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 140000 to videos/step_140000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 577 |
| ep_rew_mean | 22.1 |
| time/ | |
| fps | 383 |
| iterations | 69 |
| time_elapsed | 368 |
| total_timesteps | 141312 |
| train/ | |
| approx_kl | 0.004717985 |
| clip_fraction | 0.0145 |
| clip_range | 0.2 |
| entropy_loss | -1.08 |
| explained_variance | 0.92554545 |
| learning_rate | 0.0003 |
| loss | 12.7 |
| n_updates | 272 |
| policy_gradient_loss | -0.000954 |
| value_loss | 41.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 593 |
| ep_rew_mean | 24.6 |
| time/ | |
| fps | 382 |
| iterations | 70 |
| time_elapsed | 374 |
| total_timesteps | 143360 |
| train/ | |
| approx_kl | 0.0048742043 |
| clip_fraction | 0.0309 |
| clip_range | 0.2 |
| entropy_loss | -0.984 |
| explained_variance | 0.7605378 |
| learning_rate | 0.0003 |
| loss | 74.9 |
| n_updates | 276 |
| policy_gradient_loss | -0.0061 |
| value_loss | 183 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 600 |
| ep_rew_mean | 26 |
| time/ | |
| fps | 382 |
| iterations | 71 |
| time_elapsed | 380 |
| total_timesteps | 145408 |
| train/ | |
| approx_kl | 0.008403974 |
| clip_fraction | 0.0557 |
| clip_range | 0.2 |
| entropy_loss | -1.11 |
| explained_variance | 0.9409757 |
| learning_rate | 0.0003 |
| loss | 8.36 |
| n_updates | 280 |
| policy_gradient_loss | -0.00473 |
| value_loss | 24 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 600 |
| ep_rew_mean | 25.5 |
| time/ | |
| fps | 380 |
| iterations | 72 |
| time_elapsed | 387 |
| total_timesteps | 147456 |
| train/ | |
| approx_kl | 0.007754597 |
| clip_fraction | 0.0339 |
| clip_range | 0.2 |
| entropy_loss | -1.08 |
| explained_variance | 0.9413037 |
| learning_rate | 0.0003 |
| loss | 11.2 |
| n_updates | 284 |
| policy_gradient_loss | -0.00351 |
| value_loss | 26.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 609 |
| ep_rew_mean | 28.4 |
| time/ | |
| fps | 379 |
| iterations | 73 |
| time_elapsed | 393 |
| total_timesteps | 149504 |
| train/ | |
| approx_kl | 0.0029857582 |
| clip_fraction | 0.00464 |
| clip_range | 0.2 |
| entropy_loss | -1.1 |
| explained_variance | 0.8820114 |
| learning_rate | 0.0003 |
| loss | 36.9 |
| n_updates | 288 |
| policy_gradient_loss | -0.00091 |
| value_loss | 62.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 626 |
| ep_rew_mean | 29.7 |
| time/ | |
| fps | 378 |
| iterations | 74 |
| time_elapsed | 400 |
| total_timesteps | 151552 |
| train/ | |
| approx_kl | 0.0046141734 |
| clip_fraction | 0.0283 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.9342376 |
| learning_rate | 0.0003 |
| loss | 10.7 |
| n_updates | 292 |
| policy_gradient_loss | -0.00238 |
| value_loss | 41.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 636 |
| ep_rew_mean | 30.1 |
| time/ | |
| fps | 378 |
| iterations | 75 |
| time_elapsed | 406 |
| total_timesteps | 153600 |
| train/ | |
| approx_kl | 0.0025668796 |
| clip_fraction | 0.00562 |
| clip_range | 0.2 |
| entropy_loss | -0.975 |
| explained_variance | 0.8176935 |
| learning_rate | 0.0003 |
| loss | 47.6 |
| n_updates | 296 |
| policy_gradient_loss | -0.00156 |
| value_loss | 81.4 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 632 |
| ep_rew_mean | 28.8 |
| time/ | |
| fps | 377 |
| iterations | 76 |
| time_elapsed | 411 |
| total_timesteps | 155648 |
| train/ | |
| approx_kl | 0.00424249 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -1.08 |
| explained_variance | 0.7560506 |
| learning_rate | 0.0003 |
| loss | 92.3 |
| n_updates | 300 |
| policy_gradient_loss | -0.00249 |
| value_loss | 157 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 635 |
| ep_rew_mean | 29.9 |
| time/ | |
| fps | 377 |
| iterations | 77 |
| time_elapsed | 417 |
| total_timesteps | 157696 |
| train/ | |
| approx_kl | 0.0038080937 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -1.03 |
| explained_variance | 0.7649582 |
| learning_rate | 0.0003 |
| loss | 41.6 |
| n_updates | 304 |
| policy_gradient_loss | -0.00324 |
| value_loss | 191 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 652 |
| ep_rew_mean | 32.1 |
| time/ | |
| fps | 376 |
| iterations | 78 |
| time_elapsed | 423 |
| total_timesteps | 159744 |
| train/ | |
| approx_kl | 0.0037250658 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -1.11 |
| explained_variance | 0.8663412 |
| learning_rate | 0.0003 |
| loss | 78.5 |
| n_updates | 308 |
| policy_gradient_loss | -0.00243 |
| value_loss | 108 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 160000 to videos/step_160000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 660 |
| ep_rew_mean | 35.3 |
| time/ | |
| fps | 369 |
| iterations | 79 |
| time_elapsed | 438 |
| total_timesteps | 161792 |
| train/ | |
| approx_kl | 0.004687347 |
| clip_fraction | 0.0204 |
| clip_range | 0.2 |
| entropy_loss | -1.02 |
| explained_variance | 0.875966 |
| learning_rate | 0.0003 |
| loss | 30.5 |
| n_updates | 312 |
| policy_gradient_loss | -0.00304 |
| value_loss | 80.9 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 676 |
| ep_rew_mean | 37.8 |
| time/ | |
| fps | 369 |
| iterations | 80 |
| time_elapsed | 443 |
| total_timesteps | 163840 |
| train/ | |
| approx_kl | 0.007152658 |
| clip_fraction | 0.0358 |
| clip_range | 0.2 |
| entropy_loss | -1.12 |
| explained_variance | 0.9192853 |
| learning_rate | 0.0003 |
| loss | 16.8 |
| n_updates | 316 |
| policy_gradient_loss | -0.00217 |
| value_loss | 42.1 |
-----------------------------------------
---------------------------------------
| rollout/ | |
| ep_len_mean | 692 |
| ep_rew_mean | 39 |
| time/ | |
| fps | 367 |
| iterations | 81 |
| time_elapsed | 450 |
| total_timesteps | 165888 |
| train/ | |
| approx_kl | 0.0053707 |
| clip_fraction | 0.0404 |
| clip_range | 0.2 |
| entropy_loss | -1.08 |
| explained_variance | 0.961576 |
| learning_rate | 0.0003 |
| loss | 11.5 |
| n_updates | 320 |
| policy_gradient_loss | -0.000308 |
| value_loss | 17.9 |
---------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 707 |
| ep_rew_mean | 41.6 |
| time/ | |
| fps | 367 |
| iterations | 82 |
| time_elapsed | 457 |
| total_timesteps | 167936 |
| train/ | |
| approx_kl | 0.0030738732 |
| clip_fraction | 0.0291 |
| clip_range | 0.2 |
| entropy_loss | -0.985 |
| explained_variance | 0.94173133 |
| learning_rate | 0.0003 |
| loss | 5.86 |
| n_updates | 324 |
| policy_gradient_loss | -0.001 |
| value_loss | 19.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 715 |
| ep_rew_mean | 43.2 |
| time/ | |
| fps | 366 |
| iterations | 83 |
| time_elapsed | 463 |
| total_timesteps | 169984 |
| train/ | |
| approx_kl | 0.0027180926 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -1.08 |
| explained_variance | 0.9597426 |
| learning_rate | 0.0003 |
| loss | 23.7 |
| n_updates | 328 |
| policy_gradient_loss | -0.00122 |
| value_loss | 27.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 730 |
| ep_rew_mean | 47.4 |
| time/ | |
| fps | 366 |
| iterations | 84 |
| time_elapsed | 469 |
| total_timesteps | 172032 |
| train/ | |
| approx_kl | 0.009553924 |
| clip_fraction | 0.0558 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.9735119 |
| learning_rate | 0.0003 |
| loss | 5.36 |
| n_updates | 332 |
| policy_gradient_loss | -0.00312 |
| value_loss | 15.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 738 |
| ep_rew_mean | 49 |
| time/ | |
| fps | 365 |
| iterations | 85 |
| time_elapsed | 476 |
| total_timesteps | 174080 |
| train/ | |
| approx_kl | 0.0025081043 |
| clip_fraction | 0.00562 |
| clip_range | 0.2 |
| entropy_loss | -1.11 |
| explained_variance | 0.83761734 |
| learning_rate | 0.0003 |
| loss | 28.1 |
| n_updates | 336 |
| policy_gradient_loss | -0.00137 |
| value_loss | 133 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 752 |
| ep_rew_mean | 51.6 |
| time/ | |
| fps | 365 |
| iterations | 86 |
| time_elapsed | 482 |
| total_timesteps | 176128 |
| train/ | |
| approx_kl | 0.005504003 |
| clip_fraction | 0.0219 |
| clip_range | 0.2 |
| entropy_loss | -1.07 |
| explained_variance | 0.9596741 |
| learning_rate | 0.0003 |
| loss | 8.41 |
| n_updates | 340 |
| policy_gradient_loss | 3.52e-05 |
| value_loss | 25.7 |
-----------------------------------------
---------------------------------------
| rollout/ | |
| ep_len_mean | 762 |
| ep_rew_mean | 52.8 |
| time/ | |
| fps | 365 |
| iterations | 87 |
| time_elapsed | 487 |
| total_timesteps | 178176 |
| train/ | |
| approx_kl | 0.0096641 |
| clip_fraction | 0.0873 |
| clip_range | 0.2 |
| entropy_loss | -1.06 |
| explained_variance | 0.9647306 |
| learning_rate | 0.0003 |
| loss | 13.6 |
| n_updates | 344 |
| policy_gradient_loss | -0.00399 |
| value_loss | 28.3 |
---------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 180000 to videos/step_180000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 762 |
| ep_rew_mean | 54.2 |
| time/ | |
| fps | 359 |
| iterations | 88 |
| time_elapsed | 501 |
| total_timesteps | 180224 |
| train/ | |
| approx_kl | 0.001901669 |
| clip_fraction | 0.00977 |
| clip_range | 0.2 |
| entropy_loss | -1.06 |
| explained_variance | 0.8564778 |
| learning_rate | 0.0003 |
| loss | 60.7 |
| n_updates | 348 |
| policy_gradient_loss | -0.002 |
| value_loss | 139 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 769 |
| ep_rew_mean | 56.6 |
| time/ | |
| fps | 359 |
| iterations | 89 |
| time_elapsed | 507 |
| total_timesteps | 182272 |
| train/ | |
| approx_kl | 0.010260288 |
| clip_fraction | 0.105 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.9726843 |
| learning_rate | 0.0003 |
| loss | 6.88 |
| n_updates | 352 |
| policy_gradient_loss | -0.00369 |
| value_loss | 17.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 784 |
| ep_rew_mean | 58.4 |
| time/ | |
| fps | 359 |
| iterations | 90 |
| time_elapsed | 513 |
| total_timesteps | 184320 |
| train/ | |
| approx_kl | 0.0059805554 |
| clip_fraction | 0.0295 |
| clip_range | 0.2 |
| entropy_loss | -1.1 |
| explained_variance | 0.9660654 |
| learning_rate | 0.0003 |
| loss | 17.3 |
| n_updates | 356 |
| policy_gradient_loss | -0.002 |
| value_loss | 25.6 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 799 |
| ep_rew_mean | 60.9 |
| time/ | |
| fps | 358 |
| iterations | 91 |
| time_elapsed | 519 |
| total_timesteps | 186368 |
| train/ | |
| approx_kl | 0.006415116 |
| clip_fraction | 0.0348 |
| clip_range | 0.2 |
| entropy_loss | -0.964 |
| explained_variance | 0.96307445 |
| learning_rate | 0.0003 |
| loss | 6.45 |
| n_updates | 360 |
| policy_gradient_loss | -0.00104 |
| value_loss | 31 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 806 |
| ep_rew_mean | 62.3 |
| time/ | |
| fps | 358 |
| iterations | 92 |
| time_elapsed | 525 |
| total_timesteps | 188416 |
| train/ | |
| approx_kl | 0.008542726 |
| clip_fraction | 0.0464 |
| clip_range | 0.2 |
| entropy_loss | -1.01 |
| explained_variance | 0.97477925 |
| learning_rate | 0.0003 |
| loss | 9.81 |
| n_updates | 364 |
| policy_gradient_loss | -0.00447 |
| value_loss | 16.7 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 806 |
| ep_rew_mean | 64 |
| time/ | |
| fps | 357 |
| iterations | 93 |
| time_elapsed | 532 |
| total_timesteps | 190464 |
| train/ | |
| approx_kl | 0.00881435 |
| clip_fraction | 0.0288 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.95742637 |
| learning_rate | 0.0003 |
| loss | 13.6 |
| n_updates | 368 |
| policy_gradient_loss | -0.00127 |
| value_loss | 27.6 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 821 |
| ep_rew_mean | 66.7 |
| time/ | |
| fps | 357 |
| iterations | 94 |
| time_elapsed | 538 |
| total_timesteps | 192512 |
| train/ | |
| approx_kl | 0.005030601 |
| clip_fraction | 0.0514 |
| clip_range | 0.2 |
| entropy_loss | -1.08 |
| explained_variance | 0.97697866 |
| learning_rate | 0.0003 |
| loss | 10.8 |
| n_updates | 372 |
| policy_gradient_loss | -0.00107 |
| value_loss | 14.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 828 |
| ep_rew_mean | 67.4 |
| time/ | |
| fps | 357 |
| iterations | 95 |
| time_elapsed | 544 |
| total_timesteps | 194560 |
| train/ | |
| approx_kl | 0.0016044911 |
| clip_fraction | 0.0137 |
| clip_range | 0.2 |
| entropy_loss | -1.06 |
| explained_variance | 0.96283203 |
| learning_rate | 0.0003 |
| loss | 7.4 |
| n_updates | 376 |
| policy_gradient_loss | -0.00328 |
| value_loss | 24.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 828 |
| ep_rew_mean | 68.7 |
| time/ | |
| fps | 357 |
| iterations | 96 |
| time_elapsed | 550 |
| total_timesteps | 196608 |
| train/ | |
| approx_kl | 0.0044421116 |
| clip_fraction | 0.0131 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.98039514 |
| learning_rate | 0.0003 |
| loss | 4.47 |
| n_updates | 380 |
| policy_gradient_loss | -0.00268 |
| value_loss | 13.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 835 |
| ep_rew_mean | 70.6 |
| time/ | |
| fps | 356 |
| iterations | 97 |
| time_elapsed | 557 |
| total_timesteps | 198656 |
| train/ | |
| approx_kl | 0.0036420499 |
| clip_fraction | 0.00525 |
| clip_range | 0.2 |
| entropy_loss | -1.07 |
| explained_variance | 0.96388763 |
| learning_rate | 0.0003 |
| loss | 10.5 |
| n_updates | 384 |
| policy_gradient_loss | -0.000653 |
| value_loss | 27 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 200000 to videos/step_200000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 843 |
| ep_rew_mean | 72 |
| time/ | |
| fps | 350 |
| iterations | 98 |
| time_elapsed | 573 |
| total_timesteps | 200704 |
| train/ | |
| approx_kl | 0.0064555863 |
| clip_fraction | 0.0426 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.9816038 |
| learning_rate | 0.0003 |
| loss | 4.96 |
| n_updates | 388 |
| policy_gradient_loss | -0.00323 |
| value_loss | 11 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 843 |
| ep_rew_mean | 72.9 |
| time/ | |
| fps | 350 |
| iterations | 99 |
| time_elapsed | 579 |
| total_timesteps | 202752 |
| train/ | |
| approx_kl | 0.00656581 |
| clip_fraction | 0.0503 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.9635119 |
| learning_rate | 0.0003 |
| loss | 4.37 |
| n_updates | 392 |
| policy_gradient_loss | -0.00117 |
| value_loss | 29.5 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 855 |
| ep_rew_mean | 78.9 |
| time/ | |
| fps | 349 |
| iterations | 100 |
| time_elapsed | 585 |
| total_timesteps | 204800 |
| train/ | |
| approx_kl | 0.0070309807 |
| clip_fraction | 0.00842 |
| clip_range | 0.2 |
| entropy_loss | -1.09 |
| explained_variance | 0.9762978 |
| learning_rate | 0.0003 |
| loss | 3.7 |
| n_updates | 396 |
| policy_gradient_loss | -0.000426 |
| value_loss | 14 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 871 |
| ep_rew_mean | 81.2 |
| time/ | |
| fps | 349 |
| iterations | 101 |
| time_elapsed | 591 |
| total_timesteps | 206848 |
| train/ | |
| approx_kl | 0.004208331 |
| clip_fraction | 0.00427 |
| clip_range | 0.2 |
| entropy_loss | -1.06 |
| explained_variance | 0.9835882 |
| learning_rate | 0.0003 |
| loss | 4.33 |
| n_updates | 400 |
| policy_gradient_loss | -0.000222 |
| value_loss | 11.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 880 |
| ep_rew_mean | 83.5 |
| time/ | |
| fps | 349 |
| iterations | 102 |
| time_elapsed | 596 |
| total_timesteps | 208896 |
| train/ | |
| approx_kl | 0.010569755 |
| clip_fraction | 0.0237 |
| clip_range | 0.2 |
| entropy_loss | -1.03 |
| explained_variance | 0.9776722 |
| learning_rate | 0.0003 |
| loss | 14.1 |
| n_updates | 404 |
| policy_gradient_loss | -0.00283 |
| value_loss | 19.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 880 |
| ep_rew_mean | 83.3 |
| time/ | |
| fps | 350 |
| iterations | 103 |
| time_elapsed | 602 |
| total_timesteps | 210944 |
| train/ | |
| approx_kl | 0.0031817178 |
| clip_fraction | 0.0083 |
| clip_range | 0.2 |
| entropy_loss | -1 |
| explained_variance | 0.8425119 |
| learning_rate | 0.0003 |
| loss | 157 |
| n_updates | 408 |
| policy_gradient_loss | -0.00243 |
| value_loss | 166 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 887 |
| ep_rew_mean | 83.7 |
| time/ | |
| fps | 349 |
| iterations | 104 |
| time_elapsed | 608 |
| total_timesteps | 212992 |
| train/ | |
| approx_kl | 0.0034875958 |
| clip_fraction | 0.0153 |
| clip_range | 0.2 |
| entropy_loss | -1 |
| explained_variance | 0.98062634 |
| learning_rate | 0.0003 |
| loss | 3.33 |
| n_updates | 412 |
| policy_gradient_loss | 0.000486 |
| value_loss | 16 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 903 |
| ep_rew_mean | 86.1 |
| time/ | |
| fps | 349 |
| iterations | 105 |
| time_elapsed | 614 |
| total_timesteps | 215040 |
| train/ | |
| approx_kl | 0.0058191186 |
| clip_fraction | 0.0535 |
| clip_range | 0.2 |
| entropy_loss | -0.997 |
| explained_variance | 0.97953767 |
| learning_rate | 0.0003 |
| loss | 3.77 |
| n_updates | 416 |
| policy_gradient_loss | -0.00765 |
| value_loss | 14.2 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 910 |
| ep_rew_mean | 88.3 |
| time/ | |
| fps | 349 |
| iterations | 106 |
| time_elapsed | 620 |
| total_timesteps | 217088 |
| train/ | |
| approx_kl | 0.007037095 |
| clip_fraction | 0.031 |
| clip_range | 0.2 |
| entropy_loss | -0.957 |
| explained_variance | 0.9727411 |
| learning_rate | 0.0003 |
| loss | 8.96 |
| n_updates | 420 |
| policy_gradient_loss | -0.00143 |
| value_loss | 18.9 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 909 |
| ep_rew_mean | 88.7 |
| time/ | |
| fps | 349 |
| iterations | 107 |
| time_elapsed | 626 |
| total_timesteps | 219136 |
| train/ | |
| approx_kl | 0.003265044 |
| clip_fraction | 0.00586 |
| clip_range | 0.2 |
| entropy_loss | -0.907 |
| explained_variance | 0.9738519 |
| learning_rate | 0.0003 |
| loss | 2.63 |
| n_updates | 424 |
| policy_gradient_loss | -9.21e-05 |
| value_loss | 17.7 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 220000 to videos/step_220000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 909 |
| ep_rew_mean | 88.6 |
| time/ | |
| fps | 344 |
| iterations | 108 |
| time_elapsed | 641 |
| total_timesteps | 221184 |
| train/ | |
| approx_kl | 0.0015720768 |
| clip_fraction | 0.000244 |
| clip_range | 0.2 |
| entropy_loss | -0.912 |
| explained_variance | 0.70963806 |
| learning_rate | 0.0003 |
| loss | 277 |
| n_updates | 428 |
| policy_gradient_loss | -0.000998 |
| value_loss | 463 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 916 |
| ep_rew_mean | 90.4 |
| time/ | |
| fps | 345 |
| iterations | 109 |
| time_elapsed | 646 |
| total_timesteps | 223232 |
| train/ | |
| approx_kl | 0.0037638533 |
| clip_fraction | 0.00977 |
| clip_range | 0.2 |
| entropy_loss | -0.92 |
| explained_variance | 0.91075706 |
| learning_rate | 0.0003 |
| loss | 31.9 |
| n_updates | 432 |
| policy_gradient_loss | -0.000791 |
| value_loss | 73.5 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 916 |
| ep_rew_mean | 90.9 |
| time/ | |
| fps | 345 |
| iterations | 110 |
| time_elapsed | 652 |
| total_timesteps | 225280 |
| train/ | |
| approx_kl | 0.005898675 |
| clip_fraction | 0.0435 |
| clip_range | 0.2 |
| entropy_loss | -0.927 |
| explained_variance | 0.9696002 |
| learning_rate | 0.0003 |
| loss | 4.88 |
| n_updates | 436 |
| policy_gradient_loss | -0.000892 |
| value_loss | 18.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 922 |
| ep_rew_mean | 92.3 |
| time/ | |
| fps | 345 |
| iterations | 111 |
| time_elapsed | 658 |
| total_timesteps | 227328 |
| train/ | |
| approx_kl | 0.0051690624 |
| clip_fraction | 0.0264 |
| clip_range | 0.2 |
| entropy_loss | -0.918 |
| explained_variance | 0.9632635 |
| learning_rate | 0.0003 |
| loss | 1.5 |
| n_updates | 440 |
| policy_gradient_loss | -0.0018 |
| value_loss | 36.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 918 |
| ep_rew_mean | 90.2 |
| time/ | |
| fps | 344 |
| iterations | 112 |
| time_elapsed | 664 |
| total_timesteps | 229376 |
| train/ | |
| approx_kl | 0.0069203693 |
| clip_fraction | 0.0363 |
| clip_range | 0.2 |
| entropy_loss | -0.901 |
| explained_variance | 0.9824635 |
| learning_rate | 0.0003 |
| loss | 6.77 |
| n_updates | 444 |
| policy_gradient_loss | -0.00298 |
| value_loss | 16.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 918 |
| ep_rew_mean | 90.9 |
| time/ | |
| fps | 345 |
| iterations | 113 |
| time_elapsed | 670 |
| total_timesteps | 231424 |
| train/ | |
| approx_kl | 0.0038524868 |
| clip_fraction | 0.0072 |
| clip_range | 0.2 |
| entropy_loss | -0.884 |
| explained_variance | 0.7799375 |
| learning_rate | 0.0003 |
| loss | 86.8 |
| n_updates | 448 |
| policy_gradient_loss | -0.0012 |
| value_loss | 195 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 918 |
| ep_rew_mean | 91.8 |
| time/ | |
| fps | 345 |
| iterations | 114 |
| time_elapsed | 676 |
| total_timesteps | 233472 |
| train/ | |
| approx_kl | 0.006992953 |
| clip_fraction | 0.0275 |
| clip_range | 0.2 |
| entropy_loss | -0.851 |
| explained_variance | 0.9725562 |
| learning_rate | 0.0003 |
| loss | 1.43 |
| n_updates | 452 |
| policy_gradient_loss | -0.00114 |
| value_loss | 12.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 925 |
| ep_rew_mean | 94.9 |
| time/ | |
| fps | 345 |
| iterations | 115 |
| time_elapsed | 681 |
| total_timesteps | 235520 |
| train/ | |
| approx_kl | 0.0042324346 |
| clip_fraction | 0.0566 |
| clip_range | 0.2 |
| entropy_loss | -0.833 |
| explained_variance | 0.97358745 |
| learning_rate | 0.0003 |
| loss | 4.91 |
| n_updates | 456 |
| policy_gradient_loss | -0.00227 |
| value_loss | 18 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 925 |
| ep_rew_mean | 96.3 |
| time/ | |
| fps | 345 |
| iterations | 116 |
| time_elapsed | 687 |
| total_timesteps | 237568 |
| train/ | |
| approx_kl | 0.0067531522 |
| clip_fraction | 0.053 |
| clip_range | 0.2 |
| entropy_loss | -0.875 |
| explained_variance | 0.9655474 |
| learning_rate | 0.0003 |
| loss | 4.87 |
| n_updates | 460 |
| policy_gradient_loss | 0.00109 |
| value_loss | 22.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 925 |
| ep_rew_mean | 97 |
| time/ | |
| fps | 345 |
| iterations | 117 |
| time_elapsed | 693 |
| total_timesteps | 239616 |
| train/ | |
| approx_kl | 0.015832692 |
| clip_fraction | 0.108 |
| clip_range | 0.2 |
| entropy_loss | -0.925 |
| explained_variance | 0.97858584 |
| learning_rate | 0.0003 |
| loss | 8.22 |
| n_updates | 464 |
| policy_gradient_loss | -0.000747 |
| value_loss | 20.3 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 240000 to videos/step_240000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 918 |
| ep_rew_mean | 96.3 |
| time/ | |
| fps | 341 |
| iterations | 118 |
| time_elapsed | 708 |
| total_timesteps | 241664 |
| train/ | |
| approx_kl | 0.0049235607 |
| clip_fraction | 0.0348 |
| clip_range | 0.2 |
| entropy_loss | -0.948 |
| explained_variance | 0.9842316 |
| learning_rate | 0.0003 |
| loss | 2.4 |
| n_updates | 468 |
| policy_gradient_loss | -0.00181 |
| value_loss | 13.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 918 |
| ep_rew_mean | 97.3 |
| time/ | |
| fps | 341 |
| iterations | 119 |
| time_elapsed | 713 |
| total_timesteps | 243712 |
| train/ | |
| approx_kl | 0.002787627 |
| clip_fraction | 0.0146 |
| clip_range | 0.2 |
| entropy_loss | -0.994 |
| explained_variance | 0.80519634 |
| learning_rate | 0.0003 |
| loss | 78.6 |
| n_updates | 472 |
| policy_gradient_loss | -0.00195 |
| value_loss | 234 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 925 |
| ep_rew_mean | 99.3 |
| time/ | |
| fps | 341 |
| iterations | 120 |
| time_elapsed | 720 |
| total_timesteps | 245760 |
| train/ | |
| approx_kl | 0.0075014466 |
| clip_fraction | 0.0466 |
| clip_range | 0.2 |
| entropy_loss | -0.924 |
| explained_variance | 0.980995 |
| learning_rate | 0.0003 |
| loss | 6.73 |
| n_updates | 476 |
| policy_gradient_loss | -0.00271 |
| value_loss | 10.7 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 931 |
| ep_rew_mean | 102 |
| time/ | |
| fps | 341 |
| iterations | 121 |
| time_elapsed | 725 |
| total_timesteps | 247808 |
| train/ | |
| approx_kl | 0.002686028 |
| clip_fraction | 0.0061 |
| clip_range | 0.2 |
| entropy_loss | -0.971 |
| explained_variance | 0.84240925 |
| learning_rate | 0.0003 |
| loss | 31.3 |
| n_updates | 480 |
| policy_gradient_loss | -0.00211 |
| value_loss | 246 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 921 |
| ep_rew_mean | 101 |
| time/ | |
| fps | 341 |
| iterations | 122 |
| time_elapsed | 731 |
| total_timesteps | 249856 |
| train/ | |
| approx_kl | 0.0023052532 |
| clip_fraction | 0.026 |
| clip_range | 0.2 |
| entropy_loss | -0.974 |
| explained_variance | 0.9833261 |
| learning_rate | 0.0003 |
| loss | 5.49 |
| n_updates | 484 |
| policy_gradient_loss | -0.00246 |
| value_loss | 15.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 921 |
| ep_rew_mean | 101 |
| time/ | |
| fps | 341 |
| iterations | 123 |
| time_elapsed | 736 |
| total_timesteps | 251904 |
| train/ | |
| approx_kl | 0.0006938746 |
| clip_fraction | 0.000977 |
| clip_range | 0.2 |
| entropy_loss | -0.916 |
| explained_variance | 0.7556349 |
| learning_rate | 0.0003 |
| loss | 192 |
| n_updates | 488 |
| policy_gradient_loss | 0.000134 |
| value_loss | 407 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 914 |
| ep_rew_mean | 100 |
| time/ | |
| fps | 341 |
| iterations | 124 |
| time_elapsed | 742 |
| total_timesteps | 253952 |
| train/ | |
| approx_kl | 0.008821249 |
| clip_fraction | 0.0507 |
| clip_range | 0.2 |
| entropy_loss | -0.975 |
| explained_variance | 0.98086894 |
| learning_rate | 0.0003 |
| loss | 3.88 |
| n_updates | 492 |
| policy_gradient_loss | -0.00183 |
| value_loss | 11.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 914 |
| ep_rew_mean | 101 |
| time/ | |
| fps | 342 |
| iterations | 125 |
| time_elapsed | 748 |
| total_timesteps | 256000 |
| train/ | |
| approx_kl | 0.008290017 |
| clip_fraction | 0.054 |
| clip_range | 0.2 |
| entropy_loss | -1.01 |
| explained_variance | 0.8910267 |
| learning_rate | 0.0003 |
| loss | 16.5 |
| n_updates | 496 |
| policy_gradient_loss | -0.00291 |
| value_loss | 118 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 892 |
| ep_rew_mean | 97.7 |
| time/ | |
| fps | 342 |
| iterations | 126 |
| time_elapsed | 753 |
| total_timesteps | 258048 |
| train/ | |
| approx_kl | 0.0019536526 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -0.967 |
| explained_variance | 0.97752017 |
| learning_rate | 0.0003 |
| loss | 3.13 |
| n_updates | 500 |
| policy_gradient_loss | -0.00249 |
| value_loss | 15.6 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 260000 to videos/step_260000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 892 |
| ep_rew_mean | 97.7 |
| time/ | |
| fps | 339 |
| iterations | 127 |
| time_elapsed | 766 |
| total_timesteps | 260096 |
| train/ | |
| approx_kl | 0.00028758595 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.943 |
| explained_variance | 0.6625037 |
| learning_rate | 0.0003 |
| loss | 110 |
| n_updates | 504 |
| policy_gradient_loss | 9.43e-05 |
| value_loss | 593 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 883 |
| ep_rew_mean | 97.6 |
| time/ | |
| fps | 339 |
| iterations | 128 |
| time_elapsed | 771 |
| total_timesteps | 262144 |
| train/ | |
| approx_kl | 0.007629823 |
| clip_fraction | 0.0355 |
| clip_range | 0.2 |
| entropy_loss | -0.969 |
| explained_variance | 0.97190464 |
| learning_rate | 0.0003 |
| loss | 2.82 |
| n_updates | 508 |
| policy_gradient_loss | -0.0019 |
| value_loss | 13.9 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 883 |
| ep_rew_mean | 97.7 |
| time/ | |
| fps | 340 |
| iterations | 129 |
| time_elapsed | 776 |
| total_timesteps | 264192 |
| train/ | |
| approx_kl | 0.0062257675 |
| clip_fraction | 0.00635 |
| clip_range | 0.2 |
| entropy_loss | -0.969 |
| explained_variance | 0.8569973 |
| learning_rate | 0.0003 |
| loss | 90.7 |
| n_updates | 512 |
| policy_gradient_loss | -0.00148 |
| value_loss | 229 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 881 |
| ep_rew_mean | 98.8 |
| time/ | |
| fps | 340 |
| iterations | 130 |
| time_elapsed | 782 |
| total_timesteps | 266240 |
| train/ | |
| approx_kl | 0.0026525157 |
| clip_fraction | 0.0349 |
| clip_range | 0.2 |
| entropy_loss | -0.896 |
| explained_variance | 0.9816393 |
| learning_rate | 0.0003 |
| loss | 5.24 |
| n_updates | 516 |
| policy_gradient_loss | 0.000211 |
| value_loss | 12.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 873 |
| ep_rew_mean | 97.7 |
| time/ | |
| fps | 340 |
| iterations | 131 |
| time_elapsed | 788 |
| total_timesteps | 268288 |
| train/ | |
| approx_kl | 0.001105156 |
| clip_fraction | 0.00305 |
| clip_range | 0.2 |
| entropy_loss | -0.919 |
| explained_variance | 0.9252334 |
| learning_rate | 0.0003 |
| loss | 51.6 |
| n_updates | 520 |
| policy_gradient_loss | -0.000744 |
| value_loss | 107 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 844 |
| ep_rew_mean | 94.2 |
| time/ | |
| fps | 340 |
| iterations | 132 |
| time_elapsed | 792 |
| total_timesteps | 270336 |
| train/ | |
| approx_kl | 0.0035310413 |
| clip_fraction | 0.0187 |
| clip_range | 0.2 |
| entropy_loss | -0.919 |
| explained_variance | 0.8440285 |
| learning_rate | 0.0003 |
| loss | 61.5 |
| n_updates | 524 |
| policy_gradient_loss | -0.00334 |
| value_loss | 231 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 844 |
| ep_rew_mean | 94.9 |
| time/ | |
| fps | 340 |
| iterations | 133 |
| time_elapsed | 798 |
| total_timesteps | 272384 |
| train/ | |
| approx_kl | 0.0004107853 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.942 |
| explained_variance | 0.771625 |
| learning_rate | 0.0003 |
| loss | 142 |
| n_updates | 528 |
| policy_gradient_loss | -2.46e-05 |
| value_loss | 500 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 844 |
| ep_rew_mean | 94.9 |
| time/ | |
| fps | 341 |
| iterations | 134 |
| time_elapsed | 804 |
| total_timesteps | 274432 |
| train/ | |
| approx_kl | 0.0023841613 |
| clip_fraction | 0.005 |
| clip_range | 0.2 |
| entropy_loss | -0.886 |
| explained_variance | 0.93957955 |
| learning_rate | 0.0003 |
| loss | 3.22 |
| n_updates | 532 |
| policy_gradient_loss | -0.00131 |
| value_loss | 26.9 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 828 |
| ep_rew_mean | 92.8 |
| time/ | |
| fps | 341 |
| iterations | 135 |
| time_elapsed | 810 |
| total_timesteps | 276480 |
| train/ | |
| approx_kl | 0.01322646 |
| clip_fraction | 0.0552 |
| clip_range | 0.2 |
| entropy_loss | -0.978 |
| explained_variance | 0.9770493 |
| learning_rate | 0.0003 |
| loss | 3.79 |
| n_updates | 536 |
| policy_gradient_loss | -0.00234 |
| value_loss | 13.3 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 824 |
| ep_rew_mean | 93.8 |
| time/ | |
| fps | 341 |
| iterations | 136 |
| time_elapsed | 815 |
| total_timesteps | 278528 |
| train/ | |
| approx_kl | 0.0014058908 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -0.94 |
| explained_variance | 0.81320906 |
| learning_rate | 0.0003 |
| loss | 99.8 |
| n_updates | 540 |
| policy_gradient_loss | 0.000384 |
| value_loss | 170 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 280000 to videos/step_280000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 824 |
| ep_rew_mean | 95 |
| time/ | |
| fps | 338 |
| iterations | 137 |
| time_elapsed | 828 |
| total_timesteps | 280576 |
| train/ | |
| approx_kl | 0.0068661678 |
| clip_fraction | 0.0129 |
| clip_range | 0.2 |
| entropy_loss | -0.902 |
| explained_variance | 0.8120952 |
| learning_rate | 0.0003 |
| loss | 77.2 |
| n_updates | 544 |
| policy_gradient_loss | -0.00011 |
| value_loss | 168 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 817 |
| ep_rew_mean | 94.1 |
| time/ | |
| fps | 338 |
| iterations | 138 |
| time_elapsed | 834 |
| total_timesteps | 282624 |
| train/ | |
| approx_kl | 0.008456634 |
| clip_fraction | 0.00549 |
| clip_range | 0.2 |
| entropy_loss | -0.967 |
| explained_variance | 0.9822634 |
| learning_rate | 0.0003 |
| loss | 3.81 |
| n_updates | 548 |
| policy_gradient_loss | -0.000174 |
| value_loss | 10.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 787 |
| ep_rew_mean | 89.7 |
| time/ | |
| fps | 339 |
| iterations | 139 |
| time_elapsed | 839 |
| total_timesteps | 284672 |
| train/ | |
| approx_kl | 0.002866476 |
| clip_fraction | 0.0125 |
| clip_range | 0.2 |
| entropy_loss | -0.969 |
| explained_variance | 0.8988431 |
| learning_rate | 0.0003 |
| loss | 16.1 |
| n_updates | 552 |
| policy_gradient_loss | -0.0023 |
| value_loss | 92.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 787 |
| ep_rew_mean | 89.6 |
| time/ | |
| fps | 339 |
| iterations | 140 |
| time_elapsed | 844 |
| total_timesteps | 286720 |
| train/ | |
| approx_kl | 0.0009685765 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.981 |
| explained_variance | 0.75525427 |
| learning_rate | 0.0003 |
| loss | 330 |
| n_updates | 556 |
| policy_gradient_loss | -0.000401 |
| value_loss | 520 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 793 |
| ep_rew_mean | 91.3 |
| time/ | |
| fps | 339 |
| iterations | 141 |
| time_elapsed | 850 |
| total_timesteps | 288768 |
| train/ | |
| approx_kl | 0.006308576 |
| clip_fraction | 0.0117 |
| clip_range | 0.2 |
| entropy_loss | -0.965 |
| explained_variance | 0.9282582 |
| learning_rate | 0.0003 |
| loss | 5.46 |
| n_updates | 560 |
| policy_gradient_loss | -0.00124 |
| value_loss | 31.3 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 793 |
| ep_rew_mean | 91.8 |
| time/ | |
| fps | 339 |
| iterations | 142 |
| time_elapsed | 856 |
| total_timesteps | 290816 |
| train/ | |
| approx_kl | 0.007530058 |
| clip_fraction | 0.0345 |
| clip_range | 0.2 |
| entropy_loss | -0.903 |
| explained_variance | 0.959983 |
| learning_rate | 0.0003 |
| loss | 2.43 |
| n_updates | 564 |
| policy_gradient_loss | -0.00342 |
| value_loss | 14.3 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 793 |
| ep_rew_mean | 92.3 |
| time/ | |
| fps | 339 |
| iterations | 143 |
| time_elapsed | 861 |
| total_timesteps | 292864 |
| train/ | |
| approx_kl | 0.011391676 |
| clip_fraction | 0.156 |
| clip_range | 0.2 |
| entropy_loss | -0.969 |
| explained_variance | 0.9806464 |
| learning_rate | 0.0003 |
| loss | 4.61 |
| n_updates | 568 |
| policy_gradient_loss | -0.000658 |
| value_loss | 10.2 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 793 |
| ep_rew_mean | 92.3 |
| time/ | |
| fps | 340 |
| iterations | 144 |
| time_elapsed | 867 |
| total_timesteps | 294912 |
| train/ | |
| approx_kl | 0.006287212 |
| clip_fraction | 0.0271 |
| clip_range | 0.2 |
| entropy_loss | -0.866 |
| explained_variance | 0.96507865 |
| learning_rate | 0.0003 |
| loss | 1.9 |
| n_updates | 572 |
| policy_gradient_loss | -0.00432 |
| value_loss | 20.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 793 |
| ep_rew_mean | 92.3 |
| time/ | |
| fps | 340 |
| iterations | 145 |
| time_elapsed | 872 |
| total_timesteps | 296960 |
| train/ | |
| approx_kl | 0.0054113483 |
| clip_fraction | 0.0482 |
| clip_range | 0.2 |
| entropy_loss | -0.817 |
| explained_variance | 0.9852306 |
| learning_rate | 0.0003 |
| loss | 14.7 |
| n_updates | 576 |
| policy_gradient_loss | 0.00157 |
| value_loss | 15.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 800 |
| ep_rew_mean | 94.3 |
| time/ | |
| fps | 340 |
| iterations | 146 |
| time_elapsed | 878 |
| total_timesteps | 299008 |
| train/ | |
| approx_kl | 0.006469669 |
| clip_fraction | 0.0504 |
| clip_range | 0.2 |
| entropy_loss | -0.941 |
| explained_variance | 0.99262553 |
| learning_rate | 0.0003 |
| loss | 1.42 |
| n_updates | 580 |
| policy_gradient_loss | -0.00184 |
| value_loss | 5.46 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 300000 to videos/step_300000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 784 |
| ep_rew_mean | 92.5 |
| time/ | |
| fps | 337 |
| iterations | 147 |
| time_elapsed | 891 |
| total_timesteps | 301056 |
| train/ | |
| approx_kl | 0.006498353 |
| clip_fraction | 0.0454 |
| clip_range | 0.2 |
| entropy_loss | -0.908 |
| explained_variance | 0.9299376 |
| learning_rate | 0.0003 |
| loss | 39.4 |
| n_updates | 584 |
| policy_gradient_loss | -0.00174 |
| value_loss | 92.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 784 |
| ep_rew_mean | 92.2 |
| time/ | |
| fps | 337 |
| iterations | 148 |
| time_elapsed | 897 |
| total_timesteps | 303104 |
| train/ | |
| approx_kl | 0.0004918156 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.882 |
| explained_variance | 0.7697208 |
| learning_rate | 0.0003 |
| loss | 68 |
| n_updates | 588 |
| policy_gradient_loss | -0.000244 |
| value_loss | 401 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 777 |
| ep_rew_mean | 91.4 |
| time/ | |
| fps | 337 |
| iterations | 149 |
| time_elapsed | 903 |
| total_timesteps | 305152 |
| train/ | |
| approx_kl | 0.003115837 |
| clip_fraction | 0.0172 |
| clip_range | 0.2 |
| entropy_loss | -0.874 |
| explained_variance | 0.9700765 |
| learning_rate | 0.0003 |
| loss | 6.2 |
| n_updates | 592 |
| policy_gradient_loss | -0.000966 |
| value_loss | 14.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 776 |
| ep_rew_mean | 91.9 |
| time/ | |
| fps | 338 |
| iterations | 150 |
| time_elapsed | 908 |
| total_timesteps | 307200 |
| train/ | |
| approx_kl | 0.0034646592 |
| clip_fraction | 0.00623 |
| clip_range | 0.2 |
| entropy_loss | -0.906 |
| explained_variance | 0.8677606 |
| learning_rate | 0.0003 |
| loss | 44.2 |
| n_updates | 596 |
| policy_gradient_loss | -0.00182 |
| value_loss | 149 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 769 |
| ep_rew_mean | 91 |
| time/ | |
| fps | 338 |
| iterations | 151 |
| time_elapsed | 913 |
| total_timesteps | 309248 |
| train/ | |
| approx_kl | 0.0046790084 |
| clip_fraction | 0.00659 |
| clip_range | 0.2 |
| entropy_loss | -0.815 |
| explained_variance | 0.8474222 |
| learning_rate | 0.0003 |
| loss | 105 |
| n_updates | 600 |
| policy_gradient_loss | -0.00108 |
| value_loss | 140 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 755 |
| ep_rew_mean | 87.5 |
| time/ | |
| fps | 338 |
| iterations | 152 |
| time_elapsed | 919 |
| total_timesteps | 311296 |
| train/ | |
| approx_kl | 0.0012743897 |
| clip_fraction | 0.00232 |
| clip_range | 0.2 |
| entropy_loss | -0.937 |
| explained_variance | 0.89595014 |
| learning_rate | 0.0003 |
| loss | 33.8 |
| n_updates | 604 |
| policy_gradient_loss | -0.000858 |
| value_loss | 121 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 755 |
| ep_rew_mean | 87.4 |
| time/ | |
| fps | 338 |
| iterations | 153 |
| time_elapsed | 924 |
| total_timesteps | 313344 |
| train/ | |
| approx_kl | 0.0027567565 |
| clip_fraction | 0.0125 |
| clip_range | 0.2 |
| entropy_loss | -0.88 |
| explained_variance | 0.860183 |
| learning_rate | 0.0003 |
| loss | 44.6 |
| n_updates | 608 |
| policy_gradient_loss | -0.00278 |
| value_loss | 174 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 748 |
| ep_rew_mean | 86.1 |
| time/ | |
| fps | 338 |
| iterations | 154 |
| time_elapsed | 931 |
| total_timesteps | 315392 |
| train/ | |
| approx_kl | 0.007681025 |
| clip_fraction | 0.0376 |
| clip_range | 0.2 |
| entropy_loss | -0.902 |
| explained_variance | 0.9430257 |
| learning_rate | 0.0003 |
| loss | 11.4 |
| n_updates | 612 |
| policy_gradient_loss | -0.00151 |
| value_loss | 28.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 740 |
| ep_rew_mean | 84.8 |
| time/ | |
| fps | 339 |
| iterations | 155 |
| time_elapsed | 936 |
| total_timesteps | 317440 |
| train/ | |
| approx_kl | 0.002498658 |
| clip_fraction | 0.0083 |
| clip_range | 0.2 |
| entropy_loss | -0.889 |
| explained_variance | 0.87737596 |
| learning_rate | 0.0003 |
| loss | 17.4 |
| n_updates | 616 |
| policy_gradient_loss | -0.000365 |
| value_loss | 107 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 739 |
| ep_rew_mean | 84.2 |
| time/ | |
| fps | 339 |
| iterations | 156 |
| time_elapsed | 941 |
| total_timesteps | 319488 |
| train/ | |
| approx_kl | 0.0025510485 |
| clip_fraction | 0.01 |
| clip_range | 0.2 |
| entropy_loss | -0.91 |
| explained_variance | 0.8501159 |
| learning_rate | 0.0003 |
| loss | 107 |
| n_updates | 620 |
| policy_gradient_loss | -0.00135 |
| value_loss | 214 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 320000 to videos/step_320000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 731 |
| ep_rew_mean | 82.1 |
| time/ | |
| fps | 336 |
| iterations | 157 |
| time_elapsed | 954 |
| total_timesteps | 321536 |
| train/ | |
| approx_kl | 0.0031557847 |
| clip_fraction | 0.00708 |
| clip_range | 0.2 |
| entropy_loss | -0.872 |
| explained_variance | 0.92458975 |
| learning_rate | 0.0003 |
| loss | 5.43 |
| n_updates | 624 |
| policy_gradient_loss | -0.00193 |
| value_loss | 70.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 740 |
| ep_rew_mean | 81.7 |
| time/ | |
| fps | 336 |
| iterations | 158 |
| time_elapsed | 960 |
| total_timesteps | 323584 |
| train/ | |
| approx_kl | 0.0054531004 |
| clip_fraction | 0.0327 |
| clip_range | 0.2 |
| entropy_loss | -0.811 |
| explained_variance | 0.63275534 |
| learning_rate | 0.0003 |
| loss | 23.2 |
| n_updates | 628 |
| policy_gradient_loss | -0.000995 |
| value_loss | 362 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 735 |
| ep_rew_mean | 82.4 |
| time/ | |
| fps | 336 |
| iterations | 159 |
| time_elapsed | 966 |
| total_timesteps | 325632 |
| train/ | |
| approx_kl | 0.0030200253 |
| clip_fraction | 0.00366 |
| clip_range | 0.2 |
| entropy_loss | -0.918 |
| explained_variance | 0.7385748 |
| learning_rate | 0.0003 |
| loss | 74.5 |
| n_updates | 632 |
| policy_gradient_loss | -0.000199 |
| value_loss | 275 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 742 |
| ep_rew_mean | 83.8 |
| time/ | |
| fps | 336 |
| iterations | 160 |
| time_elapsed | 972 |
| total_timesteps | 327680 |
| train/ | |
| approx_kl | 0.019539952 |
| clip_fraction | 0.0668 |
| clip_range | 0.2 |
| entropy_loss | -0.871 |
| explained_variance | 0.59947217 |
| learning_rate | 0.0003 |
| loss | 90.2 |
| n_updates | 636 |
| policy_gradient_loss | -0.00443 |
| value_loss | 193 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 740 |
| ep_rew_mean | 83.7 |
| time/ | |
| fps | 337 |
| iterations | 161 |
| time_elapsed | 977 |
| total_timesteps | 329728 |
| train/ | |
| approx_kl | 0.0073569603 |
| clip_fraction | 0.0278 |
| clip_range | 0.2 |
| entropy_loss | -0.916 |
| explained_variance | 0.9623939 |
| learning_rate | 0.0003 |
| loss | 1.81 |
| n_updates | 640 |
| policy_gradient_loss | -0.00206 |
| value_loss | 15.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 741 |
| ep_rew_mean | 83.6 |
| time/ | |
| fps | 337 |
| iterations | 162 |
| time_elapsed | 984 |
| total_timesteps | 331776 |
| train/ | |
| approx_kl | 0.007701143 |
| clip_fraction | 0.035 |
| clip_range | 0.2 |
| entropy_loss | -0.858 |
| explained_variance | 0.7891678 |
| learning_rate | 0.0003 |
| loss | 50.2 |
| n_updates | 644 |
| policy_gradient_loss | -0.000858 |
| value_loss | 149 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 749 |
| ep_rew_mean | 84.9 |
| time/ | |
| fps | 337 |
| iterations | 163 |
| time_elapsed | 989 |
| total_timesteps | 333824 |
| train/ | |
| approx_kl | 0.0030130136 |
| clip_fraction | 0.00427 |
| clip_range | 0.2 |
| entropy_loss | -0.89 |
| explained_variance | 0.73093003 |
| learning_rate | 0.0003 |
| loss | 87.5 |
| n_updates | 648 |
| policy_gradient_loss | -0.00197 |
| value_loss | 195 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 749 |
| ep_rew_mean | 84.5 |
| time/ | |
| fps | 337 |
| iterations | 164 |
| time_elapsed | 995 |
| total_timesteps | 335872 |
| train/ | |
| approx_kl | 0.0028224268 |
| clip_fraction | 0.0277 |
| clip_range | 0.2 |
| entropy_loss | -0.836 |
| explained_variance | 0.9627773 |
| learning_rate | 0.0003 |
| loss | 3.06 |
| n_updates | 652 |
| policy_gradient_loss | -0.0018 |
| value_loss | 15.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 757 |
| ep_rew_mean | 85.4 |
| time/ | |
| fps | 337 |
| iterations | 165 |
| time_elapsed | 1001 |
| total_timesteps | 337920 |
| train/ | |
| approx_kl | 0.012622348 |
| clip_fraction | 0.117 |
| clip_range | 0.2 |
| entropy_loss | -0.863 |
| explained_variance | 0.9653945 |
| learning_rate | 0.0003 |
| loss | 3.75 |
| n_updates | 656 |
| policy_gradient_loss | -0.00343 |
| value_loss | 17.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 757 |
| ep_rew_mean | 86.5 |
| time/ | |
| fps | 337 |
| iterations | 166 |
| time_elapsed | 1007 |
| total_timesteps | 339968 |
| train/ | |
| approx_kl | 0.0044620475 |
| clip_fraction | 0.0248 |
| clip_range | 0.2 |
| entropy_loss | -0.823 |
| explained_variance | 0.98091763 |
| learning_rate | 0.0003 |
| loss | 3.45 |
| n_updates | 660 |
| policy_gradient_loss | -0.0014 |
| value_loss | 10.3 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 340000 to videos/step_340000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 760 |
| ep_rew_mean | 88 |
| time/ | |
| fps | 335 |
| iterations | 167 |
| time_elapsed | 1020 |
| total_timesteps | 342016 |
| train/ | |
| approx_kl | 0.007027425 |
| clip_fraction | 0.0599 |
| clip_range | 0.2 |
| entropy_loss | -0.855 |
| explained_variance | 0.6526216 |
| learning_rate | 0.0003 |
| loss | 181 |
| n_updates | 664 |
| policy_gradient_loss | -0.000545 |
| value_loss | 283 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 754 |
| ep_rew_mean | 88.7 |
| time/ | |
| fps | 335 |
| iterations | 168 |
| time_elapsed | 1026 |
| total_timesteps | 344064 |
| train/ | |
| approx_kl | 0.0087187905 |
| clip_fraction | 0.0659 |
| clip_range | 0.2 |
| entropy_loss | -0.817 |
| explained_variance | 0.7903959 |
| learning_rate | 0.0003 |
| loss | 33.9 |
| n_updates | 668 |
| policy_gradient_loss | -0.005 |
| value_loss | 150 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 769 |
| ep_rew_mean | 91.4 |
| time/ | |
| fps | 335 |
| iterations | 169 |
| time_elapsed | 1032 |
| total_timesteps | 346112 |
| train/ | |
| approx_kl | 0.0065441625 |
| clip_fraction | 0.102 |
| clip_range | 0.2 |
| entropy_loss | -0.815 |
| explained_variance | 0.6434953 |
| learning_rate | 0.0003 |
| loss | 259 |
| n_updates | 672 |
| policy_gradient_loss | -0.00513 |
| value_loss | 254 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 776 |
| ep_rew_mean | 92.3 |
| time/ | |
| fps | 335 |
| iterations | 170 |
| time_elapsed | 1038 |
| total_timesteps | 348160 |
| train/ | |
| approx_kl | 0.0034461343 |
| clip_fraction | 0.0527 |
| clip_range | 0.2 |
| entropy_loss | -0.851 |
| explained_variance | 0.9591208 |
| learning_rate | 0.0003 |
| loss | 2.78 |
| n_updates | 676 |
| policy_gradient_loss | 0.0002 |
| value_loss | 13.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 791 |
| ep_rew_mean | 94.7 |
| time/ | |
| fps | 335 |
| iterations | 171 |
| time_elapsed | 1043 |
| total_timesteps | 350208 |
| train/ | |
| approx_kl | 0.0040493086 |
| clip_fraction | 0.0286 |
| clip_range | 0.2 |
| entropy_loss | -0.811 |
| explained_variance | 0.97407365 |
| learning_rate | 0.0003 |
| loss | 4.6 |
| n_updates | 680 |
| policy_gradient_loss | 0.000435 |
| value_loss | 9.61 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 791 |
| ep_rew_mean | 94.7 |
| time/ | |
| fps | 335 |
| iterations | 172 |
| time_elapsed | 1049 |
| total_timesteps | 352256 |
| train/ | |
| approx_kl | 0.0043637743 |
| clip_fraction | 0.0332 |
| clip_range | 0.2 |
| entropy_loss | -0.781 |
| explained_variance | 0.98058677 |
| learning_rate | 0.0003 |
| loss | 2.01 |
| n_updates | 684 |
| policy_gradient_loss | 0.00169 |
| value_loss | 7.01 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 791 |
| ep_rew_mean | 94.3 |
| time/ | |
| fps | 335 |
| iterations | 173 |
| time_elapsed | 1054 |
| total_timesteps | 354304 |
| train/ | |
| approx_kl | 0.007112156 |
| clip_fraction | 0.0892 |
| clip_range | 0.2 |
| entropy_loss | -0.816 |
| explained_variance | 0.9909324 |
| learning_rate | 0.0003 |
| loss | 2.39 |
| n_updates | 688 |
| policy_gradient_loss | -0.00388 |
| value_loss | 5.91 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 793 |
| ep_rew_mean | 94.8 |
| time/ | |
| fps | 336 |
| iterations | 174 |
| time_elapsed | 1060 |
| total_timesteps | 356352 |
| train/ | |
| approx_kl | 0.0087589575 |
| clip_fraction | 0.0548 |
| clip_range | 0.2 |
| entropy_loss | -0.775 |
| explained_variance | 0.98384327 |
| learning_rate | 0.0003 |
| loss | 1.85 |
| n_updates | 692 |
| policy_gradient_loss | -0.00361 |
| value_loss | 9.02 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 797 |
| ep_rew_mean | 93.8 |
| time/ | |
| fps | 336 |
| iterations | 175 |
| time_elapsed | 1065 |
| total_timesteps | 358400 |
| train/ | |
| approx_kl | 0.001820323 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -0.795 |
| explained_variance | 0.77407205 |
| learning_rate | 0.0003 |
| loss | 63.6 |
| n_updates | 696 |
| policy_gradient_loss | -0.0007 |
| value_loss | 339 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 360000 to videos/step_360000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 797 |
| ep_rew_mean | 93.7 |
| time/ | |
| fps | 335 |
| iterations | 176 |
| time_elapsed | 1075 |
| total_timesteps | 360448 |
| train/ | |
| approx_kl | 0.007839827 |
| clip_fraction | 0.0498 |
| clip_range | 0.2 |
| entropy_loss | -0.83 |
| explained_variance | 0.99201995 |
| learning_rate | 0.0003 |
| loss | 1.54 |
| n_updates | 700 |
| policy_gradient_loss | -0.00241 |
| value_loss | 6.51 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 804 |
| ep_rew_mean | 94.6 |
| time/ | |
| fps | 335 |
| iterations | 177 |
| time_elapsed | 1081 |
| total_timesteps | 362496 |
| train/ | |
| approx_kl | 0.0055515654 |
| clip_fraction | 0.0792 |
| clip_range | 0.2 |
| entropy_loss | -0.794 |
| explained_variance | 0.98680186 |
| learning_rate | 0.0003 |
| loss | 7.26 |
| n_updates | 704 |
| policy_gradient_loss | -0.00211 |
| value_loss | 9.98 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 812 |
| ep_rew_mean | 96.2 |
| time/ | |
| fps | 335 |
| iterations | 178 |
| time_elapsed | 1087 |
| total_timesteps | 364544 |
| train/ | |
| approx_kl | 0.0039599864 |
| clip_fraction | 0.0585 |
| clip_range | 0.2 |
| entropy_loss | -0.868 |
| explained_variance | 0.987198 |
| learning_rate | 0.0003 |
| loss | 3.91 |
| n_updates | 708 |
| policy_gradient_loss | -0.000928 |
| value_loss | 13.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 98.6 |
| time/ | |
| fps | 335 |
| iterations | 179 |
| time_elapsed | 1092 |
| total_timesteps | 366592 |
| train/ | |
| approx_kl | 0.0051508695 |
| clip_fraction | 0.0514 |
| clip_range | 0.2 |
| entropy_loss | -0.88 |
| explained_variance | 0.98562646 |
| learning_rate | 0.0003 |
| loss | 1.34 |
| n_updates | 712 |
| policy_gradient_loss | 0.00016 |
| value_loss | 8.77 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 98.7 |
| time/ | |
| fps | 335 |
| iterations | 180 |
| time_elapsed | 1098 |
| total_timesteps | 368640 |
| train/ | |
| approx_kl | 0.0017419497 |
| clip_fraction | 0.0214 |
| clip_range | 0.2 |
| entropy_loss | -0.849 |
| explained_variance | 0.8676783 |
| learning_rate | 0.0003 |
| loss | 35.2 |
| n_updates | 716 |
| policy_gradient_loss | -5.78e-05 |
| value_loss | 135 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 99.5 |
| time/ | |
| fps | 335 |
| iterations | 181 |
| time_elapsed | 1104 |
| total_timesteps | 370688 |
| train/ | |
| approx_kl | 0.0065774955 |
| clip_fraction | 0.0312 |
| clip_range | 0.2 |
| entropy_loss | -0.842 |
| explained_variance | 0.98242 |
| learning_rate | 0.0003 |
| loss | 3.83 |
| n_updates | 720 |
| policy_gradient_loss | -0.0016 |
| value_loss | 10.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 99.4 |
| time/ | |
| fps | 335 |
| iterations | 182 |
| time_elapsed | 1109 |
| total_timesteps | 372736 |
| train/ | |
| approx_kl | 0.0040558213 |
| clip_fraction | 0.0402 |
| clip_range | 0.2 |
| entropy_loss | -0.892 |
| explained_variance | 0.9931548 |
| learning_rate | 0.0003 |
| loss | 4.56 |
| n_updates | 724 |
| policy_gradient_loss | -0.00259 |
| value_loss | 5.43 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 98.9 |
| time/ | |
| fps | 336 |
| iterations | 183 |
| time_elapsed | 1115 |
| total_timesteps | 374784 |
| train/ | |
| approx_kl | 0.004418642 |
| clip_fraction | 0.0685 |
| clip_range | 0.2 |
| entropy_loss | -0.943 |
| explained_variance | 0.99119246 |
| learning_rate | 0.0003 |
| loss | 3.84 |
| n_updates | 728 |
| policy_gradient_loss | -0.00213 |
| value_loss | 7.34 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 99.3 |
| time/ | |
| fps | 336 |
| iterations | 184 |
| time_elapsed | 1121 |
| total_timesteps | 376832 |
| train/ | |
| approx_kl | 0.009165289 |
| clip_fraction | 0.0658 |
| clip_range | 0.2 |
| entropy_loss | -0.882 |
| explained_variance | 0.9873686 |
| learning_rate | 0.0003 |
| loss | 4.15 |
| n_updates | 732 |
| policy_gradient_loss | -0.00658 |
| value_loss | 12.8 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 99.4 |
| time/ | |
| fps | 336 |
| iterations | 185 |
| time_elapsed | 1127 |
| total_timesteps | 378880 |
| train/ | |
| approx_kl | 0.004384457 |
| clip_fraction | 0.0376 |
| clip_range | 0.2 |
| entropy_loss | -0.854 |
| explained_variance | 0.98955053 |
| learning_rate | 0.0003 |
| loss | 4.79 |
| n_updates | 736 |
| policy_gradient_loss | 0.000269 |
| value_loss | 9.94 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 380000 to videos/step_380000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 827 |
| ep_rew_mean | 99.2 |
| time/ | |
| fps | 333 |
| iterations | 186 |
| time_elapsed | 1142 |
| total_timesteps | 380928 |
| train/ | |
| approx_kl | 0.0042624297 |
| clip_fraction | 0.0514 |
| clip_range | 0.2 |
| entropy_loss | -0.867 |
| explained_variance | 0.9866625 |
| learning_rate | 0.0003 |
| loss | 4.47 |
| n_updates | 740 |
| policy_gradient_loss | -0.000316 |
| value_loss | 11.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 833 |
| ep_rew_mean | 101 |
| time/ | |
| fps | 333 |
| iterations | 187 |
| time_elapsed | 1147 |
| total_timesteps | 382976 |
| train/ | |
| approx_kl | 0.0031679207 |
| clip_fraction | 0.0258 |
| clip_range | 0.2 |
| entropy_loss | -0.807 |
| explained_variance | 0.9855108 |
| learning_rate | 0.0003 |
| loss | 7.45 |
| n_updates | 744 |
| policy_gradient_loss | -0.00266 |
| value_loss | 12.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 847 |
| ep_rew_mean | 106 |
| time/ | |
| fps | 333 |
| iterations | 188 |
| time_elapsed | 1153 |
| total_timesteps | 385024 |
| train/ | |
| approx_kl | 0.0036491863 |
| clip_fraction | 0.0476 |
| clip_range | 0.2 |
| entropy_loss | -0.741 |
| explained_variance | 0.9722174 |
| learning_rate | 0.0003 |
| loss | 4.28 |
| n_updates | 748 |
| policy_gradient_loss | -0.00355 |
| value_loss | 18.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 847 |
| ep_rew_mean | 107 |
| time/ | |
| fps | 333 |
| iterations | 189 |
| time_elapsed | 1159 |
| total_timesteps | 387072 |
| train/ | |
| approx_kl | 0.0016885031 |
| clip_fraction | 0.00378 |
| clip_range | 0.2 |
| entropy_loss | -0.764 |
| explained_variance | 0.74798495 |
| learning_rate | 0.0003 |
| loss | 77.7 |
| n_updates | 752 |
| policy_gradient_loss | -0.000332 |
| value_loss | 266 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 847 |
| ep_rew_mean | 106 |
| time/ | |
| fps | 333 |
| iterations | 190 |
| time_elapsed | 1165 |
| total_timesteps | 389120 |
| train/ | |
| approx_kl | 0.003259561 |
| clip_fraction | 0.0363 |
| clip_range | 0.2 |
| entropy_loss | -0.777 |
| explained_variance | 0.97681105 |
| learning_rate | 0.0003 |
| loss | 1.97 |
| n_updates | 756 |
| policy_gradient_loss | -0.000576 |
| value_loss | 12.2 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 854 |
| ep_rew_mean | 108 |
| time/ | |
| fps | 333 |
| iterations | 191 |
| time_elapsed | 1171 |
| total_timesteps | 391168 |
| train/ | |
| approx_kl | 0.0057980507 |
| clip_fraction | 0.0645 |
| clip_range | 0.2 |
| entropy_loss | -0.787 |
| explained_variance | 0.98778343 |
| learning_rate | 0.0003 |
| loss | 3.15 |
| n_updates | 760 |
| policy_gradient_loss | -0.00491 |
| value_loss | 7.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 859 |
| ep_rew_mean | 111 |
| time/ | |
| fps | 334 |
| iterations | 192 |
| time_elapsed | 1176 |
| total_timesteps | 393216 |
| train/ | |
| approx_kl | 0.0042476226 |
| clip_fraction | 0.0442 |
| clip_range | 0.2 |
| entropy_loss | -0.811 |
| explained_variance | 0.98000324 |
| learning_rate | 0.0003 |
| loss | 6.06 |
| n_updates | 764 |
| policy_gradient_loss | -0.00229 |
| value_loss | 11.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 863 |
| ep_rew_mean | 113 |
| time/ | |
| fps | 334 |
| iterations | 193 |
| time_elapsed | 1182 |
| total_timesteps | 395264 |
| train/ | |
| approx_kl | 0.005276507 |
| clip_fraction | 0.0498 |
| clip_range | 0.2 |
| entropy_loss | -0.747 |
| explained_variance | 0.89073044 |
| learning_rate | 0.0003 |
| loss | 127 |
| n_updates | 768 |
| policy_gradient_loss | -0.00359 |
| value_loss | 116 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 860 |
| ep_rew_mean | 114 |
| time/ | |
| fps | 334 |
| iterations | 194 |
| time_elapsed | 1187 |
| total_timesteps | 397312 |
| train/ | |
| approx_kl | 0.0025811752 |
| clip_fraction | 0.0117 |
| clip_range | 0.2 |
| entropy_loss | -0.823 |
| explained_variance | 0.85430175 |
| learning_rate | 0.0003 |
| loss | 55.1 |
| n_updates | 772 |
| policy_gradient_loss | -0.00133 |
| value_loss | 93 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 868 |
| ep_rew_mean | 117 |
| time/ | |
| fps | 334 |
| iterations | 195 |
| time_elapsed | 1193 |
| total_timesteps | 399360 |
| train/ | |
| approx_kl | 0.0018339432 |
| clip_fraction | 0.011 |
| clip_range | 0.2 |
| entropy_loss | -0.67 |
| explained_variance | 0.9027069 |
| learning_rate | 0.0003 |
| loss | 71.3 |
| n_updates | 776 |
| policy_gradient_loss | -0.00154 |
| value_loss | 82.3 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 400000 to videos/step_400000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 859 |
| ep_rew_mean | 120 |
| time/ | |
| fps | 334 |
| iterations | 196 |
| time_elapsed | 1201 |
| total_timesteps | 401408 |
| train/ | |
| approx_kl | 0.00089457567 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.755 |
| explained_variance | 0.7617139 |
| learning_rate | 0.0003 |
| loss | 181 |
| n_updates | 780 |
| policy_gradient_loss | -0.000247 |
| value_loss | 229 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 855 |
| ep_rew_mean | 125 |
| time/ | |
| fps | 334 |
| iterations | 197 |
| time_elapsed | 1207 |
| total_timesteps | 403456 |
| train/ | |
| approx_kl | 0.0024267314 |
| clip_fraction | 0.005 |
| clip_range | 0.2 |
| entropy_loss | -0.769 |
| explained_variance | 0.65657926 |
| learning_rate | 0.0003 |
| loss | 209 |
| n_updates | 784 |
| policy_gradient_loss | -0.00118 |
| value_loss | 302 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 850 |
| ep_rew_mean | 131 |
| time/ | |
| fps | 334 |
| iterations | 198 |
| time_elapsed | 1211 |
| total_timesteps | 405504 |
| train/ | |
| approx_kl | 0.000339626 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.753 |
| explained_variance | 0.6088275 |
| learning_rate | 0.0003 |
| loss | 220 |
| n_updates | 788 |
| policy_gradient_loss | -0.000404 |
| value_loss | 447 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 844 |
| ep_rew_mean | 141 |
| time/ | |
| fps | 334 |
| iterations | 199 |
| time_elapsed | 1217 |
| total_timesteps | 407552 |
| train/ | |
| approx_kl | 0.0013061033 |
| clip_fraction | 0.00061 |
| clip_range | 0.2 |
| entropy_loss | -0.821 |
| explained_variance | 0.6433532 |
| learning_rate | 0.0003 |
| loss | 197 |
| n_updates | 792 |
| policy_gradient_loss | -0.000646 |
| value_loss | 343 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 839 |
| ep_rew_mean | 143 |
| time/ | |
| fps | 335 |
| iterations | 200 |
| time_elapsed | 1222 |
| total_timesteps | 409600 |
| train/ | |
| approx_kl | 0.0006927672 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.71 |
| explained_variance | 0.66363513 |
| learning_rate | 0.0003 |
| loss | 34.4 |
| n_updates | 796 |
| policy_gradient_loss | -0.000337 |
| value_loss | 223 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 817 |
| ep_rew_mean | 143 |
| time/ | |
| fps | 335 |
| iterations | 201 |
| time_elapsed | 1227 |
| total_timesteps | 411648 |
| train/ | |
| approx_kl | 0.0023855523 |
| clip_fraction | 0.00647 |
| clip_range | 0.2 |
| entropy_loss | -0.666 |
| explained_variance | 0.8159368 |
| learning_rate | 0.0003 |
| loss | 31.2 |
| n_updates | 800 |
| policy_gradient_loss | -0.000581 |
| value_loss | 114 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 795 |
| ep_rew_mean | 149 |
| time/ | |
| fps | 335 |
| iterations | 202 |
| time_elapsed | 1232 |
| total_timesteps | 413696 |
| train/ | |
| approx_kl | 0.0011690003 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.877 |
| explained_variance | 0.40535784 |
| learning_rate | 0.0003 |
| loss | 156 |
| n_updates | 804 |
| policy_gradient_loss | -0.000365 |
| value_loss | 712 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 775 |
| ep_rew_mean | 148 |
| time/ | |
| fps | 336 |
| iterations | 203 |
| time_elapsed | 1237 |
| total_timesteps | 415744 |
| train/ | |
| approx_kl | 0.0030381014 |
| clip_fraction | 0.0101 |
| clip_range | 0.2 |
| entropy_loss | -0.797 |
| explained_variance | 0.40151554 |
| learning_rate | 0.0003 |
| loss | 163 |
| n_updates | 808 |
| policy_gradient_loss | -0.00205 |
| value_loss | 461 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 767 |
| ep_rew_mean | 154 |
| time/ | |
| fps | 336 |
| iterations | 204 |
| time_elapsed | 1242 |
| total_timesteps | 417792 |
| train/ | |
| approx_kl | 0.0021285492 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.826 |
| explained_variance | 0.59571785 |
| learning_rate | 0.0003 |
| loss | 365 |
| n_updates | 812 |
| policy_gradient_loss | -0.000797 |
| value_loss | 519 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 742 |
| ep_rew_mean | 155 |
| time/ | |
| fps | 336 |
| iterations | 205 |
| time_elapsed | 1247 |
| total_timesteps | 419840 |
| train/ | |
| approx_kl | 0.00623208 |
| clip_fraction | 0.0255 |
| clip_range | 0.2 |
| entropy_loss | -0.779 |
| explained_variance | 0.7363901 |
| learning_rate | 0.0003 |
| loss | 60.9 |
| n_updates | 816 |
| policy_gradient_loss | -0.00205 |
| value_loss | 144 |
----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 420000 to videos/step_420000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 705 |
| ep_rew_mean | 159 |
| time/ | |
| fps | 335 |
| iterations | 206 |
| time_elapsed | 1255 |
| total_timesteps | 421888 |
| train/ | |
| approx_kl | 0.0020064868 |
| clip_fraction | 0.00244 |
| clip_range | 0.2 |
| entropy_loss | -0.859 |
| explained_variance | 0.5579882 |
| learning_rate | 0.0003 |
| loss | 158 |
| n_updates | 820 |
| policy_gradient_loss | -0.000194 |
| value_loss | 477 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 695 |
| ep_rew_mean | 161 |
| time/ | |
| fps | 336 |
| iterations | 207 |
| time_elapsed | 1261 |
| total_timesteps | 423936 |
| train/ | |
| approx_kl | 0.00069486303 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.793 |
| explained_variance | 0.4900617 |
| learning_rate | 0.0003 |
| loss | 112 |
| n_updates | 824 |
| policy_gradient_loss | -0.000403 |
| value_loss | 353 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 686 |
| ep_rew_mean | 165 |
| time/ | |
| fps | 336 |
| iterations | 208 |
| time_elapsed | 1266 |
| total_timesteps | 425984 |
| train/ | |
| approx_kl | 0.0030584042 |
| clip_fraction | 0.0111 |
| clip_range | 0.2 |
| entropy_loss | -0.741 |
| explained_variance | 0.7521478 |
| learning_rate | 0.0003 |
| loss | 30.6 |
| n_updates | 828 |
| policy_gradient_loss | -0.000674 |
| value_loss | 122 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 657 |
| ep_rew_mean | 169 |
| time/ | |
| fps | 336 |
| iterations | 209 |
| time_elapsed | 1271 |
| total_timesteps | 428032 |
| train/ | |
| approx_kl | 0.0033012186 |
| clip_fraction | 0.00757 |
| clip_range | 0.2 |
| entropy_loss | -0.796 |
| explained_variance | 0.49494988 |
| learning_rate | 0.0003 |
| loss | 125 |
| n_updates | 832 |
| policy_gradient_loss | -0.00174 |
| value_loss | 382 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 640 |
| ep_rew_mean | 173 |
| time/ | |
| fps | 336 |
| iterations | 210 |
| time_elapsed | 1276 |
| total_timesteps | 430080 |
| train/ | |
| approx_kl | 0.0024316013 |
| clip_fraction | 0.00415 |
| clip_range | 0.2 |
| entropy_loss | -0.865 |
| explained_variance | 0.47551578 |
| learning_rate | 0.0003 |
| loss | 154 |
| n_updates | 836 |
| policy_gradient_loss | -0.00153 |
| value_loss | 402 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 622 |
| ep_rew_mean | 170 |
| time/ | |
| fps | 337 |
| iterations | 211 |
| time_elapsed | 1281 |
| total_timesteps | 432128 |
| train/ | |
| approx_kl | 0.004289726 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -0.735 |
| explained_variance | 0.81798875 |
| learning_rate | 0.0003 |
| loss | 41.8 |
| n_updates | 840 |
| policy_gradient_loss | -0.00146 |
| value_loss | 84.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 608 |
| ep_rew_mean | 172 |
| time/ | |
| fps | 337 |
| iterations | 212 |
| time_elapsed | 1286 |
| total_timesteps | 434176 |
| train/ | |
| approx_kl | 0.0025698352 |
| clip_fraction | 0.00146 |
| clip_range | 0.2 |
| entropy_loss | -0.79 |
| explained_variance | 0.50181663 |
| learning_rate | 0.0003 |
| loss | 376 |
| n_updates | 844 |
| policy_gradient_loss | -0.00226 |
| value_loss | 921 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 596 |
| ep_rew_mean | 173 |
| time/ | |
| fps | 337 |
| iterations | 213 |
| time_elapsed | 1291 |
| total_timesteps | 436224 |
| train/ | |
| approx_kl | 0.0025924344 |
| clip_fraction | 0.0184 |
| clip_range | 0.2 |
| entropy_loss | -0.838 |
| explained_variance | 0.806567 |
| learning_rate | 0.0003 |
| loss | 272 |
| n_updates | 848 |
| policy_gradient_loss | -0.000968 |
| value_loss | 310 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 559 |
| ep_rew_mean | 174 |
| time/ | |
| fps | 337 |
| iterations | 214 |
| time_elapsed | 1296 |
| total_timesteps | 438272 |
| train/ | |
| approx_kl | 0.005685072 |
| clip_fraction | 0.0177 |
| clip_range | 0.2 |
| entropy_loss | -0.774 |
| explained_variance | 0.9289124 |
| learning_rate | 0.0003 |
| loss | 20.9 |
| n_updates | 852 |
| policy_gradient_loss | -0.00147 |
| value_loss | 66.9 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 440000 to videos/step_440000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 546 |
| ep_rew_mean | 177 |
| time/ | |
| fps | 337 |
| iterations | 215 |
| time_elapsed | 1304 |
| total_timesteps | 440320 |
| train/ | |
| approx_kl | 0.00061682635 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.871 |
| explained_variance | 0.5882236 |
| learning_rate | 0.0003 |
| loss | 186 |
| n_updates | 856 |
| policy_gradient_loss | -8.9e-05 |
| value_loss | 672 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 527 |
| ep_rew_mean | 181 |
| time/ | |
| fps | 337 |
| iterations | 216 |
| time_elapsed | 1309 |
| total_timesteps | 442368 |
| train/ | |
| approx_kl | 0.0020923952 |
| clip_fraction | 0.00671 |
| clip_range | 0.2 |
| entropy_loss | -0.776 |
| explained_variance | 0.7908546 |
| learning_rate | 0.0003 |
| loss | 41.4 |
| n_updates | 860 |
| policy_gradient_loss | -0.000323 |
| value_loss | 97 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 510 |
| ep_rew_mean | 183 |
| time/ | |
| fps | 338 |
| iterations | 217 |
| time_elapsed | 1314 |
| total_timesteps | 444416 |
| train/ | |
| approx_kl | 0.0024116333 |
| clip_fraction | 0.0167 |
| clip_range | 0.2 |
| entropy_loss | -0.675 |
| explained_variance | 0.86128956 |
| learning_rate | 0.0003 |
| loss | 43 |
| n_updates | 864 |
| policy_gradient_loss | -0.00172 |
| value_loss | 92.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 495 |
| ep_rew_mean | 185 |
| time/ | |
| fps | 338 |
| iterations | 218 |
| time_elapsed | 1319 |
| total_timesteps | 446464 |
| train/ | |
| approx_kl | 0.0033270542 |
| clip_fraction | 0.0101 |
| clip_range | 0.2 |
| entropy_loss | -0.807 |
| explained_variance | 0.8944935 |
| learning_rate | 0.0003 |
| loss | 19.1 |
| n_updates | 868 |
| policy_gradient_loss | 6.7e-05 |
| value_loss | 48.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 481 |
| ep_rew_mean | 189 |
| time/ | |
| fps | 338 |
| iterations | 219 |
| time_elapsed | 1325 |
| total_timesteps | 448512 |
| train/ | |
| approx_kl | 0.0032078533 |
| clip_fraction | 0.0172 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.8763143 |
| learning_rate | 0.0003 |
| loss | 16 |
| n_updates | 872 |
| policy_gradient_loss | -0.00167 |
| value_loss | 51.2 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 487 |
| ep_rew_mean | 187 |
| time/ | |
| fps | 338 |
| iterations | 220 |
| time_elapsed | 1329 |
| total_timesteps | 450560 |
| train/ | |
| approx_kl | 0.005268974 |
| clip_fraction | 0.0273 |
| clip_range | 0.2 |
| entropy_loss | -0.832 |
| explained_variance | 0.87973744 |
| learning_rate | 0.0003 |
| loss | 18.4 |
| n_updates | 876 |
| policy_gradient_loss | -0.00148 |
| value_loss | 54 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 476 |
| ep_rew_mean | 187 |
| time/ | |
| fps | 339 |
| iterations | 221 |
| time_elapsed | 1334 |
| total_timesteps | 452608 |
| train/ | |
| approx_kl | 0.002290433 |
| clip_fraction | 0.00488 |
| clip_range | 0.2 |
| entropy_loss | -0.77 |
| explained_variance | 0.40748376 |
| learning_rate | 0.0003 |
| loss | 492 |
| n_updates | 880 |
| policy_gradient_loss | -1.27e-06 |
| value_loss | 807 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 472 |
| ep_rew_mean | 185 |
| time/ | |
| fps | 339 |
| iterations | 222 |
| time_elapsed | 1339 |
| total_timesteps | 454656 |
| train/ | |
| approx_kl | 0.003653217 |
| clip_fraction | 0.024 |
| clip_range | 0.2 |
| entropy_loss | -0.787 |
| explained_variance | 0.72627556 |
| learning_rate | 0.0003 |
| loss | 47.3 |
| n_updates | 884 |
| policy_gradient_loss | -0.001 |
| value_loss | 302 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 460 |
| ep_rew_mean | 186 |
| time/ | |
| fps | 339 |
| iterations | 223 |
| time_elapsed | 1344 |
| total_timesteps | 456704 |
| train/ | |
| approx_kl | 0.002886461 |
| clip_fraction | 0.0166 |
| clip_range | 0.2 |
| entropy_loss | -0.743 |
| explained_variance | 0.76730394 |
| learning_rate | 0.0003 |
| loss | 90.1 |
| n_updates | 888 |
| policy_gradient_loss | -0.00142 |
| value_loss | 390 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 456 |
| ep_rew_mean | 186 |
| time/ | |
| fps | 340 |
| iterations | 224 |
| time_elapsed | 1348 |
| total_timesteps | 458752 |
| train/ | |
| approx_kl | 0.004828863 |
| clip_fraction | 0.028 |
| clip_range | 0.2 |
| entropy_loss | -0.835 |
| explained_variance | 0.82201505 |
| learning_rate | 0.0003 |
| loss | 156 |
| n_updates | 892 |
| policy_gradient_loss | -0.00153 |
| value_loss | 392 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 460000 to videos/step_460000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 448 |
| ep_rew_mean | 186 |
| time/ | |
| fps | 339 |
| iterations | 225 |
| time_elapsed | 1355 |
| total_timesteps | 460800 |
| train/ | |
| approx_kl | 0.0024109546 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -0.796 |
| explained_variance | 0.5428514 |
| learning_rate | 0.0003 |
| loss | 361 |
| n_updates | 896 |
| policy_gradient_loss | -0.00055 |
| value_loss | 1.23e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 443 |
| ep_rew_mean | 188 |
| time/ | |
| fps | 340 |
| iterations | 226 |
| time_elapsed | 1360 |
| total_timesteps | 462848 |
| train/ | |
| approx_kl | 0.0021074342 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -0.781 |
| explained_variance | 0.71103156 |
| learning_rate | 0.0003 |
| loss | 153 |
| n_updates | 900 |
| policy_gradient_loss | -0.000935 |
| value_loss | 660 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 440 |
| ep_rew_mean | 188 |
| time/ | |
| fps | 340 |
| iterations | 227 |
| time_elapsed | 1365 |
| total_timesteps | 464896 |
| train/ | |
| approx_kl | 0.0017199174 |
| clip_fraction | 0.0133 |
| clip_range | 0.2 |
| entropy_loss | -0.826 |
| explained_variance | 0.7324963 |
| learning_rate | 0.0003 |
| loss | 37.6 |
| n_updates | 904 |
| policy_gradient_loss | -0.000838 |
| value_loss | 100 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 441 |
| ep_rew_mean | 189 |
| time/ | |
| fps | 340 |
| iterations | 228 |
| time_elapsed | 1370 |
| total_timesteps | 466944 |
| train/ | |
| approx_kl | 0.0025557657 |
| clip_fraction | 0.00964 |
| clip_range | 0.2 |
| entropy_loss | -0.798 |
| explained_variance | 0.8268275 |
| learning_rate | 0.0003 |
| loss | 60.3 |
| n_updates | 908 |
| policy_gradient_loss | -0.000463 |
| value_loss | 314 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 435 |
| ep_rew_mean | 193 |
| time/ | |
| fps | 340 |
| iterations | 229 |
| time_elapsed | 1375 |
| total_timesteps | 468992 |
| train/ | |
| approx_kl | 0.0012218828 |
| clip_fraction | 0.00684 |
| clip_range | 0.2 |
| entropy_loss | -0.812 |
| explained_variance | 0.71603656 |
| learning_rate | 0.0003 |
| loss | 107 |
| n_updates | 912 |
| policy_gradient_loss | -0.000219 |
| value_loss | 369 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 426 |
| ep_rew_mean | 188 |
| time/ | |
| fps | 341 |
| iterations | 230 |
| time_elapsed | 1380 |
| total_timesteps | 471040 |
| train/ | |
| approx_kl | 0.004733691 |
| clip_fraction | 0.0681 |
| clip_range | 0.2 |
| entropy_loss | -0.785 |
| explained_variance | 0.85527104 |
| learning_rate | 0.0003 |
| loss | 16 |
| n_updates | 916 |
| policy_gradient_loss | -0.0016 |
| value_loss | 39.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 423 |
| ep_rew_mean | 184 |
| time/ | |
| fps | 341 |
| iterations | 231 |
| time_elapsed | 1385 |
| total_timesteps | 473088 |
| train/ | |
| approx_kl | 0.0010900581 |
| clip_fraction | 0.000977 |
| clip_range | 0.2 |
| entropy_loss | -0.81 |
| explained_variance | 0.65007967 |
| learning_rate | 0.0003 |
| loss | 505 |
| n_updates | 920 |
| policy_gradient_loss | -0.000819 |
| value_loss | 1.01e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 411 |
| ep_rew_mean | 184 |
| time/ | |
| fps | 341 |
| iterations | 232 |
| time_elapsed | 1390 |
| total_timesteps | 475136 |
| train/ | |
| approx_kl | 0.0023586599 |
| clip_fraction | 0.00415 |
| clip_range | 0.2 |
| entropy_loss | -0.781 |
| explained_variance | 0.8315761 |
| learning_rate | 0.0003 |
| loss | 160 |
| n_updates | 924 |
| policy_gradient_loss | -0.000924 |
| value_loss | 638 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 414 |
| ep_rew_mean | 181 |
| time/ | |
| fps | 341 |
| iterations | 233 |
| time_elapsed | 1395 |
| total_timesteps | 477184 |
| train/ | |
| approx_kl | 0.00064685976 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.845 |
| explained_variance | 0.79295504 |
| learning_rate | 0.0003 |
| loss | 275 |
| n_updates | 928 |
| policy_gradient_loss | -0.00058 |
| value_loss | 609 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 410 |
| ep_rew_mean | 178 |
| time/ | |
| fps | 342 |
| iterations | 234 |
| time_elapsed | 1400 |
| total_timesteps | 479232 |
| train/ | |
| approx_kl | 0.006268014 |
| clip_fraction | 0.0314 |
| clip_range | 0.2 |
| entropy_loss | -0.732 |
| explained_variance | 0.6891569 |
| learning_rate | 0.0003 |
| loss | 318 |
| n_updates | 932 |
| policy_gradient_loss | -0.000782 |
| value_loss | 355 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 480000 to videos/step_480000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 403 |
| ep_rew_mean | 171 |
| time/ | |
| fps | 341 |
| iterations | 235 |
| time_elapsed | 1407 |
| total_timesteps | 481280 |
| train/ | |
| approx_kl | 0.0018855615 |
| clip_fraction | 0.00354 |
| clip_range | 0.2 |
| entropy_loss | -0.86 |
| explained_variance | 0.8105307 |
| learning_rate | 0.0003 |
| loss | 457 |
| n_updates | 936 |
| policy_gradient_loss | -0.0019 |
| value_loss | 720 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 392 |
| ep_rew_mean | 172 |
| time/ | |
| fps | 342 |
| iterations | 236 |
| time_elapsed | 1412 |
| total_timesteps | 483328 |
| train/ | |
| approx_kl | 0.005119904 |
| clip_fraction | 0.0177 |
| clip_range | 0.2 |
| entropy_loss | -0.854 |
| explained_variance | 0.9036793 |
| learning_rate | 0.0003 |
| loss | 146 |
| n_updates | 940 |
| policy_gradient_loss | -0.0012 |
| value_loss | 379 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 398 |
| ep_rew_mean | 171 |
| time/ | |
| fps | 342 |
| iterations | 237 |
| time_elapsed | 1417 |
| total_timesteps | 485376 |
| train/ | |
| approx_kl | 0.0033285003 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -0.778 |
| explained_variance | 0.8631626 |
| learning_rate | 0.0003 |
| loss | 54.3 |
| n_updates | 944 |
| policy_gradient_loss | -0.00152 |
| value_loss | 182 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 397 |
| ep_rew_mean | 166 |
| time/ | |
| fps | 342 |
| iterations | 238 |
| time_elapsed | 1422 |
| total_timesteps | 487424 |
| train/ | |
| approx_kl | 0.015425668 |
| clip_fraction | 0.0676 |
| clip_range | 0.2 |
| entropy_loss | -0.776 |
| explained_variance | 0.96773916 |
| learning_rate | 0.0003 |
| loss | 9.51 |
| n_updates | 948 |
| policy_gradient_loss | -0.00192 |
| value_loss | 73.7 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 388 |
| ep_rew_mean | 169 |
| time/ | |
| fps | 342 |
| iterations | 239 |
| time_elapsed | 1427 |
| total_timesteps | 489472 |
| train/ | |
| approx_kl | 0.004257844 |
| clip_fraction | 0.0298 |
| clip_range | 0.2 |
| entropy_loss | -0.783 |
| explained_variance | 0.84093016 |
| learning_rate | 0.0003 |
| loss | 86.9 |
| n_updates | 952 |
| policy_gradient_loss | -0.00172 |
| value_loss | 256 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 395 |
| ep_rew_mean | 165 |
| time/ | |
| fps | 343 |
| iterations | 240 |
| time_elapsed | 1432 |
| total_timesteps | 491520 |
| train/ | |
| approx_kl | 0.0009157347 |
| clip_fraction | 0.00256 |
| clip_range | 0.2 |
| entropy_loss | -0.75 |
| explained_variance | 0.7250916 |
| learning_rate | 0.0003 |
| loss | 395 |
| n_updates | 956 |
| policy_gradient_loss | -0.000277 |
| value_loss | 460 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 404 |
| ep_rew_mean | 165 |
| time/ | |
| fps | 343 |
| iterations | 241 |
| time_elapsed | 1437 |
| total_timesteps | 493568 |
| train/ | |
| approx_kl | 0.0035095373 |
| clip_fraction | 0.0621 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.9006959 |
| learning_rate | 0.0003 |
| loss | 72.5 |
| n_updates | 960 |
| policy_gradient_loss | -0.00259 |
| value_loss | 164 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 397 |
| ep_rew_mean | 159 |
| time/ | |
| fps | 343 |
| iterations | 242 |
| time_elapsed | 1442 |
| total_timesteps | 495616 |
| train/ | |
| approx_kl | 0.00049223774 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.764 |
| explained_variance | 0.6621828 |
| learning_rate | 0.0003 |
| loss | 170 |
| n_updates | 964 |
| policy_gradient_loss | -0.000265 |
| value_loss | 442 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 403 |
| ep_rew_mean | 158 |
| time/ | |
| fps | 343 |
| iterations | 243 |
| time_elapsed | 1447 |
| total_timesteps | 497664 |
| train/ | |
| approx_kl | 0.0011338956 |
| clip_fraction | 0.00244 |
| clip_range | 0.2 |
| entropy_loss | -0.824 |
| explained_variance | 0.79397386 |
| learning_rate | 0.0003 |
| loss | 255 |
| n_updates | 968 |
| policy_gradient_loss | -0.000316 |
| value_loss | 482 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 408 |
| ep_rew_mean | 163 |
| time/ | |
| fps | 344 |
| iterations | 244 |
| time_elapsed | 1452 |
| total_timesteps | 499712 |
| train/ | |
| approx_kl | 0.006566366 |
| clip_fraction | 0.0562 |
| clip_range | 0.2 |
| entropy_loss | -0.829 |
| explained_variance | 0.77800524 |
| learning_rate | 0.0003 |
| loss | 141 |
| n_updates | 972 |
| policy_gradient_loss | -0.00186 |
| value_loss | 440 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 500000 to videos/step_500000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 407 |
| ep_rew_mean | 160 |
| time/ | |
| fps | 343 |
| iterations | 245 |
| time_elapsed | 1461 |
| total_timesteps | 501760 |
| train/ | |
| approx_kl | 0.0046441667 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -0.789 |
| explained_variance | 0.66376996 |
| learning_rate | 0.0003 |
| loss | 58.8 |
| n_updates | 976 |
| policy_gradient_loss | -0.00133 |
| value_loss | 222 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 406 |
| ep_rew_mean | 157 |
| time/ | |
| fps | 343 |
| iterations | 246 |
| time_elapsed | 1466 |
| total_timesteps | 503808 |
| train/ | |
| approx_kl | 0.0018281523 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -0.88 |
| explained_variance | 0.5134449 |
| learning_rate | 0.0003 |
| loss | 513 |
| n_updates | 980 |
| policy_gradient_loss | -0.000614 |
| value_loss | 961 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 406 |
| ep_rew_mean | 155 |
| time/ | |
| fps | 343 |
| iterations | 247 |
| time_elapsed | 1471 |
| total_timesteps | 505856 |
| train/ | |
| approx_kl | 0.0047939075 |
| clip_fraction | 0.025 |
| clip_range | 0.2 |
| entropy_loss | -0.765 |
| explained_variance | 0.84721345 |
| learning_rate | 0.0003 |
| loss | 30.2 |
| n_updates | 984 |
| policy_gradient_loss | -0.00267 |
| value_loss | 160 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 403 |
| ep_rew_mean | 154 |
| time/ | |
| fps | 344 |
| iterations | 248 |
| time_elapsed | 1476 |
| total_timesteps | 507904 |
| train/ | |
| approx_kl | 0.0012371407 |
| clip_fraction | 0.000854 |
| clip_range | 0.2 |
| entropy_loss | -0.85 |
| explained_variance | 0.6320432 |
| learning_rate | 0.0003 |
| loss | 339 |
| n_updates | 988 |
| policy_gradient_loss | -0.000849 |
| value_loss | 776 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 403 |
| ep_rew_mean | 157 |
| time/ | |
| fps | 344 |
| iterations | 249 |
| time_elapsed | 1481 |
| total_timesteps | 509952 |
| train/ | |
| approx_kl | 0.0037914703 |
| clip_fraction | 0.0192 |
| clip_range | 0.2 |
| entropy_loss | -0.786 |
| explained_variance | 0.915509 |
| learning_rate | 0.0003 |
| loss | 66.3 |
| n_updates | 992 |
| policy_gradient_loss | -0.00202 |
| value_loss | 121 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 405 |
| ep_rew_mean | 165 |
| time/ | |
| fps | 344 |
| iterations | 250 |
| time_elapsed | 1486 |
| total_timesteps | 512000 |
| train/ | |
| approx_kl | 0.0045639733 |
| clip_fraction | 0.0453 |
| clip_range | 0.2 |
| entropy_loss | -0.762 |
| explained_variance | 0.92608786 |
| learning_rate | 0.0003 |
| loss | 12.8 |
| n_updates | 996 |
| policy_gradient_loss | 0.000972 |
| value_loss | 31.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 401 |
| ep_rew_mean | 173 |
| time/ | |
| fps | 344 |
| iterations | 251 |
| time_elapsed | 1490 |
| total_timesteps | 514048 |
| train/ | |
| approx_kl | 0.003893061 |
| clip_fraction | 0.0327 |
| clip_range | 0.2 |
| entropy_loss | -0.803 |
| explained_variance | 0.5937791 |
| learning_rate | 0.0003 |
| loss | 37.4 |
| n_updates | 1000 |
| policy_gradient_loss | -0.000808 |
| value_loss | 77.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 410 |
| ep_rew_mean | 176 |
| time/ | |
| fps | 344 |
| iterations | 252 |
| time_elapsed | 1496 |
| total_timesteps | 516096 |
| train/ | |
| approx_kl | 0.0044840677 |
| clip_fraction | 0.0233 |
| clip_range | 0.2 |
| entropy_loss | -0.795 |
| explained_variance | 0.4856863 |
| learning_rate | 0.0003 |
| loss | 83 |
| n_updates | 1004 |
| policy_gradient_loss | -0.00234 |
| value_loss | 522 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 409 |
| ep_rew_mean | 181 |
| time/ | |
| fps | 345 |
| iterations | 253 |
| time_elapsed | 1501 |
| total_timesteps | 518144 |
| train/ | |
| approx_kl | 0.011024909 |
| clip_fraction | 0.0499 |
| clip_range | 0.2 |
| entropy_loss | -0.743 |
| explained_variance | 0.8741459 |
| learning_rate | 0.0003 |
| loss | 16 |
| n_updates | 1008 |
| policy_gradient_loss | 0.000712 |
| value_loss | 55.7 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 520000 to videos/step_520000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 416 |
| ep_rew_mean | 184 |
| time/ | |
| fps | 344 |
| iterations | 254 |
| time_elapsed | 1511 |
| total_timesteps | 520192 |
| train/ | |
| approx_kl | 0.006248506 |
| clip_fraction | 0.0422 |
| clip_range | 0.2 |
| entropy_loss | -0.702 |
| explained_variance | 0.9602749 |
| learning_rate | 0.0003 |
| loss | 7 |
| n_updates | 1012 |
| policy_gradient_loss | 0.00165 |
| value_loss | 21.7 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 426 |
| ep_rew_mean | 189 |
| time/ | |
| fps | 344 |
| iterations | 255 |
| time_elapsed | 1516 |
| total_timesteps | 522240 |
| train/ | |
| approx_kl | 0.011277264 |
| clip_fraction | 0.0914 |
| clip_range | 0.2 |
| entropy_loss | -0.703 |
| explained_variance | 0.9762211 |
| learning_rate | 0.0003 |
| loss | 5.71 |
| n_updates | 1016 |
| policy_gradient_loss | 0.00129 |
| value_loss | 14.8 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 434 |
| ep_rew_mean | 194 |
| time/ | |
| fps | 344 |
| iterations | 256 |
| time_elapsed | 1521 |
| total_timesteps | 524288 |
| train/ | |
| approx_kl | 0.005918427 |
| clip_fraction | 0.0386 |
| clip_range | 0.2 |
| entropy_loss | -0.645 |
| explained_variance | 0.9400156 |
| learning_rate | 0.0003 |
| loss | 35.4 |
| n_updates | 1020 |
| policy_gradient_loss | -0.00308 |
| value_loss | 44.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 440 |
| ep_rew_mean | 194 |
| time/ | |
| fps | 344 |
| iterations | 257 |
| time_elapsed | 1526 |
| total_timesteps | 526336 |
| train/ | |
| approx_kl | 0.0024552555 |
| clip_fraction | 0.0358 |
| clip_range | 0.2 |
| entropy_loss | -0.817 |
| explained_variance | 0.6682333 |
| learning_rate | 0.0003 |
| loss | 33.4 |
| n_updates | 1024 |
| policy_gradient_loss | -0.00108 |
| value_loss | 387 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 443 |
| ep_rew_mean | 194 |
| time/ | |
| fps | 344 |
| iterations | 258 |
| time_elapsed | 1531 |
| total_timesteps | 528384 |
| train/ | |
| approx_kl | 0.0059046727 |
| clip_fraction | 0.042 |
| clip_range | 0.2 |
| entropy_loss | -0.705 |
| explained_variance | 0.9747312 |
| learning_rate | 0.0003 |
| loss | 7.53 |
| n_updates | 1028 |
| policy_gradient_loss | -0.00271 |
| value_loss | 20.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 438 |
| ep_rew_mean | 195 |
| time/ | |
| fps | 345 |
| iterations | 259 |
| time_elapsed | 1536 |
| total_timesteps | 530432 |
| train/ | |
| approx_kl | 0.0014737659 |
| clip_fraction | 0.00952 |
| clip_range | 0.2 |
| entropy_loss | -0.768 |
| explained_variance | 0.94948494 |
| learning_rate | 0.0003 |
| loss | 13.3 |
| n_updates | 1032 |
| policy_gradient_loss | -0.000295 |
| value_loss | 33 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 446 |
| ep_rew_mean | 201 |
| time/ | |
| fps | 345 |
| iterations | 260 |
| time_elapsed | 1541 |
| total_timesteps | 532480 |
| train/ | |
| approx_kl | 0.008586538 |
| clip_fraction | 0.072 |
| clip_range | 0.2 |
| entropy_loss | -0.765 |
| explained_variance | 0.9566847 |
| learning_rate | 0.0003 |
| loss | 7.12 |
| n_updates | 1036 |
| policy_gradient_loss | -0.000437 |
| value_loss | 27.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 438 |
| ep_rew_mean | 202 |
| time/ | |
| fps | 345 |
| iterations | 261 |
| time_elapsed | 1547 |
| total_timesteps | 534528 |
| train/ | |
| approx_kl | 0.0022444203 |
| clip_fraction | 0.0156 |
| clip_range | 0.2 |
| entropy_loss | -0.692 |
| explained_variance | 0.95539486 |
| learning_rate | 0.0003 |
| loss | 26.9 |
| n_updates | 1040 |
| policy_gradient_loss | -0.00139 |
| value_loss | 36.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 437 |
| ep_rew_mean | 207 |
| time/ | |
| fps | 345 |
| iterations | 262 |
| time_elapsed | 1552 |
| total_timesteps | 536576 |
| train/ | |
| approx_kl | 0.0014538937 |
| clip_fraction | 0.00647 |
| clip_range | 0.2 |
| entropy_loss | -0.853 |
| explained_variance | 0.6123003 |
| learning_rate | 0.0003 |
| loss | 31.1 |
| n_updates | 1044 |
| policy_gradient_loss | 0.000203 |
| value_loss | 432 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 436 |
| ep_rew_mean | 211 |
| time/ | |
| fps | 345 |
| iterations | 263 |
| time_elapsed | 1556 |
| total_timesteps | 538624 |
| train/ | |
| approx_kl | 0.0021403187 |
| clip_fraction | 0.026 |
| clip_range | 0.2 |
| entropy_loss | -0.805 |
| explained_variance | 0.945073 |
| learning_rate | 0.0003 |
| loss | 11.6 |
| n_updates | 1048 |
| policy_gradient_loss | -0.00166 |
| value_loss | 39.4 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 540000 to videos/step_540000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 430 |
| ep_rew_mean | 216 |
| time/ | |
| fps | 345 |
| iterations | 264 |
| time_elapsed | 1564 |
| total_timesteps | 540672 |
| train/ | |
| approx_kl | 0.00030441536 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.85 |
| explained_variance | 0.70585203 |
| learning_rate | 0.0003 |
| loss | 244 |
| n_updates | 1052 |
| policy_gradient_loss | 0.000329 |
| value_loss | 327 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 428 |
| ep_rew_mean | 214 |
| time/ | |
| fps | 345 |
| iterations | 265 |
| time_elapsed | 1568 |
| total_timesteps | 542720 |
| train/ | |
| approx_kl | 0.00015119096 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.867 |
| explained_variance | 0.6685668 |
| learning_rate | 0.0003 |
| loss | 129 |
| n_updates | 1056 |
| policy_gradient_loss | -0.000172 |
| value_loss | 593 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 438 |
| ep_rew_mean | 218 |
| time/ | |
| fps | 346 |
| iterations | 266 |
| time_elapsed | 1574 |
| total_timesteps | 544768 |
| train/ | |
| approx_kl | 0.000530906 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.847 |
| explained_variance | 0.5765004 |
| learning_rate | 0.0003 |
| loss | 182 |
| n_updates | 1060 |
| policy_gradient_loss | -0.000253 |
| value_loss | 489 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 438 |
| ep_rew_mean | 218 |
| time/ | |
| fps | 346 |
| iterations | 267 |
| time_elapsed | 1578 |
| total_timesteps | 546816 |
| train/ | |
| approx_kl | 0.009950229 |
| clip_fraction | 0.0759 |
| clip_range | 0.2 |
| entropy_loss | -0.757 |
| explained_variance | 0.9627228 |
| learning_rate | 0.0003 |
| loss | 13.1 |
| n_updates | 1064 |
| policy_gradient_loss | -0.00208 |
| value_loss | 29.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 437 |
| ep_rew_mean | 220 |
| time/ | |
| fps | 346 |
| iterations | 268 |
| time_elapsed | 1583 |
| total_timesteps | 548864 |
| train/ | |
| approx_kl | 0.0005525501 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.856 |
| explained_variance | 0.6846489 |
| learning_rate | 0.0003 |
| loss | 251 |
| n_updates | 1068 |
| policy_gradient_loss | 0.000154 |
| value_loss | 601 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 441 |
| ep_rew_mean | 223 |
| time/ | |
| fps | 346 |
| iterations | 269 |
| time_elapsed | 1588 |
| total_timesteps | 550912 |
| train/ | |
| approx_kl | 0.0066212122 |
| clip_fraction | 0.0248 |
| clip_range | 0.2 |
| entropy_loss | -0.784 |
| explained_variance | 0.9122072 |
| learning_rate | 0.0003 |
| loss | 15 |
| n_updates | 1072 |
| policy_gradient_loss | -0.00174 |
| value_loss | 35 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 444 |
| ep_rew_mean | 225 |
| time/ | |
| fps | 347 |
| iterations | 270 |
| time_elapsed | 1593 |
| total_timesteps | 552960 |
| train/ | |
| approx_kl | 0.00850571 |
| clip_fraction | 0.0924 |
| clip_range | 0.2 |
| entropy_loss | -0.813 |
| explained_variance | 0.98254293 |
| learning_rate | 0.0003 |
| loss | 7.8 |
| n_updates | 1076 |
| policy_gradient_loss | -0.0033 |
| value_loss | 17.9 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 445 |
| ep_rew_mean | 225 |
| time/ | |
| fps | 347 |
| iterations | 271 |
| time_elapsed | 1598 |
| total_timesteps | 555008 |
| train/ | |
| approx_kl | 0.0042126696 |
| clip_fraction | 0.0236 |
| clip_range | 0.2 |
| entropy_loss | -0.762 |
| explained_variance | 0.94078207 |
| learning_rate | 0.0003 |
| loss | 11.4 |
| n_updates | 1080 |
| policy_gradient_loss | -0.0034 |
| value_loss | 28.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 446 |
| ep_rew_mean | 226 |
| time/ | |
| fps | 347 |
| iterations | 272 |
| time_elapsed | 1602 |
| total_timesteps | 557056 |
| train/ | |
| approx_kl | 0.0036365676 |
| clip_fraction | 0.0255 |
| clip_range | 0.2 |
| entropy_loss | -0.831 |
| explained_variance | 0.94567585 |
| learning_rate | 0.0003 |
| loss | 8.02 |
| n_updates | 1084 |
| policy_gradient_loss | -0.00274 |
| value_loss | 24.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 454 |
| ep_rew_mean | 227 |
| time/ | |
| fps | 347 |
| iterations | 273 |
| time_elapsed | 1607 |
| total_timesteps | 559104 |
| train/ | |
| approx_kl | 0.0024922143 |
| clip_fraction | 0.00842 |
| clip_range | 0.2 |
| entropy_loss | -0.778 |
| explained_variance | 0.91811603 |
| learning_rate | 0.0003 |
| loss | 31.6 |
| n_updates | 1088 |
| policy_gradient_loss | 0.0012 |
| value_loss | 49.6 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 560000 to videos/step_560000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 447 |
| ep_rew_mean | 227 |
| time/ | |
| fps | 346 |
| iterations | 274 |
| time_elapsed | 1620 |
| total_timesteps | 561152 |
| train/ | |
| approx_kl | 0.0025746904 |
| clip_fraction | 0.0425 |
| clip_range | 0.2 |
| entropy_loss | -0.74 |
| explained_variance | 0.9729362 |
| learning_rate | 0.0003 |
| loss | 3.86 |
| n_updates | 1092 |
| policy_gradient_loss | -0.00138 |
| value_loss | 14 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 438 |
| ep_rew_mean | 227 |
| time/ | |
| fps | 346 |
| iterations | 275 |
| time_elapsed | 1625 |
| total_timesteps | 563200 |
| train/ | |
| approx_kl | 0.005784927 |
| clip_fraction | 0.0474 |
| clip_range | 0.2 |
| entropy_loss | -0.862 |
| explained_variance | 0.9508154 |
| learning_rate | 0.0003 |
| loss | 13.8 |
| n_updates | 1096 |
| policy_gradient_loss | -0.00246 |
| value_loss | 28.6 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 428 |
| ep_rew_mean | 227 |
| time/ | |
| fps | 346 |
| iterations | 276 |
| time_elapsed | 1629 |
| total_timesteps | 565248 |
| train/ | |
| approx_kl | 0.00477351 |
| clip_fraction | 0.04 |
| clip_range | 0.2 |
| entropy_loss | -0.803 |
| explained_variance | 0.96048635 |
| learning_rate | 0.0003 |
| loss | 5.76 |
| n_updates | 1100 |
| policy_gradient_loss | -0.00269 |
| value_loss | 18.5 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 423 |
| ep_rew_mean | 229 |
| time/ | |
| fps | 346 |
| iterations | 277 |
| time_elapsed | 1635 |
| total_timesteps | 567296 |
| train/ | |
| approx_kl | 0.006210279 |
| clip_fraction | 0.0452 |
| clip_range | 0.2 |
| entropy_loss | -0.803 |
| explained_variance | 0.96510255 |
| learning_rate | 0.0003 |
| loss | 4.8 |
| n_updates | 1104 |
| policy_gradient_loss | -0.00201 |
| value_loss | 16.9 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 419 |
| ep_rew_mean | 230 |
| time/ | |
| fps | 347 |
| iterations | 278 |
| time_elapsed | 1639 |
| total_timesteps | 569344 |
| train/ | |
| approx_kl | 0.0073518544 |
| clip_fraction | 0.0415 |
| clip_range | 0.2 |
| entropy_loss | -0.876 |
| explained_variance | 0.96836424 |
| learning_rate | 0.0003 |
| loss | 4.28 |
| n_updates | 1108 |
| policy_gradient_loss | -0.00315 |
| value_loss | 12.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 408 |
| ep_rew_mean | 232 |
| time/ | |
| fps | 347 |
| iterations | 279 |
| time_elapsed | 1644 |
| total_timesteps | 571392 |
| train/ | |
| approx_kl | 0.0061610006 |
| clip_fraction | 0.0466 |
| clip_range | 0.2 |
| entropy_loss | -0.812 |
| explained_variance | 0.95754284 |
| learning_rate | 0.0003 |
| loss | 15.2 |
| n_updates | 1112 |
| policy_gradient_loss | -0.00257 |
| value_loss | 31.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 402 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 347 |
| iterations | 280 |
| time_elapsed | 1649 |
| total_timesteps | 573440 |
| train/ | |
| approx_kl | 0.0044109053 |
| clip_fraction | 0.0197 |
| clip_range | 0.2 |
| entropy_loss | -0.826 |
| explained_variance | 0.95786685 |
| learning_rate | 0.0003 |
| loss | 15.3 |
| n_updates | 1116 |
| policy_gradient_loss | -0.000505 |
| value_loss | 27.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 396 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 347 |
| iterations | 281 |
| time_elapsed | 1653 |
| total_timesteps | 575488 |
| train/ | |
| approx_kl | 0.0029312735 |
| clip_fraction | 0.0238 |
| clip_range | 0.2 |
| entropy_loss | -0.773 |
| explained_variance | 0.9833889 |
| learning_rate | 0.0003 |
| loss | 4.46 |
| n_updates | 1120 |
| policy_gradient_loss | 0.000397 |
| value_loss | 13 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 397 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 348 |
| iterations | 282 |
| time_elapsed | 1658 |
| total_timesteps | 577536 |
| train/ | |
| approx_kl | 0.0025344035 |
| clip_fraction | 0.0239 |
| clip_range | 0.2 |
| entropy_loss | -0.805 |
| explained_variance | 0.96333075 |
| learning_rate | 0.0003 |
| loss | 9.67 |
| n_updates | 1124 |
| policy_gradient_loss | -0.0012 |
| value_loss | 26.7 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 399 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 348 |
| iterations | 283 |
| time_elapsed | 1663 |
| total_timesteps | 579584 |
| train/ | |
| approx_kl | 0.002802658 |
| clip_fraction | 0.0171 |
| clip_range | 0.2 |
| entropy_loss | -0.75 |
| explained_variance | 0.9733484 |
| learning_rate | 0.0003 |
| loss | 15.4 |
| n_updates | 1128 |
| policy_gradient_loss | -0.000823 |
| value_loss | 24.7 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 580000 to videos/step_580000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 398 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 348 |
| iterations | 284 |
| time_elapsed | 1670 |
| total_timesteps | 581632 |
| train/ | |
| approx_kl | 0.0036595266 |
| clip_fraction | 0.0184 |
| clip_range | 0.2 |
| entropy_loss | -0.766 |
| explained_variance | 0.96371746 |
| learning_rate | 0.0003 |
| loss | 10.7 |
| n_updates | 1132 |
| policy_gradient_loss | -0.000442 |
| value_loss | 30.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 388 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 348 |
| iterations | 285 |
| time_elapsed | 1676 |
| total_timesteps | 583680 |
| train/ | |
| approx_kl | 0.0012979265 |
| clip_fraction | 0.00696 |
| clip_range | 0.2 |
| entropy_loss | -0.844 |
| explained_variance | 0.6580386 |
| learning_rate | 0.0003 |
| loss | 56.8 |
| n_updates | 1136 |
| policy_gradient_loss | 0.000331 |
| value_loss | 568 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 389 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 348 |
| iterations | 286 |
| time_elapsed | 1680 |
| total_timesteps | 585728 |
| train/ | |
| approx_kl | 0.00068800006 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.823 |
| explained_variance | 0.7771591 |
| learning_rate | 0.0003 |
| loss | 182 |
| n_updates | 1140 |
| policy_gradient_loss | -0.000405 |
| value_loss | 390 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 390 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 348 |
| iterations | 287 |
| time_elapsed | 1685 |
| total_timesteps | 587776 |
| train/ | |
| approx_kl | 0.0037196996 |
| clip_fraction | 0.0176 |
| clip_range | 0.2 |
| entropy_loss | -0.775 |
| explained_variance | 0.90367115 |
| learning_rate | 0.0003 |
| loss | 22.6 |
| n_updates | 1144 |
| policy_gradient_loss | -0.0019 |
| value_loss | 55.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 382 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 348 |
| iterations | 288 |
| time_elapsed | 1690 |
| total_timesteps | 589824 |
| train/ | |
| approx_kl | 0.0053053154 |
| clip_fraction | 0.033 |
| clip_range | 0.2 |
| entropy_loss | -0.765 |
| explained_variance | 0.9638064 |
| learning_rate | 0.0003 |
| loss | 13.6 |
| n_updates | 1148 |
| policy_gradient_loss | -0.00238 |
| value_loss | 17.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 380 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 349 |
| iterations | 289 |
| time_elapsed | 1694 |
| total_timesteps | 591872 |
| train/ | |
| approx_kl | 0.0011253463 |
| clip_fraction | 0.00256 |
| clip_range | 0.2 |
| entropy_loss | -0.788 |
| explained_variance | 0.6557063 |
| learning_rate | 0.0003 |
| loss | 769 |
| n_updates | 1152 |
| policy_gradient_loss | -0.000922 |
| value_loss | 511 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 380 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 349 |
| iterations | 290 |
| time_elapsed | 1699 |
| total_timesteps | 593920 |
| train/ | |
| approx_kl | 0.0013557405 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.782 |
| explained_variance | 0.7173494 |
| learning_rate | 0.0003 |
| loss | 59.5 |
| n_updates | 1156 |
| policy_gradient_loss | -0.00043 |
| value_loss | 396 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 378 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 349 |
| iterations | 291 |
| time_elapsed | 1704 |
| total_timesteps | 595968 |
| train/ | |
| approx_kl | 0.005822175 |
| clip_fraction | 0.0364 |
| clip_range | 0.2 |
| entropy_loss | -0.784 |
| explained_variance | 0.9600708 |
| learning_rate | 0.0003 |
| loss | 6.37 |
| n_updates | 1160 |
| policy_gradient_loss | -0.000571 |
| value_loss | 22.9 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 369 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 350 |
| iterations | 292 |
| time_elapsed | 1708 |
| total_timesteps | 598016 |
| train/ | |
| approx_kl | 0.004813377 |
| clip_fraction | 0.0286 |
| clip_range | 0.2 |
| entropy_loss | -0.782 |
| explained_variance | 0.96399385 |
| learning_rate | 0.0003 |
| loss | 5.86 |
| n_updates | 1164 |
| policy_gradient_loss | -0.000608 |
| value_loss | 19.2 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 600000 to videos/step_600000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 364 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 349 |
| iterations | 293 |
| time_elapsed | 1716 |
| total_timesteps | 600064 |
| train/ | |
| approx_kl | 0.003934575 |
| clip_fraction | 0.058 |
| clip_range | 0.2 |
| entropy_loss | -0.834 |
| explained_variance | 0.63396305 |
| learning_rate | 0.0003 |
| loss | 225 |
| n_updates | 1168 |
| policy_gradient_loss | -0.00166 |
| value_loss | 535 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 366 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 349 |
| iterations | 294 |
| time_elapsed | 1721 |
| total_timesteps | 602112 |
| train/ | |
| approx_kl | 0.0013957437 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.783 |
| explained_variance | 0.6126939 |
| learning_rate | 0.0003 |
| loss | 495 |
| n_updates | 1172 |
| policy_gradient_loss | -0.000717 |
| value_loss | 462 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 370 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 349 |
| iterations | 295 |
| time_elapsed | 1726 |
| total_timesteps | 604160 |
| train/ | |
| approx_kl | 0.0046395804 |
| clip_fraction | 0.0256 |
| clip_range | 0.2 |
| entropy_loss | -0.781 |
| explained_variance | 0.97449285 |
| learning_rate | 0.0003 |
| loss | 12.8 |
| n_updates | 1176 |
| policy_gradient_loss | -0.00341 |
| value_loss | 21.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 369 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 350 |
| iterations | 296 |
| time_elapsed | 1731 |
| total_timesteps | 606208 |
| train/ | |
| approx_kl | 0.0016476751 |
| clip_fraction | 0.00562 |
| clip_range | 0.2 |
| entropy_loss | -0.729 |
| explained_variance | 0.9585453 |
| learning_rate | 0.0003 |
| loss | 31.2 |
| n_updates | 1180 |
| policy_gradient_loss | -0.000741 |
| value_loss | 64.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 371 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 350 |
| iterations | 297 |
| time_elapsed | 1735 |
| total_timesteps | 608256 |
| train/ | |
| approx_kl | 0.004324998 |
| clip_fraction | 0.0475 |
| clip_range | 0.2 |
| entropy_loss | -0.756 |
| explained_variance | 0.9718915 |
| learning_rate | 0.0003 |
| loss | 7.15 |
| n_updates | 1184 |
| policy_gradient_loss | -0.00235 |
| value_loss | 20.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 372 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 350 |
| iterations | 298 |
| time_elapsed | 1740 |
| total_timesteps | 610304 |
| train/ | |
| approx_kl | 0.0035852904 |
| clip_fraction | 0.0339 |
| clip_range | 0.2 |
| entropy_loss | -0.799 |
| explained_variance | 0.96982086 |
| learning_rate | 0.0003 |
| loss | 4.47 |
| n_updates | 1188 |
| policy_gradient_loss | -0.000592 |
| value_loss | 17.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 370 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 350 |
| iterations | 299 |
| time_elapsed | 1745 |
| total_timesteps | 612352 |
| train/ | |
| approx_kl | 0.0066330153 |
| clip_fraction | 0.0447 |
| clip_range | 0.2 |
| entropy_loss | -0.816 |
| explained_variance | 0.9684166 |
| learning_rate | 0.0003 |
| loss | 7.06 |
| n_updates | 1192 |
| policy_gradient_loss | -0.00102 |
| value_loss | 14.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 370 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 351 |
| iterations | 300 |
| time_elapsed | 1749 |
| total_timesteps | 614400 |
| train/ | |
| approx_kl | 0.002531549 |
| clip_fraction | 0.0115 |
| clip_range | 0.2 |
| entropy_loss | -0.823 |
| explained_variance | 0.6511619 |
| learning_rate | 0.0003 |
| loss | 59.4 |
| n_updates | 1196 |
| policy_gradient_loss | -0.00123 |
| value_loss | 396 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 372 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 351 |
| iterations | 301 |
| time_elapsed | 1755 |
| total_timesteps | 616448 |
| train/ | |
| approx_kl | 0.0019515279 |
| clip_fraction | 0.000732 |
| clip_range | 0.2 |
| entropy_loss | -0.839 |
| explained_variance | 0.6736957 |
| learning_rate | 0.0003 |
| loss | 119 |
| n_updates | 1200 |
| policy_gradient_loss | -0.00144 |
| value_loss | 406 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 376 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 351 |
| iterations | 302 |
| time_elapsed | 1759 |
| total_timesteps | 618496 |
| train/ | |
| approx_kl | 0.0019328761 |
| clip_fraction | 0.00403 |
| clip_range | 0.2 |
| entropy_loss | -0.81 |
| explained_variance | 0.9167411 |
| learning_rate | 0.0003 |
| loss | 13.3 |
| n_updates | 1204 |
| policy_gradient_loss | -0.000284 |
| value_loss | 43.7 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 620000 to videos/step_620000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 378 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 351 |
| iterations | 303 |
| time_elapsed | 1767 |
| total_timesteps | 620544 |
| train/ | |
| approx_kl | 0.000681952 |
| clip_fraction | 0.000366 |
| clip_range | 0.2 |
| entropy_loss | -0.815 |
| explained_variance | 0.9449134 |
| learning_rate | 0.0003 |
| loss | 31.9 |
| n_updates | 1208 |
| policy_gradient_loss | -0.00106 |
| value_loss | 65.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 384 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 351 |
| iterations | 304 |
| time_elapsed | 1772 |
| total_timesteps | 622592 |
| train/ | |
| approx_kl | 0.0057974174 |
| clip_fraction | 0.0466 |
| clip_range | 0.2 |
| entropy_loss | -0.777 |
| explained_variance | 0.9708614 |
| learning_rate | 0.0003 |
| loss | 8.65 |
| n_updates | 1212 |
| policy_gradient_loss | -0.00281 |
| value_loss | 16.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 384 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 351 |
| iterations | 305 |
| time_elapsed | 1777 |
| total_timesteps | 624640 |
| train/ | |
| approx_kl | 0.010070858 |
| clip_fraction | 0.0823 |
| clip_range | 0.2 |
| entropy_loss | -0.754 |
| explained_variance | 0.79100096 |
| learning_rate | 0.0003 |
| loss | 346 |
| n_updates | 1216 |
| policy_gradient_loss | -0.00462 |
| value_loss | 507 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 391 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 351 |
| iterations | 306 |
| time_elapsed | 1782 |
| total_timesteps | 626688 |
| train/ | |
| approx_kl | 0.005909621 |
| clip_fraction | 0.0612 |
| clip_range | 0.2 |
| entropy_loss | -0.886 |
| explained_variance | 0.9675523 |
| learning_rate | 0.0003 |
| loss | 5.18 |
| n_updates | 1220 |
| policy_gradient_loss | -0.00205 |
| value_loss | 12.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 392 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 351 |
| iterations | 307 |
| time_elapsed | 1786 |
| total_timesteps | 628736 |
| train/ | |
| approx_kl | 0.005853832 |
| clip_fraction | 0.0314 |
| clip_range | 0.2 |
| entropy_loss | -0.74 |
| explained_variance | 0.9415673 |
| learning_rate | 0.0003 |
| loss | 11.2 |
| n_updates | 1224 |
| policy_gradient_loss | -0.00275 |
| value_loss | 47.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 392 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 352 |
| iterations | 308 |
| time_elapsed | 1791 |
| total_timesteps | 630784 |
| train/ | |
| approx_kl | 0.0055165626 |
| clip_fraction | 0.0454 |
| clip_range | 0.2 |
| entropy_loss | -0.778 |
| explained_variance | 0.978958 |
| learning_rate | 0.0003 |
| loss | 5.42 |
| n_updates | 1228 |
| policy_gradient_loss | -0.00118 |
| value_loss | 16.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 392 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 352 |
| iterations | 309 |
| time_elapsed | 1795 |
| total_timesteps | 632832 |
| train/ | |
| approx_kl | 0.0017051477 |
| clip_fraction | 0.0201 |
| clip_range | 0.2 |
| entropy_loss | -0.899 |
| explained_variance | 0.5668086 |
| learning_rate | 0.0003 |
| loss | 271 |
| n_updates | 1232 |
| policy_gradient_loss | 8.48e-05 |
| value_loss | 511 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 389 |
| ep_rew_mean | 230 |
| time/ | |
| fps | 352 |
| iterations | 310 |
| time_elapsed | 1800 |
| total_timesteps | 634880 |
| train/ | |
| approx_kl | 0.0026133289 |
| clip_fraction | 0.0181 |
| clip_range | 0.2 |
| entropy_loss | -0.8 |
| explained_variance | 0.9750948 |
| learning_rate | 0.0003 |
| loss | 10.8 |
| n_updates | 1236 |
| policy_gradient_loss | -0.00155 |
| value_loss | 19.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 389 |
| ep_rew_mean | 232 |
| time/ | |
| fps | 352 |
| iterations | 311 |
| time_elapsed | 1805 |
| total_timesteps | 636928 |
| train/ | |
| approx_kl | 0.0010109926 |
| clip_fraction | 0.00256 |
| clip_range | 0.2 |
| entropy_loss | -0.806 |
| explained_variance | 0.61024016 |
| learning_rate | 0.0003 |
| loss | 459 |
| n_updates | 1240 |
| policy_gradient_loss | 0.00102 |
| value_loss | 818 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 389 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 353 |
| iterations | 312 |
| time_elapsed | 1809 |
| total_timesteps | 638976 |
| train/ | |
| approx_kl | 0.0034891414 |
| clip_fraction | 0.0148 |
| clip_range | 0.2 |
| entropy_loss | -0.733 |
| explained_variance | 0.9405303 |
| learning_rate | 0.0003 |
| loss | 12 |
| n_updates | 1244 |
| policy_gradient_loss | -0.00119 |
| value_loss | 31.7 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 640000 to videos/step_640000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 386 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 352 |
| iterations | 313 |
| time_elapsed | 1818 |
| total_timesteps | 641024 |
| train/ | |
| approx_kl | 0.004025467 |
| clip_fraction | 0.0466 |
| clip_range | 0.2 |
| entropy_loss | -0.761 |
| explained_variance | 0.94799393 |
| learning_rate | 0.0003 |
| loss | 11 |
| n_updates | 1248 |
| policy_gradient_loss | -0.00335 |
| value_loss | 28.2 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 378 |
| ep_rew_mean | 231 |
| time/ | |
| fps | 352 |
| iterations | 314 |
| time_elapsed | 1822 |
| total_timesteps | 643072 |
| train/ | |
| approx_kl | 0.0051186047 |
| clip_fraction | 0.0305 |
| clip_range | 0.2 |
| entropy_loss | -0.808 |
| explained_variance | 0.9702966 |
| learning_rate | 0.0003 |
| loss | 8.63 |
| n_updates | 1252 |
| policy_gradient_loss | -0.000471 |
| value_loss | 18.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 377 |
| ep_rew_mean | 230 |
| time/ | |
| fps | 353 |
| iterations | 315 |
| time_elapsed | 1826 |
| total_timesteps | 645120 |
| train/ | |
| approx_kl | 0.005027827 |
| clip_fraction | 0.0529 |
| clip_range | 0.2 |
| entropy_loss | -0.796 |
| explained_variance | 0.5718798 |
| learning_rate | 0.0003 |
| loss | 228 |
| n_updates | 1256 |
| policy_gradient_loss | -0.003 |
| value_loss | 493 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 375 |
| ep_rew_mean | 228 |
| time/ | |
| fps | 353 |
| iterations | 316 |
| time_elapsed | 1831 |
| total_timesteps | 647168 |
| train/ | |
| approx_kl | 0.0020331633 |
| clip_fraction | 0.00378 |
| clip_range | 0.2 |
| entropy_loss | -0.829 |
| explained_variance | 0.6584902 |
| learning_rate | 0.0003 |
| loss | 237 |
| n_updates | 1260 |
| policy_gradient_loss | -0.000455 |
| value_loss | 451 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 374 |
| ep_rew_mean | 229 |
| time/ | |
| fps | 353 |
| iterations | 317 |
| time_elapsed | 1836 |
| total_timesteps | 649216 |
| train/ | |
| approx_kl | 0.0015396525 |
| clip_fraction | 0.00647 |
| clip_range | 0.2 |
| entropy_loss | -0.832 |
| explained_variance | 0.6493981 |
| learning_rate | 0.0003 |
| loss | 86.4 |
| n_updates | 1264 |
| policy_gradient_loss | 0.000155 |
| value_loss | 491 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 374 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 353 |
| iterations | 318 |
| time_elapsed | 1840 |
| total_timesteps | 651264 |
| train/ | |
| approx_kl | 0.0049825544 |
| clip_fraction | 0.0427 |
| clip_range | 0.2 |
| entropy_loss | -0.817 |
| explained_variance | 0.95405245 |
| learning_rate | 0.0003 |
| loss | 7.54 |
| n_updates | 1268 |
| policy_gradient_loss | -0.00205 |
| value_loss | 20.1 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 373 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 353 |
| iterations | 319 |
| time_elapsed | 1845 |
| total_timesteps | 653312 |
| train/ | |
| approx_kl | 0.002884435 |
| clip_fraction | 0.0239 |
| clip_range | 0.2 |
| entropy_loss | -0.773 |
| explained_variance | 0.97524804 |
| learning_rate | 0.0003 |
| loss | 5.38 |
| n_updates | 1272 |
| policy_gradient_loss | -0.00031 |
| value_loss | 16.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 369 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 354 |
| iterations | 320 |
| time_elapsed | 1850 |
| total_timesteps | 655360 |
| train/ | |
| approx_kl | 0.007132809 |
| clip_fraction | 0.061 |
| clip_range | 0.2 |
| entropy_loss | -0.8 |
| explained_variance | 0.9760993 |
| learning_rate | 0.0003 |
| loss | 5.28 |
| n_updates | 1276 |
| policy_gradient_loss | -0.00356 |
| value_loss | 13.2 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 367 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 354 |
| iterations | 321 |
| time_elapsed | 1855 |
| total_timesteps | 657408 |
| train/ | |
| approx_kl | 0.0059518293 |
| clip_fraction | 0.0562 |
| clip_range | 0.2 |
| entropy_loss | -0.826 |
| explained_variance | 0.9725843 |
| learning_rate | 0.0003 |
| loss | 6.53 |
| n_updates | 1280 |
| policy_gradient_loss | -0.000775 |
| value_loss | 15.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 361 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 354 |
| iterations | 322 |
| time_elapsed | 1859 |
| total_timesteps | 659456 |
| train/ | |
| approx_kl | 0.004291858 |
| clip_fraction | 0.0403 |
| clip_range | 0.2 |
| entropy_loss | -0.744 |
| explained_variance | 0.97959465 |
| learning_rate | 0.0003 |
| loss | 6.95 |
| n_updates | 1284 |
| policy_gradient_loss | -0.00145 |
| value_loss | 14.4 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 660000 to videos/step_660000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 354 |
| ep_rew_mean | 240 |
| time/ | |
| fps | 354 |
| iterations | 323 |
| time_elapsed | 1867 |
| total_timesteps | 661504 |
| train/ | |
| approx_kl | 0.0043321503 |
| clip_fraction | 0.036 |
| clip_range | 0.2 |
| entropy_loss | -0.788 |
| explained_variance | 0.9796149 |
| learning_rate | 0.0003 |
| loss | 3.62 |
| n_updates | 1288 |
| policy_gradient_loss | -0.00104 |
| value_loss | 14.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 354 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 354 |
| iterations | 324 |
| time_elapsed | 1872 |
| total_timesteps | 663552 |
| train/ | |
| approx_kl | 0.005180819 |
| clip_fraction | 0.0448 |
| clip_range | 0.2 |
| entropy_loss | -0.77 |
| explained_variance | 0.9830743 |
| learning_rate | 0.0003 |
| loss | 5.17 |
| n_updates | 1292 |
| policy_gradient_loss | -0.00312 |
| value_loss | 11.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 351 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 354 |
| iterations | 325 |
| time_elapsed | 1876 |
| total_timesteps | 665600 |
| train/ | |
| approx_kl | 0.0041947328 |
| clip_fraction | 0.0275 |
| clip_range | 0.2 |
| entropy_loss | -0.773 |
| explained_variance | 0.9783102 |
| learning_rate | 0.0003 |
| loss | 7.57 |
| n_updates | 1296 |
| policy_gradient_loss | -0.000581 |
| value_loss | 16.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 349 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 354 |
| iterations | 326 |
| time_elapsed | 1881 |
| total_timesteps | 667648 |
| train/ | |
| approx_kl | 0.0037134096 |
| clip_fraction | 0.0203 |
| clip_range | 0.2 |
| entropy_loss | -0.79 |
| explained_variance | 0.66636944 |
| learning_rate | 0.0003 |
| loss | 91 |
| n_updates | 1300 |
| policy_gradient_loss | -0.00299 |
| value_loss | 532 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 349 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 355 |
| iterations | 327 |
| time_elapsed | 1886 |
| total_timesteps | 669696 |
| train/ | |
| approx_kl | 0.005216861 |
| clip_fraction | 0.0436 |
| clip_range | 0.2 |
| entropy_loss | -0.811 |
| explained_variance | 0.9790488 |
| learning_rate | 0.0003 |
| loss | 10.1 |
| n_updates | 1304 |
| policy_gradient_loss | -0.00064 |
| value_loss | 17.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 355 |
| iterations | 328 |
| time_elapsed | 1890 |
| total_timesteps | 671744 |
| train/ | |
| approx_kl | 0.0016657235 |
| clip_fraction | 0.0083 |
| clip_range | 0.2 |
| entropy_loss | -0.77 |
| explained_variance | 0.75512 |
| learning_rate | 0.0003 |
| loss | 12.4 |
| n_updates | 1308 |
| policy_gradient_loss | -0.00057 |
| value_loss | 369 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 346 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 355 |
| iterations | 329 |
| time_elapsed | 1895 |
| total_timesteps | 673792 |
| train/ | |
| approx_kl | 0.0016318145 |
| clip_fraction | 0.00183 |
| clip_range | 0.2 |
| entropy_loss | -0.737 |
| explained_variance | 0.8566395 |
| learning_rate | 0.0003 |
| loss | 14.1 |
| n_updates | 1312 |
| policy_gradient_loss | -0.00166 |
| value_loss | 188 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 344 |
| ep_rew_mean | 240 |
| time/ | |
| fps | 355 |
| iterations | 330 |
| time_elapsed | 1899 |
| total_timesteps | 675840 |
| train/ | |
| approx_kl | 0.0027083224 |
| clip_fraction | 0.0236 |
| clip_range | 0.2 |
| entropy_loss | -0.735 |
| explained_variance | 0.9570414 |
| learning_rate | 0.0003 |
| loss | 8.24 |
| n_updates | 1316 |
| policy_gradient_loss | -0.00116 |
| value_loss | 32.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 344 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 356 |
| iterations | 331 |
| time_elapsed | 1904 |
| total_timesteps | 677888 |
| train/ | |
| approx_kl | 0.0006679391 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.788 |
| explained_variance | 0.70951414 |
| learning_rate | 0.0003 |
| loss | 63.2 |
| n_updates | 1320 |
| policy_gradient_loss | -0.000114 |
| value_loss | 291 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 341 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 356 |
| iterations | 332 |
| time_elapsed | 1909 |
| total_timesteps | 679936 |
| train/ | |
| approx_kl | 0.0019355878 |
| clip_fraction | 0.0122 |
| clip_range | 0.2 |
| entropy_loss | -0.797 |
| explained_variance | 0.9760887 |
| learning_rate | 0.0003 |
| loss | 11.1 |
| n_updates | 1324 |
| policy_gradient_loss | -0.000768 |
| value_loss | 19.7 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 680000 to videos/step_680000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 343 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 355 |
| iterations | 333 |
| time_elapsed | 1916 |
| total_timesteps | 681984 |
| train/ | |
| approx_kl | 0.004820833 |
| clip_fraction | 0.0128 |
| clip_range | 0.2 |
| entropy_loss | -0.718 |
| explained_variance | 0.6706275 |
| learning_rate | 0.0003 |
| loss | 152 |
| n_updates | 1328 |
| policy_gradient_loss | -0.00347 |
| value_loss | 562 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 342 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 355 |
| iterations | 334 |
| time_elapsed | 1921 |
| total_timesteps | 684032 |
| train/ | |
| approx_kl | 0.00401313 |
| clip_fraction | 0.0167 |
| clip_range | 0.2 |
| entropy_loss | -0.827 |
| explained_variance | 0.9032712 |
| learning_rate | 0.0003 |
| loss | 26.2 |
| n_updates | 1332 |
| policy_gradient_loss | -0.00194 |
| value_loss | 84.4 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 341 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 356 |
| iterations | 335 |
| time_elapsed | 1926 |
| total_timesteps | 686080 |
| train/ | |
| approx_kl | 0.008010032 |
| clip_fraction | 0.0891 |
| clip_range | 0.2 |
| entropy_loss | -0.769 |
| explained_variance | 0.97045636 |
| learning_rate | 0.0003 |
| loss | 3.83 |
| n_updates | 1336 |
| policy_gradient_loss | -0.00373 |
| value_loss | 14.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 339 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 356 |
| iterations | 336 |
| time_elapsed | 1930 |
| total_timesteps | 688128 |
| train/ | |
| approx_kl | 0.0047063027 |
| clip_fraction | 0.0388 |
| clip_range | 0.2 |
| entropy_loss | -0.809 |
| explained_variance | 0.98045003 |
| learning_rate | 0.0003 |
| loss | 4.62 |
| n_updates | 1340 |
| policy_gradient_loss | -0.0026 |
| value_loss | 12.7 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 356 |
| iterations | 337 |
| time_elapsed | 1935 |
| total_timesteps | 690176 |
| train/ | |
| approx_kl | 0.005852525 |
| clip_fraction | 0.0609 |
| clip_range | 0.2 |
| entropy_loss | -0.778 |
| explained_variance | 0.9811432 |
| learning_rate | 0.0003 |
| loss | 5.17 |
| n_updates | 1344 |
| policy_gradient_loss | -0.000855 |
| value_loss | 13.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 356 |
| iterations | 338 |
| time_elapsed | 1940 |
| total_timesteps | 692224 |
| train/ | |
| approx_kl | 0.0060338024 |
| clip_fraction | 0.051 |
| clip_range | 0.2 |
| entropy_loss | -0.714 |
| explained_variance | 0.9822418 |
| learning_rate | 0.0003 |
| loss | 6.6 |
| n_updates | 1348 |
| policy_gradient_loss | -0.00318 |
| value_loss | 10.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 346 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 356 |
| iterations | 339 |
| time_elapsed | 1945 |
| total_timesteps | 694272 |
| train/ | |
| approx_kl | 0.0042729634 |
| clip_fraction | 0.0121 |
| clip_range | 0.2 |
| entropy_loss | -0.738 |
| explained_variance | 0.98503476 |
| learning_rate | 0.0003 |
| loss | 14.1 |
| n_updates | 1352 |
| policy_gradient_loss | 0.000229 |
| value_loss | 34.2 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 346 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 357 |
| iterations | 340 |
| time_elapsed | 1950 |
| total_timesteps | 696320 |
| train/ | |
| approx_kl | 0.0053418 |
| clip_fraction | 0.0575 |
| clip_range | 0.2 |
| entropy_loss | -0.776 |
| explained_variance | 0.98503906 |
| learning_rate | 0.0003 |
| loss | 4.8 |
| n_updates | 1356 |
| policy_gradient_loss | -0.000162 |
| value_loss | 11.9 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 346 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 357 |
| iterations | 341 |
| time_elapsed | 1954 |
| total_timesteps | 698368 |
| train/ | |
| approx_kl | 0.0032474927 |
| clip_fraction | 0.0165 |
| clip_range | 0.2 |
| entropy_loss | -0.775 |
| explained_variance | 0.98242295 |
| learning_rate | 0.0003 |
| loss | 8.94 |
| n_updates | 1360 |
| policy_gradient_loss | 0.000241 |
| value_loss | 19 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 700000 to videos/step_700000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 356 |
| iterations | 342 |
| time_elapsed | 1962 |
| total_timesteps | 700416 |
| train/ | |
| approx_kl | 0.0008714805 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.748 |
| explained_variance | 0.7303984 |
| learning_rate | 0.0003 |
| loss | 142 |
| n_updates | 1364 |
| policy_gradient_loss | -2.42e-05 |
| value_loss | 281 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 357 |
| iterations | 343 |
| time_elapsed | 1967 |
| total_timesteps | 702464 |
| train/ | |
| approx_kl | 0.003927693 |
| clip_fraction | 0.0133 |
| clip_range | 0.2 |
| entropy_loss | -0.787 |
| explained_variance | 0.9121846 |
| learning_rate | 0.0003 |
| loss | 86.8 |
| n_updates | 1368 |
| policy_gradient_loss | -0.0019 |
| value_loss | 162 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 357 |
| iterations | 344 |
| time_elapsed | 1972 |
| total_timesteps | 704512 |
| train/ | |
| approx_kl | 0.005259488 |
| clip_fraction | 0.0254 |
| clip_range | 0.2 |
| entropy_loss | -0.736 |
| explained_variance | 0.923389 |
| learning_rate | 0.0003 |
| loss | 13 |
| n_updates | 1372 |
| policy_gradient_loss | -0.000567 |
| value_loss | 52.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 348 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 357 |
| iterations | 345 |
| time_elapsed | 1976 |
| total_timesteps | 706560 |
| train/ | |
| approx_kl | 0.0019141638 |
| clip_fraction | 0.00171 |
| clip_range | 0.2 |
| entropy_loss | -0.755 |
| explained_variance | 0.827454 |
| learning_rate | 0.0003 |
| loss | 339 |
| n_updates | 1376 |
| policy_gradient_loss | -0.000673 |
| value_loss | 334 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 349 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 357 |
| iterations | 346 |
| time_elapsed | 1980 |
| total_timesteps | 708608 |
| train/ | |
| approx_kl | 0.0051747523 |
| clip_fraction | 0.0328 |
| clip_range | 0.2 |
| entropy_loss | -0.739 |
| explained_variance | 0.82929915 |
| learning_rate | 0.0003 |
| loss | 33.2 |
| n_updates | 1380 |
| policy_gradient_loss | -0.00454 |
| value_loss | 103 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 349 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 357 |
| iterations | 347 |
| time_elapsed | 1985 |
| total_timesteps | 710656 |
| train/ | |
| approx_kl | 0.004571468 |
| clip_fraction | 0.0311 |
| clip_range | 0.2 |
| entropy_loss | -0.753 |
| explained_variance | 0.9475397 |
| learning_rate | 0.0003 |
| loss | 7.62 |
| n_updates | 1384 |
| policy_gradient_loss | 0.000415 |
| value_loss | 22.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 350 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 358 |
| iterations | 348 |
| time_elapsed | 1990 |
| total_timesteps | 712704 |
| train/ | |
| approx_kl | 0.0030245897 |
| clip_fraction | 0.0153 |
| clip_range | 0.2 |
| entropy_loss | -0.744 |
| explained_variance | 0.9228674 |
| learning_rate | 0.0003 |
| loss | 11.9 |
| n_updates | 1388 |
| policy_gradient_loss | -0.00159 |
| value_loss | 44.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 351 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 358 |
| iterations | 349 |
| time_elapsed | 1994 |
| total_timesteps | 714752 |
| train/ | |
| approx_kl | 0.0032365583 |
| clip_fraction | 0.00671 |
| clip_range | 0.2 |
| entropy_loss | -0.712 |
| explained_variance | 0.6805098 |
| learning_rate | 0.0003 |
| loss | 38.2 |
| n_updates | 1392 |
| policy_gradient_loss | -0.000253 |
| value_loss | 529 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 348 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 358 |
| iterations | 350 |
| time_elapsed | 1999 |
| total_timesteps | 716800 |
| train/ | |
| approx_kl | 0.004640894 |
| clip_fraction | 0.0314 |
| clip_range | 0.2 |
| entropy_loss | -0.745 |
| explained_variance | 0.97506076 |
| learning_rate | 0.0003 |
| loss | 5.55 |
| n_updates | 1396 |
| policy_gradient_loss | -0.000951 |
| value_loss | 17.9 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 348 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 358 |
| iterations | 351 |
| time_elapsed | 2003 |
| total_timesteps | 718848 |
| train/ | |
| approx_kl | 0.00473055 |
| clip_fraction | 0.012 |
| clip_range | 0.2 |
| entropy_loss | -0.748 |
| explained_variance | 0.6963248 |
| learning_rate | 0.0003 |
| loss | 390 |
| n_updates | 1400 |
| policy_gradient_loss | -0.00127 |
| value_loss | 513 |
----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 720000 to videos/step_720000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 358 |
| iterations | 352 |
| time_elapsed | 2012 |
| total_timesteps | 720896 |
| train/ | |
| approx_kl | 0.006914623 |
| clip_fraction | 0.0472 |
| clip_range | 0.2 |
| entropy_loss | -0.794 |
| explained_variance | 0.9809174 |
| learning_rate | 0.0003 |
| loss | 5.29 |
| n_updates | 1404 |
| policy_gradient_loss | 0.000516 |
| value_loss | 15 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 349 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 358 |
| iterations | 353 |
| time_elapsed | 2016 |
| total_timesteps | 722944 |
| train/ | |
| approx_kl | 0.004171028 |
| clip_fraction | 0.0455 |
| clip_range | 0.2 |
| entropy_loss | -0.737 |
| explained_variance | 0.9806354 |
| learning_rate | 0.0003 |
| loss | 8.3 |
| n_updates | 1408 |
| policy_gradient_loss | -0.00246 |
| value_loss | 22.7 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 358 |
| iterations | 354 |
| time_elapsed | 2021 |
| total_timesteps | 724992 |
| train/ | |
| approx_kl | 0.010247223 |
| clip_fraction | 0.0927 |
| clip_range | 0.2 |
| entropy_loss | -0.745 |
| explained_variance | 0.98730606 |
| learning_rate | 0.0003 |
| loss | 6.23 |
| n_updates | 1412 |
| policy_gradient_loss | -0.00651 |
| value_loss | 11.2 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 345 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 358 |
| iterations | 355 |
| time_elapsed | 2026 |
| total_timesteps | 727040 |
| train/ | |
| approx_kl | 0.0034771306 |
| clip_fraction | 0.028 |
| clip_range | 0.2 |
| entropy_loss | -0.781 |
| explained_variance | 0.9864496 |
| learning_rate | 0.0003 |
| loss | 6.34 |
| n_updates | 1416 |
| policy_gradient_loss | -0.00117 |
| value_loss | 11.6 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 350 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 358 |
| iterations | 356 |
| time_elapsed | 2031 |
| total_timesteps | 729088 |
| train/ | |
| approx_kl | 0.008557617 |
| clip_fraction | 0.0564 |
| clip_range | 0.2 |
| entropy_loss | -0.719 |
| explained_variance | 0.89249957 |
| learning_rate | 0.0003 |
| loss | 197 |
| n_updates | 1420 |
| policy_gradient_loss | -0.0022 |
| value_loss | 465 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 349 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 359 |
| iterations | 357 |
| time_elapsed | 2036 |
| total_timesteps | 731136 |
| train/ | |
| approx_kl | 0.0016802303 |
| clip_fraction | 0.0232 |
| clip_range | 0.2 |
| entropy_loss | -0.705 |
| explained_variance | 0.9885143 |
| learning_rate | 0.0003 |
| loss | 3.57 |
| n_updates | 1424 |
| policy_gradient_loss | -0.00089 |
| value_loss | 22.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 359 |
| iterations | 358 |
| time_elapsed | 2040 |
| total_timesteps | 733184 |
| train/ | |
| approx_kl | 0.0032928637 |
| clip_fraction | 0.021 |
| clip_range | 0.2 |
| entropy_loss | -0.749 |
| explained_variance | 0.9825839 |
| learning_rate | 0.0003 |
| loss | 8.54 |
| n_updates | 1428 |
| policy_gradient_loss | -0.000773 |
| value_loss | 16.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 348 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 359 |
| iterations | 359 |
| time_elapsed | 2045 |
| total_timesteps | 735232 |
| train/ | |
| approx_kl | 0.0051062074 |
| clip_fraction | 0.0416 |
| clip_range | 0.2 |
| entropy_loss | -0.751 |
| explained_variance | 0.97738224 |
| learning_rate | 0.0003 |
| loss | 5.96 |
| n_updates | 1432 |
| policy_gradient_loss | -0.00342 |
| value_loss | 16.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 348 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 359 |
| iterations | 360 |
| time_elapsed | 2050 |
| total_timesteps | 737280 |
| train/ | |
| approx_kl | 0.0034594238 |
| clip_fraction | 0.0238 |
| clip_range | 0.2 |
| entropy_loss | -0.765 |
| explained_variance | 0.9656447 |
| learning_rate | 0.0003 |
| loss | 4.4 |
| n_updates | 1436 |
| policy_gradient_loss | -0.00155 |
| value_loss | 16.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 352 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 359 |
| iterations | 361 |
| time_elapsed | 2054 |
| total_timesteps | 739328 |
| train/ | |
| approx_kl | 0.004877569 |
| clip_fraction | 0.0143 |
| clip_range | 0.2 |
| entropy_loss | -0.728 |
| explained_variance | 0.96872383 |
| learning_rate | 0.0003 |
| loss | 10.2 |
| n_updates | 1440 |
| policy_gradient_loss | 0.000135 |
| value_loss | 45.5 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 740000 to videos/step_740000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 351 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 359 |
| iterations | 362 |
| time_elapsed | 2062 |
| total_timesteps | 741376 |
| train/ | |
| approx_kl | 0.0011838108 |
| clip_fraction | 0.00208 |
| clip_range | 0.2 |
| entropy_loss | -0.738 |
| explained_variance | 0.9108796 |
| learning_rate | 0.0003 |
| loss | 13.1 |
| n_updates | 1444 |
| policy_gradient_loss | -0.00015 |
| value_loss | 146 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 350 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 359 |
| iterations | 363 |
| time_elapsed | 2066 |
| total_timesteps | 743424 |
| train/ | |
| approx_kl | 0.0020755494 |
| clip_fraction | 0.0232 |
| clip_range | 0.2 |
| entropy_loss | -0.707 |
| explained_variance | 0.9904421 |
| learning_rate | 0.0003 |
| loss | 3.58 |
| n_updates | 1448 |
| policy_gradient_loss | -0.00127 |
| value_loss | 8.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 359 |
| iterations | 364 |
| time_elapsed | 2070 |
| total_timesteps | 745472 |
| train/ | |
| approx_kl | 0.0045491015 |
| clip_fraction | 0.0651 |
| clip_range | 0.2 |
| entropy_loss | -0.753 |
| explained_variance | 0.99033326 |
| learning_rate | 0.0003 |
| loss | 4.87 |
| n_updates | 1452 |
| policy_gradient_loss | -0.00117 |
| value_loss | 10.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 347 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 360 |
| iterations | 365 |
| time_elapsed | 2076 |
| total_timesteps | 747520 |
| train/ | |
| approx_kl | 0.0012898277 |
| clip_fraction | 0.00171 |
| clip_range | 0.2 |
| entropy_loss | -0.762 |
| explained_variance | 0.63966733 |
| learning_rate | 0.0003 |
| loss | 157 |
| n_updates | 1456 |
| policy_gradient_loss | -8.44e-05 |
| value_loss | 705 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 345 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 360 |
| iterations | 366 |
| time_elapsed | 2080 |
| total_timesteps | 749568 |
| train/ | |
| approx_kl | 0.0038088085 |
| clip_fraction | 0.0195 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.97917044 |
| learning_rate | 0.0003 |
| loss | 3.74 |
| n_updates | 1460 |
| policy_gradient_loss | -0.00216 |
| value_loss | 17.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 346 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 360 |
| iterations | 367 |
| time_elapsed | 2084 |
| total_timesteps | 751616 |
| train/ | |
| approx_kl | 0.0019251582 |
| clip_fraction | 0.0115 |
| clip_range | 0.2 |
| entropy_loss | -0.723 |
| explained_variance | 0.974817 |
| learning_rate | 0.0003 |
| loss | 6.78 |
| n_updates | 1464 |
| policy_gradient_loss | -0.00011 |
| value_loss | 15.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 344 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 360 |
| iterations | 368 |
| time_elapsed | 2089 |
| total_timesteps | 753664 |
| train/ | |
| approx_kl | 0.0040914295 |
| clip_fraction | 0.0295 |
| clip_range | 0.2 |
| entropy_loss | -0.755 |
| explained_variance | 0.9796867 |
| learning_rate | 0.0003 |
| loss | 5.6 |
| n_updates | 1468 |
| policy_gradient_loss | -0.00132 |
| value_loss | 18.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 360 |
| iterations | 369 |
| time_elapsed | 2094 |
| total_timesteps | 755712 |
| train/ | |
| approx_kl | 0.0022560395 |
| clip_fraction | 0.0148 |
| clip_range | 0.2 |
| entropy_loss | -0.679 |
| explained_variance | 0.99035263 |
| learning_rate | 0.0003 |
| loss | 5.57 |
| n_updates | 1472 |
| policy_gradient_loss | -0.00195 |
| value_loss | 11.7 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 361 |
| iterations | 370 |
| time_elapsed | 2098 |
| total_timesteps | 757760 |
| train/ | |
| approx_kl | 0.00487844 |
| clip_fraction | 0.0413 |
| clip_range | 0.2 |
| entropy_loss | -0.712 |
| explained_variance | 0.98996216 |
| learning_rate | 0.0003 |
| loss | 5.79 |
| n_updates | 1476 |
| policy_gradient_loss | -0.00102 |
| value_loss | 12.6 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 361 |
| iterations | 371 |
| time_elapsed | 2103 |
| total_timesteps | 759808 |
| train/ | |
| approx_kl | 0.0031391406 |
| clip_fraction | 0.0255 |
| clip_range | 0.2 |
| entropy_loss | -0.714 |
| explained_variance | 0.98906106 |
| learning_rate | 0.0003 |
| loss | 6.49 |
| n_updates | 1480 |
| policy_gradient_loss | -0.000531 |
| value_loss | 10.2 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 760000 to videos/step_760000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 360 |
| iterations | 372 |
| time_elapsed | 2110 |
| total_timesteps | 761856 |
| train/ | |
| approx_kl | 0.003589118 |
| clip_fraction | 0.022 |
| clip_range | 0.2 |
| entropy_loss | -0.678 |
| explained_variance | 0.99132305 |
| learning_rate | 0.0003 |
| loss | 3.39 |
| n_updates | 1484 |
| policy_gradient_loss | -0.00167 |
| value_loss | 10.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 326 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 361 |
| iterations | 373 |
| time_elapsed | 2115 |
| total_timesteps | 763904 |
| train/ | |
| approx_kl | 0.0013482929 |
| clip_fraction | 0.0022 |
| clip_range | 0.2 |
| entropy_loss | -0.726 |
| explained_variance | 0.7479675 |
| learning_rate | 0.0003 |
| loss | 106 |
| n_updates | 1488 |
| policy_gradient_loss | 7.27e-05 |
| value_loss | 547 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 326 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 361 |
| iterations | 374 |
| time_elapsed | 2119 |
| total_timesteps | 765952 |
| train/ | |
| approx_kl | 0.0033043032 |
| clip_fraction | 0.0271 |
| clip_range | 0.2 |
| entropy_loss | -0.699 |
| explained_variance | 0.9728153 |
| learning_rate | 0.0003 |
| loss | 10.4 |
| n_updates | 1492 |
| policy_gradient_loss | -0.00169 |
| value_loss | 26.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 326 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 361 |
| iterations | 375 |
| time_elapsed | 2124 |
| total_timesteps | 768000 |
| train/ | |
| approx_kl | 0.0050451066 |
| clip_fraction | 0.0496 |
| clip_range | 0.2 |
| entropy_loss | -0.724 |
| explained_variance | 0.98855287 |
| learning_rate | 0.0003 |
| loss | 5 |
| n_updates | 1496 |
| policy_gradient_loss | -0.00312 |
| value_loss | 12.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 332 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 361 |
| iterations | 376 |
| time_elapsed | 2129 |
| total_timesteps | 770048 |
| train/ | |
| approx_kl | 0.0038243588 |
| clip_fraction | 0.041 |
| clip_range | 0.2 |
| entropy_loss | -0.786 |
| explained_variance | 0.9867241 |
| learning_rate | 0.0003 |
| loss | 5.42 |
| n_updates | 1500 |
| policy_gradient_loss | -0.00262 |
| value_loss | 9.08 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 361 |
| iterations | 377 |
| time_elapsed | 2133 |
| total_timesteps | 772096 |
| train/ | |
| approx_kl | 0.006833162 |
| clip_fraction | 0.0525 |
| clip_range | 0.2 |
| entropy_loss | -0.726 |
| explained_variance | 0.9901487 |
| learning_rate | 0.0003 |
| loss | 3.36 |
| n_updates | 1504 |
| policy_gradient_loss | 0.000196 |
| value_loss | 13.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 362 |
| iterations | 378 |
| time_elapsed | 2138 |
| total_timesteps | 774144 |
| train/ | |
| approx_kl | 0.0052339444 |
| clip_fraction | 0.0532 |
| clip_range | 0.2 |
| entropy_loss | -0.674 |
| explained_variance | 0.9896123 |
| learning_rate | 0.0003 |
| loss | 8.55 |
| n_updates | 1508 |
| policy_gradient_loss | -0.00481 |
| value_loss | 12.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 325 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 362 |
| iterations | 379 |
| time_elapsed | 2143 |
| total_timesteps | 776192 |
| train/ | |
| approx_kl | 0.0022930535 |
| clip_fraction | 0.0123 |
| clip_range | 0.2 |
| entropy_loss | -0.752 |
| explained_variance | 0.9912522 |
| learning_rate | 0.0003 |
| loss | 3.84 |
| n_updates | 1512 |
| policy_gradient_loss | -0.00169 |
| value_loss | 9.12 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 362 |
| iterations | 380 |
| time_elapsed | 2147 |
| total_timesteps | 778240 |
| train/ | |
| approx_kl | 0.0005265275 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.725 |
| explained_variance | 0.59920835 |
| learning_rate | 0.0003 |
| loss | 612 |
| n_updates | 1516 |
| policy_gradient_loss | -4.22e-05 |
| value_loss | 900 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 780000 to videos/step_780000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 326 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 362 |
| iterations | 381 |
| time_elapsed | 2154 |
| total_timesteps | 780288 |
| train/ | |
| approx_kl | 0.0070818597 |
| clip_fraction | 0.0513 |
| clip_range | 0.2 |
| entropy_loss | -0.734 |
| explained_variance | 0.97908604 |
| learning_rate | 0.0003 |
| loss | 3.92 |
| n_updates | 1520 |
| policy_gradient_loss | -0.00287 |
| value_loss | 12.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 362 |
| iterations | 382 |
| time_elapsed | 2159 |
| total_timesteps | 782336 |
| train/ | |
| approx_kl | 0.0034916669 |
| clip_fraction | 0.0359 |
| clip_range | 0.2 |
| entropy_loss | -0.744 |
| explained_variance | 0.9869154 |
| learning_rate | 0.0003 |
| loss | 4.11 |
| n_updates | 1524 |
| policy_gradient_loss | -0.000932 |
| value_loss | 12.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 333 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 362 |
| iterations | 383 |
| time_elapsed | 2164 |
| total_timesteps | 784384 |
| train/ | |
| approx_kl | 0.0014772997 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.717 |
| explained_variance | 0.87000984 |
| learning_rate | 0.0003 |
| loss | 51 |
| n_updates | 1528 |
| policy_gradient_loss | -0.000438 |
| value_loss | 453 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 362 |
| iterations | 384 |
| time_elapsed | 2168 |
| total_timesteps | 786432 |
| train/ | |
| approx_kl | 0.0009633456 |
| clip_fraction | 0.000244 |
| clip_range | 0.2 |
| entropy_loss | -0.661 |
| explained_variance | 0.9548095 |
| learning_rate | 0.0003 |
| loss | 11.4 |
| n_updates | 1532 |
| policy_gradient_loss | 2.85e-05 |
| value_loss | 48.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 362 |
| iterations | 385 |
| time_elapsed | 2173 |
| total_timesteps | 788480 |
| train/ | |
| approx_kl | 0.004960408 |
| clip_fraction | 0.0426 |
| clip_range | 0.2 |
| entropy_loss | -0.585 |
| explained_variance | 0.9930148 |
| learning_rate | 0.0003 |
| loss | 4.76 |
| n_updates | 1536 |
| policy_gradient_loss | -0.000272 |
| value_loss | 10.1 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 338 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 362 |
| iterations | 386 |
| time_elapsed | 2178 |
| total_timesteps | 790528 |
| train/ | |
| approx_kl | 0.004900491 |
| clip_fraction | 0.0314 |
| clip_range | 0.2 |
| entropy_loss | -0.757 |
| explained_variance | 0.98923606 |
| learning_rate | 0.0003 |
| loss | 3.26 |
| n_updates | 1540 |
| policy_gradient_loss | -0.00133 |
| value_loss | 7.67 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 337 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 363 |
| iterations | 387 |
| time_elapsed | 2182 |
| total_timesteps | 792576 |
| train/ | |
| approx_kl | 0.0052049486 |
| clip_fraction | 0.0577 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.9914009 |
| learning_rate | 0.0003 |
| loss | 3.69 |
| n_updates | 1544 |
| policy_gradient_loss | -0.0027 |
| value_loss | 7.23 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 341 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 363 |
| iterations | 388 |
| time_elapsed | 2186 |
| total_timesteps | 794624 |
| train/ | |
| approx_kl | 0.00480517 |
| clip_fraction | 0.0385 |
| clip_range | 0.2 |
| entropy_loss | -0.71 |
| explained_variance | 0.9824582 |
| learning_rate | 0.0003 |
| loss | 2.99 |
| n_updates | 1548 |
| policy_gradient_loss | -0.000884 |
| value_loss | 11.9 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 363 |
| iterations | 389 |
| time_elapsed | 2191 |
| total_timesteps | 796672 |
| train/ | |
| approx_kl | 0.004204149 |
| clip_fraction | 0.0369 |
| clip_range | 0.2 |
| entropy_loss | -0.689 |
| explained_variance | 0.9864323 |
| learning_rate | 0.0003 |
| loss | 3.98 |
| n_updates | 1552 |
| policy_gradient_loss | -0.00314 |
| value_loss | 13.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 342 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 363 |
| iterations | 390 |
| time_elapsed | 2196 |
| total_timesteps | 798720 |
| train/ | |
| approx_kl | 0.0023938492 |
| clip_fraction | 0.0295 |
| clip_range | 0.2 |
| entropy_loss | -0.696 |
| explained_variance | 0.98479295 |
| learning_rate | 0.0003 |
| loss | 6.64 |
| n_updates | 1556 |
| policy_gradient_loss | -0.0037 |
| value_loss | 14.8 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 800000 to videos/step_800000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 337 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 363 |
| iterations | 391 |
| time_elapsed | 2203 |
| total_timesteps | 800768 |
| train/ | |
| approx_kl | 0.0036202618 |
| clip_fraction | 0.0228 |
| clip_range | 0.2 |
| entropy_loss | -0.619 |
| explained_variance | 0.7978068 |
| learning_rate | 0.0003 |
| loss | 381 |
| n_updates | 1560 |
| policy_gradient_loss | -0.00206 |
| value_loss | 485 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 343 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 363 |
| iterations | 392 |
| time_elapsed | 2208 |
| total_timesteps | 802816 |
| train/ | |
| approx_kl | 0.0025485407 |
| clip_fraction | 0.00476 |
| clip_range | 0.2 |
| entropy_loss | -0.668 |
| explained_variance | 0.7845533 |
| learning_rate | 0.0003 |
| loss | 298 |
| n_updates | 1564 |
| policy_gradient_loss | -0.00163 |
| value_loss | 487 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 333 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 363 |
| iterations | 393 |
| time_elapsed | 2212 |
| total_timesteps | 804864 |
| train/ | |
| approx_kl | 0.0015882335 |
| clip_fraction | 0.021 |
| clip_range | 0.2 |
| entropy_loss | -0.602 |
| explained_variance | 0.9872067 |
| learning_rate | 0.0003 |
| loss | 3.73 |
| n_updates | 1568 |
| policy_gradient_loss | 0.000632 |
| value_loss | 15 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 331 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 363 |
| iterations | 394 |
| time_elapsed | 2217 |
| total_timesteps | 806912 |
| train/ | |
| approx_kl | 0.0032635583 |
| clip_fraction | 0.0211 |
| clip_range | 0.2 |
| entropy_loss | -0.681 |
| explained_variance | 0.9882224 |
| learning_rate | 0.0003 |
| loss | 4.62 |
| n_updates | 1572 |
| policy_gradient_loss | -0.000157 |
| value_loss | 16.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 329 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 364 |
| iterations | 395 |
| time_elapsed | 2221 |
| total_timesteps | 808960 |
| train/ | |
| approx_kl | 0.0033263047 |
| clip_fraction | 0.0256 |
| clip_range | 0.2 |
| entropy_loss | -0.677 |
| explained_variance | 0.99258417 |
| learning_rate | 0.0003 |
| loss | 4.26 |
| n_updates | 1576 |
| policy_gradient_loss | -0.00111 |
| value_loss | 11 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 364 |
| iterations | 396 |
| time_elapsed | 2225 |
| total_timesteps | 811008 |
| train/ | |
| approx_kl | 0.002015676 |
| clip_fraction | 0.0275 |
| clip_range | 0.2 |
| entropy_loss | -0.694 |
| explained_variance | 0.99022436 |
| learning_rate | 0.0003 |
| loss | 3.49 |
| n_updates | 1580 |
| policy_gradient_loss | -0.00166 |
| value_loss | 9.77 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 331 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 364 |
| iterations | 397 |
| time_elapsed | 2230 |
| total_timesteps | 813056 |
| train/ | |
| approx_kl | 0.005186012 |
| clip_fraction | 0.047 |
| clip_range | 0.2 |
| entropy_loss | -0.664 |
| explained_variance | 0.9919727 |
| learning_rate | 0.0003 |
| loss | 8.86 |
| n_updates | 1584 |
| policy_gradient_loss | -0.00275 |
| value_loss | 10.8 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 320 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 364 |
| iterations | 398 |
| time_elapsed | 2235 |
| total_timesteps | 815104 |
| train/ | |
| approx_kl | 0.002323662 |
| clip_fraction | 0.021 |
| clip_range | 0.2 |
| entropy_loss | -0.524 |
| explained_variance | 0.9929954 |
| learning_rate | 0.0003 |
| loss | 2.02 |
| n_updates | 1588 |
| policy_gradient_loss | -0.000645 |
| value_loss | 5.5 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 310 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 364 |
| iterations | 399 |
| time_elapsed | 2239 |
| total_timesteps | 817152 |
| train/ | |
| approx_kl | 0.00091226946 |
| clip_fraction | 0.00378 |
| clip_range | 0.2 |
| entropy_loss | -0.749 |
| explained_variance | 0.6166959 |
| learning_rate | 0.0003 |
| loss | 700 |
| n_updates | 1592 |
| policy_gradient_loss | -5.11e-05 |
| value_loss | 1.3e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 306 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 365 |
| iterations | 400 |
| time_elapsed | 2244 |
| total_timesteps | 819200 |
| train/ | |
| approx_kl | 0.0037276004 |
| clip_fraction | 0.0112 |
| clip_range | 0.2 |
| entropy_loss | -0.645 |
| explained_variance | 0.8215059 |
| learning_rate | 0.0003 |
| loss | 167 |
| n_updates | 1596 |
| policy_gradient_loss | -0.0009 |
| value_loss | 439 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 820000 to videos/step_820000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 310 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 364 |
| iterations | 401 |
| time_elapsed | 2250 |
| total_timesteps | 821248 |
| train/ | |
| approx_kl | 0.0011954146 |
| clip_fraction | 0.000977 |
| clip_range | 0.2 |
| entropy_loss | -0.691 |
| explained_variance | 0.73219526 |
| learning_rate | 0.0003 |
| loss | 407 |
| n_updates | 1600 |
| policy_gradient_loss | -0.000444 |
| value_loss | 790 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 365 |
| iterations | 402 |
| time_elapsed | 2255 |
| total_timesteps | 823296 |
| train/ | |
| approx_kl | 0.0019031337 |
| clip_fraction | 0.00269 |
| clip_range | 0.2 |
| entropy_loss | -0.738 |
| explained_variance | 0.81006956 |
| learning_rate | 0.0003 |
| loss | 149 |
| n_updates | 1604 |
| policy_gradient_loss | -0.000788 |
| value_loss | 523 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 365 |
| iterations | 403 |
| time_elapsed | 2259 |
| total_timesteps | 825344 |
| train/ | |
| approx_kl | 0.0010521375 |
| clip_fraction | 0.0175 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.9413286 |
| learning_rate | 0.0003 |
| loss | 8.39 |
| n_updates | 1608 |
| policy_gradient_loss | -0.00061 |
| value_loss | 51.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 293 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 365 |
| iterations | 404 |
| time_elapsed | 2263 |
| total_timesteps | 827392 |
| train/ | |
| approx_kl | 0.0034499918 |
| clip_fraction | 0.0198 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.93995535 |
| learning_rate | 0.0003 |
| loss | 12.5 |
| n_updates | 1612 |
| policy_gradient_loss | -0.0016 |
| value_loss | 58.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 365 |
| iterations | 405 |
| time_elapsed | 2268 |
| total_timesteps | 829440 |
| train/ | |
| approx_kl | 0.0011214647 |
| clip_fraction | 0.00232 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.6989145 |
| learning_rate | 0.0003 |
| loss | 276 |
| n_updates | 1616 |
| policy_gradient_loss | -0.000713 |
| value_loss | 757 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 365 |
| iterations | 406 |
| time_elapsed | 2272 |
| total_timesteps | 831488 |
| train/ | |
| approx_kl | 0.00084153155 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.698 |
| explained_variance | 0.7804512 |
| learning_rate | 0.0003 |
| loss | 99.6 |
| n_updates | 1620 |
| policy_gradient_loss | -0.000205 |
| value_loss | 408 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 366 |
| iterations | 407 |
| time_elapsed | 2277 |
| total_timesteps | 833536 |
| train/ | |
| approx_kl | 0.0011622899 |
| clip_fraction | 0.000244 |
| clip_range | 0.2 |
| entropy_loss | -0.706 |
| explained_variance | 0.63112533 |
| learning_rate | 0.0003 |
| loss | 458 |
| n_updates | 1624 |
| policy_gradient_loss | -0.000809 |
| value_loss | 931 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 366 |
| iterations | 408 |
| time_elapsed | 2282 |
| total_timesteps | 835584 |
| train/ | |
| approx_kl | 0.0042520855 |
| clip_fraction | 0.0232 |
| clip_range | 0.2 |
| entropy_loss | -0.624 |
| explained_variance | 0.92768407 |
| learning_rate | 0.0003 |
| loss | 12.3 |
| n_updates | 1628 |
| policy_gradient_loss | -0.00225 |
| value_loss | 75.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 277 |
| ep_rew_mean | 231 |
| time/ | |
| fps | 366 |
| iterations | 409 |
| time_elapsed | 2286 |
| total_timesteps | 837632 |
| train/ | |
| approx_kl | 0.0056965393 |
| clip_fraction | 0.0269 |
| clip_range | 0.2 |
| entropy_loss | -0.7 |
| explained_variance | 0.9487153 |
| learning_rate | 0.0003 |
| loss | 4.16 |
| n_updates | 1632 |
| policy_gradient_loss | -0.000547 |
| value_loss | 39.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 231 |
| time/ | |
| fps | 366 |
| iterations | 410 |
| time_elapsed | 2290 |
| total_timesteps | 839680 |
| train/ | |
| approx_kl | 0.0010918728 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.699 |
| explained_variance | 0.6507932 |
| learning_rate | 0.0003 |
| loss | 92.7 |
| n_updates | 1636 |
| policy_gradient_loss | -6.64e-05 |
| value_loss | 965 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 840000 to videos/step_840000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 279 |
| ep_rew_mean | 233 |
| time/ | |
| fps | 366 |
| iterations | 411 |
| time_elapsed | 2298 |
| total_timesteps | 841728 |
| train/ | |
| approx_kl | 0.011142266 |
| clip_fraction | 0.128 |
| clip_range | 0.2 |
| entropy_loss | -0.656 |
| explained_variance | 0.98763245 |
| learning_rate | 0.0003 |
| loss | 5.82 |
| n_updates | 1640 |
| policy_gradient_loss | -0.00154 |
| value_loss | 19.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 366 |
| iterations | 412 |
| time_elapsed | 2302 |
| total_timesteps | 843776 |
| train/ | |
| approx_kl | 0.0031822242 |
| clip_fraction | 0.0284 |
| clip_range | 0.2 |
| entropy_loss | -0.613 |
| explained_variance | 0.8442122 |
| learning_rate | 0.0003 |
| loss | 77.8 |
| n_updates | 1644 |
| policy_gradient_loss | -0.000242 |
| value_loss | 311 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 279 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 366 |
| iterations | 413 |
| time_elapsed | 2307 |
| total_timesteps | 845824 |
| train/ | |
| approx_kl | 0.004072821 |
| clip_fraction | 0.0369 |
| clip_range | 0.2 |
| entropy_loss | -0.678 |
| explained_variance | 0.9872698 |
| learning_rate | 0.0003 |
| loss | 8.46 |
| n_updates | 1648 |
| policy_gradient_loss | -0.0014 |
| value_loss | 16.3 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 279 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 366 |
| iterations | 414 |
| time_elapsed | 2311 |
| total_timesteps | 847872 |
| train/ | |
| approx_kl | 0.003502687 |
| clip_fraction | 0.0353 |
| clip_range | 0.2 |
| entropy_loss | -0.659 |
| explained_variance | 0.7242153 |
| learning_rate | 0.0003 |
| loss | 212 |
| n_updates | 1652 |
| policy_gradient_loss | -0.00129 |
| value_loss | 854 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 366 |
| iterations | 415 |
| time_elapsed | 2316 |
| total_timesteps | 849920 |
| train/ | |
| approx_kl | 0.0005636704 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.702 |
| explained_variance | 0.7444695 |
| learning_rate | 0.0003 |
| loss | 63 |
| n_updates | 1656 |
| policy_gradient_loss | -0.000525 |
| value_loss | 838 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 233 |
| time/ | |
| fps | 367 |
| iterations | 416 |
| time_elapsed | 2320 |
| total_timesteps | 851968 |
| train/ | |
| approx_kl | 0.004141529 |
| clip_fraction | 0.0343 |
| clip_range | 0.2 |
| entropy_loss | -0.684 |
| explained_variance | 0.9638596 |
| learning_rate | 0.0003 |
| loss | 9.63 |
| n_updates | 1660 |
| policy_gradient_loss | -0.00271 |
| value_loss | 47.8 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 367 |
| iterations | 417 |
| time_elapsed | 2325 |
| total_timesteps | 854016 |
| train/ | |
| approx_kl | 0.00096288853 |
| clip_fraction | 0.00183 |
| clip_range | 0.2 |
| entropy_loss | -0.689 |
| explained_variance | 0.8831927 |
| learning_rate | 0.0003 |
| loss | 118 |
| n_updates | 1664 |
| policy_gradient_loss | -0.000115 |
| value_loss | 337 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 367 |
| iterations | 418 |
| time_elapsed | 2329 |
| total_timesteps | 856064 |
| train/ | |
| approx_kl | 0.0025402731 |
| clip_fraction | 0.0187 |
| clip_range | 0.2 |
| entropy_loss | -0.641 |
| explained_variance | 0.93007135 |
| learning_rate | 0.0003 |
| loss | 14.9 |
| n_updates | 1668 |
| policy_gradient_loss | -0.00265 |
| value_loss | 50.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 277 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 367 |
| iterations | 419 |
| time_elapsed | 2334 |
| total_timesteps | 858112 |
| train/ | |
| approx_kl | 0.0018682466 |
| clip_fraction | 0.0217 |
| clip_range | 0.2 |
| entropy_loss | -0.704 |
| explained_variance | 0.77666974 |
| learning_rate | 0.0003 |
| loss | 80.7 |
| n_updates | 1672 |
| policy_gradient_loss | -0.000502 |
| value_loss | 450 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 860000 to videos/step_860000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 275 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 367 |
| iterations | 420 |
| time_elapsed | 2339 |
| total_timesteps | 860160 |
| train/ | |
| approx_kl | 0.003333956 |
| clip_fraction | 0.019 |
| clip_range | 0.2 |
| entropy_loss | -0.64 |
| explained_variance | 0.95221066 |
| learning_rate | 0.0003 |
| loss | 7.08 |
| n_updates | 1676 |
| policy_gradient_loss | -0.000656 |
| value_loss | 37.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 273 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 367 |
| iterations | 421 |
| time_elapsed | 2343 |
| total_timesteps | 862208 |
| train/ | |
| approx_kl | 0.0036402745 |
| clip_fraction | 0.0444 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.679049 |
| learning_rate | 0.0003 |
| loss | 200 |
| n_updates | 1680 |
| policy_gradient_loss | -0.000796 |
| value_loss | 717 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 367 |
| iterations | 422 |
| time_elapsed | 2348 |
| total_timesteps | 864256 |
| train/ | |
| approx_kl | 0.0005143841 |
| clip_fraction | 0.0022 |
| clip_range | 0.2 |
| entropy_loss | -0.674 |
| explained_variance | 0.78008366 |
| learning_rate | 0.0003 |
| loss | 298 |
| n_updates | 1684 |
| policy_gradient_loss | -0.00121 |
| value_loss | 575 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 275 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 368 |
| iterations | 423 |
| time_elapsed | 2353 |
| total_timesteps | 866304 |
| train/ | |
| approx_kl | 0.0018616992 |
| clip_fraction | 0.00122 |
| clip_range | 0.2 |
| entropy_loss | -0.691 |
| explained_variance | 0.8606976 |
| learning_rate | 0.0003 |
| loss | 132 |
| n_updates | 1688 |
| policy_gradient_loss | -0.00162 |
| value_loss | 374 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 368 |
| iterations | 424 |
| time_elapsed | 2357 |
| total_timesteps | 868352 |
| train/ | |
| approx_kl | 0.0008798698 |
| clip_fraction | 0.00195 |
| clip_range | 0.2 |
| entropy_loss | -0.682 |
| explained_variance | 0.7262344 |
| learning_rate | 0.0003 |
| loss | 424 |
| n_updates | 1692 |
| policy_gradient_loss | -0.00011 |
| value_loss | 794 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 228 |
| time/ | |
| fps | 368 |
| iterations | 425 |
| time_elapsed | 2362 |
| total_timesteps | 870400 |
| train/ | |
| approx_kl | 0.0015050112 |
| clip_fraction | 0.000977 |
| clip_range | 0.2 |
| entropy_loss | -0.693 |
| explained_variance | 0.862304 |
| learning_rate | 0.0003 |
| loss | 164 |
| n_updates | 1696 |
| policy_gradient_loss | -0.00105 |
| value_loss | 384 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 273 |
| ep_rew_mean | 233 |
| time/ | |
| fps | 368 |
| iterations | 426 |
| time_elapsed | 2366 |
| total_timesteps | 872448 |
| train/ | |
| approx_kl | 0.0019155119 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.741 |
| explained_variance | 0.7174738 |
| learning_rate | 0.0003 |
| loss | 620 |
| n_updates | 1700 |
| policy_gradient_loss | -0.00151 |
| value_loss | 1.27e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 368 |
| iterations | 427 |
| time_elapsed | 2370 |
| total_timesteps | 874496 |
| train/ | |
| approx_kl | 0.0011322948 |
| clip_fraction | 0.00159 |
| clip_range | 0.2 |
| entropy_loss | -0.606 |
| explained_variance | 0.73990023 |
| learning_rate | 0.0003 |
| loss | 78.2 |
| n_updates | 1704 |
| policy_gradient_loss | -0.000241 |
| value_loss | 227 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 368 |
| iterations | 428 |
| time_elapsed | 2375 |
| total_timesteps | 876544 |
| train/ | |
| approx_kl | 0.004325562 |
| clip_fraction | 0.019 |
| clip_range | 0.2 |
| entropy_loss | -0.698 |
| explained_variance | 0.8544205 |
| learning_rate | 0.0003 |
| loss | 34.3 |
| n_updates | 1708 |
| policy_gradient_loss | -0.00238 |
| value_loss | 290 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 369 |
| iterations | 429 |
| time_elapsed | 2379 |
| total_timesteps | 878592 |
| train/ | |
| approx_kl | 0.010483902 |
| clip_fraction | 0.106 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.9875209 |
| learning_rate | 0.0003 |
| loss | 5.23 |
| n_updates | 1712 |
| policy_gradient_loss | -0.00232 |
| value_loss | 30.3 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 880000 to videos/step_880000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 240 |
| time/ | |
| fps | 368 |
| iterations | 430 |
| time_elapsed | 2392 |
| total_timesteps | 880640 |
| train/ | |
| approx_kl | 0.006896326 |
| clip_fraction | 0.0593 |
| clip_range | 0.2 |
| entropy_loss | -0.677 |
| explained_variance | 0.9500431 |
| learning_rate | 0.0003 |
| loss | 7.92 |
| n_updates | 1716 |
| policy_gradient_loss | -0.00286 |
| value_loss | 47.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 368 |
| iterations | 431 |
| time_elapsed | 2396 |
| total_timesteps | 882688 |
| train/ | |
| approx_kl | 0.0024488328 |
| clip_fraction | 0.021 |
| clip_range | 0.2 |
| entropy_loss | -0.686 |
| explained_variance | 0.964355 |
| learning_rate | 0.0003 |
| loss | 9.93 |
| n_updates | 1720 |
| policy_gradient_loss | 0.000525 |
| value_loss | 28.8 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 368 |
| iterations | 432 |
| time_elapsed | 2401 |
| total_timesteps | 884736 |
| train/ | |
| approx_kl | 0.00087326515 |
| clip_fraction | 0.0033 |
| clip_range | 0.2 |
| entropy_loss | -0.69 |
| explained_variance | 0.65208507 |
| learning_rate | 0.0003 |
| loss | 328 |
| n_updates | 1724 |
| policy_gradient_loss | -0.000145 |
| value_loss | 874 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 368 |
| iterations | 433 |
| time_elapsed | 2405 |
| total_timesteps | 886784 |
| train/ | |
| approx_kl | 0.006124866 |
| clip_fraction | 0.0507 |
| clip_range | 0.2 |
| entropy_loss | -0.656 |
| explained_variance | 0.96904325 |
| learning_rate | 0.0003 |
| loss | 15.3 |
| n_updates | 1728 |
| policy_gradient_loss | -0.00105 |
| value_loss | 32.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 368 |
| iterations | 434 |
| time_elapsed | 2410 |
| total_timesteps | 888832 |
| train/ | |
| approx_kl | 0.0043716887 |
| clip_fraction | 0.0879 |
| clip_range | 0.2 |
| entropy_loss | -0.609 |
| explained_variance | 0.79097766 |
| learning_rate | 0.0003 |
| loss | 205 |
| n_updates | 1732 |
| policy_gradient_loss | -0.00163 |
| value_loss | 446 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 286 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 368 |
| iterations | 435 |
| time_elapsed | 2414 |
| total_timesteps | 890880 |
| train/ | |
| approx_kl | 0.0018961537 |
| clip_fraction | 0.00403 |
| clip_range | 0.2 |
| entropy_loss | -0.704 |
| explained_variance | 0.7715373 |
| learning_rate | 0.0003 |
| loss | 427 |
| n_updates | 1736 |
| policy_gradient_loss | -0.000353 |
| value_loss | 481 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 293 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 369 |
| iterations | 436 |
| time_elapsed | 2418 |
| total_timesteps | 892928 |
| train/ | |
| approx_kl | 0.0025808054 |
| clip_fraction | 0.0177 |
| clip_range | 0.2 |
| entropy_loss | -0.7 |
| explained_variance | 0.7777089 |
| learning_rate | 0.0003 |
| loss | 565 |
| n_updates | 1740 |
| policy_gradient_loss | -0.00173 |
| value_loss | 467 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 369 |
| iterations | 437 |
| time_elapsed | 2423 |
| total_timesteps | 894976 |
| train/ | |
| approx_kl | 0.008121224 |
| clip_fraction | 0.0751 |
| clip_range | 0.2 |
| entropy_loss | -0.641 |
| explained_variance | 0.97027516 |
| learning_rate | 0.0003 |
| loss | 4.72 |
| n_updates | 1744 |
| policy_gradient_loss | 0.000217 |
| value_loss | 31.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 369 |
| iterations | 438 |
| time_elapsed | 2428 |
| total_timesteps | 897024 |
| train/ | |
| approx_kl | 0.0012593564 |
| clip_fraction | 0.00952 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.765888 |
| learning_rate | 0.0003 |
| loss | 459 |
| n_updates | 1748 |
| policy_gradient_loss | 0.000425 |
| value_loss | 505 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 369 |
| iterations | 439 |
| time_elapsed | 2432 |
| total_timesteps | 899072 |
| train/ | |
| approx_kl | 0.007986353 |
| clip_fraction | 0.053 |
| clip_range | 0.2 |
| entropy_loss | -0.628 |
| explained_variance | 0.9601669 |
| learning_rate | 0.0003 |
| loss | 13.1 |
| n_updates | 1752 |
| policy_gradient_loss | -0.00217 |
| value_loss | 35.7 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 900000 to videos/step_900000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 369 |
| iterations | 440 |
| time_elapsed | 2439 |
| total_timesteps | 901120 |
| train/ | |
| approx_kl | 0.019567067 |
| clip_fraction | 0.0708 |
| clip_range | 0.2 |
| entropy_loss | -0.613 |
| explained_variance | 0.98732454 |
| learning_rate | 0.0003 |
| loss | 13.7 |
| n_updates | 1756 |
| policy_gradient_loss | -0.00329 |
| value_loss | 22.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 369 |
| iterations | 441 |
| time_elapsed | 2444 |
| total_timesteps | 903168 |
| train/ | |
| approx_kl | 0.0070142383 |
| clip_fraction | 0.067 |
| clip_range | 0.2 |
| entropy_loss | -0.651 |
| explained_variance | 0.9906675 |
| learning_rate | 0.0003 |
| loss | 3.14 |
| n_updates | 1760 |
| policy_gradient_loss | -0.000503 |
| value_loss | 9.92 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 369 |
| iterations | 442 |
| time_elapsed | 2448 |
| total_timesteps | 905216 |
| train/ | |
| approx_kl | 0.0057299607 |
| clip_fraction | 0.0626 |
| clip_range | 0.2 |
| entropy_loss | -0.667 |
| explained_variance | 0.9943433 |
| learning_rate | 0.0003 |
| loss | 3.9 |
| n_updates | 1764 |
| policy_gradient_loss | -0.000181 |
| value_loss | 8.75 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 296 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 369 |
| iterations | 443 |
| time_elapsed | 2453 |
| total_timesteps | 907264 |
| train/ | |
| approx_kl | 0.0049866615 |
| clip_fraction | 0.118 |
| clip_range | 0.2 |
| entropy_loss | -0.682 |
| explained_variance | 0.6882183 |
| learning_rate | 0.0003 |
| loss | 188 |
| n_updates | 1768 |
| policy_gradient_loss | 0.00188 |
| value_loss | 917 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 370 |
| iterations | 444 |
| time_elapsed | 2457 |
| total_timesteps | 909312 |
| train/ | |
| approx_kl | 0.0015868344 |
| clip_fraction | 0.0033 |
| clip_range | 0.2 |
| entropy_loss | -0.648 |
| explained_variance | 0.91273344 |
| learning_rate | 0.0003 |
| loss | 140 |
| n_updates | 1772 |
| policy_gradient_loss | -0.000918 |
| value_loss | 347 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 296 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 370 |
| iterations | 445 |
| time_elapsed | 2461 |
| total_timesteps | 911360 |
| train/ | |
| approx_kl | 0.0047932565 |
| clip_fraction | 0.0269 |
| clip_range | 0.2 |
| entropy_loss | -0.658 |
| explained_variance | 0.97781986 |
| learning_rate | 0.0003 |
| loss | 4.89 |
| n_updates | 1776 |
| policy_gradient_loss | -0.00166 |
| value_loss | 20.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 370 |
| iterations | 446 |
| time_elapsed | 2466 |
| total_timesteps | 913408 |
| train/ | |
| approx_kl | 0.0024213353 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -0.57 |
| explained_variance | 0.84722626 |
| learning_rate | 0.0003 |
| loss | 361 |
| n_updates | 1780 |
| policy_gradient_loss | -0.000187 |
| value_loss | 434 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 370 |
| iterations | 447 |
| time_elapsed | 2470 |
| total_timesteps | 915456 |
| train/ | |
| approx_kl | 0.00022902657 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.712 |
| explained_variance | 0.7744173 |
| learning_rate | 0.0003 |
| loss | 139 |
| n_updates | 1784 |
| policy_gradient_loss | -0.000159 |
| value_loss | 574 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 370 |
| iterations | 448 |
| time_elapsed | 2475 |
| total_timesteps | 917504 |
| train/ | |
| approx_kl | 0.004289898 |
| clip_fraction | 0.0334 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.9524715 |
| learning_rate | 0.0003 |
| loss | 15.2 |
| n_updates | 1788 |
| policy_gradient_loss | -0.0028 |
| value_loss | 35.9 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 370 |
| iterations | 449 |
| time_elapsed | 2479 |
| total_timesteps | 919552 |
| train/ | |
| approx_kl | 0.004489936 |
| clip_fraction | 0.0476 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.98990196 |
| learning_rate | 0.0003 |
| loss | 5.04 |
| n_updates | 1792 |
| policy_gradient_loss | -0.00262 |
| value_loss | 11.2 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 920000 to videos/step_920000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 370 |
| iterations | 450 |
| time_elapsed | 2486 |
| total_timesteps | 921600 |
| train/ | |
| approx_kl | 0.0032501344 |
| clip_fraction | 0.0269 |
| clip_range | 0.2 |
| entropy_loss | -0.717 |
| explained_variance | 0.7940023 |
| learning_rate | 0.0003 |
| loss | 41.6 |
| n_updates | 1796 |
| policy_gradient_loss | -0.000985 |
| value_loss | 449 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 294 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 370 |
| iterations | 451 |
| time_elapsed | 2491 |
| total_timesteps | 923648 |
| train/ | |
| approx_kl | 0.007891713 |
| clip_fraction | 0.0519 |
| clip_range | 0.2 |
| entropy_loss | -0.664 |
| explained_variance | 0.9537682 |
| learning_rate | 0.0003 |
| loss | 4.54 |
| n_updates | 1800 |
| policy_gradient_loss | -0.00188 |
| value_loss | 28.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 293 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 370 |
| iterations | 452 |
| time_elapsed | 2495 |
| total_timesteps | 925696 |
| train/ | |
| approx_kl | 0.0062941625 |
| clip_fraction | 0.0228 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.7972037 |
| learning_rate | 0.0003 |
| loss | 404 |
| n_updates | 1804 |
| policy_gradient_loss | -0.00257 |
| value_loss | 472 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 371 |
| iterations | 453 |
| time_elapsed | 2499 |
| total_timesteps | 927744 |
| train/ | |
| approx_kl | 0.00560656 |
| clip_fraction | 0.058 |
| clip_range | 0.2 |
| entropy_loss | -0.669 |
| explained_variance | 0.98453695 |
| learning_rate | 0.0003 |
| loss | 3.24 |
| n_updates | 1808 |
| policy_gradient_loss | -0.00339 |
| value_loss | 15.6 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 289 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 371 |
| iterations | 454 |
| time_elapsed | 2504 |
| total_timesteps | 929792 |
| train/ | |
| approx_kl | 0.0055565136 |
| clip_fraction | 0.0355 |
| clip_range | 0.2 |
| entropy_loss | -0.687 |
| explained_variance | 0.99140865 |
| learning_rate | 0.0003 |
| loss | 4.33 |
| n_updates | 1812 |
| policy_gradient_loss | -0.000768 |
| value_loss | 11.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 371 |
| iterations | 455 |
| time_elapsed | 2508 |
| total_timesteps | 931840 |
| train/ | |
| approx_kl | 0.0033552635 |
| clip_fraction | 0.0293 |
| clip_range | 0.2 |
| entropy_loss | -0.636 |
| explained_variance | 0.97931504 |
| learning_rate | 0.0003 |
| loss | 7.74 |
| n_updates | 1816 |
| policy_gradient_loss | -0.000373 |
| value_loss | 17.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 371 |
| iterations | 456 |
| time_elapsed | 2512 |
| total_timesteps | 933888 |
| train/ | |
| approx_kl | 0.0017997008 |
| clip_fraction | 0.00439 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.82325125 |
| learning_rate | 0.0003 |
| loss | 164 |
| n_updates | 1820 |
| policy_gradient_loss | -0.000168 |
| value_loss | 373 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 371 |
| iterations | 457 |
| time_elapsed | 2517 |
| total_timesteps | 935936 |
| train/ | |
| approx_kl | 0.008561261 |
| clip_fraction | 0.0504 |
| clip_range | 0.2 |
| entropy_loss | -0.687 |
| explained_variance | 0.9820781 |
| learning_rate | 0.0003 |
| loss | 5.92 |
| n_updates | 1824 |
| policy_gradient_loss | -0.000997 |
| value_loss | 19.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 371 |
| iterations | 458 |
| time_elapsed | 2521 |
| total_timesteps | 937984 |
| train/ | |
| approx_kl | 0.0032618854 |
| clip_fraction | 0.0267 |
| clip_range | 0.2 |
| entropy_loss | -0.681 |
| explained_variance | 0.63220274 |
| learning_rate | 0.0003 |
| loss | 198 |
| n_updates | 1828 |
| policy_gradient_loss | -0.00344 |
| value_loss | 1.01e+03 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 940000 to videos/step_940000.mp4
----------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 371 |
| iterations | 459 |
| time_elapsed | 2528 |
| total_timesteps | 940032 |
| train/ | |
| approx_kl | 0.00197126 |
| clip_fraction | 0.000854 |
| clip_range | 0.2 |
| entropy_loss | -0.634 |
| explained_variance | 0.8263209 |
| learning_rate | 0.0003 |
| loss | 64.6 |
| n_updates | 1832 |
| policy_gradient_loss | -0.00163 |
| value_loss | 401 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 371 |
| iterations | 460 |
| time_elapsed | 2532 |
| total_timesteps | 942080 |
| train/ | |
| approx_kl | 0.0028091434 |
| clip_fraction | 0.037 |
| clip_range | 0.2 |
| entropy_loss | -0.63 |
| explained_variance | 0.9496259 |
| learning_rate | 0.0003 |
| loss | 9.55 |
| n_updates | 1836 |
| policy_gradient_loss | -0.00179 |
| value_loss | 33.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 372 |
| iterations | 461 |
| time_elapsed | 2536 |
| total_timesteps | 944128 |
| train/ | |
| approx_kl | 0.005372977 |
| clip_fraction | 0.0378 |
| clip_range | 0.2 |
| entropy_loss | -0.618 |
| explained_variance | 0.96333754 |
| learning_rate | 0.0003 |
| loss | 4.35 |
| n_updates | 1840 |
| policy_gradient_loss | -0.00251 |
| value_loss | 21.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 278 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 372 |
| iterations | 462 |
| time_elapsed | 2541 |
| total_timesteps | 946176 |
| train/ | |
| approx_kl | 0.0032834513 |
| clip_fraction | 0.0327 |
| clip_range | 0.2 |
| entropy_loss | -0.636 |
| explained_variance | 0.97733593 |
| learning_rate | 0.0003 |
| loss | 5.25 |
| n_updates | 1844 |
| policy_gradient_loss | -0.00101 |
| value_loss | 21.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 278 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 372 |
| iterations | 463 |
| time_elapsed | 2545 |
| total_timesteps | 948224 |
| train/ | |
| approx_kl | 0.0016871546 |
| clip_fraction | 0.00903 |
| clip_range | 0.2 |
| entropy_loss | -0.761 |
| explained_variance | 0.5572896 |
| learning_rate | 0.0003 |
| loss | 843 |
| n_updates | 1848 |
| policy_gradient_loss | -0.000503 |
| value_loss | 1.43e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 276 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 372 |
| iterations | 464 |
| time_elapsed | 2549 |
| total_timesteps | 950272 |
| train/ | |
| approx_kl | 0.0016734393 |
| clip_fraction | 0.0151 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.7902692 |
| learning_rate | 0.0003 |
| loss | 158 |
| n_updates | 1852 |
| policy_gradient_loss | -2.83e-05 |
| value_loss | 398 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 372 |
| iterations | 465 |
| time_elapsed | 2554 |
| total_timesteps | 952320 |
| train/ | |
| approx_kl | 0.005536101 |
| clip_fraction | 0.0385 |
| clip_range | 0.2 |
| entropy_loss | -0.724 |
| explained_variance | 0.97820187 |
| learning_rate | 0.0003 |
| loss | 3.95 |
| n_updates | 1856 |
| policy_gradient_loss | -0.000313 |
| value_loss | 17.9 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 240 |
| time/ | |
| fps | 372 |
| iterations | 466 |
| time_elapsed | 2558 |
| total_timesteps | 954368 |
| train/ | |
| approx_kl | 0.002762457 |
| clip_fraction | 0.0281 |
| clip_range | 0.2 |
| entropy_loss | -0.695 |
| explained_variance | 0.6991345 |
| learning_rate | 0.0003 |
| loss | 235 |
| n_updates | 1860 |
| policy_gradient_loss | -5.67e-05 |
| value_loss | 431 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 373 |
| iterations | 467 |
| time_elapsed | 2562 |
| total_timesteps | 956416 |
| train/ | |
| approx_kl | 0.0008579525 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.711 |
| explained_variance | 0.6597846 |
| learning_rate | 0.0003 |
| loss | 230 |
| n_updates | 1864 |
| policy_gradient_loss | -0.0013 |
| value_loss | 736 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 231 |
| time/ | |
| fps | 373 |
| iterations | 468 |
| time_elapsed | 2568 |
| total_timesteps | 958464 |
| train/ | |
| approx_kl | 0.0035703024 |
| clip_fraction | 0.0204 |
| clip_range | 0.2 |
| entropy_loss | -0.716 |
| explained_variance | 0.8212256 |
| learning_rate | 0.0003 |
| loss | 110 |
| n_updates | 1868 |
| policy_gradient_loss | -0.000233 |
| value_loss | 358 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 960000 to videos/step_960000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 233 |
| time/ | |
| fps | 373 |
| iterations | 469 |
| time_elapsed | 2574 |
| total_timesteps | 960512 |
| train/ | |
| approx_kl | 0.0011983886 |
| clip_fraction | 0.000732 |
| clip_range | 0.2 |
| entropy_loss | -0.704 |
| explained_variance | 0.8537992 |
| learning_rate | 0.0003 |
| loss | 104 |
| n_updates | 1872 |
| policy_gradient_loss | -0.000816 |
| value_loss | 330 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 373 |
| iterations | 470 |
| time_elapsed | 2580 |
| total_timesteps | 962560 |
| train/ | |
| approx_kl | 0.0001718059 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.713 |
| explained_variance | 0.42352766 |
| learning_rate | 0.0003 |
| loss | 113 |
| n_updates | 1876 |
| policy_gradient_loss | 1.57e-05 |
| value_loss | 601 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 373 |
| iterations | 471 |
| time_elapsed | 2584 |
| total_timesteps | 964608 |
| train/ | |
| approx_kl | 0.003943193 |
| clip_fraction | 0.0118 |
| clip_range | 0.2 |
| entropy_loss | -0.622 |
| explained_variance | 0.8275503 |
| learning_rate | 0.0003 |
| loss | 20.9 |
| n_updates | 1880 |
| policy_gradient_loss | -0.00206 |
| value_loss | 116 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 373 |
| iterations | 472 |
| time_elapsed | 2588 |
| total_timesteps | 966656 |
| train/ | |
| approx_kl | 0.008117125 |
| clip_fraction | 0.0499 |
| clip_range | 0.2 |
| entropy_loss | -0.678 |
| explained_variance | 0.9107527 |
| learning_rate | 0.0003 |
| loss | 6.69 |
| n_updates | 1884 |
| policy_gradient_loss | -0.00337 |
| value_loss | 37.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 236 |
| time/ | |
| fps | 373 |
| iterations | 473 |
| time_elapsed | 2593 |
| total_timesteps | 968704 |
| train/ | |
| approx_kl | 0.0033065695 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -0.661 |
| explained_variance | 0.92935514 |
| learning_rate | 0.0003 |
| loss | 9.95 |
| n_updates | 1888 |
| policy_gradient_loss | 0.000532 |
| value_loss | 39.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 237 |
| time/ | |
| fps | 373 |
| iterations | 474 |
| time_elapsed | 2597 |
| total_timesteps | 970752 |
| train/ | |
| approx_kl | 0.0044245576 |
| clip_fraction | 0.027 |
| clip_range | 0.2 |
| entropy_loss | -0.69 |
| explained_variance | 0.97394097 |
| learning_rate | 0.0003 |
| loss | 3.42 |
| n_updates | 1892 |
| policy_gradient_loss | -0.00102 |
| value_loss | 12.7 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 233 |
| time/ | |
| fps | 373 |
| iterations | 475 |
| time_elapsed | 2602 |
| total_timesteps | 972800 |
| train/ | |
| approx_kl | 0.010404276 |
| clip_fraction | 0.0597 |
| clip_range | 0.2 |
| entropy_loss | -0.688 |
| explained_variance | 0.97804517 |
| learning_rate | 0.0003 |
| loss | 3.89 |
| n_updates | 1896 |
| policy_gradient_loss | -0.00122 |
| value_loss | 13.8 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 373 |
| iterations | 476 |
| time_elapsed | 2606 |
| total_timesteps | 974848 |
| train/ | |
| approx_kl | 0.002540129 |
| clip_fraction | 0.0322 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.85413593 |
| learning_rate | 0.0003 |
| loss | 177 |
| n_updates | 1900 |
| policy_gradient_loss | 0.000471 |
| value_loss | 343 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 294 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 374 |
| iterations | 477 |
| time_elapsed | 2611 |
| total_timesteps | 976896 |
| train/ | |
| approx_kl | 0.0009522344 |
| clip_fraction | 0.000244 |
| clip_range | 0.2 |
| entropy_loss | -0.68 |
| explained_variance | 0.8490927 |
| learning_rate | 0.0003 |
| loss | 394 |
| n_updates | 1904 |
| policy_gradient_loss | -0.000769 |
| value_loss | 373 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 374 |
| iterations | 478 |
| time_elapsed | 2615 |
| total_timesteps | 978944 |
| train/ | |
| approx_kl | 0.0012304761 |
| clip_fraction | 0.00134 |
| clip_range | 0.2 |
| entropy_loss | -0.701 |
| explained_variance | 0.731645 |
| learning_rate | 0.0003 |
| loss | 587 |
| n_updates | 1908 |
| policy_gradient_loss | -0.000221 |
| value_loss | 543 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 980000 to videos/step_980000.mp4
----------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 374 |
| iterations | 479 |
| time_elapsed | 2622 |
| total_timesteps | 980992 |
| train/ | |
| approx_kl | 0.00656572 |
| clip_fraction | 0.0442 |
| clip_range | 0.2 |
| entropy_loss | -0.624 |
| explained_variance | 0.9788895 |
| learning_rate | 0.0003 |
| loss | 11.7 |
| n_updates | 1912 |
| policy_gradient_loss | 0.00135 |
| value_loss | 21.1 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 374 |
| iterations | 480 |
| time_elapsed | 2627 |
| total_timesteps | 983040 |
| train/ | |
| approx_kl | 0.005797052 |
| clip_fraction | 0.0591 |
| clip_range | 0.2 |
| entropy_loss | -0.705 |
| explained_variance | 0.9899039 |
| learning_rate | 0.0003 |
| loss | 3.84 |
| n_updates | 1916 |
| policy_gradient_loss | -0.00129 |
| value_loss | 10.8 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 305 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 374 |
| iterations | 481 |
| time_elapsed | 2632 |
| total_timesteps | 985088 |
| train/ | |
| approx_kl | 0.004933938 |
| clip_fraction | 0.0428 |
| clip_range | 0.2 |
| entropy_loss | -0.7 |
| explained_variance | 0.9900496 |
| learning_rate | 0.0003 |
| loss | 3.59 |
| n_updates | 1920 |
| policy_gradient_loss | 0.000653 |
| value_loss | 11.1 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 313 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 374 |
| iterations | 482 |
| time_elapsed | 2636 |
| total_timesteps | 987136 |
| train/ | |
| approx_kl | 0.005134928 |
| clip_fraction | 0.025 |
| clip_range | 0.2 |
| entropy_loss | -0.664 |
| explained_variance | 0.99178475 |
| learning_rate | 0.0003 |
| loss | 2.4 |
| n_updates | 1924 |
| policy_gradient_loss | -0.00137 |
| value_loss | 9.74 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 309 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 374 |
| iterations | 483 |
| time_elapsed | 2640 |
| total_timesteps | 989184 |
| train/ | |
| approx_kl | 0.0060933037 |
| clip_fraction | 0.0477 |
| clip_range | 0.2 |
| entropy_loss | -0.675 |
| explained_variance | 0.9910136 |
| learning_rate | 0.0003 |
| loss | 2.9 |
| n_updates | 1928 |
| policy_gradient_loss | -0.000819 |
| value_loss | 8.97 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 374 |
| iterations | 484 |
| time_elapsed | 2645 |
| total_timesteps | 991232 |
| train/ | |
| approx_kl | 0.0051599587 |
| clip_fraction | 0.0491 |
| clip_range | 0.2 |
| entropy_loss | -0.691 |
| explained_variance | 0.99348634 |
| learning_rate | 0.0003 |
| loss | 7.13 |
| n_updates | 1932 |
| policy_gradient_loss | -0.00183 |
| value_loss | 10.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 374 |
| iterations | 485 |
| time_elapsed | 2649 |
| total_timesteps | 993280 |
| train/ | |
| approx_kl | 0.0043587266 |
| clip_fraction | 0.0304 |
| clip_range | 0.2 |
| entropy_loss | -0.651 |
| explained_variance | 0.99379957 |
| learning_rate | 0.0003 |
| loss | 2.69 |
| n_updates | 1936 |
| policy_gradient_loss | -0.000262 |
| value_loss | 7.74 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 315 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 374 |
| iterations | 486 |
| time_elapsed | 2654 |
| total_timesteps | 995328 |
| train/ | |
| approx_kl | 0.006317065 |
| clip_fraction | 0.0968 |
| clip_range | 0.2 |
| entropy_loss | -0.799 |
| explained_variance | 0.6639592 |
| learning_rate | 0.0003 |
| loss | 120 |
| n_updates | 1940 |
| policy_gradient_loss | 6.17e-06 |
| value_loss | 766 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 319 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 375 |
| iterations | 487 |
| time_elapsed | 2658 |
| total_timesteps | 997376 |
| train/ | |
| approx_kl | 0.004985626 |
| clip_fraction | 0.0245 |
| clip_range | 0.2 |
| entropy_loss | -0.625 |
| explained_variance | 0.97692317 |
| learning_rate | 0.0003 |
| loss | 4.15 |
| n_updates | 1944 |
| policy_gradient_loss | 0.000192 |
| value_loss | 25.6 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 318 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 375 |
| iterations | 488 |
| time_elapsed | 2663 |
| total_timesteps | 999424 |
| train/ | |
| approx_kl | 0.004790224 |
| clip_fraction | 0.0377 |
| clip_range | 0.2 |
| entropy_loss | -0.627 |
| explained_variance | 0.99013746 |
| learning_rate | 0.0003 |
| loss | 4.69 |
| n_updates | 1948 |
| policy_gradient_loss | -0.00308 |
| value_loss | 13 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1000000 to videos/step_1000000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 325 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 374 |
| iterations | 489 |
| time_elapsed | 2671 |
| total_timesteps | 1001472 |
| train/ | |
| approx_kl | 0.00032519468 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.72 |
| explained_variance | 0.7027923 |
| learning_rate | 0.0003 |
| loss | 372 |
| n_updates | 1952 |
| policy_gradient_loss | -0.000617 |
| value_loss | 870 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 375 |
| iterations | 490 |
| time_elapsed | 2675 |
| total_timesteps | 1003520 |
| train/ | |
| approx_kl | 0.0035429895 |
| clip_fraction | 0.0204 |
| clip_range | 0.2 |
| entropy_loss | -0.69 |
| explained_variance | 0.9775604 |
| learning_rate | 0.0003 |
| loss | 5.05 |
| n_updates | 1956 |
| policy_gradient_loss | -0.000972 |
| value_loss | 15.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 336 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 375 |
| iterations | 491 |
| time_elapsed | 2679 |
| total_timesteps | 1005568 |
| train/ | |
| approx_kl | 0.004805016 |
| clip_fraction | 0.0402 |
| clip_range | 0.2 |
| entropy_loss | -0.626 |
| explained_variance | 0.98593503 |
| learning_rate | 0.0003 |
| loss | 7.65 |
| n_updates | 1960 |
| policy_gradient_loss | -0.00145 |
| value_loss | 13.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 330 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 375 |
| iterations | 492 |
| time_elapsed | 2684 |
| total_timesteps | 1007616 |
| train/ | |
| approx_kl | 0.0016685352 |
| clip_fraction | 0.0314 |
| clip_range | 0.2 |
| entropy_loss | -0.607 |
| explained_variance | 0.99245834 |
| learning_rate | 0.0003 |
| loss | 2.44 |
| n_updates | 1964 |
| policy_gradient_loss | -0.00138 |
| value_loss | 6.32 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 329 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 375 |
| iterations | 493 |
| time_elapsed | 2688 |
| total_timesteps | 1009664 |
| train/ | |
| approx_kl | 0.0024860785 |
| clip_fraction | 0.016 |
| clip_range | 0.2 |
| entropy_loss | -0.656 |
| explained_variance | 0.6578618 |
| learning_rate | 0.0003 |
| loss | 465 |
| n_updates | 1968 |
| policy_gradient_loss | 0.000251 |
| value_loss | 790 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 332 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 375 |
| iterations | 494 |
| time_elapsed | 2693 |
| total_timesteps | 1011712 |
| train/ | |
| approx_kl | 0.00026168866 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.674 |
| explained_variance | 0.8633376 |
| learning_rate | 0.0003 |
| loss | 43.2 |
| n_updates | 1972 |
| policy_gradient_loss | -6.52e-05 |
| value_loss | 334 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 338 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 375 |
| iterations | 495 |
| time_elapsed | 2698 |
| total_timesteps | 1013760 |
| train/ | |
| approx_kl | 0.0037814863 |
| clip_fraction | 0.0242 |
| clip_range | 0.2 |
| entropy_loss | -0.647 |
| explained_variance | 0.9725711 |
| learning_rate | 0.0003 |
| loss | 4.85 |
| n_updates | 1976 |
| policy_gradient_loss | -0.0015 |
| value_loss | 21.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 336 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 375 |
| iterations | 496 |
| time_elapsed | 2702 |
| total_timesteps | 1015808 |
| train/ | |
| approx_kl | 0.0085973935 |
| clip_fraction | 0.0624 |
| clip_range | 0.2 |
| entropy_loss | -0.725 |
| explained_variance | 0.9882897 |
| learning_rate | 0.0003 |
| loss | 3.32 |
| n_updates | 1980 |
| policy_gradient_loss | -0.00105 |
| value_loss | 10.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 338 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 375 |
| iterations | 497 |
| time_elapsed | 2707 |
| total_timesteps | 1017856 |
| train/ | |
| approx_kl | 0.0016737849 |
| clip_fraction | 0.00488 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.7986352 |
| learning_rate | 0.0003 |
| loss | 12.8 |
| n_updates | 1984 |
| policy_gradient_loss | -0.000522 |
| value_loss | 471 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 332 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 376 |
| iterations | 498 |
| time_elapsed | 2711 |
| total_timesteps | 1019904 |
| train/ | |
| approx_kl | 0.0031191814 |
| clip_fraction | 0.0242 |
| clip_range | 0.2 |
| entropy_loss | -0.671 |
| explained_variance | 0.9576386 |
| learning_rate | 0.0003 |
| loss | 10.4 |
| n_updates | 1988 |
| policy_gradient_loss | -0.000779 |
| value_loss | 29 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1020000 to videos/step_1020000.mp4
----------------------------------------
| rollout/ | |
| ep_len_mean | 330 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 375 |
| iterations | 499 |
| time_elapsed | 2718 |
| total_timesteps | 1021952 |
| train/ | |
| approx_kl | 0.0060887 |
| clip_fraction | 0.0581 |
| clip_range | 0.2 |
| entropy_loss | -0.72 |
| explained_variance | 0.99121433 |
| learning_rate | 0.0003 |
| loss | 4.27 |
| n_updates | 1992 |
| policy_gradient_loss | -0.00191 |
| value_loss | 9.13 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 325 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 376 |
| iterations | 500 |
| time_elapsed | 2722 |
| total_timesteps | 1024000 |
| train/ | |
| approx_kl | 0.0037525038 |
| clip_fraction | 0.0491 |
| clip_range | 0.2 |
| entropy_loss | -0.623 |
| explained_variance | 0.9932979 |
| learning_rate | 0.0003 |
| loss | 2.52 |
| n_updates | 1996 |
| policy_gradient_loss | -0.000825 |
| value_loss | 8.59 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 333 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 376 |
| iterations | 501 |
| time_elapsed | 2727 |
| total_timesteps | 1026048 |
| train/ | |
| approx_kl | 0.0062667336 |
| clip_fraction | 0.0458 |
| clip_range | 0.2 |
| entropy_loss | -0.685 |
| explained_variance | 0.9917569 |
| learning_rate | 0.0003 |
| loss | 3.45 |
| n_updates | 2000 |
| policy_gradient_loss | -0.00162 |
| value_loss | 8.94 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 328 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 376 |
| iterations | 502 |
| time_elapsed | 2731 |
| total_timesteps | 1028096 |
| train/ | |
| approx_kl | 0.004965486 |
| clip_fraction | 0.0623 |
| clip_range | 0.2 |
| entropy_loss | -0.63 |
| explained_variance | 0.81613624 |
| learning_rate | 0.0003 |
| loss | 45.5 |
| n_updates | 2004 |
| policy_gradient_loss | -0.00138 |
| value_loss | 438 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 324 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 376 |
| iterations | 503 |
| time_elapsed | 2736 |
| total_timesteps | 1030144 |
| train/ | |
| approx_kl | 0.008041456 |
| clip_fraction | 0.0536 |
| clip_range | 0.2 |
| entropy_loss | -0.701 |
| explained_variance | 0.9865399 |
| learning_rate | 0.0003 |
| loss | 3.42 |
| n_updates | 2008 |
| policy_gradient_loss | -0.00157 |
| value_loss | 10.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 318 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 376 |
| iterations | 504 |
| time_elapsed | 2740 |
| total_timesteps | 1032192 |
| train/ | |
| approx_kl | 0.0052666496 |
| clip_fraction | 0.0341 |
| clip_range | 0.2 |
| entropy_loss | -0.693 |
| explained_variance | 0.99439436 |
| learning_rate | 0.0003 |
| loss | 2.76 |
| n_updates | 2012 |
| policy_gradient_loss | 1.52e-05 |
| value_loss | 6.49 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 317 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 376 |
| iterations | 505 |
| time_elapsed | 2745 |
| total_timesteps | 1034240 |
| train/ | |
| approx_kl | 0.0023076483 |
| clip_fraction | 0.0399 |
| clip_range | 0.2 |
| entropy_loss | -0.68 |
| explained_variance | 0.76318145 |
| learning_rate | 0.0003 |
| loss | 537 |
| n_updates | 2016 |
| policy_gradient_loss | -0.000546 |
| value_loss | 509 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 307 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 376 |
| iterations | 506 |
| time_elapsed | 2749 |
| total_timesteps | 1036288 |
| train/ | |
| approx_kl | 0.0047341725 |
| clip_fraction | 0.0316 |
| clip_range | 0.2 |
| entropy_loss | -0.633 |
| explained_variance | 0.9911354 |
| learning_rate | 0.0003 |
| loss | 2.39 |
| n_updates | 2020 |
| policy_gradient_loss | -0.0032 |
| value_loss | 12.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 306 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 377 |
| iterations | 507 |
| time_elapsed | 2753 |
| total_timesteps | 1038336 |
| train/ | |
| approx_kl | 0.0034657465 |
| clip_fraction | 0.0538 |
| clip_range | 0.2 |
| entropy_loss | -0.702 |
| explained_variance | 0.9932506 |
| learning_rate | 0.0003 |
| loss | 2.54 |
| n_updates | 2024 |
| policy_gradient_loss | -0.00148 |
| value_loss | 8.58 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1040000 to videos/step_1040000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 306 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 376 |
| iterations | 508 |
| time_elapsed | 2761 |
| total_timesteps | 1040384 |
| train/ | |
| approx_kl | 0.004723194 |
| clip_fraction | 0.0375 |
| clip_range | 0.2 |
| entropy_loss | -0.681 |
| explained_variance | 0.99091387 |
| learning_rate | 0.0003 |
| loss | 8.76 |
| n_updates | 2028 |
| policy_gradient_loss | -0.00123 |
| value_loss | 12.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 307 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 376 |
| iterations | 509 |
| time_elapsed | 2766 |
| total_timesteps | 1042432 |
| train/ | |
| approx_kl | 0.0065856627 |
| clip_fraction | 0.0269 |
| clip_range | 0.2 |
| entropy_loss | -0.65 |
| explained_variance | 0.99393463 |
| learning_rate | 0.0003 |
| loss | 2.97 |
| n_updates | 2032 |
| policy_gradient_loss | -0.000524 |
| value_loss | 8.04 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 306 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 376 |
| iterations | 510 |
| time_elapsed | 2770 |
| total_timesteps | 1044480 |
| train/ | |
| approx_kl | 0.002580005 |
| clip_fraction | 0.0143 |
| clip_range | 0.2 |
| entropy_loss | -0.621 |
| explained_variance | 0.99100953 |
| learning_rate | 0.0003 |
| loss | 1.65 |
| n_updates | 2036 |
| policy_gradient_loss | -0.000107 |
| value_loss | 9.69 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 377 |
| iterations | 511 |
| time_elapsed | 2775 |
| total_timesteps | 1046528 |
| train/ | |
| approx_kl | 0.0012628958 |
| clip_fraction | 0.000732 |
| clip_range | 0.2 |
| entropy_loss | -0.688 |
| explained_variance | 0.79333025 |
| learning_rate | 0.0003 |
| loss | 39.6 |
| n_updates | 2040 |
| policy_gradient_loss | -0.000142 |
| value_loss | 509 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 313 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 377 |
| iterations | 512 |
| time_elapsed | 2779 |
| total_timesteps | 1048576 |
| train/ | |
| approx_kl | 0.006276975 |
| clip_fraction | 0.0547 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.9896488 |
| learning_rate | 0.0003 |
| loss | 4.79 |
| n_updates | 2044 |
| policy_gradient_loss | -0.00275 |
| value_loss | 13.9 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 312 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 377 |
| iterations | 513 |
| time_elapsed | 2784 |
| total_timesteps | 1050624 |
| train/ | |
| approx_kl | 0.0014061516 |
| clip_fraction | 0.00366 |
| clip_range | 0.2 |
| entropy_loss | -0.663 |
| explained_variance | 0.82022893 |
| learning_rate | 0.0003 |
| loss | 26.6 |
| n_updates | 2048 |
| policy_gradient_loss | -0.000864 |
| value_loss | 480 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 320 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 377 |
| iterations | 514 |
| time_elapsed | 2789 |
| total_timesteps | 1052672 |
| train/ | |
| approx_kl | 0.0047393963 |
| clip_fraction | 0.038 |
| clip_range | 0.2 |
| entropy_loss | -0.65 |
| explained_variance | 0.9860468 |
| learning_rate | 0.0003 |
| loss | 4.49 |
| n_updates | 2052 |
| policy_gradient_loss | -0.00099 |
| value_loss | 13.1 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 322 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 377 |
| iterations | 515 |
| time_elapsed | 2793 |
| total_timesteps | 1054720 |
| train/ | |
| approx_kl | 0.007160393 |
| clip_fraction | 0.0687 |
| clip_range | 0.2 |
| entropy_loss | -0.718 |
| explained_variance | 0.99140906 |
| learning_rate | 0.0003 |
| loss | 2.26 |
| n_updates | 2056 |
| policy_gradient_loss | -0.00203 |
| value_loss | 7.68 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 321 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 377 |
| iterations | 516 |
| time_elapsed | 2798 |
| total_timesteps | 1056768 |
| train/ | |
| approx_kl | 0.003812958 |
| clip_fraction | 0.0345 |
| clip_range | 0.2 |
| entropy_loss | -0.669 |
| explained_variance | 0.98906565 |
| learning_rate | 0.0003 |
| loss | 10.1 |
| n_updates | 2060 |
| policy_gradient_loss | -0.00329 |
| value_loss | 16.9 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 320 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 377 |
| iterations | 517 |
| time_elapsed | 2803 |
| total_timesteps | 1058816 |
| train/ | |
| approx_kl | 0.0032870052 |
| clip_fraction | 0.0133 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.81947744 |
| learning_rate | 0.0003 |
| loss | 283 |
| n_updates | 2064 |
| policy_gradient_loss | -0.00257 |
| value_loss | 447 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1060000 to videos/step_1060000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 319 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 377 |
| iterations | 518 |
| time_elapsed | 2809 |
| total_timesteps | 1060864 |
| train/ | |
| approx_kl | 0.0056743333 |
| clip_fraction | 0.0586 |
| clip_range | 0.2 |
| entropy_loss | -0.669 |
| explained_variance | 0.9811914 |
| learning_rate | 0.0003 |
| loss | 5.06 |
| n_updates | 2068 |
| policy_gradient_loss | -0.00199 |
| value_loss | 18.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 377 |
| iterations | 519 |
| time_elapsed | 2814 |
| total_timesteps | 1062912 |
| train/ | |
| approx_kl | 0.0052104415 |
| clip_fraction | 0.0267 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.7639547 |
| learning_rate | 0.0003 |
| loss | 398 |
| n_updates | 2072 |
| policy_gradient_loss | -0.00477 |
| value_loss | 468 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 329 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 377 |
| iterations | 520 |
| time_elapsed | 2818 |
| total_timesteps | 1064960 |
| train/ | |
| approx_kl | 0.0075798784 |
| clip_fraction | 0.0594 |
| clip_range | 0.2 |
| entropy_loss | -0.659 |
| explained_variance | 0.9878073 |
| learning_rate | 0.0003 |
| loss | 3.92 |
| n_updates | 2076 |
| policy_gradient_loss | -0.00482 |
| value_loss | 12.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 330 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 377 |
| iterations | 521 |
| time_elapsed | 2823 |
| total_timesteps | 1067008 |
| train/ | |
| approx_kl | 0.0070603574 |
| clip_fraction | 0.0771 |
| clip_range | 0.2 |
| entropy_loss | -0.724 |
| explained_variance | 0.9896074 |
| learning_rate | 0.0003 |
| loss | 5.48 |
| n_updates | 2080 |
| policy_gradient_loss | -0.00348 |
| value_loss | 11.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 336 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 378 |
| iterations | 522 |
| time_elapsed | 2828 |
| total_timesteps | 1069056 |
| train/ | |
| approx_kl | 0.0020515905 |
| clip_fraction | 0.0167 |
| clip_range | 0.2 |
| entropy_loss | -0.688 |
| explained_variance | 0.9856668 |
| learning_rate | 0.0003 |
| loss | 5.04 |
| n_updates | 2084 |
| policy_gradient_loss | -0.000273 |
| value_loss | 15.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 343 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 378 |
| iterations | 523 |
| time_elapsed | 2832 |
| total_timesteps | 1071104 |
| train/ | |
| approx_kl | 0.0037178812 |
| clip_fraction | 0.0267 |
| clip_range | 0.2 |
| entropy_loss | -0.707 |
| explained_variance | 0.98855996 |
| learning_rate | 0.0003 |
| loss | 3.86 |
| n_updates | 2088 |
| policy_gradient_loss | -0.000976 |
| value_loss | 9.06 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 343 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 378 |
| iterations | 524 |
| time_elapsed | 2837 |
| total_timesteps | 1073152 |
| train/ | |
| approx_kl | 0.0064367047 |
| clip_fraction | 0.0345 |
| clip_range | 0.2 |
| entropy_loss | -0.646 |
| explained_variance | 0.99494714 |
| learning_rate | 0.0003 |
| loss | 1.82 |
| n_updates | 2092 |
| policy_gradient_loss | 5.42e-05 |
| value_loss | 5.99 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 338 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 378 |
| iterations | 525 |
| time_elapsed | 2841 |
| total_timesteps | 1075200 |
| train/ | |
| approx_kl | 0.003043198 |
| clip_fraction | 0.0356 |
| clip_range | 0.2 |
| entropy_loss | -0.643 |
| explained_variance | 0.9942089 |
| learning_rate | 0.0003 |
| loss | 1.91 |
| n_updates | 2096 |
| policy_gradient_loss | 0.000259 |
| value_loss | 7.28 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 331 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 378 |
| iterations | 526 |
| time_elapsed | 2845 |
| total_timesteps | 1077248 |
| train/ | |
| approx_kl | 0.00258855 |
| clip_fraction | 0.0214 |
| clip_range | 0.2 |
| entropy_loss | -0.706 |
| explained_variance | 0.99239254 |
| learning_rate | 0.0003 |
| loss | 3.87 |
| n_updates | 2100 |
| policy_gradient_loss | -0.000406 |
| value_loss | 7.32 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 322 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 378 |
| iterations | 527 |
| time_elapsed | 2850 |
| total_timesteps | 1079296 |
| train/ | |
| approx_kl | 0.0009854741 |
| clip_fraction | 0.00146 |
| clip_range | 0.2 |
| entropy_loss | -0.687 |
| explained_variance | 0.83623004 |
| learning_rate | 0.0003 |
| loss | 42.6 |
| n_updates | 2104 |
| policy_gradient_loss | 9.03e-05 |
| value_loss | 442 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1080000 to videos/step_1080000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 322 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 378 |
| iterations | 528 |
| time_elapsed | 2857 |
| total_timesteps | 1081344 |
| train/ | |
| approx_kl | 0.001873723 |
| clip_fraction | 0.0117 |
| clip_range | 0.2 |
| entropy_loss | -0.736 |
| explained_variance | 0.9893369 |
| learning_rate | 0.0003 |
| loss | 3.71 |
| n_updates | 2108 |
| policy_gradient_loss | -0.000629 |
| value_loss | 11.2 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 312 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 378 |
| iterations | 529 |
| time_elapsed | 2862 |
| total_timesteps | 1083392 |
| train/ | |
| approx_kl | 0.0012767015 |
| clip_fraction | 0.00293 |
| clip_range | 0.2 |
| entropy_loss | -0.65 |
| explained_variance | 0.8473334 |
| learning_rate | 0.0003 |
| loss | 65.6 |
| n_updates | 2112 |
| policy_gradient_loss | -0.0021 |
| value_loss | 429 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 311 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 378 |
| iterations | 530 |
| time_elapsed | 2866 |
| total_timesteps | 1085440 |
| train/ | |
| approx_kl | 0.0016064093 |
| clip_fraction | 0.00269 |
| clip_range | 0.2 |
| entropy_loss | -0.75 |
| explained_variance | 0.82579327 |
| learning_rate | 0.0003 |
| loss | 89.1 |
| n_updates | 2116 |
| policy_gradient_loss | -0.000821 |
| value_loss | 367 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 303 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 378 |
| iterations | 531 |
| time_elapsed | 2870 |
| total_timesteps | 1087488 |
| train/ | |
| approx_kl | 0.0006931607 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.719 |
| explained_variance | 0.8989637 |
| learning_rate | 0.0003 |
| loss | 190 |
| n_updates | 2120 |
| policy_gradient_loss | -0.000539 |
| value_loss | 248 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 378 |
| iterations | 532 |
| time_elapsed | 2875 |
| total_timesteps | 1089536 |
| train/ | |
| approx_kl | 0.00735748 |
| clip_fraction | 0.0527 |
| clip_range | 0.2 |
| entropy_loss | -0.705 |
| explained_variance | 0.95912683 |
| learning_rate | 0.0003 |
| loss | 7.14 |
| n_updates | 2124 |
| policy_gradient_loss | -0.000576 |
| value_loss | 52.9 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 379 |
| iterations | 533 |
| time_elapsed | 2879 |
| total_timesteps | 1091584 |
| train/ | |
| approx_kl | 0.0015365549 |
| clip_fraction | 0.00928 |
| clip_range | 0.2 |
| entropy_loss | -0.678 |
| explained_variance | 0.9343468 |
| learning_rate | 0.0003 |
| loss | 5.7 |
| n_updates | 2128 |
| policy_gradient_loss | -0.000609 |
| value_loss | 46.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 379 |
| iterations | 534 |
| time_elapsed | 2883 |
| total_timesteps | 1093632 |
| train/ | |
| approx_kl | 0.0040758513 |
| clip_fraction | 0.0425 |
| clip_range | 0.2 |
| entropy_loss | -0.646 |
| explained_variance | 0.9959389 |
| learning_rate | 0.0003 |
| loss | 2.22 |
| n_updates | 2132 |
| policy_gradient_loss | -0.000941 |
| value_loss | 6.82 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 379 |
| iterations | 535 |
| time_elapsed | 2888 |
| total_timesteps | 1095680 |
| train/ | |
| approx_kl | 0.0017174664 |
| clip_fraction | 0.0133 |
| clip_range | 0.2 |
| entropy_loss | -0.653 |
| explained_variance | 0.98966616 |
| learning_rate | 0.0003 |
| loss | 8.93 |
| n_updates | 2136 |
| policy_gradient_loss | 0.000247 |
| value_loss | 16.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 379 |
| iterations | 536 |
| time_elapsed | 2892 |
| total_timesteps | 1097728 |
| train/ | |
| approx_kl | 0.0023029281 |
| clip_fraction | 0.0251 |
| clip_range | 0.2 |
| entropy_loss | -0.669 |
| explained_variance | 0.9871709 |
| learning_rate | 0.0003 |
| loss | 3.69 |
| n_updates | 2140 |
| policy_gradient_loss | -0.00222 |
| value_loss | 14.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 278 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 379 |
| iterations | 537 |
| time_elapsed | 2897 |
| total_timesteps | 1099776 |
| train/ | |
| approx_kl | 0.0019035105 |
| clip_fraction | 0.0518 |
| clip_range | 0.2 |
| entropy_loss | -0.647 |
| explained_variance | 0.80922496 |
| learning_rate | 0.0003 |
| loss | 271 |
| n_updates | 2144 |
| policy_gradient_loss | -0.000818 |
| value_loss | 520 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1100000 to videos/step_1100000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 379 |
| iterations | 538 |
| time_elapsed | 2904 |
| total_timesteps | 1101824 |
| train/ | |
| approx_kl | 0.0013524886 |
| clip_fraction | 0.00354 |
| clip_range | 0.2 |
| entropy_loss | -0.582 |
| explained_variance | 0.8902523 |
| learning_rate | 0.0003 |
| loss | 104 |
| n_updates | 2148 |
| policy_gradient_loss | -0.000455 |
| value_loss | 215 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 379 |
| iterations | 539 |
| time_elapsed | 2908 |
| total_timesteps | 1103872 |
| train/ | |
| approx_kl | 0.004527575 |
| clip_fraction | 0.0332 |
| clip_range | 0.2 |
| entropy_loss | -0.728 |
| explained_variance | 0.9815872 |
| learning_rate | 0.0003 |
| loss | 3.8 |
| n_updates | 2152 |
| policy_gradient_loss | -0.0014 |
| value_loss | 14.3 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 286 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 379 |
| iterations | 540 |
| time_elapsed | 2913 |
| total_timesteps | 1105920 |
| train/ | |
| approx_kl | 0.00545552 |
| clip_fraction | 0.0472 |
| clip_range | 0.2 |
| entropy_loss | -0.636 |
| explained_variance | 0.9904714 |
| learning_rate | 0.0003 |
| loss | 3.24 |
| n_updates | 2156 |
| policy_gradient_loss | -0.00148 |
| value_loss | 9.74 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 289 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 379 |
| iterations | 541 |
| time_elapsed | 2918 |
| total_timesteps | 1107968 |
| train/ | |
| approx_kl | 0.0052494355 |
| clip_fraction | 0.0249 |
| clip_range | 0.2 |
| entropy_loss | -0.619 |
| explained_variance | 0.9923855 |
| learning_rate | 0.0003 |
| loss | 3.9 |
| n_updates | 2160 |
| policy_gradient_loss | 0.000554 |
| value_loss | 9.69 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 289 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 379 |
| iterations | 542 |
| time_elapsed | 2922 |
| total_timesteps | 1110016 |
| train/ | |
| approx_kl | 0.0026358664 |
| clip_fraction | 0.0242 |
| clip_range | 0.2 |
| entropy_loss | -0.657 |
| explained_variance | 0.9948716 |
| learning_rate | 0.0003 |
| loss | 1.76 |
| n_updates | 2164 |
| policy_gradient_loss | -0.00122 |
| value_loss | 6.63 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 289 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 379 |
| iterations | 543 |
| time_elapsed | 2926 |
| total_timesteps | 1112064 |
| train/ | |
| approx_kl | 0.0027341289 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -0.702 |
| explained_variance | 0.81250197 |
| learning_rate | 0.0003 |
| loss | 195 |
| n_updates | 2168 |
| policy_gradient_loss | 6.75e-05 |
| value_loss | 484 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 288 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 380 |
| iterations | 544 |
| time_elapsed | 2931 |
| total_timesteps | 1114112 |
| train/ | |
| approx_kl | 0.00164938 |
| clip_fraction | 0.00159 |
| clip_range | 0.2 |
| entropy_loss | -0.692 |
| explained_variance | 0.864685 |
| learning_rate | 0.0003 |
| loss | 15.3 |
| n_updates | 2172 |
| policy_gradient_loss | -0.000632 |
| value_loss | 218 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 380 |
| iterations | 545 |
| time_elapsed | 2935 |
| total_timesteps | 1116160 |
| train/ | |
| approx_kl | 0.004506397 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -0.696 |
| explained_variance | 0.9069065 |
| learning_rate | 0.0003 |
| loss | 8.13 |
| n_updates | 2176 |
| policy_gradient_loss | -0.00286 |
| value_loss | 60.9 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 380 |
| iterations | 546 |
| time_elapsed | 2940 |
| total_timesteps | 1118208 |
| train/ | |
| approx_kl | 0.00063722034 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.649 |
| explained_variance | 0.7699863 |
| learning_rate | 0.0003 |
| loss | 634 |
| n_updates | 2180 |
| policy_gradient_loss | 1.77e-05 |
| value_loss | 840 |
-------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1120000 to videos/step_1120000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 380 |
| iterations | 547 |
| time_elapsed | 2947 |
| total_timesteps | 1120256 |
| train/ | |
| approx_kl | 0.0013196521 |
| clip_fraction | 0.00305 |
| clip_range | 0.2 |
| entropy_loss | -0.661 |
| explained_variance | 0.8427825 |
| learning_rate | 0.0003 |
| loss | 283 |
| n_updates | 2184 |
| policy_gradient_loss | -0.000733 |
| value_loss | 425 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 380 |
| iterations | 548 |
| time_elapsed | 2951 |
| total_timesteps | 1122304 |
| train/ | |
| approx_kl | 0.003703156 |
| clip_fraction | 0.0242 |
| clip_range | 0.2 |
| entropy_loss | -0.685 |
| explained_variance | 0.98362505 |
| learning_rate | 0.0003 |
| loss | 3.83 |
| n_updates | 2188 |
| policy_gradient_loss | -0.000978 |
| value_loss | 17.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 298 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 380 |
| iterations | 549 |
| time_elapsed | 2956 |
| total_timesteps | 1124352 |
| train/ | |
| approx_kl | 0.0062829885 |
| clip_fraction | 0.0502 |
| clip_range | 0.2 |
| entropy_loss | -0.743 |
| explained_variance | 0.98827857 |
| learning_rate | 0.0003 |
| loss | 3.78 |
| n_updates | 2192 |
| policy_gradient_loss | -0.00281 |
| value_loss | 9.66 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 298 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 380 |
| iterations | 550 |
| time_elapsed | 2960 |
| total_timesteps | 1126400 |
| train/ | |
| approx_kl | 0.0057650455 |
| clip_fraction | 0.0417 |
| clip_range | 0.2 |
| entropy_loss | -0.658 |
| explained_variance | 0.9912121 |
| learning_rate | 0.0003 |
| loss | 3.13 |
| n_updates | 2196 |
| policy_gradient_loss | -0.00219 |
| value_loss | 9.24 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 305 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 380 |
| iterations | 551 |
| time_elapsed | 2964 |
| total_timesteps | 1128448 |
| train/ | |
| approx_kl | 0.0033776427 |
| clip_fraction | 0.0299 |
| clip_range | 0.2 |
| entropy_loss | -0.656 |
| explained_variance | 0.99047416 |
| learning_rate | 0.0003 |
| loss | 3.91 |
| n_updates | 2200 |
| policy_gradient_loss | 0.000557 |
| value_loss | 11.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 380 |
| iterations | 552 |
| time_elapsed | 2969 |
| total_timesteps | 1130496 |
| train/ | |
| approx_kl | 0.004585095 |
| clip_fraction | 0.0562 |
| clip_range | 0.2 |
| entropy_loss | -0.633 |
| explained_variance | 0.99356675 |
| learning_rate | 0.0003 |
| loss | 3.69 |
| n_updates | 2204 |
| policy_gradient_loss | -0.00173 |
| value_loss | 7.16 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 380 |
| iterations | 553 |
| time_elapsed | 2973 |
| total_timesteps | 1132544 |
| train/ | |
| approx_kl | 0.003578351 |
| clip_fraction | 0.036 |
| clip_range | 0.2 |
| entropy_loss | -0.718 |
| explained_variance | 0.82331216 |
| learning_rate | 0.0003 |
| loss | 24.2 |
| n_updates | 2208 |
| policy_gradient_loss | -0.00068 |
| value_loss | 350 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 307 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 380 |
| iterations | 554 |
| time_elapsed | 2978 |
| total_timesteps | 1134592 |
| train/ | |
| approx_kl | 0.0021615245 |
| clip_fraction | 0.00635 |
| clip_range | 0.2 |
| entropy_loss | -0.67 |
| explained_variance | 0.9611883 |
| learning_rate | 0.0003 |
| loss | 11.6 |
| n_updates | 2212 |
| policy_gradient_loss | -0.000468 |
| value_loss | 47 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 304 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 381 |
| iterations | 555 |
| time_elapsed | 2982 |
| total_timesteps | 1136640 |
| train/ | |
| approx_kl | 0.0045592985 |
| clip_fraction | 0.0244 |
| clip_range | 0.2 |
| entropy_loss | -0.693 |
| explained_variance | 0.982991 |
| learning_rate | 0.0003 |
| loss | 12.7 |
| n_updates | 2216 |
| policy_gradient_loss | -0.00214 |
| value_loss | 22.3 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 381 |
| iterations | 556 |
| time_elapsed | 2987 |
| total_timesteps | 1138688 |
| train/ | |
| approx_kl | 0.00033564246 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.8215687 |
| learning_rate | 0.0003 |
| loss | 251 |
| n_updates | 2220 |
| policy_gradient_loss | 0.000109 |
| value_loss | 332 |
-------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1140000 to videos/step_1140000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 309 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 380 |
| iterations | 557 |
| time_elapsed | 2994 |
| total_timesteps | 1140736 |
| train/ | |
| approx_kl | 0.0018815234 |
| clip_fraction | 0.0033 |
| clip_range | 0.2 |
| entropy_loss | -0.603 |
| explained_variance | 0.9736632 |
| learning_rate | 0.0003 |
| loss | 20.5 |
| n_updates | 2224 |
| policy_gradient_loss | -0.000612 |
| value_loss | 40.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 309 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 381 |
| iterations | 558 |
| time_elapsed | 2998 |
| total_timesteps | 1142784 |
| train/ | |
| approx_kl | 0.0034224223 |
| clip_fraction | 0.0239 |
| clip_range | 0.2 |
| entropy_loss | -0.681 |
| explained_variance | 0.9876276 |
| learning_rate | 0.0003 |
| loss | 3.61 |
| n_updates | 2228 |
| policy_gradient_loss | -0.00182 |
| value_loss | 18.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 309 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 381 |
| iterations | 559 |
| time_elapsed | 3002 |
| total_timesteps | 1144832 |
| train/ | |
| approx_kl | 0.0037091197 |
| clip_fraction | 0.0194 |
| clip_range | 0.2 |
| entropy_loss | -0.726 |
| explained_variance | 0.8068105 |
| learning_rate | 0.0003 |
| loss | 80.3 |
| n_updates | 2232 |
| policy_gradient_loss | -0.00101 |
| value_loss | 423 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 313 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 381 |
| iterations | 560 |
| time_elapsed | 3007 |
| total_timesteps | 1146880 |
| train/ | |
| approx_kl | 0.0030964885 |
| clip_fraction | 0.0243 |
| clip_range | 0.2 |
| entropy_loss | -0.685 |
| explained_variance | 0.9810265 |
| learning_rate | 0.0003 |
| loss | 6.76 |
| n_updates | 2236 |
| policy_gradient_loss | -0.00132 |
| value_loss | 23.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 381 |
| iterations | 561 |
| time_elapsed | 3011 |
| total_timesteps | 1148928 |
| train/ | |
| approx_kl | 0.0048935665 |
| clip_fraction | 0.0405 |
| clip_range | 0.2 |
| entropy_loss | -0.623 |
| explained_variance | 0.9822338 |
| learning_rate | 0.0003 |
| loss | 14 |
| n_updates | 2240 |
| policy_gradient_loss | -0.00322 |
| value_loss | 45.6 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 316 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 381 |
| iterations | 562 |
| time_elapsed | 3015 |
| total_timesteps | 1150976 |
| train/ | |
| approx_kl | 0.00077067607 |
| clip_fraction | 0.00061 |
| clip_range | 0.2 |
| entropy_loss | -0.742 |
| explained_variance | 0.8496719 |
| learning_rate | 0.0003 |
| loss | 135 |
| n_updates | 2244 |
| policy_gradient_loss | -0.000666 |
| value_loss | 352 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 315 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 381 |
| iterations | 563 |
| time_elapsed | 3020 |
| total_timesteps | 1153024 |
| train/ | |
| approx_kl | 0.0038501176 |
| clip_fraction | 0.0289 |
| clip_range | 0.2 |
| entropy_loss | -0.681 |
| explained_variance | 0.94996893 |
| learning_rate | 0.0003 |
| loss | 14.4 |
| n_updates | 2248 |
| policy_gradient_loss | -0.00136 |
| value_loss | 54.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 315 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 381 |
| iterations | 564 |
| time_elapsed | 3024 |
| total_timesteps | 1155072 |
| train/ | |
| approx_kl | 0.0042431327 |
| clip_fraction | 0.0259 |
| clip_range | 0.2 |
| entropy_loss | -0.768 |
| explained_variance | 0.80745924 |
| learning_rate | 0.0003 |
| loss | 34.5 |
| n_updates | 2252 |
| policy_gradient_loss | -0.000907 |
| value_loss | 435 |
------------------------------------------
--------------------------------------------
| rollout/ | |
| ep_len_mean | 318 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 381 |
| iterations | 565 |
| time_elapsed | 3029 |
| total_timesteps | 1157120 |
| train/ | |
| approx_kl | 0.000105739135 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.601 |
| explained_variance | 0.9077578 |
| learning_rate | 0.0003 |
| loss | 185 |
| n_updates | 2256 |
| policy_gradient_loss | 9.42e-05 |
| value_loss | 265 |
--------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 316 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 382 |
| iterations | 566 |
| time_elapsed | 3033 |
| total_timesteps | 1159168 |
| train/ | |
| approx_kl | 0.0054181265 |
| clip_fraction | 0.0216 |
| clip_range | 0.2 |
| entropy_loss | -0.689 |
| explained_variance | 0.9352891 |
| learning_rate | 0.0003 |
| loss | 27.9 |
| n_updates | 2260 |
| policy_gradient_loss | -0.00418 |
| value_loss | 100 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1160000 to videos/step_1160000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 313 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 381 |
| iterations | 567 |
| time_elapsed | 3039 |
| total_timesteps | 1161216 |
| train/ | |
| approx_kl | 0.0030409177 |
| clip_fraction | 0.0267 |
| clip_range | 0.2 |
| entropy_loss | -0.706 |
| explained_variance | 0.9508057 |
| learning_rate | 0.0003 |
| loss | 14 |
| n_updates | 2264 |
| policy_gradient_loss | -0.00169 |
| value_loss | 37.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 382 |
| iterations | 568 |
| time_elapsed | 3044 |
| total_timesteps | 1163264 |
| train/ | |
| approx_kl | 0.007949158 |
| clip_fraction | 0.0582 |
| clip_range | 0.2 |
| entropy_loss | -0.666 |
| explained_variance | 0.98920304 |
| learning_rate | 0.0003 |
| loss | 4.58 |
| n_updates | 2268 |
| policy_gradient_loss | -0.00158 |
| value_loss | 12.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 300 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 382 |
| iterations | 569 |
| time_elapsed | 3048 |
| total_timesteps | 1165312 |
| train/ | |
| approx_kl | 0.0014451041 |
| clip_fraction | 0.0011 |
| clip_range | 0.2 |
| entropy_loss | -0.711 |
| explained_variance | 0.8259391 |
| learning_rate | 0.0003 |
| loss | 38.2 |
| n_updates | 2272 |
| policy_gradient_loss | -0.000827 |
| value_loss | 321 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 300 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 382 |
| iterations | 570 |
| time_elapsed | 3052 |
| total_timesteps | 1167360 |
| train/ | |
| approx_kl | 0.0027159695 |
| clip_fraction | 0.0118 |
| clip_range | 0.2 |
| entropy_loss | -0.699 |
| explained_variance | 0.9426929 |
| learning_rate | 0.0003 |
| loss | 5.31 |
| n_updates | 2276 |
| policy_gradient_loss | -0.00143 |
| value_loss | 42.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 300 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 382 |
| iterations | 571 |
| time_elapsed | 3057 |
| total_timesteps | 1169408 |
| train/ | |
| approx_kl | 0.0045883846 |
| clip_fraction | 0.041 |
| clip_range | 0.2 |
| entropy_loss | -0.733 |
| explained_variance | 0.982212 |
| learning_rate | 0.0003 |
| loss | 3.83 |
| n_updates | 2280 |
| policy_gradient_loss | 0.000134 |
| value_loss | 12.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 382 |
| iterations | 572 |
| time_elapsed | 3062 |
| total_timesteps | 1171456 |
| train/ | |
| approx_kl | 0.0027604438 |
| clip_fraction | 0.0341 |
| clip_range | 0.2 |
| entropy_loss | -0.612 |
| explained_variance | 0.99213564 |
| learning_rate | 0.0003 |
| loss | 3.78 |
| n_updates | 2284 |
| policy_gradient_loss | -0.000888 |
| value_loss | 10.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 382 |
| iterations | 573 |
| time_elapsed | 3066 |
| total_timesteps | 1173504 |
| train/ | |
| approx_kl | 0.007947007 |
| clip_fraction | 0.0568 |
| clip_range | 0.2 |
| entropy_loss | -0.709 |
| explained_variance | 0.98746985 |
| learning_rate | 0.0003 |
| loss | 3.96 |
| n_updates | 2288 |
| policy_gradient_loss | -0.00288 |
| value_loss | 11.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 382 |
| iterations | 574 |
| time_elapsed | 3071 |
| total_timesteps | 1175552 |
| train/ | |
| approx_kl | 0.0056247646 |
| clip_fraction | 0.0519 |
| clip_range | 0.2 |
| entropy_loss | -0.685 |
| explained_variance | 0.9917087 |
| learning_rate | 0.0003 |
| loss | 3.13 |
| n_updates | 2292 |
| policy_gradient_loss | -0.00287 |
| value_loss | 8.52 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 304 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 382 |
| iterations | 575 |
| time_elapsed | 3075 |
| total_timesteps | 1177600 |
| train/ | |
| approx_kl | 0.003676531 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -0.723 |
| explained_variance | 0.9835669 |
| learning_rate | 0.0003 |
| loss | 2.59 |
| n_updates | 2296 |
| policy_gradient_loss | -0.000304 |
| value_loss | 10.9 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 303 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 383 |
| iterations | 576 |
| time_elapsed | 3079 |
| total_timesteps | 1179648 |
| train/ | |
| approx_kl | 0.002974084 |
| clip_fraction | 0.0322 |
| clip_range | 0.2 |
| entropy_loss | -0.637 |
| explained_variance | 0.99407524 |
| learning_rate | 0.0003 |
| loss | 3.23 |
| n_updates | 2300 |
| policy_gradient_loss | -0.00167 |
| value_loss | 8.36 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1180000 to videos/step_1180000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 303 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 382 |
| iterations | 577 |
| time_elapsed | 3086 |
| total_timesteps | 1181696 |
| train/ | |
| approx_kl | 0.0033889282 |
| clip_fraction | 0.0166 |
| clip_range | 0.2 |
| entropy_loss | -0.677 |
| explained_variance | 0.7521596 |
| learning_rate | 0.0003 |
| loss | 365 |
| n_updates | 2304 |
| policy_gradient_loss | -0.00218 |
| value_loss | 546 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 305 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 382 |
| iterations | 578 |
| time_elapsed | 3090 |
| total_timesteps | 1183744 |
| train/ | |
| approx_kl | 0.0073713395 |
| clip_fraction | 0.0553 |
| clip_range | 0.2 |
| entropy_loss | -0.751 |
| explained_variance | 0.9855469 |
| learning_rate | 0.0003 |
| loss | 4.41 |
| n_updates | 2308 |
| policy_gradient_loss | -0.00123 |
| value_loss | 11.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 383 |
| iterations | 579 |
| time_elapsed | 3095 |
| total_timesteps | 1185792 |
| train/ | |
| approx_kl | 0.0048805336 |
| clip_fraction | 0.0256 |
| clip_range | 0.2 |
| entropy_loss | -0.73 |
| explained_variance | 0.98934317 |
| learning_rate | 0.0003 |
| loss | 2.55 |
| n_updates | 2312 |
| policy_gradient_loss | -2.14e-06 |
| value_loss | 9.01 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 383 |
| iterations | 580 |
| time_elapsed | 3100 |
| total_timesteps | 1187840 |
| train/ | |
| approx_kl | 0.0032926705 |
| clip_fraction | 0.0155 |
| clip_range | 0.2 |
| entropy_loss | -0.695 |
| explained_variance | 0.76745 |
| learning_rate | 0.0003 |
| loss | 300 |
| n_updates | 2316 |
| policy_gradient_loss | -0.00162 |
| value_loss | 464 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 383 |
| iterations | 581 |
| time_elapsed | 3104 |
| total_timesteps | 1189888 |
| train/ | |
| approx_kl | 0.0032408354 |
| clip_fraction | 0.0176 |
| clip_range | 0.2 |
| entropy_loss | -0.742 |
| explained_variance | 0.99431944 |
| learning_rate | 0.0003 |
| loss | 1.84 |
| n_updates | 2320 |
| policy_gradient_loss | -0.00015 |
| value_loss | 5.98 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 303 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 383 |
| iterations | 582 |
| time_elapsed | 3109 |
| total_timesteps | 1191936 |
| train/ | |
| approx_kl | 0.0042639156 |
| clip_fraction | 0.103 |
| clip_range | 0.2 |
| entropy_loss | -0.805 |
| explained_variance | 0.7873766 |
| learning_rate | 0.0003 |
| loss | 194 |
| n_updates | 2324 |
| policy_gradient_loss | -1.43e-05 |
| value_loss | 446 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 305 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 383 |
| iterations | 583 |
| time_elapsed | 3113 |
| total_timesteps | 1193984 |
| train/ | |
| approx_kl | 0.0026122374 |
| clip_fraction | 0.0238 |
| clip_range | 0.2 |
| entropy_loss | -0.679 |
| explained_variance | 0.991037 |
| learning_rate | 0.0003 |
| loss | 15.2 |
| n_updates | 2328 |
| policy_gradient_loss | -0.00215 |
| value_loss | 16.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 305 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 383 |
| iterations | 584 |
| time_elapsed | 3117 |
| total_timesteps | 1196032 |
| train/ | |
| approx_kl | 0.0051201596 |
| clip_fraction | 0.0515 |
| clip_range | 0.2 |
| entropy_loss | -0.72 |
| explained_variance | 0.991548 |
| learning_rate | 0.0003 |
| loss | 2.79 |
| n_updates | 2332 |
| policy_gradient_loss | -0.00248 |
| value_loss | 7.76 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 306 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 383 |
| iterations | 585 |
| time_elapsed | 3122 |
| total_timesteps | 1198080 |
| train/ | |
| approx_kl | 0.003852033 |
| clip_fraction | 0.0289 |
| clip_range | 0.2 |
| entropy_loss | -0.654 |
| explained_variance | 0.992254 |
| learning_rate | 0.0003 |
| loss | 3.62 |
| n_updates | 2336 |
| policy_gradient_loss | -0.0011 |
| value_loss | 8.64 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1200000 to videos/step_1200000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 383 |
| iterations | 586 |
| time_elapsed | 3128 |
| total_timesteps | 1200128 |
| train/ | |
| approx_kl | 0.009570309 |
| clip_fraction | 0.0728 |
| clip_range | 0.2 |
| entropy_loss | -0.705 |
| explained_variance | 0.9956383 |
| learning_rate | 0.0003 |
| loss | 2.74 |
| n_updates | 2340 |
| policy_gradient_loss | -0.0012 |
| value_loss | 5.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 298 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 383 |
| iterations | 587 |
| time_elapsed | 3133 |
| total_timesteps | 1202176 |
| train/ | |
| approx_kl | 0.0045955367 |
| clip_fraction | 0.0365 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.9913692 |
| learning_rate | 0.0003 |
| loss | 4.19 |
| n_updates | 2344 |
| policy_gradient_loss | -0.00203 |
| value_loss | 12.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 296 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 383 |
| iterations | 588 |
| time_elapsed | 3137 |
| total_timesteps | 1204224 |
| train/ | |
| approx_kl | 0.0031462559 |
| clip_fraction | 0.0157 |
| clip_range | 0.2 |
| entropy_loss | -0.679 |
| explained_variance | 0.7802969 |
| learning_rate | 0.0003 |
| loss | 99.9 |
| n_updates | 2348 |
| policy_gradient_loss | -0.00283 |
| value_loss | 519 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 383 |
| iterations | 589 |
| time_elapsed | 3141 |
| total_timesteps | 1206272 |
| train/ | |
| approx_kl | 0.00034674772 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.688 |
| explained_variance | 0.8194174 |
| learning_rate | 0.0003 |
| loss | 407 |
| n_updates | 2352 |
| policy_gradient_loss | -0.000197 |
| value_loss | 452 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 286 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 384 |
| iterations | 590 |
| time_elapsed | 3146 |
| total_timesteps | 1208320 |
| train/ | |
| approx_kl | 0.0042869765 |
| clip_fraction | 0.0288 |
| clip_range | 0.2 |
| entropy_loss | -0.729 |
| explained_variance | 0.9613544 |
| learning_rate | 0.0003 |
| loss | 3.56 |
| n_updates | 2356 |
| policy_gradient_loss | -0.000287 |
| value_loss | 28.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 384 |
| iterations | 591 |
| time_elapsed | 3150 |
| total_timesteps | 1210368 |
| train/ | |
| approx_kl | 0.004911702 |
| clip_fraction | 0.0322 |
| clip_range | 0.2 |
| entropy_loss | -0.687 |
| explained_variance | 0.9921642 |
| learning_rate | 0.0003 |
| loss | 5.17 |
| n_updates | 2360 |
| policy_gradient_loss | -0.00135 |
| value_loss | 8.04 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 277 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 384 |
| iterations | 592 |
| time_elapsed | 3154 |
| total_timesteps | 1212416 |
| train/ | |
| approx_kl | 0.0049807373 |
| clip_fraction | 0.0876 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.6452054 |
| learning_rate | 0.0003 |
| loss | 1.07e+03 |
| n_updates | 2364 |
| policy_gradient_loss | -0.000974 |
| value_loss | 900 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 384 |
| iterations | 593 |
| time_elapsed | 3159 |
| total_timesteps | 1214464 |
| train/ | |
| approx_kl | 0.0052937246 |
| clip_fraction | 0.0261 |
| clip_range | 0.2 |
| entropy_loss | -0.714 |
| explained_variance | 0.96811175 |
| learning_rate | 0.0003 |
| loss | 5.48 |
| n_updates | 2368 |
| policy_gradient_loss | -0.00234 |
| value_loss | 44.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 384 |
| iterations | 594 |
| time_elapsed | 3164 |
| total_timesteps | 1216512 |
| train/ | |
| approx_kl | 0.0049794065 |
| clip_fraction | 0.0155 |
| clip_range | 0.2 |
| entropy_loss | -0.691 |
| explained_variance | 0.9896989 |
| learning_rate | 0.0003 |
| loss | 11.3 |
| n_updates | 2372 |
| policy_gradient_loss | -0.000803 |
| value_loss | 30.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 384 |
| iterations | 595 |
| time_elapsed | 3168 |
| total_timesteps | 1218560 |
| train/ | |
| approx_kl | 0.0073360093 |
| clip_fraction | 0.0951 |
| clip_range | 0.2 |
| entropy_loss | -0.675 |
| explained_variance | 0.94576305 |
| learning_rate | 0.0003 |
| loss | 22.5 |
| n_updates | 2376 |
| policy_gradient_loss | -0.00136 |
| value_loss | 192 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1220000 to videos/step_1220000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 384 |
| iterations | 596 |
| time_elapsed | 3175 |
| total_timesteps | 1220608 |
| train/ | |
| approx_kl | 0.0040189754 |
| clip_fraction | 0.0131 |
| clip_range | 0.2 |
| entropy_loss | -0.603 |
| explained_variance | 0.9354177 |
| learning_rate | 0.0003 |
| loss | 5.89 |
| n_updates | 2380 |
| policy_gradient_loss | -0.00124 |
| value_loss | 67.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 384 |
| iterations | 597 |
| time_elapsed | 3179 |
| total_timesteps | 1222656 |
| train/ | |
| approx_kl | 0.0038658862 |
| clip_fraction | 0.0322 |
| clip_range | 0.2 |
| entropy_loss | -0.656 |
| explained_variance | 0.98576695 |
| learning_rate | 0.0003 |
| loss | 5.77 |
| n_updates | 2384 |
| policy_gradient_loss | -0.00097 |
| value_loss | 13.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 384 |
| iterations | 598 |
| time_elapsed | 3183 |
| total_timesteps | 1224704 |
| train/ | |
| approx_kl | 0.0040271506 |
| clip_fraction | 0.0438 |
| clip_range | 0.2 |
| entropy_loss | -0.725 |
| explained_variance | 0.9927791 |
| learning_rate | 0.0003 |
| loss | 1.76 |
| n_updates | 2388 |
| policy_gradient_loss | -0.000185 |
| value_loss | 6.22 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 384 |
| iterations | 599 |
| time_elapsed | 3188 |
| total_timesteps | 1226752 |
| train/ | |
| approx_kl | 0.006764356 |
| clip_fraction | 0.0597 |
| clip_range | 0.2 |
| entropy_loss | -0.689 |
| explained_variance | 0.9925801 |
| learning_rate | 0.0003 |
| loss | 2.49 |
| n_updates | 2392 |
| policy_gradient_loss | -0.00269 |
| value_loss | 7.16 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 384 |
| iterations | 600 |
| time_elapsed | 3192 |
| total_timesteps | 1228800 |
| train/ | |
| approx_kl | 0.0055658165 |
| clip_fraction | 0.164 |
| clip_range | 0.2 |
| entropy_loss | -0.626 |
| explained_variance | 0.7998903 |
| learning_rate | 0.0003 |
| loss | 126 |
| n_updates | 2396 |
| policy_gradient_loss | -0.00077 |
| value_loss | 448 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 385 |
| iterations | 601 |
| time_elapsed | 3196 |
| total_timesteps | 1230848 |
| train/ | |
| approx_kl | 0.0054545347 |
| clip_fraction | 0.0367 |
| clip_range | 0.2 |
| entropy_loss | -0.709 |
| explained_variance | 0.9907895 |
| learning_rate | 0.0003 |
| loss | 1.44 |
| n_updates | 2400 |
| policy_gradient_loss | -0.000884 |
| value_loss | 8.51 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 385 |
| iterations | 602 |
| time_elapsed | 3201 |
| total_timesteps | 1232896 |
| train/ | |
| approx_kl | 0.0053939335 |
| clip_fraction | 0.0427 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.98203343 |
| learning_rate | 0.0003 |
| loss | 3.34 |
| n_updates | 2404 |
| policy_gradient_loss | -0.00105 |
| value_loss | 12.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 385 |
| iterations | 603 |
| time_elapsed | 3205 |
| total_timesteps | 1234944 |
| train/ | |
| approx_kl | 0.0032913787 |
| clip_fraction | 0.0222 |
| clip_range | 0.2 |
| entropy_loss | -0.615 |
| explained_variance | 0.955864 |
| learning_rate | 0.0003 |
| loss | 16.2 |
| n_updates | 2408 |
| policy_gradient_loss | -0.0012 |
| value_loss | 92.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 296 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 385 |
| iterations | 604 |
| time_elapsed | 3210 |
| total_timesteps | 1236992 |
| train/ | |
| approx_kl | 0.0034185858 |
| clip_fraction | 0.0205 |
| clip_range | 0.2 |
| entropy_loss | -0.7 |
| explained_variance | 0.9798781 |
| learning_rate | 0.0003 |
| loss | 3.19 |
| n_updates | 2412 |
| policy_gradient_loss | -0.00233 |
| value_loss | 12.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 385 |
| iterations | 605 |
| time_elapsed | 3214 |
| total_timesteps | 1239040 |
| train/ | |
| approx_kl | 0.0042094085 |
| clip_fraction | 0.0233 |
| clip_range | 0.2 |
| entropy_loss | -0.684 |
| explained_variance | 0.9923123 |
| learning_rate | 0.0003 |
| loss | 3.46 |
| n_updates | 2416 |
| policy_gradient_loss | -0.00126 |
| value_loss | 9.48 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1240000 to videos/step_1240000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 385 |
| iterations | 606 |
| time_elapsed | 3220 |
| total_timesteps | 1241088 |
| train/ | |
| approx_kl | 0.0033813268 |
| clip_fraction | 0.0389 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.99388754 |
| learning_rate | 0.0003 |
| loss | 1.96 |
| n_updates | 2420 |
| policy_gradient_loss | -0.000225 |
| value_loss | 6.46 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 385 |
| iterations | 607 |
| time_elapsed | 3225 |
| total_timesteps | 1243136 |
| train/ | |
| approx_kl | 0.005891851 |
| clip_fraction | 0.0519 |
| clip_range | 0.2 |
| entropy_loss | -0.654 |
| explained_variance | 0.9954293 |
| learning_rate | 0.0003 |
| loss | 2 |
| n_updates | 2424 |
| policy_gradient_loss | -0.000705 |
| value_loss | 5.98 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 385 |
| iterations | 608 |
| time_elapsed | 3229 |
| total_timesteps | 1245184 |
| train/ | |
| approx_kl | 0.0037489072 |
| clip_fraction | 0.0421 |
| clip_range | 0.2 |
| entropy_loss | -0.717 |
| explained_variance | 0.9960397 |
| learning_rate | 0.0003 |
| loss | 1.27 |
| n_updates | 2428 |
| policy_gradient_loss | -0.00229 |
| value_loss | 4.88 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 385 |
| iterations | 609 |
| time_elapsed | 3234 |
| total_timesteps | 1247232 |
| train/ | |
| approx_kl | 0.0015716772 |
| clip_fraction | 0.0156 |
| clip_range | 0.2 |
| entropy_loss | -0.633 |
| explained_variance | 0.9961026 |
| learning_rate | 0.0003 |
| loss | 2.41 |
| n_updates | 2432 |
| policy_gradient_loss | 0.000431 |
| value_loss | 5.96 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 385 |
| iterations | 610 |
| time_elapsed | 3239 |
| total_timesteps | 1249280 |
| train/ | |
| approx_kl | 0.0071332646 |
| clip_fraction | 0.0675 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.9950488 |
| learning_rate | 0.0003 |
| loss | 2.04 |
| n_updates | 2436 |
| policy_gradient_loss | 0.000636 |
| value_loss | 5.05 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 286 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 385 |
| iterations | 611 |
| time_elapsed | 3243 |
| total_timesteps | 1251328 |
| train/ | |
| approx_kl | 0.005271438 |
| clip_fraction | 0.0623 |
| clip_range | 0.2 |
| entropy_loss | -0.699 |
| explained_variance | 0.99479073 |
| learning_rate | 0.0003 |
| loss | 2.02 |
| n_updates | 2440 |
| policy_gradient_loss | -0.0018 |
| value_loss | 6.78 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 385 |
| iterations | 612 |
| time_elapsed | 3247 |
| total_timesteps | 1253376 |
| train/ | |
| approx_kl | 0.0012877986 |
| clip_fraction | 0.005 |
| clip_range | 0.2 |
| entropy_loss | -0.723 |
| explained_variance | 0.7904221 |
| learning_rate | 0.0003 |
| loss | 40.4 |
| n_updates | 2444 |
| policy_gradient_loss | -0.000652 |
| value_loss | 480 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 288 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 386 |
| iterations | 613 |
| time_elapsed | 3252 |
| total_timesteps | 1255424 |
| train/ | |
| approx_kl | 0.0036129407 |
| clip_fraction | 0.025 |
| clip_range | 0.2 |
| entropy_loss | -0.676 |
| explained_variance | 0.9855988 |
| learning_rate | 0.0003 |
| loss | 3.56 |
| n_updates | 2448 |
| policy_gradient_loss | -0.00269 |
| value_loss | 13.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 296 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 386 |
| iterations | 614 |
| time_elapsed | 3256 |
| total_timesteps | 1257472 |
| train/ | |
| approx_kl | 0.0043813447 |
| clip_fraction | 0.0215 |
| clip_range | 0.2 |
| entropy_loss | -0.633 |
| explained_variance | 0.98595184 |
| learning_rate | 0.0003 |
| loss | 4.73 |
| n_updates | 2452 |
| policy_gradient_loss | -0.000662 |
| value_loss | 12.7 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 300 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 386 |
| iterations | 615 |
| time_elapsed | 3260 |
| total_timesteps | 1259520 |
| train/ | |
| approx_kl | 0.007991209 |
| clip_fraction | 0.0928 |
| clip_range | 0.2 |
| entropy_loss | -0.68 |
| explained_variance | 0.9931738 |
| learning_rate | 0.0003 |
| loss | 1.84 |
| n_updates | 2456 |
| policy_gradient_loss | -0.00202 |
| value_loss | 6.01 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1260000 to videos/step_1260000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 293 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 386 |
| iterations | 616 |
| time_elapsed | 3267 |
| total_timesteps | 1261568 |
| train/ | |
| approx_kl | 0.0063912617 |
| clip_fraction | 0.0413 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.9812416 |
| learning_rate | 0.0003 |
| loss | 2.06 |
| n_updates | 2460 |
| policy_gradient_loss | -0.000925 |
| value_loss | 15.6 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 386 |
| iterations | 617 |
| time_elapsed | 3271 |
| total_timesteps | 1263616 |
| train/ | |
| approx_kl | 0.003276039 |
| clip_fraction | 0.0275 |
| clip_range | 0.2 |
| entropy_loss | -0.646 |
| explained_variance | 0.9939773 |
| learning_rate | 0.0003 |
| loss | 4.52 |
| n_updates | 2464 |
| policy_gradient_loss | -0.00124 |
| value_loss | 7.31 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 386 |
| iterations | 618 |
| time_elapsed | 3276 |
| total_timesteps | 1265664 |
| train/ | |
| approx_kl | 0.0020131548 |
| clip_fraction | 0.0144 |
| clip_range | 0.2 |
| entropy_loss | -0.639 |
| explained_variance | 0.9844865 |
| learning_rate | 0.0003 |
| loss | 2.43 |
| n_updates | 2468 |
| policy_gradient_loss | -0.000785 |
| value_loss | 15.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 289 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 386 |
| iterations | 619 |
| time_elapsed | 3280 |
| total_timesteps | 1267712 |
| train/ | |
| approx_kl | 0.0025731781 |
| clip_fraction | 0.0107 |
| clip_range | 0.2 |
| entropy_loss | -0.685 |
| explained_variance | 0.683859 |
| learning_rate | 0.0003 |
| loss | 613 |
| n_updates | 2472 |
| policy_gradient_loss | -0.000772 |
| value_loss | 925 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 297 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 386 |
| iterations | 620 |
| time_elapsed | 3284 |
| total_timesteps | 1269760 |
| train/ | |
| approx_kl | 0.004549235 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -0.699 |
| explained_variance | 0.8262527 |
| learning_rate | 0.0003 |
| loss | 304 |
| n_updates | 2476 |
| policy_gradient_loss | -0.00161 |
| value_loss | 405 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 304 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 386 |
| iterations | 621 |
| time_elapsed | 3290 |
| total_timesteps | 1271808 |
| train/ | |
| approx_kl | 0.004031927 |
| clip_fraction | 0.016 |
| clip_range | 0.2 |
| entropy_loss | -0.652 |
| explained_variance | 0.9873336 |
| learning_rate | 0.0003 |
| loss | 11.3 |
| n_updates | 2480 |
| policy_gradient_loss | -0.00184 |
| value_loss | 30.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 386 |
| iterations | 622 |
| time_elapsed | 3294 |
| total_timesteps | 1273856 |
| train/ | |
| approx_kl | 0.0028138016 |
| clip_fraction | 0.005 |
| clip_range | 0.2 |
| entropy_loss | -0.635 |
| explained_variance | 0.8886844 |
| learning_rate | 0.0003 |
| loss | 155 |
| n_updates | 2484 |
| policy_gradient_loss | -0.00117 |
| value_loss | 414 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 386 |
| iterations | 623 |
| time_elapsed | 3298 |
| total_timesteps | 1275904 |
| train/ | |
| approx_kl | 0.004276824 |
| clip_fraction | 0.0278 |
| clip_range | 0.2 |
| entropy_loss | -0.709 |
| explained_variance | 0.8357909 |
| learning_rate | 0.0003 |
| loss | 285 |
| n_updates | 2488 |
| policy_gradient_loss | -0.00343 |
| value_loss | 357 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 386 |
| iterations | 624 |
| time_elapsed | 3303 |
| total_timesteps | 1277952 |
| train/ | |
| approx_kl | 0.006196391 |
| clip_fraction | 0.0341 |
| clip_range | 0.2 |
| entropy_loss | -0.546 |
| explained_variance | 0.98365676 |
| learning_rate | 0.0003 |
| loss | 2.28 |
| n_updates | 2492 |
| policy_gradient_loss | -0.00119 |
| value_loss | 23.8 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1280000 to videos/step_1280000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 386 |
| iterations | 625 |
| time_elapsed | 3309 |
| total_timesteps | 1280000 |
| train/ | |
| approx_kl | 0.0032792538 |
| clip_fraction | 0.0161 |
| clip_range | 0.2 |
| entropy_loss | -0.657 |
| explained_variance | 0.97895616 |
| learning_rate | 0.0003 |
| loss | 14.3 |
| n_updates | 2496 |
| policy_gradient_loss | -0.00168 |
| value_loss | 27.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 386 |
| iterations | 626 |
| time_elapsed | 3313 |
| total_timesteps | 1282048 |
| train/ | |
| approx_kl | 0.0029064477 |
| clip_fraction | 0.0179 |
| clip_range | 0.2 |
| entropy_loss | -0.643 |
| explained_variance | 0.97668654 |
| learning_rate | 0.0003 |
| loss | 16.8 |
| n_updates | 2500 |
| policy_gradient_loss | -0.00165 |
| value_loss | 35.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 386 |
| iterations | 627 |
| time_elapsed | 3318 |
| total_timesteps | 1284096 |
| train/ | |
| approx_kl | 0.003044284 |
| clip_fraction | 0.0164 |
| clip_range | 0.2 |
| entropy_loss | -0.612 |
| explained_variance | 0.98878187 |
| learning_rate | 0.0003 |
| loss | 2.01 |
| n_updates | 2504 |
| policy_gradient_loss | -0.00154 |
| value_loss | 11.3 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 387 |
| iterations | 628 |
| time_elapsed | 3322 |
| total_timesteps | 1286144 |
| train/ | |
| approx_kl | 0.010896137 |
| clip_fraction | 0.133 |
| clip_range | 0.2 |
| entropy_loss | -0.665 |
| explained_variance | 0.9906387 |
| learning_rate | 0.0003 |
| loss | 3.37 |
| n_updates | 2508 |
| policy_gradient_loss | -0.00248 |
| value_loss | 10.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 387 |
| iterations | 629 |
| time_elapsed | 3327 |
| total_timesteps | 1288192 |
| train/ | |
| approx_kl | 0.0025254644 |
| clip_fraction | 0.0195 |
| clip_range | 0.2 |
| entropy_loss | -0.686 |
| explained_variance | 0.97950095 |
| learning_rate | 0.0003 |
| loss | 21.6 |
| n_updates | 2512 |
| policy_gradient_loss | -0.000185 |
| value_loss | 52.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 298 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 387 |
| iterations | 630 |
| time_elapsed | 3331 |
| total_timesteps | 1290240 |
| train/ | |
| approx_kl | 0.0037924862 |
| clip_fraction | 0.0231 |
| clip_range | 0.2 |
| entropy_loss | -0.614 |
| explained_variance | 0.98822635 |
| learning_rate | 0.0003 |
| loss | 4.27 |
| n_updates | 2516 |
| policy_gradient_loss | -0.00042 |
| value_loss | 20 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 306 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 387 |
| iterations | 631 |
| time_elapsed | 3336 |
| total_timesteps | 1292288 |
| train/ | |
| approx_kl | 0.0023084707 |
| clip_fraction | 0.0125 |
| clip_range | 0.2 |
| entropy_loss | -0.616 |
| explained_variance | 0.98890895 |
| learning_rate | 0.0003 |
| loss | 11.8 |
| n_updates | 2520 |
| policy_gradient_loss | -0.000813 |
| value_loss | 26.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 312 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 387 |
| iterations | 632 |
| time_elapsed | 3341 |
| total_timesteps | 1294336 |
| train/ | |
| approx_kl | 0.0012912375 |
| clip_fraction | 0.00378 |
| clip_range | 0.2 |
| entropy_loss | -0.651 |
| explained_variance | 0.98124653 |
| learning_rate | 0.0003 |
| loss | 23.3 |
| n_updates | 2524 |
| policy_gradient_loss | -0.000786 |
| value_loss | 59.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 320 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 387 |
| iterations | 633 |
| time_elapsed | 3345 |
| total_timesteps | 1296384 |
| train/ | |
| approx_kl | 0.002509032 |
| clip_fraction | 0.0466 |
| clip_range | 0.2 |
| entropy_loss | -0.522 |
| explained_variance | 0.992965 |
| learning_rate | 0.0003 |
| loss | 15.5 |
| n_updates | 2528 |
| policy_gradient_loss | -0.002 |
| value_loss | 19.5 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 322 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 387 |
| iterations | 634 |
| time_elapsed | 3349 |
| total_timesteps | 1298432 |
| train/ | |
| approx_kl | 0.007921219 |
| clip_fraction | 0.0593 |
| clip_range | 0.2 |
| entropy_loss | -0.644 |
| explained_variance | 0.9960064 |
| learning_rate | 0.0003 |
| loss | 2.75 |
| n_updates | 2532 |
| policy_gradient_loss | -0.000141 |
| value_loss | 9.34 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1300000 to videos/step_1300000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 330 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 387 |
| iterations | 635 |
| time_elapsed | 3356 |
| total_timesteps | 1300480 |
| train/ | |
| approx_kl | 0.0061084675 |
| clip_fraction | 0.0447 |
| clip_range | 0.2 |
| entropy_loss | -0.653 |
| explained_variance | 0.9829966 |
| learning_rate | 0.0003 |
| loss | 6.66 |
| n_updates | 2536 |
| policy_gradient_loss | -0.00151 |
| value_loss | 17.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 387 |
| iterations | 636 |
| time_elapsed | 3360 |
| total_timesteps | 1302528 |
| train/ | |
| approx_kl | 0.0021082594 |
| clip_fraction | 0.0249 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.9837553 |
| learning_rate | 0.0003 |
| loss | 32.4 |
| n_updates | 2540 |
| policy_gradient_loss | -0.00103 |
| value_loss | 59.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 387 |
| iterations | 637 |
| time_elapsed | 3365 |
| total_timesteps | 1304576 |
| train/ | |
| approx_kl | 0.0018637141 |
| clip_fraction | 0.00256 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.8353124 |
| learning_rate | 0.0003 |
| loss | 52.8 |
| n_updates | 2544 |
| policy_gradient_loss | -0.000713 |
| value_loss | 247 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 387 |
| iterations | 638 |
| time_elapsed | 3369 |
| total_timesteps | 1306624 |
| train/ | |
| approx_kl | 0.003256733 |
| clip_fraction | 0.0199 |
| clip_range | 0.2 |
| entropy_loss | -0.648 |
| explained_variance | 0.9850452 |
| learning_rate | 0.0003 |
| loss | 4.27 |
| n_updates | 2548 |
| policy_gradient_loss | -0.00173 |
| value_loss | 18.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 387 |
| iterations | 639 |
| time_elapsed | 3373 |
| total_timesteps | 1308672 |
| train/ | |
| approx_kl | 0.0058527635 |
| clip_fraction | 0.0432 |
| clip_range | 0.2 |
| entropy_loss | -0.651 |
| explained_variance | 0.9907397 |
| learning_rate | 0.0003 |
| loss | 2.63 |
| n_updates | 2552 |
| policy_gradient_loss | -0.000323 |
| value_loss | 10.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 387 |
| iterations | 640 |
| time_elapsed | 3378 |
| total_timesteps | 1310720 |
| train/ | |
| approx_kl | 0.0058209226 |
| clip_fraction | 0.0435 |
| clip_range | 0.2 |
| entropy_loss | -0.606 |
| explained_variance | 0.99535197 |
| learning_rate | 0.0003 |
| loss | 3.81 |
| n_updates | 2556 |
| policy_gradient_loss | -0.00101 |
| value_loss | 7.76 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 309 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 388 |
| iterations | 641 |
| time_elapsed | 3382 |
| total_timesteps | 1312768 |
| train/ | |
| approx_kl | 0.004681644 |
| clip_fraction | 0.0439 |
| clip_range | 0.2 |
| entropy_loss | -0.621 |
| explained_variance | 0.9941239 |
| learning_rate | 0.0003 |
| loss | 7.62 |
| n_updates | 2560 |
| policy_gradient_loss | -0.00282 |
| value_loss | 11.2 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 308 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 388 |
| iterations | 642 |
| time_elapsed | 3387 |
| total_timesteps | 1314816 |
| train/ | |
| approx_kl | 0.0014993362 |
| clip_fraction | 0.000488 |
| clip_range | 0.2 |
| entropy_loss | -0.635 |
| explained_variance | 0.8154776 |
| learning_rate | 0.0003 |
| loss | 159 |
| n_updates | 2564 |
| policy_gradient_loss | -0.00108 |
| value_loss | 479 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 302 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 388 |
| iterations | 643 |
| time_elapsed | 3392 |
| total_timesteps | 1316864 |
| train/ | |
| approx_kl | 0.036470607 |
| clip_fraction | 0.156 |
| clip_range | 0.2 |
| entropy_loss | -0.641 |
| explained_variance | 0.9948953 |
| learning_rate | 0.0003 |
| loss | 3 |
| n_updates | 2568 |
| policy_gradient_loss | -0.0126 |
| value_loss | 8.02 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 309 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 388 |
| iterations | 644 |
| time_elapsed | 3396 |
| total_timesteps | 1318912 |
| train/ | |
| approx_kl | 0.0026506423 |
| clip_fraction | 0.0128 |
| clip_range | 0.2 |
| entropy_loss | -0.64 |
| explained_variance | 0.8407806 |
| learning_rate | 0.0003 |
| loss | 141 |
| n_updates | 2572 |
| policy_gradient_loss | -0.00167 |
| value_loss | 451 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1320000 to videos/step_1320000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 388 |
| iterations | 645 |
| time_elapsed | 3403 |
| total_timesteps | 1320960 |
| train/ | |
| approx_kl | 0.0044366308 |
| clip_fraction | 0.0371 |
| clip_range | 0.2 |
| entropy_loss | -0.632 |
| explained_variance | 0.99234813 |
| learning_rate | 0.0003 |
| loss | 2.11 |
| n_updates | 2576 |
| policy_gradient_loss | -0.00122 |
| value_loss | 11.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 292 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 388 |
| iterations | 646 |
| time_elapsed | 3407 |
| total_timesteps | 1323008 |
| train/ | |
| approx_kl | 0.004079932 |
| clip_fraction | 0.0404 |
| clip_range | 0.2 |
| entropy_loss | -0.667 |
| explained_variance | 0.99489623 |
| learning_rate | 0.0003 |
| loss | 2.2 |
| n_updates | 2580 |
| policy_gradient_loss | 0.000181 |
| value_loss | 6.46 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 286 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 388 |
| iterations | 647 |
| time_elapsed | 3411 |
| total_timesteps | 1325056 |
| train/ | |
| approx_kl | 0.0024701543 |
| clip_fraction | 0.0541 |
| clip_range | 0.2 |
| entropy_loss | -0.721 |
| explained_variance | 0.99442434 |
| learning_rate | 0.0003 |
| loss | 3.46 |
| n_updates | 2584 |
| policy_gradient_loss | -0.00189 |
| value_loss | 7.82 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 296 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 388 |
| iterations | 648 |
| time_elapsed | 3416 |
| total_timesteps | 1327104 |
| train/ | |
| approx_kl | 0.0023839502 |
| clip_fraction | 0.032 |
| clip_range | 0.2 |
| entropy_loss | -0.625 |
| explained_variance | 0.9932854 |
| learning_rate | 0.0003 |
| loss | 3.48 |
| n_updates | 2588 |
| policy_gradient_loss | 0.000309 |
| value_loss | 9.49 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 388 |
| iterations | 649 |
| time_elapsed | 3421 |
| total_timesteps | 1329152 |
| train/ | |
| approx_kl | 0.015468331 |
| clip_fraction | 0.171 |
| clip_range | 0.2 |
| entropy_loss | -0.65 |
| explained_variance | 0.9945722 |
| learning_rate | 0.0003 |
| loss | 4.89 |
| n_updates | 2592 |
| policy_gradient_loss | -0.00656 |
| value_loss | 10.1 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 388 |
| iterations | 650 |
| time_elapsed | 3425 |
| total_timesteps | 1331200 |
| train/ | |
| approx_kl | 0.004659936 |
| clip_fraction | 0.0476 |
| clip_range | 0.2 |
| entropy_loss | -0.648 |
| explained_variance | 0.995663 |
| learning_rate | 0.0003 |
| loss | 1.57 |
| n_updates | 2596 |
| policy_gradient_loss | -0.00193 |
| value_loss | 4.87 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 388 |
| iterations | 651 |
| time_elapsed | 3430 |
| total_timesteps | 1333248 |
| train/ | |
| approx_kl | 0.0040516206 |
| clip_fraction | 0.0294 |
| clip_range | 0.2 |
| entropy_loss | -0.628 |
| explained_variance | 0.99586326 |
| learning_rate | 0.0003 |
| loss | 1.78 |
| n_updates | 2600 |
| policy_gradient_loss | -0.00107 |
| value_loss | 5.72 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 300 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 388 |
| iterations | 652 |
| time_elapsed | 3435 |
| total_timesteps | 1335296 |
| train/ | |
| approx_kl | 0.013267802 |
| clip_fraction | 0.0447 |
| clip_range | 0.2 |
| entropy_loss | -0.699 |
| explained_variance | 0.99504113 |
| learning_rate | 0.0003 |
| loss | 3.41 |
| n_updates | 2604 |
| policy_gradient_loss | -0.0037 |
| value_loss | 7.35 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 317 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 388 |
| iterations | 653 |
| time_elapsed | 3439 |
| total_timesteps | 1337344 |
| train/ | |
| approx_kl | 0.008421072 |
| clip_fraction | 0.053 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.99494714 |
| learning_rate | 0.0003 |
| loss | 5.32 |
| n_updates | 2608 |
| policy_gradient_loss | -0.00147 |
| value_loss | 7.81 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 323 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 388 |
| iterations | 654 |
| time_elapsed | 3444 |
| total_timesteps | 1339392 |
| train/ | |
| approx_kl | 0.0047957436 |
| clip_fraction | 0.0226 |
| clip_range | 0.2 |
| entropy_loss | -0.625 |
| explained_variance | 0.9964648 |
| learning_rate | 0.0003 |
| loss | 2.88 |
| n_updates | 2612 |
| policy_gradient_loss | -0.000175 |
| value_loss | 5.1 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1340000 to videos/step_1340000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 330 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 388 |
| iterations | 655 |
| time_elapsed | 3450 |
| total_timesteps | 1341440 |
| train/ | |
| approx_kl | 0.0057353135 |
| clip_fraction | 0.0596 |
| clip_range | 0.2 |
| entropy_loss | -0.639 |
| explained_variance | 0.99499476 |
| learning_rate | 0.0003 |
| loss | 2.9 |
| n_updates | 2616 |
| policy_gradient_loss | -0.00497 |
| value_loss | 8.12 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 334 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 388 |
| iterations | 656 |
| time_elapsed | 3455 |
| total_timesteps | 1343488 |
| train/ | |
| approx_kl | 0.0011389242 |
| clip_fraction | 0.00244 |
| clip_range | 0.2 |
| entropy_loss | -0.589 |
| explained_variance | 0.86723953 |
| learning_rate | 0.0003 |
| loss | 37.8 |
| n_updates | 2620 |
| policy_gradient_loss | -0.000776 |
| value_loss | 378 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 335 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 388 |
| iterations | 657 |
| time_elapsed | 3459 |
| total_timesteps | 1345536 |
| train/ | |
| approx_kl | 0.0019702916 |
| clip_fraction | 0.0201 |
| clip_range | 0.2 |
| entropy_loss | -0.64 |
| explained_variance | 0.97791296 |
| learning_rate | 0.0003 |
| loss | 3.19 |
| n_updates | 2624 |
| policy_gradient_loss | 0.000743 |
| value_loss | 16 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 332 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 389 |
| iterations | 658 |
| time_elapsed | 3463 |
| total_timesteps | 1347584 |
| train/ | |
| approx_kl | 0.004579045 |
| clip_fraction | 0.0505 |
| clip_range | 0.2 |
| entropy_loss | -0.671 |
| explained_variance | 0.9924093 |
| learning_rate | 0.0003 |
| loss | 3.46 |
| n_updates | 2628 |
| policy_gradient_loss | -0.00157 |
| value_loss | 8.99 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 333 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 389 |
| iterations | 659 |
| time_elapsed | 3468 |
| total_timesteps | 1349632 |
| train/ | |
| approx_kl | 0.0018873042 |
| clip_fraction | 0.0211 |
| clip_range | 0.2 |
| entropy_loss | -0.596 |
| explained_variance | 0.9924744 |
| learning_rate | 0.0003 |
| loss | 2.36 |
| n_updates | 2632 |
| policy_gradient_loss | -0.00108 |
| value_loss | 9.95 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 322 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 389 |
| iterations | 660 |
| time_elapsed | 3472 |
| total_timesteps | 1351680 |
| train/ | |
| approx_kl | 0.0072434563 |
| clip_fraction | 0.048 |
| clip_range | 0.2 |
| entropy_loss | -0.622 |
| explained_variance | 0.98976153 |
| learning_rate | 0.0003 |
| loss | 5.75 |
| n_updates | 2636 |
| policy_gradient_loss | -0.00409 |
| value_loss | 13.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 326 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 389 |
| iterations | 661 |
| time_elapsed | 3476 |
| total_timesteps | 1353728 |
| train/ | |
| approx_kl | 0.004819939 |
| clip_fraction | 0.0441 |
| clip_range | 0.2 |
| entropy_loss | -0.695 |
| explained_variance | 0.9958946 |
| learning_rate | 0.0003 |
| loss | 1.64 |
| n_updates | 2640 |
| policy_gradient_loss | -0.000354 |
| value_loss | 5.71 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 327 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 389 |
| iterations | 662 |
| time_elapsed | 3481 |
| total_timesteps | 1355776 |
| train/ | |
| approx_kl | 0.0018156868 |
| clip_fraction | 0.00867 |
| clip_range | 0.2 |
| entropy_loss | -0.716 |
| explained_variance | 0.7946298 |
| learning_rate | 0.0003 |
| loss | 68.3 |
| n_updates | 2644 |
| policy_gradient_loss | -1.33e-06 |
| value_loss | 449 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 319 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 389 |
| iterations | 663 |
| time_elapsed | 3485 |
| total_timesteps | 1357824 |
| train/ | |
| approx_kl | 0.0057681296 |
| clip_fraction | 0.0382 |
| clip_range | 0.2 |
| entropy_loss | -0.655 |
| explained_variance | 0.9926801 |
| learning_rate | 0.0003 |
| loss | 2.12 |
| n_updates | 2648 |
| policy_gradient_loss | -0.00344 |
| value_loss | 7.98 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 317 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 389 |
| iterations | 664 |
| time_elapsed | 3489 |
| total_timesteps | 1359872 |
| train/ | |
| approx_kl | 0.0013095047 |
| clip_fraction | 0.0129 |
| clip_range | 0.2 |
| entropy_loss | -0.665 |
| explained_variance | 0.98220575 |
| learning_rate | 0.0003 |
| loss | 3.47 |
| n_updates | 2652 |
| policy_gradient_loss | -0.000381 |
| value_loss | 16 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1360000 to videos/step_1360000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 311 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 389 |
| iterations | 665 |
| time_elapsed | 3496 |
| total_timesteps | 1361920 |
| train/ | |
| approx_kl | 0.0043460806 |
| clip_fraction | 0.0299 |
| clip_range | 0.2 |
| entropy_loss | -0.716 |
| explained_variance | 0.9940832 |
| learning_rate | 0.0003 |
| loss | 2.72 |
| n_updates | 2656 |
| policy_gradient_loss | -0.00079 |
| value_loss | 6.95 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 317 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 389 |
| iterations | 666 |
| time_elapsed | 3501 |
| total_timesteps | 1363968 |
| train/ | |
| approx_kl | 0.0028404112 |
| clip_fraction | 0.0117 |
| clip_range | 0.2 |
| entropy_loss | -0.69 |
| explained_variance | 0.86387175 |
| learning_rate | 0.0003 |
| loss | 78.7 |
| n_updates | 2660 |
| policy_gradient_loss | -0.000193 |
| value_loss | 192 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 298 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 389 |
| iterations | 667 |
| time_elapsed | 3505 |
| total_timesteps | 1366016 |
| train/ | |
| approx_kl | 0.00064635766 |
| clip_fraction | 0.000244 |
| clip_range | 0.2 |
| entropy_loss | -0.796 |
| explained_variance | 0.75969946 |
| learning_rate | 0.0003 |
| loss | 226 |
| n_updates | 2664 |
| policy_gradient_loss | -0.000533 |
| value_loss | 468 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 389 |
| iterations | 668 |
| time_elapsed | 3510 |
| total_timesteps | 1368064 |
| train/ | |
| approx_kl | 0.004507986 |
| clip_fraction | 0.0204 |
| clip_range | 0.2 |
| entropy_loss | -0.632 |
| explained_variance | 0.97847486 |
| learning_rate | 0.0003 |
| loss | 3.68 |
| n_updates | 2668 |
| policy_gradient_loss | -0.00115 |
| value_loss | 21 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 291 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 389 |
| iterations | 669 |
| time_elapsed | 3514 |
| total_timesteps | 1370112 |
| train/ | |
| approx_kl | 0.005863975 |
| clip_fraction | 0.0626 |
| clip_range | 0.2 |
| entropy_loss | -0.654 |
| explained_variance | 0.99311703 |
| learning_rate | 0.0003 |
| loss | 2.24 |
| n_updates | 2672 |
| policy_gradient_loss | 0.00113 |
| value_loss | 10 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 288 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 389 |
| iterations | 670 |
| time_elapsed | 3518 |
| total_timesteps | 1372160 |
| train/ | |
| approx_kl | 0.002526504 |
| clip_fraction | 0.0426 |
| clip_range | 0.2 |
| entropy_loss | -0.652 |
| explained_variance | 0.9897164 |
| learning_rate | 0.0003 |
| loss | 8.26 |
| n_updates | 2676 |
| policy_gradient_loss | 0.000399 |
| value_loss | 15.8 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 389 |
| iterations | 671 |
| time_elapsed | 3523 |
| total_timesteps | 1374208 |
| train/ | |
| approx_kl | 0.0042028306 |
| clip_fraction | 0.0319 |
| clip_range | 0.2 |
| entropy_loss | -0.615 |
| explained_variance | 0.99474144 |
| learning_rate | 0.0003 |
| loss | 2.56 |
| n_updates | 2680 |
| policy_gradient_loss | -0.00201 |
| value_loss | 7.25 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 279 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 390 |
| iterations | 672 |
| time_elapsed | 3527 |
| total_timesteps | 1376256 |
| train/ | |
| approx_kl | 0.0039215046 |
| clip_fraction | 0.0403 |
| clip_range | 0.2 |
| entropy_loss | -0.646 |
| explained_variance | 0.99516356 |
| learning_rate | 0.0003 |
| loss | 2.46 |
| n_updates | 2684 |
| policy_gradient_loss | -0.000683 |
| value_loss | 5.64 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 286 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 390 |
| iterations | 673 |
| time_elapsed | 3532 |
| total_timesteps | 1378304 |
| train/ | |
| approx_kl | 0.0016937004 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -0.664 |
| explained_variance | 0.7255012 |
| learning_rate | 0.0003 |
| loss | 33 |
| n_updates | 2688 |
| policy_gradient_loss | 0.000284 |
| value_loss | 503 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1380000 to videos/step_1380000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 390 |
| iterations | 674 |
| time_elapsed | 3538 |
| total_timesteps | 1380352 |
| train/ | |
| approx_kl | 0.004305351 |
| clip_fraction | 0.0283 |
| clip_range | 0.2 |
| entropy_loss | -0.579 |
| explained_variance | 0.9880536 |
| learning_rate | 0.0003 |
| loss | 3.29 |
| n_updates | 2692 |
| policy_gradient_loss | -0.00151 |
| value_loss | 10.5 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 390 |
| iterations | 675 |
| time_elapsed | 3542 |
| total_timesteps | 1382400 |
| train/ | |
| approx_kl | 0.004061859 |
| clip_fraction | 0.0278 |
| clip_range | 0.2 |
| entropy_loss | -0.675 |
| explained_variance | 0.9676556 |
| learning_rate | 0.0003 |
| loss | 7.17 |
| n_updates | 2696 |
| policy_gradient_loss | -0.00134 |
| value_loss | 23.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 390 |
| iterations | 676 |
| time_elapsed | 3547 |
| total_timesteps | 1384448 |
| train/ | |
| approx_kl | 0.0049687745 |
| clip_fraction | 0.0294 |
| clip_range | 0.2 |
| entropy_loss | -0.695 |
| explained_variance | 0.9896339 |
| learning_rate | 0.0003 |
| loss | 3.16 |
| n_updates | 2700 |
| policy_gradient_loss | -0.00122 |
| value_loss | 8.54 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 390 |
| iterations | 677 |
| time_elapsed | 3551 |
| total_timesteps | 1386496 |
| train/ | |
| approx_kl | 0.0043798424 |
| clip_fraction | 0.0255 |
| clip_range | 0.2 |
| entropy_loss | -0.648 |
| explained_variance | 0.98741835 |
| learning_rate | 0.0003 |
| loss | 5.1 |
| n_updates | 2704 |
| policy_gradient_loss | -0.000402 |
| value_loss | 13.6 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 390 |
| iterations | 678 |
| time_elapsed | 3555 |
| total_timesteps | 1388544 |
| train/ | |
| approx_kl | 0.0041079577 |
| clip_fraction | 0.0321 |
| clip_range | 0.2 |
| entropy_loss | -0.627 |
| explained_variance | 0.9949776 |
| learning_rate | 0.0003 |
| loss | 2.46 |
| n_updates | 2708 |
| policy_gradient_loss | -0.0028 |
| value_loss | 6.28 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 390 |
| iterations | 679 |
| time_elapsed | 3560 |
| total_timesteps | 1390592 |
| train/ | |
| approx_kl | 0.00326363 |
| clip_fraction | 0.0255 |
| clip_range | 0.2 |
| entropy_loss | -0.604 |
| explained_variance | 0.9950385 |
| learning_rate | 0.0003 |
| loss | 2.76 |
| n_updates | 2712 |
| policy_gradient_loss | -0.00155 |
| value_loss | 7.41 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 277 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 390 |
| iterations | 680 |
| time_elapsed | 3564 |
| total_timesteps | 1392640 |
| train/ | |
| approx_kl | 0.004500663 |
| clip_fraction | 0.0422 |
| clip_range | 0.2 |
| entropy_loss | -0.627 |
| explained_variance | 0.9972717 |
| learning_rate | 0.0003 |
| loss | 2.09 |
| n_updates | 2716 |
| policy_gradient_loss | -0.00161 |
| value_loss | 4.67 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 390 |
| iterations | 681 |
| time_elapsed | 3568 |
| total_timesteps | 1394688 |
| train/ | |
| approx_kl | 0.0027247877 |
| clip_fraction | 0.00952 |
| clip_range | 0.2 |
| entropy_loss | -0.683 |
| explained_variance | 0.81746674 |
| learning_rate | 0.0003 |
| loss | 87.3 |
| n_updates | 2720 |
| policy_gradient_loss | -0.000798 |
| value_loss | 305 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 390 |
| iterations | 682 |
| time_elapsed | 3573 |
| total_timesteps | 1396736 |
| train/ | |
| approx_kl | 0.003198687 |
| clip_fraction | 0.0132 |
| clip_range | 0.2 |
| entropy_loss | -0.658 |
| explained_variance | 0.8225952 |
| learning_rate | 0.0003 |
| loss | 492 |
| n_updates | 2724 |
| policy_gradient_loss | -0.00124 |
| value_loss | 491 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 391 |
| iterations | 683 |
| time_elapsed | 3577 |
| total_timesteps | 1398784 |
| train/ | |
| approx_kl | 0.004086039 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -0.644 |
| explained_variance | 0.98647165 |
| learning_rate | 0.0003 |
| loss | 2.3 |
| n_updates | 2728 |
| policy_gradient_loss | -0.000759 |
| value_loss | 13.2 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1400000 to videos/step_1400000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 262 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 390 |
| iterations | 684 |
| time_elapsed | 3584 |
| total_timesteps | 1400832 |
| train/ | |
| approx_kl | 0.004154918 |
| clip_fraction | 0.0596 |
| clip_range | 0.2 |
| entropy_loss | -0.641 |
| explained_variance | 0.9940812 |
| learning_rate | 0.0003 |
| loss | 1.7 |
| n_updates | 2732 |
| policy_gradient_loss | 0.00151 |
| value_loss | 8.39 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 270 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 390 |
| iterations | 685 |
| time_elapsed | 3588 |
| total_timesteps | 1402880 |
| train/ | |
| approx_kl | 0.009836486 |
| clip_fraction | 0.0813 |
| clip_range | 0.2 |
| entropy_loss | -0.598 |
| explained_variance | 0.97594464 |
| learning_rate | 0.0003 |
| loss | 4.4 |
| n_updates | 2736 |
| policy_gradient_loss | -0.00189 |
| value_loss | 25.5 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 391 |
| iterations | 686 |
| time_elapsed | 3592 |
| total_timesteps | 1404928 |
| train/ | |
| approx_kl | 0.001988841 |
| clip_fraction | 0.00476 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.8387521 |
| learning_rate | 0.0003 |
| loss | 290 |
| n_updates | 2740 |
| policy_gradient_loss | -0.00253 |
| value_loss | 395 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 391 |
| iterations | 687 |
| time_elapsed | 3597 |
| total_timesteps | 1406976 |
| train/ | |
| approx_kl | 0.00025090255 |
| clip_fraction | 0.000732 |
| clip_range | 0.2 |
| entropy_loss | -0.649 |
| explained_variance | 0.8718238 |
| learning_rate | 0.0003 |
| loss | 3.52 |
| n_updates | 2744 |
| policy_gradient_loss | -0.000182 |
| value_loss | 164 |
-------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 391 |
| iterations | 688 |
| time_elapsed | 3601 |
| total_timesteps | 1409024 |
| train/ | |
| approx_kl | 0.00086915644 |
| clip_fraction | 0.0011 |
| clip_range | 0.2 |
| entropy_loss | -0.637 |
| explained_variance | 0.92824787 |
| learning_rate | 0.0003 |
| loss | 24.2 |
| n_updates | 2748 |
| policy_gradient_loss | -0.000291 |
| value_loss | 143 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 276 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 391 |
| iterations | 689 |
| time_elapsed | 3606 |
| total_timesteps | 1411072 |
| train/ | |
| approx_kl | 0.0037091696 |
| clip_fraction | 0.0138 |
| clip_range | 0.2 |
| entropy_loss | -0.636 |
| explained_variance | 0.9548951 |
| learning_rate | 0.0003 |
| loss | 24.2 |
| n_updates | 2752 |
| policy_gradient_loss | -0.00126 |
| value_loss | 88.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 391 |
| iterations | 690 |
| time_elapsed | 3610 |
| total_timesteps | 1413120 |
| train/ | |
| approx_kl | 0.0028498783 |
| clip_fraction | 0.0153 |
| clip_range | 0.2 |
| entropy_loss | -0.72 |
| explained_variance | 0.9098606 |
| learning_rate | 0.0003 |
| loss | 3.92 |
| n_updates | 2756 |
| policy_gradient_loss | -0.00163 |
| value_loss | 33.9 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 391 |
| iterations | 691 |
| time_elapsed | 3615 |
| total_timesteps | 1415168 |
| train/ | |
| approx_kl | 0.012056267 |
| clip_fraction | 0.139 |
| clip_range | 0.2 |
| entropy_loss | -0.633 |
| explained_variance | 0.9913794 |
| learning_rate | 0.0003 |
| loss | 14 |
| n_updates | 2760 |
| policy_gradient_loss | -0.00641 |
| value_loss | 32.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 391 |
| iterations | 692 |
| time_elapsed | 3619 |
| total_timesteps | 1417216 |
| train/ | |
| approx_kl | 0.0041996166 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -0.606 |
| explained_variance | 0.84413016 |
| learning_rate | 0.0003 |
| loss | 241 |
| n_updates | 2764 |
| policy_gradient_loss | -0.00239 |
| value_loss | 355 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 391 |
| iterations | 693 |
| time_elapsed | 3623 |
| total_timesteps | 1419264 |
| train/ | |
| approx_kl | 0.0042103655 |
| clip_fraction | 0.0245 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.9869571 |
| learning_rate | 0.0003 |
| loss | 2.47 |
| n_updates | 2768 |
| policy_gradient_loss | -0.00035 |
| value_loss | 10.2 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1420000 to videos/step_1420000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 391 |
| iterations | 694 |
| time_elapsed | 3629 |
| total_timesteps | 1421312 |
| train/ | |
| approx_kl | 0.0039007468 |
| clip_fraction | 0.0269 |
| clip_range | 0.2 |
| entropy_loss | -0.625 |
| explained_variance | 0.9734799 |
| learning_rate | 0.0003 |
| loss | 6.64 |
| n_updates | 2772 |
| policy_gradient_loss | -0.0017 |
| value_loss | 23.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 391 |
| iterations | 695 |
| time_elapsed | 3633 |
| total_timesteps | 1423360 |
| train/ | |
| approx_kl | 0.0022087716 |
| clip_fraction | 0.0061 |
| clip_range | 0.2 |
| entropy_loss | -0.691 |
| explained_variance | 0.8169008 |
| learning_rate | 0.0003 |
| loss | 118 |
| n_updates | 2776 |
| policy_gradient_loss | -0.000192 |
| value_loss | 307 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 391 |
| iterations | 696 |
| time_elapsed | 3638 |
| total_timesteps | 1425408 |
| train/ | |
| approx_kl | 0.0050296756 |
| clip_fraction | 0.0472 |
| clip_range | 0.2 |
| entropy_loss | -0.632 |
| explained_variance | 0.9854786 |
| learning_rate | 0.0003 |
| loss | 2.15 |
| n_updates | 2780 |
| policy_gradient_loss | -0.00157 |
| value_loss | 11.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 281 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 391 |
| iterations | 697 |
| time_elapsed | 3642 |
| total_timesteps | 1427456 |
| train/ | |
| approx_kl | 0.0032467113 |
| clip_fraction | 0.0348 |
| clip_range | 0.2 |
| entropy_loss | -0.591 |
| explained_variance | 0.98894244 |
| learning_rate | 0.0003 |
| loss | 2.02 |
| n_updates | 2784 |
| policy_gradient_loss | -0.000388 |
| value_loss | 8.54 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 391 |
| iterations | 698 |
| time_elapsed | 3646 |
| total_timesteps | 1429504 |
| train/ | |
| approx_kl | 0.004968533 |
| clip_fraction | 0.0615 |
| clip_range | 0.2 |
| entropy_loss | -0.652 |
| explained_variance | 0.99219596 |
| learning_rate | 0.0003 |
| loss | 1.56 |
| n_updates | 2788 |
| policy_gradient_loss | -0.00127 |
| value_loss | 8.21 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 392 |
| iterations | 699 |
| time_elapsed | 3651 |
| total_timesteps | 1431552 |
| train/ | |
| approx_kl | 0.0027611232 |
| clip_fraction | 0.0365 |
| clip_range | 0.2 |
| entropy_loss | -0.648 |
| explained_variance | 0.98703957 |
| learning_rate | 0.0003 |
| loss | 6.46 |
| n_updates | 2792 |
| policy_gradient_loss | -0.00233 |
| value_loss | 21.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 392 |
| iterations | 700 |
| time_elapsed | 3655 |
| total_timesteps | 1433600 |
| train/ | |
| approx_kl | 0.006519519 |
| clip_fraction | 0.0575 |
| clip_range | 0.2 |
| entropy_loss | -0.629 |
| explained_variance | 0.9962302 |
| learning_rate | 0.0003 |
| loss | 1.63 |
| n_updates | 2796 |
| policy_gradient_loss | -0.000989 |
| value_loss | 4.62 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 277 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 392 |
| iterations | 701 |
| time_elapsed | 3659 |
| total_timesteps | 1435648 |
| train/ | |
| approx_kl | 0.0042621717 |
| clip_fraction | 0.056 |
| clip_range | 0.2 |
| entropy_loss | -0.672 |
| explained_variance | 0.99695927 |
| learning_rate | 0.0003 |
| loss | 1.07 |
| n_updates | 2800 |
| policy_gradient_loss | -0.000407 |
| value_loss | 3.69 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 269 |
| time/ | |
| fps | 392 |
| iterations | 702 |
| time_elapsed | 3664 |
| total_timesteps | 1437696 |
| train/ | |
| approx_kl | 0.004831264 |
| clip_fraction | 0.0422 |
| clip_range | 0.2 |
| entropy_loss | -0.704 |
| explained_variance | 0.9976316 |
| learning_rate | 0.0003 |
| loss | 3.61 |
| n_updates | 2804 |
| policy_gradient_loss | -0.000247 |
| value_loss | 4.74 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 392 |
| iterations | 703 |
| time_elapsed | 3668 |
| total_timesteps | 1439744 |
| train/ | |
| approx_kl | 0.0015847088 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -0.581 |
| explained_variance | 0.9932552 |
| learning_rate | 0.0003 |
| loss | 4.56 |
| n_updates | 2808 |
| policy_gradient_loss | -0.000201 |
| value_loss | 11.3 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1440000 to videos/step_1440000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 392 |
| iterations | 704 |
| time_elapsed | 3674 |
| total_timesteps | 1441792 |
| train/ | |
| approx_kl | 0.004215926 |
| clip_fraction | 0.0378 |
| clip_range | 0.2 |
| entropy_loss | -0.595 |
| explained_variance | 0.9980561 |
| learning_rate | 0.0003 |
| loss | 2.84 |
| n_updates | 2812 |
| policy_gradient_loss | -0.000938 |
| value_loss | 5.21 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 392 |
| iterations | 705 |
| time_elapsed | 3679 |
| total_timesteps | 1443840 |
| train/ | |
| approx_kl | 0.0035940018 |
| clip_fraction | 0.0319 |
| clip_range | 0.2 |
| entropy_loss | -0.64 |
| explained_variance | 0.9941391 |
| learning_rate | 0.0003 |
| loss | 3.6 |
| n_updates | 2816 |
| policy_gradient_loss | -0.002 |
| value_loss | 9.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 392 |
| iterations | 706 |
| time_elapsed | 3683 |
| total_timesteps | 1445888 |
| train/ | |
| approx_kl | 0.0032453383 |
| clip_fraction | 0.0542 |
| clip_range | 0.2 |
| entropy_loss | -0.624 |
| explained_variance | 0.9964419 |
| learning_rate | 0.0003 |
| loss | 1.94 |
| n_updates | 2820 |
| policy_gradient_loss | -0.00406 |
| value_loss | 6.04 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 275 |
| time/ | |
| fps | 392 |
| iterations | 707 |
| time_elapsed | 3687 |
| total_timesteps | 1447936 |
| train/ | |
| approx_kl | 0.0037989821 |
| clip_fraction | 0.0249 |
| clip_range | 0.2 |
| entropy_loss | -0.647 |
| explained_variance | 0.9972043 |
| learning_rate | 0.0003 |
| loss | 1.82 |
| n_updates | 2824 |
| policy_gradient_loss | -0.000505 |
| value_loss | 5.19 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 274 |
| time/ | |
| fps | 392 |
| iterations | 708 |
| time_elapsed | 3691 |
| total_timesteps | 1449984 |
| train/ | |
| approx_kl | 0.0059284316 |
| clip_fraction | 0.0405 |
| clip_range | 0.2 |
| entropy_loss | -0.637 |
| explained_variance | 0.9956856 |
| learning_rate | 0.0003 |
| loss | 3.08 |
| n_updates | 2828 |
| policy_gradient_loss | -0.00335 |
| value_loss | 8.27 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 274 |
| time/ | |
| fps | 392 |
| iterations | 709 |
| time_elapsed | 3695 |
| total_timesteps | 1452032 |
| train/ | |
| approx_kl | 0.004127157 |
| clip_fraction | 0.0249 |
| clip_range | 0.2 |
| entropy_loss | -0.64 |
| explained_variance | 0.99662834 |
| learning_rate | 0.0003 |
| loss | 3.24 |
| n_updates | 2832 |
| policy_gradient_loss | -0.000378 |
| value_loss | 6.47 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 265 |
| ep_rew_mean | 274 |
| time/ | |
| fps | 392 |
| iterations | 710 |
| time_elapsed | 3700 |
| total_timesteps | 1454080 |
| train/ | |
| approx_kl | 0.0027565486 |
| clip_fraction | 0.0388 |
| clip_range | 0.2 |
| entropy_loss | -0.597 |
| explained_variance | 0.9971468 |
| learning_rate | 0.0003 |
| loss | 1.8 |
| n_updates | 2836 |
| policy_gradient_loss | -0.00223 |
| value_loss | 5.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 263 |
| ep_rew_mean | 274 |
| time/ | |
| fps | 393 |
| iterations | 711 |
| time_elapsed | 3704 |
| total_timesteps | 1456128 |
| train/ | |
| approx_kl | 0.0029892144 |
| clip_fraction | 0.0219 |
| clip_range | 0.2 |
| entropy_loss | -0.607 |
| explained_variance | 0.9966973 |
| learning_rate | 0.0003 |
| loss | 4.21 |
| n_updates | 2840 |
| policy_gradient_loss | -0.00103 |
| value_loss | 5.6 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 393 |
| iterations | 712 |
| time_elapsed | 3708 |
| total_timesteps | 1458176 |
| train/ | |
| approx_kl | 0.003388902 |
| clip_fraction | 0.0311 |
| clip_range | 0.2 |
| entropy_loss | -0.644 |
| explained_variance | 0.9938521 |
| learning_rate | 0.0003 |
| loss | 1.93 |
| n_updates | 2844 |
| policy_gradient_loss | -0.00263 |
| value_loss | 6.84 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1460000 to videos/step_1460000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 392 |
| iterations | 713 |
| time_elapsed | 3715 |
| total_timesteps | 1460224 |
| train/ | |
| approx_kl | 0.0016215213 |
| clip_fraction | 0.00659 |
| clip_range | 0.2 |
| entropy_loss | -0.592 |
| explained_variance | 0.84522563 |
| learning_rate | 0.0003 |
| loss | 31.1 |
| n_updates | 2848 |
| policy_gradient_loss | -0.000881 |
| value_loss | 298 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 393 |
| iterations | 714 |
| time_elapsed | 3719 |
| total_timesteps | 1462272 |
| train/ | |
| approx_kl | 0.004580777 |
| clip_fraction | 0.0264 |
| clip_range | 0.2 |
| entropy_loss | -0.584 |
| explained_variance | 0.9535775 |
| learning_rate | 0.0003 |
| loss | 4.8 |
| n_updates | 2852 |
| policy_gradient_loss | -0.00272 |
| value_loss | 35.6 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 252 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 393 |
| iterations | 715 |
| time_elapsed | 3723 |
| total_timesteps | 1464320 |
| train/ | |
| approx_kl | 0.008408526 |
| clip_fraction | 0.0558 |
| clip_range | 0.2 |
| entropy_loss | -0.619 |
| explained_variance | 0.99355406 |
| learning_rate | 0.0003 |
| loss | 1.83 |
| n_updates | 2856 |
| policy_gradient_loss | -0.00235 |
| value_loss | 7.92 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 252 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 393 |
| iterations | 716 |
| time_elapsed | 3728 |
| total_timesteps | 1466368 |
| train/ | |
| approx_kl | 0.003573222 |
| clip_fraction | 0.0194 |
| clip_range | 0.2 |
| entropy_loss | -0.638 |
| explained_variance | 0.84298146 |
| learning_rate | 0.0003 |
| loss | 519 |
| n_updates | 2860 |
| policy_gradient_loss | -0.00124 |
| value_loss | 468 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 253 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 393 |
| iterations | 717 |
| time_elapsed | 3732 |
| total_timesteps | 1468416 |
| train/ | |
| approx_kl | 0.0045451485 |
| clip_fraction | 0.032 |
| clip_range | 0.2 |
| entropy_loss | -0.617 |
| explained_variance | 0.9796441 |
| learning_rate | 0.0003 |
| loss | 5.62 |
| n_updates | 2864 |
| policy_gradient_loss | -0.000454 |
| value_loss | 25.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 269 |
| time/ | |
| fps | 393 |
| iterations | 718 |
| time_elapsed | 3736 |
| total_timesteps | 1470464 |
| train/ | |
| approx_kl | 0.0024789968 |
| clip_fraction | 0.00781 |
| clip_range | 0.2 |
| entropy_loss | -0.595 |
| explained_variance | 0.9569934 |
| learning_rate | 0.0003 |
| loss | 14 |
| n_updates | 2868 |
| policy_gradient_loss | -0.00157 |
| value_loss | 35 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 393 |
| iterations | 719 |
| time_elapsed | 3741 |
| total_timesteps | 1472512 |
| train/ | |
| approx_kl | 0.0003410796 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.574 |
| explained_variance | 0.8299763 |
| learning_rate | 0.0003 |
| loss | 155 |
| n_updates | 2872 |
| policy_gradient_loss | -8.04e-05 |
| value_loss | 417 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 393 |
| iterations | 720 |
| time_elapsed | 3745 |
| total_timesteps | 1474560 |
| train/ | |
| approx_kl | 0.000622927 |
| clip_fraction | 0.000977 |
| clip_range | 0.2 |
| entropy_loss | -0.6 |
| explained_variance | 0.8057435 |
| learning_rate | 0.0003 |
| loss | 26.9 |
| n_updates | 2876 |
| policy_gradient_loss | -0.00108 |
| value_loss | 543 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 393 |
| iterations | 721 |
| time_elapsed | 3748 |
| total_timesteps | 1476608 |
| train/ | |
| approx_kl | 0.0028444233 |
| clip_fraction | 0.0172 |
| clip_range | 0.2 |
| entropy_loss | -0.619 |
| explained_variance | 0.9906143 |
| learning_rate | 0.0003 |
| loss | 4.65 |
| n_updates | 2880 |
| policy_gradient_loss | 6.58e-05 |
| value_loss | 11.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 393 |
| iterations | 722 |
| time_elapsed | 3753 |
| total_timesteps | 1478656 |
| train/ | |
| approx_kl | 0.0047926256 |
| clip_fraction | 0.0369 |
| clip_range | 0.2 |
| entropy_loss | -0.575 |
| explained_variance | 0.99540055 |
| learning_rate | 0.0003 |
| loss | 2.2 |
| n_updates | 2884 |
| policy_gradient_loss | -0.00168 |
| value_loss | 5.79 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1480000 to videos/step_1480000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 393 |
| iterations | 723 |
| time_elapsed | 3759 |
| total_timesteps | 1480704 |
| train/ | |
| approx_kl | 0.0011786895 |
| clip_fraction | 0.0134 |
| clip_range | 0.2 |
| entropy_loss | -0.632 |
| explained_variance | 0.8360879 |
| learning_rate | 0.0003 |
| loss | 145 |
| n_updates | 2888 |
| policy_gradient_loss | -0.00034 |
| value_loss | 409 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 256 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 393 |
| iterations | 724 |
| time_elapsed | 3763 |
| total_timesteps | 1482752 |
| train/ | |
| approx_kl | 0.0027758095 |
| clip_fraction | 0.0103 |
| clip_range | 0.2 |
| entropy_loss | -0.59 |
| explained_variance | 0.91027606 |
| learning_rate | 0.0003 |
| loss | 37.8 |
| n_updates | 2892 |
| policy_gradient_loss | -0.00283 |
| value_loss | 205 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 394 |
| iterations | 725 |
| time_elapsed | 3767 |
| total_timesteps | 1484800 |
| train/ | |
| approx_kl | 0.0028239544 |
| clip_fraction | 0.025 |
| clip_range | 0.2 |
| entropy_loss | -0.595 |
| explained_variance | 0.9422463 |
| learning_rate | 0.0003 |
| loss | 14.5 |
| n_updates | 2896 |
| policy_gradient_loss | -0.00105 |
| value_loss | 83.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 394 |
| iterations | 726 |
| time_elapsed | 3772 |
| total_timesteps | 1486848 |
| train/ | |
| approx_kl | 0.004491948 |
| clip_fraction | 0.0309 |
| clip_range | 0.2 |
| entropy_loss | -0.618 |
| explained_variance | 0.95258117 |
| learning_rate | 0.0003 |
| loss | 7.99 |
| n_updates | 2900 |
| policy_gradient_loss | -0.00128 |
| value_loss | 50.4 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 265 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 394 |
| iterations | 727 |
| time_elapsed | 3776 |
| total_timesteps | 1488896 |
| train/ | |
| approx_kl | 0.006146407 |
| clip_fraction | 0.0693 |
| clip_range | 0.2 |
| entropy_loss | -0.607 |
| explained_variance | 0.9954057 |
| learning_rate | 0.0003 |
| loss | 1.39 |
| n_updates | 2904 |
| policy_gradient_loss | -0.0011 |
| value_loss | 7.47 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 394 |
| iterations | 728 |
| time_elapsed | 3781 |
| total_timesteps | 1490944 |
| train/ | |
| approx_kl | 0.00056083826 |
| clip_fraction | 0.00427 |
| clip_range | 0.2 |
| entropy_loss | -0.581 |
| explained_variance | 0.84029055 |
| learning_rate | 0.0003 |
| loss | 62.4 |
| n_updates | 2908 |
| policy_gradient_loss | -0.000303 |
| value_loss | 433 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 394 |
| iterations | 729 |
| time_elapsed | 3785 |
| total_timesteps | 1492992 |
| train/ | |
| approx_kl | 0.0050530042 |
| clip_fraction | 0.0253 |
| clip_range | 0.2 |
| entropy_loss | -0.623 |
| explained_variance | 0.9818198 |
| learning_rate | 0.0003 |
| loss | 1.44 |
| n_updates | 2912 |
| policy_gradient_loss | -0.000769 |
| value_loss | 13.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 263 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 394 |
| iterations | 730 |
| time_elapsed | 3789 |
| total_timesteps | 1495040 |
| train/ | |
| approx_kl | 0.0011275883 |
| clip_fraction | 0.0161 |
| clip_range | 0.2 |
| entropy_loss | -0.563 |
| explained_variance | 0.98072875 |
| learning_rate | 0.0003 |
| loss | 6.38 |
| n_updates | 2916 |
| policy_gradient_loss | -0.000145 |
| value_loss | 22.6 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 394 |
| iterations | 731 |
| time_elapsed | 3793 |
| total_timesteps | 1497088 |
| train/ | |
| approx_kl | 0.00517329 |
| clip_fraction | 0.0361 |
| clip_range | 0.2 |
| entropy_loss | -0.585 |
| explained_variance | 0.99533594 |
| learning_rate | 0.0003 |
| loss | 1.78 |
| n_updates | 2920 |
| policy_gradient_loss | 0.000415 |
| value_loss | 6.99 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 394 |
| iterations | 732 |
| time_elapsed | 3797 |
| total_timesteps | 1499136 |
| train/ | |
| approx_kl | 0.0021764273 |
| clip_fraction | 0.0256 |
| clip_range | 0.2 |
| entropy_loss | -0.597 |
| explained_variance | 0.9904587 |
| learning_rate | 0.0003 |
| loss | 1.42 |
| n_updates | 2924 |
| policy_gradient_loss | -0.00057 |
| value_loss | 20.1 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1500000 to videos/step_1500000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 394 |
| iterations | 733 |
| time_elapsed | 3804 |
| total_timesteps | 1501184 |
| train/ | |
| approx_kl | 0.0030499129 |
| clip_fraction | 0.0184 |
| clip_range | 0.2 |
| entropy_loss | -0.557 |
| explained_variance | 0.9900172 |
| learning_rate | 0.0003 |
| loss | 3.9 |
| n_updates | 2928 |
| policy_gradient_loss | -0.00267 |
| value_loss | 15.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 266 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 394 |
| iterations | 734 |
| time_elapsed | 3808 |
| total_timesteps | 1503232 |
| train/ | |
| approx_kl | 0.0031805607 |
| clip_fraction | 0.0145 |
| clip_range | 0.2 |
| entropy_loss | -0.596 |
| explained_variance | 0.8289238 |
| learning_rate | 0.0003 |
| loss | 19.4 |
| n_updates | 2932 |
| policy_gradient_loss | -0.00154 |
| value_loss | 553 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 394 |
| iterations | 735 |
| time_elapsed | 3812 |
| total_timesteps | 1505280 |
| train/ | |
| approx_kl | 0.0005191387 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.641 |
| explained_variance | 0.7343677 |
| learning_rate | 0.0003 |
| loss | 247 |
| n_updates | 2936 |
| policy_gradient_loss | 3.91e-06 |
| value_loss | 877 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 262 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 394 |
| iterations | 736 |
| time_elapsed | 3816 |
| total_timesteps | 1507328 |
| train/ | |
| approx_kl | 0.0013307481 |
| clip_fraction | 0.0033 |
| clip_range | 0.2 |
| entropy_loss | -0.619 |
| explained_variance | 0.8509443 |
| learning_rate | 0.0003 |
| loss | 34 |
| n_updates | 2940 |
| policy_gradient_loss | -0.000884 |
| value_loss | 214 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 395 |
| iterations | 737 |
| time_elapsed | 3821 |
| total_timesteps | 1509376 |
| train/ | |
| approx_kl | 0.0056602443 |
| clip_fraction | 0.0205 |
| clip_range | 0.2 |
| entropy_loss | -0.598 |
| explained_variance | 0.9662442 |
| learning_rate | 0.0003 |
| loss | 6.35 |
| n_updates | 2944 |
| policy_gradient_loss | -0.000947 |
| value_loss | 29.7 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 395 |
| iterations | 738 |
| time_elapsed | 3825 |
| total_timesteps | 1511424 |
| train/ | |
| approx_kl | 0.00086633273 |
| clip_fraction | 0.00159 |
| clip_range | 0.2 |
| entropy_loss | -0.631 |
| explained_variance | 0.75032264 |
| learning_rate | 0.0003 |
| loss | 254 |
| n_updates | 2948 |
| policy_gradient_loss | -0.000383 |
| value_loss | 734 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 255 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 395 |
| iterations | 739 |
| time_elapsed | 3829 |
| total_timesteps | 1513472 |
| train/ | |
| approx_kl | 0.0041499957 |
| clip_fraction | 0.0181 |
| clip_range | 0.2 |
| entropy_loss | -0.571 |
| explained_variance | 0.9738935 |
| learning_rate | 0.0003 |
| loss | 19.5 |
| n_updates | 2952 |
| policy_gradient_loss | -0.000842 |
| value_loss | 31.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 247 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 395 |
| iterations | 740 |
| time_elapsed | 3833 |
| total_timesteps | 1515520 |
| train/ | |
| approx_kl | 0.0018024195 |
| clip_fraction | 0.0336 |
| clip_range | 0.2 |
| entropy_loss | -0.618 |
| explained_variance | 0.8395061 |
| learning_rate | 0.0003 |
| loss | 71.7 |
| n_updates | 2956 |
| policy_gradient_loss | -0.000454 |
| value_loss | 436 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 248 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 395 |
| iterations | 741 |
| time_elapsed | 3837 |
| total_timesteps | 1517568 |
| train/ | |
| approx_kl | 0.0033984627 |
| clip_fraction | 0.0194 |
| clip_range | 0.2 |
| entropy_loss | -0.633 |
| explained_variance | 0.8342404 |
| learning_rate | 0.0003 |
| loss | 299 |
| n_updates | 2960 |
| policy_gradient_loss | -0.00133 |
| value_loss | 516 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 395 |
| iterations | 742 |
| time_elapsed | 3842 |
| total_timesteps | 1519616 |
| train/ | |
| approx_kl | 0.0041169194 |
| clip_fraction | 0.0193 |
| clip_range | 0.2 |
| entropy_loss | -0.601 |
| explained_variance | 0.9713325 |
| learning_rate | 0.0003 |
| loss | 9.21 |
| n_updates | 2964 |
| policy_gradient_loss | -0.000533 |
| value_loss | 41 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1520000 to videos/step_1520000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 395 |
| iterations | 743 |
| time_elapsed | 3848 |
| total_timesteps | 1521664 |
| train/ | |
| approx_kl | 0.006377744 |
| clip_fraction | 0.0502 |
| clip_range | 0.2 |
| entropy_loss | -0.546 |
| explained_variance | 0.9950376 |
| learning_rate | 0.0003 |
| loss | 1.14 |
| n_updates | 2968 |
| policy_gradient_loss | 0.0013 |
| value_loss | 7.84 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 253 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 395 |
| iterations | 744 |
| time_elapsed | 3852 |
| total_timesteps | 1523712 |
| train/ | |
| approx_kl | 0.0048488886 |
| clip_fraction | 0.0833 |
| clip_range | 0.2 |
| entropy_loss | -0.557 |
| explained_variance | 0.8113289 |
| learning_rate | 0.0003 |
| loss | 35 |
| n_updates | 2972 |
| policy_gradient_loss | -0.000952 |
| value_loss | 511 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 254 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 395 |
| iterations | 745 |
| time_elapsed | 3857 |
| total_timesteps | 1525760 |
| train/ | |
| approx_kl | 0.0020443709 |
| clip_fraction | 0.0072 |
| clip_range | 0.2 |
| entropy_loss | -0.577 |
| explained_variance | 0.84467936 |
| learning_rate | 0.0003 |
| loss | 76.9 |
| n_updates | 2976 |
| policy_gradient_loss | -0.000736 |
| value_loss | 460 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 395 |
| iterations | 746 |
| time_elapsed | 3862 |
| total_timesteps | 1527808 |
| train/ | |
| approx_kl | 0.002459976 |
| clip_fraction | 0.0182 |
| clip_range | 0.2 |
| entropy_loss | -0.583 |
| explained_variance | 0.83412415 |
| learning_rate | 0.0003 |
| loss | 123 |
| n_updates | 2980 |
| policy_gradient_loss | -0.000609 |
| value_loss | 561 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 268 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 395 |
| iterations | 747 |
| time_elapsed | 3866 |
| total_timesteps | 1529856 |
| train/ | |
| approx_kl | 0.0024110745 |
| clip_fraction | 0.005 |
| clip_range | 0.2 |
| entropy_loss | -0.61 |
| explained_variance | 0.9638849 |
| learning_rate | 0.0003 |
| loss | 52.9 |
| n_updates | 2984 |
| policy_gradient_loss | -0.000499 |
| value_loss | 65.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 276 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 395 |
| iterations | 748 |
| time_elapsed | 3871 |
| total_timesteps | 1531904 |
| train/ | |
| approx_kl | 0.0023144605 |
| clip_fraction | 0.0146 |
| clip_range | 0.2 |
| entropy_loss | -0.601 |
| explained_variance | 0.77582407 |
| learning_rate | 0.0003 |
| loss | 359 |
| n_updates | 2988 |
| policy_gradient_loss | -0.00211 |
| value_loss | 953 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 239 |
| time/ | |
| fps | 395 |
| iterations | 749 |
| time_elapsed | 3875 |
| total_timesteps | 1533952 |
| train/ | |
| approx_kl | 0.00096742436 |
| clip_fraction | 0.00122 |
| clip_range | 0.2 |
| entropy_loss | -0.624 |
| explained_variance | 0.6970885 |
| learning_rate | 0.0003 |
| loss | 534 |
| n_updates | 2992 |
| policy_gradient_loss | -0.00054 |
| value_loss | 865 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 395 |
| iterations | 750 |
| time_elapsed | 3879 |
| total_timesteps | 1536000 |
| train/ | |
| approx_kl | 0.003580173 |
| clip_fraction | 0.0208 |
| clip_range | 0.2 |
| entropy_loss | -0.656 |
| explained_variance | 0.7586651 |
| learning_rate | 0.0003 |
| loss | 64.7 |
| n_updates | 2996 |
| policy_gradient_loss | -0.00171 |
| value_loss | 317 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 395 |
| iterations | 751 |
| time_elapsed | 3884 |
| total_timesteps | 1538048 |
| train/ | |
| approx_kl | 0.0036181253 |
| clip_fraction | 0.0215 |
| clip_range | 0.2 |
| entropy_loss | -0.558 |
| explained_variance | 0.9769035 |
| learning_rate | 0.0003 |
| loss | 6.02 |
| n_updates | 3000 |
| policy_gradient_loss | -0.00297 |
| value_loss | 28 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1540000 to videos/step_1540000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 395 |
| iterations | 752 |
| time_elapsed | 3890 |
| total_timesteps | 1540096 |
| train/ | |
| approx_kl | 0.009465795 |
| clip_fraction | 0.0562 |
| clip_range | 0.2 |
| entropy_loss | -0.577 |
| explained_variance | 0.9748012 |
| learning_rate | 0.0003 |
| loss | 4.35 |
| n_updates | 3004 |
| policy_gradient_loss | -0.0052 |
| value_loss | 28 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 294 |
| ep_rew_mean | 240 |
| time/ | |
| fps | 395 |
| iterations | 753 |
| time_elapsed | 3895 |
| total_timesteps | 1542144 |
| train/ | |
| approx_kl | 0.005149789 |
| clip_fraction | 0.0294 |
| clip_range | 0.2 |
| entropy_loss | -0.66 |
| explained_variance | 0.9811777 |
| learning_rate | 0.0003 |
| loss | 3.91 |
| n_updates | 3008 |
| policy_gradient_loss | -0.00119 |
| value_loss | 11.7 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 303 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 396 |
| iterations | 754 |
| time_elapsed | 3899 |
| total_timesteps | 1544192 |
| train/ | |
| approx_kl | 0.002155921 |
| clip_fraction | 0.019 |
| clip_range | 0.2 |
| entropy_loss | -0.598 |
| explained_variance | 0.8207224 |
| learning_rate | 0.0003 |
| loss | 83.4 |
| n_updates | 3012 |
| policy_gradient_loss | -0.000216 |
| value_loss | 431 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 304 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 396 |
| iterations | 755 |
| time_elapsed | 3903 |
| total_timesteps | 1546240 |
| train/ | |
| approx_kl | 0.0049965978 |
| clip_fraction | 0.0432 |
| clip_range | 0.2 |
| entropy_loss | -0.544 |
| explained_variance | 0.98781073 |
| learning_rate | 0.0003 |
| loss | 5.32 |
| n_updates | 3016 |
| policy_gradient_loss | -0.00162 |
| value_loss | 18 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 310 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 396 |
| iterations | 756 |
| time_elapsed | 3908 |
| total_timesteps | 1548288 |
| train/ | |
| approx_kl | 0.004996009 |
| clip_fraction | 0.0333 |
| clip_range | 0.2 |
| entropy_loss | -0.553 |
| explained_variance | 0.991483 |
| learning_rate | 0.0003 |
| loss | 2.64 |
| n_updates | 3020 |
| policy_gradient_loss | 0.000952 |
| value_loss | 12.6 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 396 |
| iterations | 757 |
| time_elapsed | 3912 |
| total_timesteps | 1550336 |
| train/ | |
| approx_kl | 0.004401019 |
| clip_fraction | 0.051 |
| clip_range | 0.2 |
| entropy_loss | -0.504 |
| explained_variance | 0.99672115 |
| learning_rate | 0.0003 |
| loss | 1.53 |
| n_updates | 3024 |
| policy_gradient_loss | -0.00139 |
| value_loss | 5.69 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 303 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 396 |
| iterations | 758 |
| time_elapsed | 3916 |
| total_timesteps | 1552384 |
| train/ | |
| approx_kl | 0.001904845 |
| clip_fraction | 0.0509 |
| clip_range | 0.2 |
| entropy_loss | -0.574 |
| explained_variance | 0.85863143 |
| learning_rate | 0.0003 |
| loss | 377 |
| n_updates | 3028 |
| policy_gradient_loss | 0.00345 |
| value_loss | 474 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 396 |
| iterations | 759 |
| time_elapsed | 3921 |
| total_timesteps | 1554432 |
| train/ | |
| approx_kl | 0.0006054827 |
| clip_fraction | 0.00159 |
| clip_range | 0.2 |
| entropy_loss | -0.537 |
| explained_variance | 0.8467863 |
| learning_rate | 0.0003 |
| loss | 268 |
| n_updates | 3032 |
| policy_gradient_loss | -0.000596 |
| value_loss | 498 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 323 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 396 |
| iterations | 760 |
| time_elapsed | 3925 |
| total_timesteps | 1556480 |
| train/ | |
| approx_kl | 0.003590916 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -0.501 |
| explained_variance | 0.9850743 |
| learning_rate | 0.0003 |
| loss | 3.97 |
| n_updates | 3036 |
| policy_gradient_loss | -0.000726 |
| value_loss | 25.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 313 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 396 |
| iterations | 761 |
| time_elapsed | 3929 |
| total_timesteps | 1558528 |
| train/ | |
| approx_kl | 0.0017908404 |
| clip_fraction | 0.00317 |
| clip_range | 0.2 |
| entropy_loss | -0.557 |
| explained_variance | 0.9762625 |
| learning_rate | 0.0003 |
| loss | 12.8 |
| n_updates | 3040 |
| policy_gradient_loss | 0.000277 |
| value_loss | 78.9 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1560000 to videos/step_1560000.mp4
-------------------------------------------
| rollout/ | |
| ep_len_mean | 299 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 396 |
| iterations | 762 |
| time_elapsed | 3936 |
| total_timesteps | 1560576 |
| train/ | |
| approx_kl | 5.6131103e-05 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.607 |
| explained_variance | 0.70197445 |
| learning_rate | 0.0003 |
| loss | 526 |
| n_updates | 3044 |
| policy_gradient_loss | 0.000278 |
| value_loss | 1.12e+03 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 301 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 396 |
| iterations | 763 |
| time_elapsed | 3940 |
| total_timesteps | 1562624 |
| train/ | |
| approx_kl | 0.0019091468 |
| clip_fraction | 0.00659 |
| clip_range | 0.2 |
| entropy_loss | -0.567 |
| explained_variance | 0.8196249 |
| learning_rate | 0.0003 |
| loss | 30.9 |
| n_updates | 3048 |
| policy_gradient_loss | -0.00266 |
| value_loss | 493 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 295 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 396 |
| iterations | 764 |
| time_elapsed | 3944 |
| total_timesteps | 1564672 |
| train/ | |
| approx_kl | 0.003055185 |
| clip_fraction | 0.0216 |
| clip_range | 0.2 |
| entropy_loss | -0.561 |
| explained_variance | 0.9401352 |
| learning_rate | 0.0003 |
| loss | 8.36 |
| n_updates | 3052 |
| policy_gradient_loss | -0.00314 |
| value_loss | 59.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 293 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 396 |
| iterations | 765 |
| time_elapsed | 3948 |
| total_timesteps | 1566720 |
| train/ | |
| approx_kl | 0.0036616218 |
| clip_fraction | 0.0315 |
| clip_range | 0.2 |
| entropy_loss | -0.535 |
| explained_variance | 0.9823583 |
| learning_rate | 0.0003 |
| loss | 5.69 |
| n_updates | 3056 |
| policy_gradient_loss | -0.00135 |
| value_loss | 24.2 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 304 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 396 |
| iterations | 766 |
| time_elapsed | 3953 |
| total_timesteps | 1568768 |
| train/ | |
| approx_kl | 0.012625932 |
| clip_fraction | 0.0645 |
| clip_range | 0.2 |
| entropy_loss | -0.598 |
| explained_variance | 0.9942024 |
| learning_rate | 0.0003 |
| loss | 2.12 |
| n_updates | 3060 |
| policy_gradient_loss | -0.00123 |
| value_loss | 9.52 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 314 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 396 |
| iterations | 767 |
| time_elapsed | 3958 |
| total_timesteps | 1570816 |
| train/ | |
| approx_kl | 0.010354817 |
| clip_fraction | 0.105 |
| clip_range | 0.2 |
| entropy_loss | -0.608 |
| explained_variance | 0.98756415 |
| learning_rate | 0.0003 |
| loss | 1.15 |
| n_updates | 3064 |
| policy_gradient_loss | 0.00029 |
| value_loss | 13.3 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 319 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 396 |
| iterations | 768 |
| time_elapsed | 3962 |
| total_timesteps | 1572864 |
| train/ | |
| approx_kl | 0.004532901 |
| clip_fraction | 0.0688 |
| clip_range | 0.2 |
| entropy_loss | -0.542 |
| explained_variance | 0.99355894 |
| learning_rate | 0.0003 |
| loss | 1.78 |
| n_updates | 3068 |
| policy_gradient_loss | 0.00176 |
| value_loss | 6.02 |
-----------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 329 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 396 |
| iterations | 769 |
| time_elapsed | 3967 |
| total_timesteps | 1574912 |
| train/ | |
| approx_kl | 0.00318205 |
| clip_fraction | 0.0369 |
| clip_range | 0.2 |
| entropy_loss | -0.442 |
| explained_variance | 0.99367183 |
| learning_rate | 0.0003 |
| loss | 1.77 |
| n_updates | 3072 |
| policy_gradient_loss | -0.00169 |
| value_loss | 8.83 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 396 |
| iterations | 770 |
| time_elapsed | 3972 |
| total_timesteps | 1576960 |
| train/ | |
| approx_kl | 0.003869291 |
| clip_fraction | 0.0488 |
| clip_range | 0.2 |
| entropy_loss | -0.537 |
| explained_variance | 0.99635637 |
| learning_rate | 0.0003 |
| loss | 2.53 |
| n_updates | 3076 |
| policy_gradient_loss | -0.000636 |
| value_loss | 6.64 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 338 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 397 |
| iterations | 771 |
| time_elapsed | 3976 |
| total_timesteps | 1579008 |
| train/ | |
| approx_kl | 0.005801455 |
| clip_fraction | 0.0576 |
| clip_range | 0.2 |
| entropy_loss | -0.538 |
| explained_variance | 0.99520904 |
| learning_rate | 0.0003 |
| loss | 3.64 |
| n_updates | 3080 |
| policy_gradient_loss | -0.00279 |
| value_loss | 11.8 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1580000 to videos/step_1580000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 339 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 396 |
| iterations | 772 |
| time_elapsed | 3983 |
| total_timesteps | 1581056 |
| train/ | |
| approx_kl | 0.004817994 |
| clip_fraction | 0.0713 |
| clip_range | 0.2 |
| entropy_loss | -0.488 |
| explained_variance | 0.99298817 |
| learning_rate | 0.0003 |
| loss | 7.69 |
| n_updates | 3084 |
| policy_gradient_loss | -0.00632 |
| value_loss | 18.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 339 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 397 |
| iterations | 773 |
| time_elapsed | 3987 |
| total_timesteps | 1583104 |
| train/ | |
| approx_kl | 0.0040104026 |
| clip_fraction | 0.0353 |
| clip_range | 0.2 |
| entropy_loss | -0.511 |
| explained_variance | 0.9937507 |
| learning_rate | 0.0003 |
| loss | 9.38 |
| n_updates | 3088 |
| policy_gradient_loss | -0.00293 |
| value_loss | 16.8 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 337 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 397 |
| iterations | 774 |
| time_elapsed | 3991 |
| total_timesteps | 1585152 |
| train/ | |
| approx_kl | 0.005530652 |
| clip_fraction | 0.0332 |
| clip_range | 0.2 |
| entropy_loss | -0.636 |
| explained_variance | 0.9885835 |
| learning_rate | 0.0003 |
| loss | 3.8 |
| n_updates | 3092 |
| policy_gradient_loss | -0.00142 |
| value_loss | 9.85 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 331 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 397 |
| iterations | 775 |
| time_elapsed | 3995 |
| total_timesteps | 1587200 |
| train/ | |
| approx_kl | 0.0037049586 |
| clip_fraction | 0.0225 |
| clip_range | 0.2 |
| entropy_loss | -0.585 |
| explained_variance | 0.7436567 |
| learning_rate | 0.0003 |
| loss | 370 |
| n_updates | 3096 |
| policy_gradient_loss | -0.00227 |
| value_loss | 878 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 329 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 397 |
| iterations | 776 |
| time_elapsed | 4000 |
| total_timesteps | 1589248 |
| train/ | |
| approx_kl | 0.0056831427 |
| clip_fraction | 0.047 |
| clip_range | 0.2 |
| entropy_loss | -0.57 |
| explained_variance | 0.9937094 |
| learning_rate | 0.0003 |
| loss | 6.19 |
| n_updates | 3100 |
| policy_gradient_loss | -0.00252 |
| value_loss | 12.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 332 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 397 |
| iterations | 777 |
| time_elapsed | 4004 |
| total_timesteps | 1591296 |
| train/ | |
| approx_kl | 0.0041535087 |
| clip_fraction | 0.0237 |
| clip_range | 0.2 |
| entropy_loss | -0.6 |
| explained_variance | 0.9839924 |
| learning_rate | 0.0003 |
| loss | 25.5 |
| n_updates | 3104 |
| policy_gradient_loss | -0.00368 |
| value_loss | 68.3 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 333 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 397 |
| iterations | 778 |
| time_elapsed | 4009 |
| total_timesteps | 1593344 |
| train/ | |
| approx_kl | 0.001644151 |
| clip_fraction | 0.00525 |
| clip_range | 0.2 |
| entropy_loss | -0.597 |
| explained_variance | 0.8462776 |
| learning_rate | 0.0003 |
| loss | 427 |
| n_updates | 3108 |
| policy_gradient_loss | -0.00162 |
| value_loss | 465 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 340 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 397 |
| iterations | 779 |
| time_elapsed | 4013 |
| total_timesteps | 1595392 |
| train/ | |
| approx_kl | 0.0034656655 |
| clip_fraction | 0.0269 |
| clip_range | 0.2 |
| entropy_loss | -0.61 |
| explained_variance | 0.9777452 |
| learning_rate | 0.0003 |
| loss | 10.8 |
| n_updates | 3112 |
| policy_gradient_loss | -0.00144 |
| value_loss | 25.1 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 338 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 397 |
| iterations | 780 |
| time_elapsed | 4017 |
| total_timesteps | 1597440 |
| train/ | |
| approx_kl | 0.019092817 |
| clip_fraction | 0.138 |
| clip_range | 0.2 |
| entropy_loss | -0.612 |
| explained_variance | 0.99230605 |
| learning_rate | 0.0003 |
| loss | 2.06 |
| n_updates | 3116 |
| policy_gradient_loss | -0.00862 |
| value_loss | 7.34 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 337 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 397 |
| iterations | 781 |
| time_elapsed | 4022 |
| total_timesteps | 1599488 |
| train/ | |
| approx_kl | 0.0015304127 |
| clip_fraction | 0.0315 |
| clip_range | 0.2 |
| entropy_loss | -0.618 |
| explained_variance | 0.8319112 |
| learning_rate | 0.0003 |
| loss | 316 |
| n_updates | 3120 |
| policy_gradient_loss | -0.000135 |
| value_loss | 445 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1600000 to videos/step_1600000.mp4
----------------------------------------
| rollout/ | |
| ep_len_mean | 321 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 397 |
| iterations | 782 |
| time_elapsed | 4028 |
| total_timesteps | 1601536 |
| train/ | |
| approx_kl | 0.00297886 |
| clip_fraction | 0.0123 |
| clip_range | 0.2 |
| entropy_loss | -0.559 |
| explained_variance | 0.96583855 |
| learning_rate | 0.0003 |
| loss | 7.71 |
| n_updates | 3124 |
| policy_gradient_loss | -0.00044 |
| value_loss | 44.5 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 290 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 397 |
| iterations | 783 |
| time_elapsed | 4032 |
| total_timesteps | 1603584 |
| train/ | |
| approx_kl | 0.007285265 |
| clip_fraction | 0.0564 |
| clip_range | 0.2 |
| entropy_loss | -0.614 |
| explained_variance | 0.9957987 |
| learning_rate | 0.0003 |
| loss | 2.17 |
| n_updates | 3128 |
| policy_gradient_loss | -0.0014 |
| value_loss | 5.44 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 397 |
| iterations | 784 |
| time_elapsed | 4036 |
| total_timesteps | 1605632 |
| train/ | |
| approx_kl | 0.0028210774 |
| clip_fraction | 0.0266 |
| clip_range | 0.2 |
| entropy_loss | -0.529 |
| explained_variance | 0.98666 |
| learning_rate | 0.0003 |
| loss | 5.47 |
| n_updates | 3132 |
| policy_gradient_loss | -0.00148 |
| value_loss | 19.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 270 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 397 |
| iterations | 785 |
| time_elapsed | 4040 |
| total_timesteps | 1607680 |
| train/ | |
| approx_kl | 0.0022829087 |
| clip_fraction | 0.0184 |
| clip_range | 0.2 |
| entropy_loss | -0.597 |
| explained_variance | 0.834154 |
| learning_rate | 0.0003 |
| loss | 59.6 |
| n_updates | 3136 |
| policy_gradient_loss | -0.0021 |
| value_loss | 465 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 259 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 397 |
| iterations | 786 |
| time_elapsed | 4045 |
| total_timesteps | 1609728 |
| train/ | |
| approx_kl | 0.0012038739 |
| clip_fraction | 0.00183 |
| clip_range | 0.2 |
| entropy_loss | -0.587 |
| explained_variance | 0.8640431 |
| learning_rate | 0.0003 |
| loss | 245 |
| n_updates | 3140 |
| policy_gradient_loss | -0.000927 |
| value_loss | 396 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 397 |
| iterations | 787 |
| time_elapsed | 4049 |
| total_timesteps | 1611776 |
| train/ | |
| approx_kl | 0.00010998722 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.697 |
| explained_variance | 0.80049837 |
| learning_rate | 0.0003 |
| loss | 41.3 |
| n_updates | 3144 |
| policy_gradient_loss | -0.000164 |
| value_loss | 489 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 258 |
| time/ | |
| fps | 398 |
| iterations | 788 |
| time_elapsed | 4054 |
| total_timesteps | 1613824 |
| train/ | |
| approx_kl | 0.0019516845 |
| clip_fraction | 0.0072 |
| clip_range | 0.2 |
| entropy_loss | -0.627 |
| explained_variance | 0.87508345 |
| learning_rate | 0.0003 |
| loss | 25 |
| n_updates | 3148 |
| policy_gradient_loss | -0.000937 |
| value_loss | 125 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 398 |
| iterations | 789 |
| time_elapsed | 4058 |
| total_timesteps | 1615872 |
| train/ | |
| approx_kl | 0.002619762 |
| clip_fraction | 0.015 |
| clip_range | 0.2 |
| entropy_loss | -0.649 |
| explained_variance | 0.96601987 |
| learning_rate | 0.0003 |
| loss | 3.21 |
| n_updates | 3152 |
| policy_gradient_loss | 0.000548 |
| value_loss | 30.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 398 |
| iterations | 790 |
| time_elapsed | 4063 |
| total_timesteps | 1617920 |
| train/ | |
| approx_kl | 0.0013155942 |
| clip_fraction | 0.0135 |
| clip_range | 0.2 |
| entropy_loss | -0.644 |
| explained_variance | 0.8658554 |
| learning_rate | 0.0003 |
| loss | 230 |
| n_updates | 3156 |
| policy_gradient_loss | -0.000388 |
| value_loss | 364 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 276 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 398 |
| iterations | 791 |
| time_elapsed | 4067 |
| total_timesteps | 1619968 |
| train/ | |
| approx_kl | 0.0037940538 |
| clip_fraction | 0.0215 |
| clip_range | 0.2 |
| entropy_loss | -0.584 |
| explained_variance | 0.9481982 |
| learning_rate | 0.0003 |
| loss | 15.2 |
| n_updates | 3160 |
| policy_gradient_loss | -0.000812 |
| value_loss | 59.8 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1620000 to videos/step_1620000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 266 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 398 |
| iterations | 792 |
| time_elapsed | 4073 |
| total_timesteps | 1622016 |
| train/ | |
| approx_kl | 0.0013771625 |
| clip_fraction | 0.00427 |
| clip_range | 0.2 |
| entropy_loss | -0.675 |
| explained_variance | 0.7739105 |
| learning_rate | 0.0003 |
| loss | 234 |
| n_updates | 3164 |
| policy_gradient_loss | -0.00142 |
| value_loss | 424 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 398 |
| iterations | 793 |
| time_elapsed | 4078 |
| total_timesteps | 1624064 |
| train/ | |
| approx_kl | 0.0026594023 |
| clip_fraction | 0.00928 |
| clip_range | 0.2 |
| entropy_loss | -0.586 |
| explained_variance | 0.8290772 |
| learning_rate | 0.0003 |
| loss | 221 |
| n_updates | 3168 |
| policy_gradient_loss | -0.00273 |
| value_loss | 395 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 398 |
| iterations | 794 |
| time_elapsed | 4082 |
| total_timesteps | 1626112 |
| train/ | |
| approx_kl | 0.013110102 |
| clip_fraction | 0.116 |
| clip_range | 0.2 |
| entropy_loss | -0.634 |
| explained_variance | 0.98731256 |
| learning_rate | 0.0003 |
| loss | 3.51 |
| n_updates | 3172 |
| policy_gradient_loss | -0.0012 |
| value_loss | 16.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 282 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 398 |
| iterations | 795 |
| time_elapsed | 4086 |
| total_timesteps | 1628160 |
| train/ | |
| approx_kl | 0.0026573087 |
| clip_fraction | 0.0438 |
| clip_range | 0.2 |
| entropy_loss | -0.617 |
| explained_variance | 0.98117226 |
| learning_rate | 0.0003 |
| loss | 5.07 |
| n_updates | 3176 |
| policy_gradient_loss | -6.48e-05 |
| value_loss | 25 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 398 |
| iterations | 796 |
| time_elapsed | 4090 |
| total_timesteps | 1630208 |
| train/ | |
| approx_kl | 0.0017384613 |
| clip_fraction | 0.0422 |
| clip_range | 0.2 |
| entropy_loss | -0.544 |
| explained_variance | 0.715835 |
| learning_rate | 0.0003 |
| loss | 80.8 |
| n_updates | 3180 |
| policy_gradient_loss | -0.00128 |
| value_loss | 890 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 287 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 398 |
| iterations | 797 |
| time_elapsed | 4095 |
| total_timesteps | 1632256 |
| train/ | |
| approx_kl | 0.004386698 |
| clip_fraction | 0.0311 |
| clip_range | 0.2 |
| entropy_loss | -0.566 |
| explained_variance | 0.9640812 |
| learning_rate | 0.0003 |
| loss | 16.7 |
| n_updates | 3184 |
| policy_gradient_loss | -0.00243 |
| value_loss | 55.1 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 398 |
| iterations | 798 |
| time_elapsed | 4099 |
| total_timesteps | 1634304 |
| train/ | |
| approx_kl | 0.001258018 |
| clip_fraction | 0.00403 |
| clip_range | 0.2 |
| entropy_loss | -0.621 |
| explained_variance | 0.8426976 |
| learning_rate | 0.0003 |
| loss | 247 |
| n_updates | 3188 |
| policy_gradient_loss | -0.00139 |
| value_loss | 631 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 284 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 398 |
| iterations | 799 |
| time_elapsed | 4104 |
| total_timesteps | 1636352 |
| train/ | |
| approx_kl | 0.0031420176 |
| clip_fraction | 0.0115 |
| clip_range | 0.2 |
| entropy_loss | -0.577 |
| explained_variance | 0.8660121 |
| learning_rate | 0.0003 |
| loss | 117 |
| n_updates | 3192 |
| policy_gradient_loss | -0.00126 |
| value_loss | 398 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 285 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 398 |
| iterations | 800 |
| time_elapsed | 4108 |
| total_timesteps | 1638400 |
| train/ | |
| approx_kl | 0.0102773085 |
| clip_fraction | 0.0791 |
| clip_range | 0.2 |
| entropy_loss | -0.687 |
| explained_variance | 0.983272 |
| learning_rate | 0.0003 |
| loss | 4.02 |
| n_updates | 3196 |
| policy_gradient_loss | -0.00211 |
| value_loss | 19 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1640000 to videos/step_1640000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 398 |
| iterations | 801 |
| time_elapsed | 4115 |
| total_timesteps | 1640448 |
| train/ | |
| approx_kl | 0.0051639304 |
| clip_fraction | 0.0569 |
| clip_range | 0.2 |
| entropy_loss | -0.628 |
| explained_variance | 0.8403573 |
| learning_rate | 0.0003 |
| loss | 249 |
| n_updates | 3200 |
| policy_gradient_loss | -0.00211 |
| value_loss | 478 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 398 |
| iterations | 802 |
| time_elapsed | 4118 |
| total_timesteps | 1642496 |
| train/ | |
| approx_kl | 0.0007210319 |
| clip_fraction | 0.000122 |
| clip_range | 0.2 |
| entropy_loss | -0.537 |
| explained_variance | 0.84844834 |
| learning_rate | 0.0003 |
| loss | 59.7 |
| n_updates | 3204 |
| policy_gradient_loss | 4.63e-05 |
| value_loss | 431 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 398 |
| iterations | 803 |
| time_elapsed | 4122 |
| total_timesteps | 1644544 |
| train/ | |
| approx_kl | 0.0025388254 |
| clip_fraction | 0.0281 |
| clip_range | 0.2 |
| entropy_loss | -0.556 |
| explained_variance | 0.9904995 |
| learning_rate | 0.0003 |
| loss | 4.83 |
| n_updates | 3208 |
| policy_gradient_loss | -0.000137 |
| value_loss | 14.6 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 270 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 398 |
| iterations | 804 |
| time_elapsed | 4127 |
| total_timesteps | 1646592 |
| train/ | |
| approx_kl | 0.003068504 |
| clip_fraction | 0.0234 |
| clip_range | 0.2 |
| entropy_loss | -0.581 |
| explained_variance | 0.77935684 |
| learning_rate | 0.0003 |
| loss | 240 |
| n_updates | 3212 |
| policy_gradient_loss | -0.0011 |
| value_loss | 822 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 399 |
| iterations | 805 |
| time_elapsed | 4131 |
| total_timesteps | 1648640 |
| train/ | |
| approx_kl | 0.004622833 |
| clip_fraction | 0.0282 |
| clip_range | 0.2 |
| entropy_loss | -0.678 |
| explained_variance | 0.9465113 |
| learning_rate | 0.0003 |
| loss | 8.13 |
| n_updates | 3216 |
| policy_gradient_loss | -0.00181 |
| value_loss | 64 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 276 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 399 |
| iterations | 806 |
| time_elapsed | 4136 |
| total_timesteps | 1650688 |
| train/ | |
| approx_kl | 0.0020569125 |
| clip_fraction | 0.0215 |
| clip_range | 0.2 |
| entropy_loss | -0.662 |
| explained_variance | 0.69076383 |
| learning_rate | 0.0003 |
| loss | 406 |
| n_updates | 3220 |
| policy_gradient_loss | -0.00037 |
| value_loss | 1.14e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 262 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 399 |
| iterations | 807 |
| time_elapsed | 4140 |
| total_timesteps | 1652736 |
| train/ | |
| approx_kl | 0.0055237333 |
| clip_fraction | 0.0699 |
| clip_range | 0.2 |
| entropy_loss | -0.621 |
| explained_variance | 0.9732897 |
| learning_rate | 0.0003 |
| loss | 2.55 |
| n_updates | 3224 |
| policy_gradient_loss | -0.0036 |
| value_loss | 30.1 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 262 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 399 |
| iterations | 808 |
| time_elapsed | 4144 |
| total_timesteps | 1654784 |
| train/ | |
| approx_kl | 0.001696477 |
| clip_fraction | 0.0254 |
| clip_range | 0.2 |
| entropy_loss | -0.529 |
| explained_variance | 0.87306255 |
| learning_rate | 0.0003 |
| loss | 124 |
| n_updates | 3228 |
| policy_gradient_loss | -0.00121 |
| value_loss | 432 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 399 |
| iterations | 809 |
| time_elapsed | 4149 |
| total_timesteps | 1656832 |
| train/ | |
| approx_kl | 0.0017096486 |
| clip_fraction | 0.0164 |
| clip_range | 0.2 |
| entropy_loss | -0.554 |
| explained_variance | 0.9372842 |
| learning_rate | 0.0003 |
| loss | 39.5 |
| n_updates | 3232 |
| policy_gradient_loss | -0.000464 |
| value_loss | 158 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 399 |
| iterations | 810 |
| time_elapsed | 4154 |
| total_timesteps | 1658880 |
| train/ | |
| approx_kl | 0.0070530986 |
| clip_fraction | 0.0199 |
| clip_range | 0.2 |
| entropy_loss | -0.599 |
| explained_variance | 0.9089455 |
| learning_rate | 0.0003 |
| loss | 224 |
| n_updates | 3236 |
| policy_gradient_loss | -0.00174 |
| value_loss | 289 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1660000 to videos/step_1660000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 399 |
| iterations | 811 |
| time_elapsed | 4159 |
| total_timesteps | 1660928 |
| train/ | |
| approx_kl | 0.0024693948 |
| clip_fraction | 0.0116 |
| clip_range | 0.2 |
| entropy_loss | -0.631 |
| explained_variance | 0.77257156 |
| learning_rate | 0.0003 |
| loss | 472 |
| n_updates | 3240 |
| policy_gradient_loss | -0.00109 |
| value_loss | 639 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 263 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 399 |
| iterations | 812 |
| time_elapsed | 4164 |
| total_timesteps | 1662976 |
| train/ | |
| approx_kl | 0.0020911014 |
| clip_fraction | 0.0121 |
| clip_range | 0.2 |
| entropy_loss | -0.585 |
| explained_variance | 0.94869876 |
| learning_rate | 0.0003 |
| loss | 54.7 |
| n_updates | 3244 |
| policy_gradient_loss | -0.0008 |
| value_loss | 77.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 399 |
| iterations | 813 |
| time_elapsed | 4168 |
| total_timesteps | 1665024 |
| train/ | |
| approx_kl | 0.0012973576 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -0.638 |
| explained_variance | 0.81675005 |
| learning_rate | 0.0003 |
| loss | 111 |
| n_updates | 3248 |
| policy_gradient_loss | -8.28e-05 |
| value_loss | 267 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 399 |
| iterations | 814 |
| time_elapsed | 4172 |
| total_timesteps | 1667072 |
| train/ | |
| approx_kl | 0.0044364827 |
| clip_fraction | 0.0193 |
| clip_range | 0.2 |
| entropy_loss | -0.691 |
| explained_variance | 0.96321195 |
| learning_rate | 0.0003 |
| loss | 3.45 |
| n_updates | 3252 |
| policy_gradient_loss | 0.000541 |
| value_loss | 31.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 272 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 399 |
| iterations | 815 |
| time_elapsed | 4177 |
| total_timesteps | 1669120 |
| train/ | |
| approx_kl | 0.006377222 |
| clip_fraction | 0.0442 |
| clip_range | 0.2 |
| entropy_loss | -0.556 |
| explained_variance | 0.9910336 |
| learning_rate | 0.0003 |
| loss | 2.81 |
| n_updates | 3256 |
| policy_gradient_loss | -0.000798 |
| value_loss | 9.95 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 399 |
| iterations | 816 |
| time_elapsed | 4181 |
| total_timesteps | 1671168 |
| train/ | |
| approx_kl | 0.0035259416 |
| clip_fraction | 0.113 |
| clip_range | 0.2 |
| entropy_loss | -0.62 |
| explained_variance | 0.7459362 |
| learning_rate | 0.0003 |
| loss | 250 |
| n_updates | 3260 |
| policy_gradient_loss | 0.00275 |
| value_loss | 817 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 247 |
| time/ | |
| fps | 399 |
| iterations | 817 |
| time_elapsed | 4185 |
| total_timesteps | 1673216 |
| train/ | |
| approx_kl | 0.010034464 |
| clip_fraction | 0.0615 |
| clip_range | 0.2 |
| entropy_loss | -0.588 |
| explained_variance | 0.993968 |
| learning_rate | 0.0003 |
| loss | 3.03 |
| n_updates | 3264 |
| policy_gradient_loss | 0.000318 |
| value_loss | 10.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 256 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 399 |
| iterations | 818 |
| time_elapsed | 4189 |
| total_timesteps | 1675264 |
| train/ | |
| approx_kl | 0.0010253722 |
| clip_fraction | 0.00122 |
| clip_range | 0.2 |
| entropy_loss | -0.561 |
| explained_variance | 0.77614164 |
| learning_rate | 0.0003 |
| loss | 583 |
| n_updates | 3268 |
| policy_gradient_loss | -0.0005 |
| value_loss | 846 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 262 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 399 |
| iterations | 819 |
| time_elapsed | 4194 |
| total_timesteps | 1677312 |
| train/ | |
| approx_kl | 0.0021045743 |
| clip_fraction | 0.0123 |
| clip_range | 0.2 |
| entropy_loss | -0.566 |
| explained_variance | 0.96617055 |
| learning_rate | 0.0003 |
| loss | 24.3 |
| n_updates | 3272 |
| policy_gradient_loss | -0.00203 |
| value_loss | 84.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 259 |
| ep_rew_mean | 249 |
| time/ | |
| fps | 400 |
| iterations | 820 |
| time_elapsed | 4197 |
| total_timesteps | 1679360 |
| train/ | |
| approx_kl | 0.0043966644 |
| clip_fraction | 0.0271 |
| clip_range | 0.2 |
| entropy_loss | -0.63 |
| explained_variance | 0.94001734 |
| learning_rate | 0.0003 |
| loss | 37.4 |
| n_updates | 3276 |
| policy_gradient_loss | -0.000899 |
| value_loss | 243 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1680000 to videos/step_1680000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 399 |
| iterations | 821 |
| time_elapsed | 4204 |
| total_timesteps | 1681408 |
| train/ | |
| approx_kl | 0.0016361785 |
| clip_fraction | 0.00647 |
| clip_range | 0.2 |
| entropy_loss | -0.595 |
| explained_variance | 0.87002057 |
| learning_rate | 0.0003 |
| loss | 357 |
| n_updates | 3280 |
| policy_gradient_loss | -0.00175 |
| value_loss | 408 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 263 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 399 |
| iterations | 822 |
| time_elapsed | 4209 |
| total_timesteps | 1683456 |
| train/ | |
| approx_kl | 0.0021853873 |
| clip_fraction | 0.015 |
| clip_range | 0.2 |
| entropy_loss | -0.552 |
| explained_variance | 0.98020935 |
| learning_rate | 0.0003 |
| loss | 8.8 |
| n_updates | 3284 |
| policy_gradient_loss | -0.000372 |
| value_loss | 28.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 250 |
| time/ | |
| fps | 400 |
| iterations | 823 |
| time_elapsed | 4213 |
| total_timesteps | 1685504 |
| train/ | |
| approx_kl | 0.0012705285 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.72 |
| explained_variance | 0.7144191 |
| learning_rate | 0.0003 |
| loss | 432 |
| n_updates | 3288 |
| policy_gradient_loss | 4.89e-05 |
| value_loss | 755 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 261 |
| ep_rew_mean | 244 |
| time/ | |
| fps | 400 |
| iterations | 824 |
| time_elapsed | 4218 |
| total_timesteps | 1687552 |
| train/ | |
| approx_kl | 0.007957136 |
| clip_fraction | 0.11 |
| clip_range | 0.2 |
| entropy_loss | -0.614 |
| explained_variance | 0.9896178 |
| learning_rate | 0.0003 |
| loss | 23 |
| n_updates | 3292 |
| policy_gradient_loss | -0.00383 |
| value_loss | 63.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 254 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 400 |
| iterations | 825 |
| time_elapsed | 4221 |
| total_timesteps | 1689600 |
| train/ | |
| approx_kl | 0.0007680362 |
| clip_fraction | 0.000366 |
| clip_range | 0.2 |
| entropy_loss | -0.621 |
| explained_variance | 0.79188144 |
| learning_rate | 0.0003 |
| loss | 179 |
| n_updates | 3296 |
| policy_gradient_loss | -0.00086 |
| value_loss | 837 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 250 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 400 |
| iterations | 826 |
| time_elapsed | 4225 |
| total_timesteps | 1691648 |
| train/ | |
| approx_kl | 0.0042392076 |
| clip_fraction | 0.0422 |
| clip_range | 0.2 |
| entropy_loss | -0.624 |
| explained_variance | 0.92127615 |
| learning_rate | 0.0003 |
| loss | 240 |
| n_updates | 3300 |
| policy_gradient_loss | -0.00176 |
| value_loss | 205 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 249 |
| ep_rew_mean | 248 |
| time/ | |
| fps | 400 |
| iterations | 827 |
| time_elapsed | 4230 |
| total_timesteps | 1693696 |
| train/ | |
| approx_kl | 0.0013639141 |
| clip_fraction | 0.00537 |
| clip_range | 0.2 |
| entropy_loss | -0.535 |
| explained_variance | 0.8761378 |
| learning_rate | 0.0003 |
| loss | 79.3 |
| n_updates | 3304 |
| policy_gradient_loss | -0.000482 |
| value_loss | 189 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 400 |
| iterations | 828 |
| time_elapsed | 4234 |
| total_timesteps | 1695744 |
| train/ | |
| approx_kl | 0.0033181873 |
| clip_fraction | 0.0179 |
| clip_range | 0.2 |
| entropy_loss | -0.554 |
| explained_variance | 0.8046403 |
| learning_rate | 0.0003 |
| loss | 171 |
| n_updates | 3308 |
| policy_gradient_loss | -0.00264 |
| value_loss | 496 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 246 |
| time/ | |
| fps | 400 |
| iterations | 829 |
| time_elapsed | 4238 |
| total_timesteps | 1697792 |
| train/ | |
| approx_kl | 0.007834468 |
| clip_fraction | 0.0586 |
| clip_range | 0.2 |
| entropy_loss | -0.718 |
| explained_variance | 0.9474403 |
| learning_rate | 0.0003 |
| loss | 4.63 |
| n_updates | 3312 |
| policy_gradient_loss | 0.00155 |
| value_loss | 30.9 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 262 |
| ep_rew_mean | 243 |
| time/ | |
| fps | 400 |
| iterations | 830 |
| time_elapsed | 4243 |
| total_timesteps | 1699840 |
| train/ | |
| approx_kl | 0.0046162396 |
| clip_fraction | 0.0398 |
| clip_range | 0.2 |
| entropy_loss | -0.575 |
| explained_variance | 0.9877464 |
| learning_rate | 0.0003 |
| loss | 13.5 |
| n_updates | 3316 |
| policy_gradient_loss | -0.000475 |
| value_loss | 27.6 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1700000 to videos/step_1700000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 269 |
| ep_rew_mean | 242 |
| time/ | |
| fps | 400 |
| iterations | 831 |
| time_elapsed | 4249 |
| total_timesteps | 1701888 |
| train/ | |
| approx_kl | 0.0010959171 |
| clip_fraction | 0.0121 |
| clip_range | 0.2 |
| entropy_loss | -0.581 |
| explained_variance | 0.7110528 |
| learning_rate | 0.0003 |
| loss | 169 |
| n_updates | 3320 |
| policy_gradient_loss | 0.000602 |
| value_loss | 1.32e+03 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 270 |
| ep_rew_mean | 241 |
| time/ | |
| fps | 400 |
| iterations | 832 |
| time_elapsed | 4253 |
| total_timesteps | 1703936 |
| train/ | |
| approx_kl | 0.0041504437 |
| clip_fraction | 0.0182 |
| clip_range | 0.2 |
| entropy_loss | -0.529 |
| explained_variance | 0.9419404 |
| learning_rate | 0.0003 |
| loss | 62 |
| n_updates | 3324 |
| policy_gradient_loss | -0.00344 |
| value_loss | 135 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 235 |
| time/ | |
| fps | 400 |
| iterations | 833 |
| time_elapsed | 4258 |
| total_timesteps | 1705984 |
| train/ | |
| approx_kl | 0.0011457012 |
| clip_fraction | 0.00391 |
| clip_range | 0.2 |
| entropy_loss | -0.594 |
| explained_variance | 0.82252586 |
| learning_rate | 0.0003 |
| loss | 29.5 |
| n_updates | 3328 |
| policy_gradient_loss | -0.00135 |
| value_loss | 464 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 229 |
| time/ | |
| fps | 400 |
| iterations | 834 |
| time_elapsed | 4262 |
| total_timesteps | 1708032 |
| train/ | |
| approx_kl | 0.002713311 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -0.594 |
| explained_variance | 0.79938877 |
| learning_rate | 0.0003 |
| loss | 60.3 |
| n_updates | 3332 |
| policy_gradient_loss | -0.000961 |
| value_loss | 563 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 229 |
| time/ | |
| fps | 400 |
| iterations | 835 |
| time_elapsed | 4266 |
| total_timesteps | 1710080 |
| train/ | |
| approx_kl | 0.0010527789 |
| clip_fraction | 0.005 |
| clip_range | 0.2 |
| entropy_loss | -0.58 |
| explained_variance | 0.88204265 |
| learning_rate | 0.0003 |
| loss | 92.1 |
| n_updates | 3336 |
| policy_gradient_loss | -0.000727 |
| value_loss | 373 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 254 |
| ep_rew_mean | 233 |
| time/ | |
| fps | 400 |
| iterations | 836 |
| time_elapsed | 4271 |
| total_timesteps | 1712128 |
| train/ | |
| approx_kl | 0.0017906905 |
| clip_fraction | 0.00159 |
| clip_range | 0.2 |
| entropy_loss | -0.579 |
| explained_variance | 0.81062126 |
| learning_rate | 0.0003 |
| loss | 770 |
| n_updates | 3340 |
| policy_gradient_loss | -0.000741 |
| value_loss | 829 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 258 |
| ep_rew_mean | 234 |
| time/ | |
| fps | 400 |
| iterations | 837 |
| time_elapsed | 4275 |
| total_timesteps | 1714176 |
| train/ | |
| approx_kl | 0.0010501331 |
| clip_fraction | 0.00195 |
| clip_range | 0.2 |
| entropy_loss | -0.524 |
| explained_variance | 0.57972705 |
| learning_rate | 0.0003 |
| loss | 105 |
| n_updates | 3344 |
| policy_gradient_loss | -0.000375 |
| value_loss | 468 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 256 |
| ep_rew_mean | 229 |
| time/ | |
| fps | 400 |
| iterations | 838 |
| time_elapsed | 4280 |
| total_timesteps | 1716224 |
| train/ | |
| approx_kl | 0.0028892835 |
| clip_fraction | 0.0146 |
| clip_range | 0.2 |
| entropy_loss | -0.563 |
| explained_variance | 0.87386966 |
| learning_rate | 0.0003 |
| loss | 173 |
| n_updates | 3348 |
| policy_gradient_loss | -0.000591 |
| value_loss | 364 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 266 |
| ep_rew_mean | 223 |
| time/ | |
| fps | 401 |
| iterations | 839 |
| time_elapsed | 4284 |
| total_timesteps | 1718272 |
| train/ | |
| approx_kl | 0.0042996556 |
| clip_fraction | 0.0273 |
| clip_range | 0.2 |
| entropy_loss | -0.503 |
| explained_variance | 0.8321183 |
| learning_rate | 0.0003 |
| loss | 151 |
| n_updates | 3352 |
| policy_gradient_loss | -0.00241 |
| value_loss | 608 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1720000 to videos/step_1720000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 271 |
| ep_rew_mean | 224 |
| time/ | |
| fps | 400 |
| iterations | 840 |
| time_elapsed | 4290 |
| total_timesteps | 1720320 |
| train/ | |
| approx_kl | 0.0024890467 |
| clip_fraction | 0.0216 |
| clip_range | 0.2 |
| entropy_loss | -0.482 |
| explained_variance | 0.8215256 |
| learning_rate | 0.0003 |
| loss | 606 |
| n_updates | 3356 |
| policy_gradient_loss | -0.00377 |
| value_loss | 696 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 216 |
| time/ | |
| fps | 401 |
| iterations | 841 |
| time_elapsed | 4294 |
| total_timesteps | 1722368 |
| train/ | |
| approx_kl | 0.0023756414 |
| clip_fraction | 0.0126 |
| clip_range | 0.2 |
| entropy_loss | -0.539 |
| explained_variance | 0.92623925 |
| learning_rate | 0.0003 |
| loss | 47.4 |
| n_updates | 3360 |
| policy_gradient_loss | -0.00121 |
| value_loss | 225 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 220 |
| time/ | |
| fps | 401 |
| iterations | 842 |
| time_elapsed | 4298 |
| total_timesteps | 1724416 |
| train/ | |
| approx_kl | 0.000757373 |
| clip_fraction | 0.00061 |
| clip_range | 0.2 |
| entropy_loss | -0.583 |
| explained_variance | 0.7372059 |
| learning_rate | 0.0003 |
| loss | 535 |
| n_updates | 3364 |
| policy_gradient_loss | -0.000497 |
| value_loss | 1.12e+03 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 241 |
| ep_rew_mean | 221 |
| time/ | |
| fps | 401 |
| iterations | 843 |
| time_elapsed | 4302 |
| total_timesteps | 1726464 |
| train/ | |
| approx_kl | 0.002776795 |
| clip_fraction | 0.0137 |
| clip_range | 0.2 |
| entropy_loss | -0.568 |
| explained_variance | 0.93769073 |
| learning_rate | 0.0003 |
| loss | 75.2 |
| n_updates | 3368 |
| policy_gradient_loss | -0.00128 |
| value_loss | 173 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 221 |
| time/ | |
| fps | 401 |
| iterations | 844 |
| time_elapsed | 4307 |
| total_timesteps | 1728512 |
| train/ | |
| approx_kl | 0.0017578065 |
| clip_fraction | 0.0159 |
| clip_range | 0.2 |
| entropy_loss | -0.52 |
| explained_variance | 0.8493264 |
| learning_rate | 0.0003 |
| loss | 80 |
| n_updates | 3372 |
| policy_gradient_loss | -0.000105 |
| value_loss | 328 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 239 |
| ep_rew_mean | 223 |
| time/ | |
| fps | 401 |
| iterations | 845 |
| time_elapsed | 4311 |
| total_timesteps | 1730560 |
| train/ | |
| approx_kl | 0.0028607878 |
| clip_fraction | 0.0229 |
| clip_range | 0.2 |
| entropy_loss | -0.596 |
| explained_variance | 0.798327 |
| learning_rate | 0.0003 |
| loss | 62 |
| n_updates | 3376 |
| policy_gradient_loss | -0.00247 |
| value_loss | 561 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 239 |
| ep_rew_mean | 229 |
| time/ | |
| fps | 401 |
| iterations | 846 |
| time_elapsed | 4315 |
| total_timesteps | 1732608 |
| train/ | |
| approx_kl | 0.0027599689 |
| clip_fraction | 0.0137 |
| clip_range | 0.2 |
| entropy_loss | -0.573 |
| explained_variance | 0.9037639 |
| learning_rate | 0.0003 |
| loss | 100 |
| n_updates | 3380 |
| policy_gradient_loss | -0.00138 |
| value_loss | 377 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 240 |
| ep_rew_mean | 230 |
| time/ | |
| fps | 401 |
| iterations | 847 |
| time_elapsed | 4320 |
| total_timesteps | 1734656 |
| train/ | |
| approx_kl | 0.002232979 |
| clip_fraction | 0.00403 |
| clip_range | 0.2 |
| entropy_loss | -0.58 |
| explained_variance | 0.81973124 |
| learning_rate | 0.0003 |
| loss | 322 |
| n_updates | 3384 |
| policy_gradient_loss | -6.44e-05 |
| value_loss | 639 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 241 |
| ep_rew_mean | 230 |
| time/ | |
| fps | 401 |
| iterations | 848 |
| time_elapsed | 4324 |
| total_timesteps | 1736704 |
| train/ | |
| approx_kl | 0.0036073243 |
| clip_fraction | 0.0194 |
| clip_range | 0.2 |
| entropy_loss | -0.498 |
| explained_variance | 0.8753712 |
| learning_rate | 0.0003 |
| loss | 68.3 |
| n_updates | 3388 |
| policy_gradient_loss | -0.00231 |
| value_loss | 302 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 246 |
| ep_rew_mean | 230 |
| time/ | |
| fps | 401 |
| iterations | 849 |
| time_elapsed | 4328 |
| total_timesteps | 1738752 |
| train/ | |
| approx_kl | 0.0025892213 |
| clip_fraction | 0.0154 |
| clip_range | 0.2 |
| entropy_loss | -0.548 |
| explained_variance | 0.9495505 |
| learning_rate | 0.0003 |
| loss | 43.3 |
| n_updates | 3392 |
| policy_gradient_loss | -0.00213 |
| value_loss | 177 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1740000 to videos/step_1740000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 401 |
| iterations | 850 |
| time_elapsed | 4334 |
| total_timesteps | 1740800 |
| train/ | |
| approx_kl | 0.0064298143 |
| clip_fraction | 0.047 |
| clip_range | 0.2 |
| entropy_loss | -0.555 |
| explained_variance | 0.97505885 |
| learning_rate | 0.0003 |
| loss | 10.7 |
| n_updates | 3396 |
| policy_gradient_loss | -0.000121 |
| value_loss | 57.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 239 |
| ep_rew_mean | 238 |
| time/ | |
| fps | 401 |
| iterations | 851 |
| time_elapsed | 4338 |
| total_timesteps | 1742848 |
| train/ | |
| approx_kl | 0.0034931311 |
| clip_fraction | 0.0175 |
| clip_range | 0.2 |
| entropy_loss | -0.543 |
| explained_variance | 0.83667696 |
| learning_rate | 0.0003 |
| loss | 20.3 |
| n_updates | 3400 |
| policy_gradient_loss | -0.00146 |
| value_loss | 142 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 236 |
| ep_rew_mean | 245 |
| time/ | |
| fps | 401 |
| iterations | 852 |
| time_elapsed | 4342 |
| total_timesteps | 1744896 |
| train/ | |
| approx_kl | 0.0017364924 |
| clip_fraction | 0.0276 |
| clip_range | 0.2 |
| entropy_loss | -0.552 |
| explained_variance | 0.73014635 |
| learning_rate | 0.0003 |
| loss | 441 |
| n_updates | 3404 |
| policy_gradient_loss | 0.000753 |
| value_loss | 939 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 239 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 401 |
| iterations | 853 |
| time_elapsed | 4346 |
| total_timesteps | 1746944 |
| train/ | |
| approx_kl | 0.0047123865 |
| clip_fraction | 0.045 |
| clip_range | 0.2 |
| entropy_loss | -0.541 |
| explained_variance | 0.92866766 |
| learning_rate | 0.0003 |
| loss | 6.2 |
| n_updates | 3408 |
| policy_gradient_loss | -0.00046 |
| value_loss | 74.5 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 241 |
| ep_rew_mean | 253 |
| time/ | |
| fps | 401 |
| iterations | 854 |
| time_elapsed | 4350 |
| total_timesteps | 1748992 |
| train/ | |
| approx_kl | 0.006044563 |
| clip_fraction | 0.0654 |
| clip_range | 0.2 |
| entropy_loss | -0.584 |
| explained_variance | 0.9866812 |
| learning_rate | 0.0003 |
| loss | 2.87 |
| n_updates | 3412 |
| policy_gradient_loss | 0.00226 |
| value_loss | 12.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 402 |
| iterations | 855 |
| time_elapsed | 4354 |
| total_timesteps | 1751040 |
| train/ | |
| approx_kl | 0.0034118646 |
| clip_fraction | 0.0497 |
| clip_range | 0.2 |
| entropy_loss | -0.556 |
| explained_variance | 0.9922499 |
| learning_rate | 0.0003 |
| loss | 3.01 |
| n_updates | 3416 |
| policy_gradient_loss | 0.00107 |
| value_loss | 16.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 241 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 402 |
| iterations | 856 |
| time_elapsed | 4359 |
| total_timesteps | 1753088 |
| train/ | |
| approx_kl | 0.0027525558 |
| clip_fraction | 0.042 |
| clip_range | 0.2 |
| entropy_loss | -0.558 |
| explained_variance | 0.833099 |
| learning_rate | 0.0003 |
| loss | 209 |
| n_updates | 3420 |
| policy_gradient_loss | 0.00153 |
| value_loss | 611 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 249 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 402 |
| iterations | 857 |
| time_elapsed | 4363 |
| total_timesteps | 1755136 |
| train/ | |
| approx_kl | 0.0024010898 |
| clip_fraction | 0.0166 |
| clip_range | 0.2 |
| entropy_loss | -0.524 |
| explained_variance | 0.84857637 |
| learning_rate | 0.0003 |
| loss | 251 |
| n_updates | 3424 |
| policy_gradient_loss | -0.00213 |
| value_loss | 443 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 250 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 402 |
| iterations | 858 |
| time_elapsed | 4367 |
| total_timesteps | 1757184 |
| train/ | |
| approx_kl | 0.0043717395 |
| clip_fraction | 0.0414 |
| clip_range | 0.2 |
| entropy_loss | -0.587 |
| explained_variance | 0.98429096 |
| learning_rate | 0.0003 |
| loss | 6.45 |
| n_updates | 3428 |
| policy_gradient_loss | 0.00184 |
| value_loss | 11.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 247 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 402 |
| iterations | 859 |
| time_elapsed | 4372 |
| total_timesteps | 1759232 |
| train/ | |
| approx_kl | 0.0021061865 |
| clip_fraction | 0.0446 |
| clip_range | 0.2 |
| entropy_loss | -0.499 |
| explained_variance | 0.87588257 |
| learning_rate | 0.0003 |
| loss | 165 |
| n_updates | 3432 |
| policy_gradient_loss | -0.00139 |
| value_loss | 409 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1760000 to videos/step_1760000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 246 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 402 |
| iterations | 860 |
| time_elapsed | 4378 |
| total_timesteps | 1761280 |
| train/ | |
| approx_kl | 0.002710655 |
| clip_fraction | 0.00439 |
| clip_range | 0.2 |
| entropy_loss | -0.594 |
| explained_variance | 0.8847939 |
| learning_rate | 0.0003 |
| loss | 143 |
| n_updates | 3436 |
| policy_gradient_loss | -0.00124 |
| value_loss | 330 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 402 |
| iterations | 861 |
| time_elapsed | 4382 |
| total_timesteps | 1763328 |
| train/ | |
| approx_kl | 0.0033348394 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -0.565 |
| explained_variance | 0.8454652 |
| learning_rate | 0.0003 |
| loss | 293 |
| n_updates | 3440 |
| policy_gradient_loss | -0.00114 |
| value_loss | 497 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 402 |
| iterations | 862 |
| time_elapsed | 4386 |
| total_timesteps | 1765376 |
| train/ | |
| approx_kl | 0.0020690945 |
| clip_fraction | 0.00879 |
| clip_range | 0.2 |
| entropy_loss | -0.559 |
| explained_variance | 0.87442744 |
| learning_rate | 0.0003 |
| loss | 240 |
| n_updates | 3444 |
| policy_gradient_loss | -0.000921 |
| value_loss | 386 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 245 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 402 |
| iterations | 863 |
| time_elapsed | 4391 |
| total_timesteps | 1767424 |
| train/ | |
| approx_kl | 0.002682955 |
| clip_fraction | 0.0161 |
| clip_range | 0.2 |
| entropy_loss | -0.581 |
| explained_variance | 0.92072755 |
| learning_rate | 0.0003 |
| loss | 86.3 |
| n_updates | 3448 |
| policy_gradient_loss | -0.00297 |
| value_loss | 302 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 244 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 402 |
| iterations | 864 |
| time_elapsed | 4395 |
| total_timesteps | 1769472 |
| train/ | |
| approx_kl | 0.0036212967 |
| clip_fraction | 0.0281 |
| clip_range | 0.2 |
| entropy_loss | -0.731 |
| explained_variance | 0.93945956 |
| learning_rate | 0.0003 |
| loss | 3.67 |
| n_updates | 3452 |
| policy_gradient_loss | -0.00106 |
| value_loss | 62.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 244 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 402 |
| iterations | 865 |
| time_elapsed | 4399 |
| total_timesteps | 1771520 |
| train/ | |
| approx_kl | 0.0021074447 |
| clip_fraction | 0.0353 |
| clip_range | 0.2 |
| entropy_loss | -0.543 |
| explained_variance | 0.8449467 |
| learning_rate | 0.0003 |
| loss | 297 |
| n_updates | 3456 |
| policy_gradient_loss | -0.00236 |
| value_loss | 511 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 249 |
| ep_rew_mean | 252 |
| time/ | |
| fps | 402 |
| iterations | 866 |
| time_elapsed | 4404 |
| total_timesteps | 1773568 |
| train/ | |
| approx_kl | 0.0021660966 |
| clip_fraction | 0.0118 |
| clip_range | 0.2 |
| entropy_loss | -0.568 |
| explained_variance | 0.8860859 |
| learning_rate | 0.0003 |
| loss | 56 |
| n_updates | 3460 |
| policy_gradient_loss | -0.0027 |
| value_loss | 394 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 247 |
| ep_rew_mean | 251 |
| time/ | |
| fps | 402 |
| iterations | 867 |
| time_elapsed | 4408 |
| total_timesteps | 1775616 |
| train/ | |
| approx_kl | 0.0037792996 |
| clip_fraction | 0.0256 |
| clip_range | 0.2 |
| entropy_loss | -0.517 |
| explained_variance | 0.9717626 |
| learning_rate | 0.0003 |
| loss | 229 |
| n_updates | 3464 |
| policy_gradient_loss | -0.00077 |
| value_loss | 71.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 247 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 402 |
| iterations | 868 |
| time_elapsed | 4412 |
| total_timesteps | 1777664 |
| train/ | |
| approx_kl | 0.004714123 |
| clip_fraction | 0.0571 |
| clip_range | 0.2 |
| entropy_loss | -0.587 |
| explained_variance | 0.9667165 |
| learning_rate | 0.0003 |
| loss | 3.4 |
| n_updates | 3468 |
| policy_gradient_loss | -0.00131 |
| value_loss | 79.5 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 402 |
| iterations | 869 |
| time_elapsed | 4416 |
| total_timesteps | 1779712 |
| train/ | |
| approx_kl | 0.00054777984 |
| clip_fraction | 0.00598 |
| clip_range | 0.2 |
| entropy_loss | -0.571 |
| explained_variance | 0.86357224 |
| learning_rate | 0.0003 |
| loss | 224 |
| n_updates | 3472 |
| policy_gradient_loss | -0.000284 |
| value_loss | 490 |
-------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1780000 to videos/step_1780000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 402 |
| iterations | 870 |
| time_elapsed | 4423 |
| total_timesteps | 1781760 |
| train/ | |
| approx_kl | 0.008034635 |
| clip_fraction | 0.0702 |
| clip_range | 0.2 |
| entropy_loss | -0.559 |
| explained_variance | 0.989861 |
| learning_rate | 0.0003 |
| loss | 5 |
| n_updates | 3476 |
| policy_gradient_loss | -0.00262 |
| value_loss | 12.3 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 402 |
| iterations | 871 |
| time_elapsed | 4427 |
| total_timesteps | 1783808 |
| train/ | |
| approx_kl | 0.0026760125 |
| clip_fraction | 0.0162 |
| clip_range | 0.2 |
| entropy_loss | -0.606 |
| explained_variance | 0.8202264 |
| learning_rate | 0.0003 |
| loss | 619 |
| n_updates | 3480 |
| policy_gradient_loss | -0.000804 |
| value_loss | 543 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 402 |
| iterations | 872 |
| time_elapsed | 4431 |
| total_timesteps | 1785856 |
| train/ | |
| approx_kl | 0.013179025 |
| clip_fraction | 0.0759 |
| clip_range | 0.2 |
| entropy_loss | -0.584 |
| explained_variance | 0.9926122 |
| learning_rate | 0.0003 |
| loss | 6.12 |
| n_updates | 3484 |
| policy_gradient_loss | -0.00453 |
| value_loss | 13.5 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 243 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 403 |
| iterations | 873 |
| time_elapsed | 4436 |
| total_timesteps | 1787904 |
| train/ | |
| approx_kl | 0.0019006393 |
| clip_fraction | 0.0198 |
| clip_range | 0.2 |
| entropy_loss | -0.572 |
| explained_variance | 0.8861045 |
| learning_rate | 0.0003 |
| loss | 49.2 |
| n_updates | 3488 |
| policy_gradient_loss | -0.00209 |
| value_loss | 264 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 251 |
| ep_rew_mean | 256 |
| time/ | |
| fps | 403 |
| iterations | 874 |
| time_elapsed | 4440 |
| total_timesteps | 1789952 |
| train/ | |
| approx_kl | 0.0019784993 |
| clip_fraction | 0.0288 |
| clip_range | 0.2 |
| entropy_loss | -0.627 |
| explained_variance | 0.8684229 |
| learning_rate | 0.0003 |
| loss | 98.8 |
| n_updates | 3492 |
| policy_gradient_loss | -0.000722 |
| value_loss | 408 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 243 |
| ep_rew_mean | 255 |
| time/ | |
| fps | 403 |
| iterations | 875 |
| time_elapsed | 4444 |
| total_timesteps | 1792000 |
| train/ | |
| approx_kl | 0.0024971247 |
| clip_fraction | 0.024 |
| clip_range | 0.2 |
| entropy_loss | -0.557 |
| explained_variance | 0.9562164 |
| learning_rate | 0.0003 |
| loss | 7.24 |
| n_updates | 3496 |
| policy_gradient_loss | -0.000341 |
| value_loss | 49.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 250 |
| ep_rew_mean | 254 |
| time/ | |
| fps | 403 |
| iterations | 876 |
| time_elapsed | 4449 |
| total_timesteps | 1794048 |
| train/ | |
| approx_kl | 0.0018128331 |
| clip_fraction | 0.0145 |
| clip_range | 0.2 |
| entropy_loss | -0.572 |
| explained_variance | 0.8888047 |
| learning_rate | 0.0003 |
| loss | 65.6 |
| n_updates | 3500 |
| policy_gradient_loss | -0.00138 |
| value_loss | 246 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 251 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 403 |
| iterations | 877 |
| time_elapsed | 4453 |
| total_timesteps | 1796096 |
| train/ | |
| approx_kl | 0.0016047934 |
| clip_fraction | 0.00476 |
| clip_range | 0.2 |
| entropy_loss | -0.58 |
| explained_variance | 0.8955335 |
| learning_rate | 0.0003 |
| loss | 77.3 |
| n_updates | 3504 |
| policy_gradient_loss | -0.00113 |
| value_loss | 314 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 258 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 403 |
| iterations | 878 |
| time_elapsed | 4457 |
| total_timesteps | 1798144 |
| train/ | |
| approx_kl | 0.0032270164 |
| clip_fraction | 0.0188 |
| clip_range | 0.2 |
| entropy_loss | -0.669 |
| explained_variance | 0.81106514 |
| learning_rate | 0.0003 |
| loss | 96.9 |
| n_updates | 3508 |
| policy_gradient_loss | -0.000884 |
| value_loss | 507 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1800000 to videos/step_1800000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 256 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 403 |
| iterations | 879 |
| time_elapsed | 4464 |
| total_timesteps | 1800192 |
| train/ | |
| approx_kl | 0.008018012 |
| clip_fraction | 0.0441 |
| clip_range | 0.2 |
| entropy_loss | -0.554 |
| explained_variance | 0.9861362 |
| learning_rate | 0.0003 |
| loss | 5.56 |
| n_updates | 3512 |
| policy_gradient_loss | -0.00032 |
| value_loss | 20.1 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 403 |
| iterations | 880 |
| time_elapsed | 4468 |
| total_timesteps | 1802240 |
| train/ | |
| approx_kl | 0.0052738446 |
| clip_fraction | 0.0723 |
| clip_range | 0.2 |
| entropy_loss | -0.58 |
| explained_variance | 0.9071923 |
| learning_rate | 0.0003 |
| loss | 27.1 |
| n_updates | 3516 |
| policy_gradient_loss | -0.00197 |
| value_loss | 165 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 403 |
| iterations | 881 |
| time_elapsed | 4472 |
| total_timesteps | 1804288 |
| train/ | |
| approx_kl | 0.002443085 |
| clip_fraction | 0.0112 |
| clip_range | 0.2 |
| entropy_loss | -0.657 |
| explained_variance | 0.8481055 |
| learning_rate | 0.0003 |
| loss | 112 |
| n_updates | 3520 |
| policy_gradient_loss | -0.00162 |
| value_loss | 487 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 403 |
| iterations | 882 |
| time_elapsed | 4477 |
| total_timesteps | 1806336 |
| train/ | |
| approx_kl | 0.005661169 |
| clip_fraction | 0.0459 |
| clip_range | 0.2 |
| entropy_loss | -0.587 |
| explained_variance | 0.98616976 |
| learning_rate | 0.0003 |
| loss | 5.23 |
| n_updates | 3524 |
| policy_gradient_loss | 0.000792 |
| value_loss | 15.3 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 265 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 403 |
| iterations | 883 |
| time_elapsed | 4481 |
| total_timesteps | 1808384 |
| train/ | |
| approx_kl | 0.006026434 |
| clip_fraction | 0.0872 |
| clip_range | 0.2 |
| entropy_loss | -0.617 |
| explained_variance | 0.9952954 |
| learning_rate | 0.0003 |
| loss | 2.76 |
| n_updates | 3528 |
| policy_gradient_loss | 0.00412 |
| value_loss | 7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 403 |
| iterations | 884 |
| time_elapsed | 4485 |
| total_timesteps | 1810432 |
| train/ | |
| approx_kl | 0.0027355286 |
| clip_fraction | 0.0415 |
| clip_range | 0.2 |
| entropy_loss | -0.651 |
| explained_variance | 0.87121 |
| learning_rate | 0.0003 |
| loss | 192 |
| n_updates | 3532 |
| policy_gradient_loss | 0.00039 |
| value_loss | 351 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 280 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 403 |
| iterations | 885 |
| time_elapsed | 4490 |
| total_timesteps | 1812480 |
| train/ | |
| approx_kl | 0.0053784596 |
| clip_fraction | 0.0574 |
| clip_range | 0.2 |
| entropy_loss | -0.614 |
| explained_variance | 0.99490994 |
| learning_rate | 0.0003 |
| loss | 1.9 |
| n_updates | 3536 |
| policy_gradient_loss | -0.00223 |
| value_loss | 9.68 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 278 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 403 |
| iterations | 886 |
| time_elapsed | 4494 |
| total_timesteps | 1814528 |
| train/ | |
| approx_kl | 0.0071990862 |
| clip_fraction | 0.0842 |
| clip_range | 0.2 |
| entropy_loss | -0.617 |
| explained_variance | 0.99450386 |
| learning_rate | 0.0003 |
| loss | 1.77 |
| n_updates | 3540 |
| policy_gradient_loss | 0.000558 |
| value_loss | 7.24 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 283 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 403 |
| iterations | 887 |
| time_elapsed | 4498 |
| total_timesteps | 1816576 |
| train/ | |
| approx_kl | 0.0015123705 |
| clip_fraction | 0.00891 |
| clip_range | 0.2 |
| entropy_loss | -0.557 |
| explained_variance | 0.8026234 |
| learning_rate | 0.0003 |
| loss | 221 |
| n_updates | 3544 |
| policy_gradient_loss | -0.000551 |
| value_loss | 815 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 277 |
| ep_rew_mean | 257 |
| time/ | |
| fps | 403 |
| iterations | 888 |
| time_elapsed | 4502 |
| total_timesteps | 1818624 |
| train/ | |
| approx_kl | 0.0024282713 |
| clip_fraction | 0.0125 |
| clip_range | 0.2 |
| entropy_loss | -0.589 |
| explained_variance | 0.8729852 |
| learning_rate | 0.0003 |
| loss | 120 |
| n_updates | 3548 |
| policy_gradient_loss | -0.00368 |
| value_loss | 396 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1820000 to videos/step_1820000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 274 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 403 |
| iterations | 889 |
| time_elapsed | 4508 |
| total_timesteps | 1820672 |
| train/ | |
| approx_kl | 0.0025102724 |
| clip_fraction | 0.0135 |
| clip_range | 0.2 |
| entropy_loss | -0.575 |
| explained_variance | 0.9653846 |
| learning_rate | 0.0003 |
| loss | 16.4 |
| n_updates | 3552 |
| policy_gradient_loss | -0.00096 |
| value_loss | 54.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 276 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 403 |
| iterations | 890 |
| time_elapsed | 4513 |
| total_timesteps | 1822720 |
| train/ | |
| approx_kl | 0.0012329258 |
| clip_fraction | 0.00684 |
| clip_range | 0.2 |
| entropy_loss | -0.543 |
| explained_variance | 0.92005646 |
| learning_rate | 0.0003 |
| loss | 6.03 |
| n_updates | 3556 |
| policy_gradient_loss | -0.000308 |
| value_loss | 70.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 264 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 403 |
| iterations | 891 |
| time_elapsed | 4517 |
| total_timesteps | 1824768 |
| train/ | |
| approx_kl | 0.0012477876 |
| clip_fraction | 0.00391 |
| clip_range | 0.2 |
| entropy_loss | -0.55 |
| explained_variance | 0.7512953 |
| learning_rate | 0.0003 |
| loss | 415 |
| n_updates | 3560 |
| policy_gradient_loss | -0.000823 |
| value_loss | 795 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 404 |
| iterations | 892 |
| time_elapsed | 4521 |
| total_timesteps | 1826816 |
| train/ | |
| approx_kl | 0.0031574145 |
| clip_fraction | 0.0278 |
| clip_range | 0.2 |
| entropy_loss | -0.557 |
| explained_variance | 0.9926779 |
| learning_rate | 0.0003 |
| loss | 4.48 |
| n_updates | 3564 |
| policy_gradient_loss | -0.00135 |
| value_loss | 14.5 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 256 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 404 |
| iterations | 893 |
| time_elapsed | 4525 |
| total_timesteps | 1828864 |
| train/ | |
| approx_kl | 0.005165954 |
| clip_fraction | 0.0513 |
| clip_range | 0.2 |
| entropy_loss | -0.6 |
| explained_variance | 0.99805987 |
| learning_rate | 0.0003 |
| loss | 3.19 |
| n_updates | 3568 |
| policy_gradient_loss | 0.00036 |
| value_loss | 4.98 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 257 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 404 |
| iterations | 894 |
| time_elapsed | 4529 |
| total_timesteps | 1830912 |
| train/ | |
| approx_kl | 0.0031339028 |
| clip_fraction | 0.0303 |
| clip_range | 0.2 |
| entropy_loss | -0.563 |
| explained_variance | 0.98235875 |
| learning_rate | 0.0003 |
| loss | 4.03 |
| n_updates | 3572 |
| policy_gradient_loss | -0.000908 |
| value_loss | 24.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 260 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 404 |
| iterations | 895 |
| time_elapsed | 4533 |
| total_timesteps | 1832960 |
| train/ | |
| approx_kl | 0.0045441827 |
| clip_fraction | 0.0547 |
| clip_range | 0.2 |
| entropy_loss | -0.565 |
| explained_variance | 0.9915079 |
| learning_rate | 0.0003 |
| loss | 7.04 |
| n_updates | 3576 |
| policy_gradient_loss | -0.00338 |
| value_loss | 15.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 404 |
| iterations | 896 |
| time_elapsed | 4538 |
| total_timesteps | 1835008 |
| train/ | |
| approx_kl | 0.0044818507 |
| clip_fraction | 0.0691 |
| clip_range | 0.2 |
| entropy_loss | -0.552 |
| explained_variance | 0.99327284 |
| learning_rate | 0.0003 |
| loss | 3.06 |
| n_updates | 3580 |
| policy_gradient_loss | -0.00211 |
| value_loss | 10.8 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 267 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 404 |
| iterations | 897 |
| time_elapsed | 4542 |
| total_timesteps | 1837056 |
| train/ | |
| approx_kl | 0.0043013166 |
| clip_fraction | 0.0103 |
| clip_range | 0.2 |
| entropy_loss | -0.641 |
| explained_variance | 0.86211145 |
| learning_rate | 0.0003 |
| loss | 116 |
| n_updates | 3584 |
| policy_gradient_loss | -0.000788 |
| value_loss | 452 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 254 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 404 |
| iterations | 898 |
| time_elapsed | 4546 |
| total_timesteps | 1839104 |
| train/ | |
| approx_kl | 0.003004441 |
| clip_fraction | 0.0228 |
| clip_range | 0.2 |
| entropy_loss | -0.505 |
| explained_variance | 0.98699474 |
| learning_rate | 0.0003 |
| loss | 6.27 |
| n_updates | 3588 |
| policy_gradient_loss | -0.000742 |
| value_loss | 19.9 |
-----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1840000 to videos/step_1840000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 251 |
| ep_rew_mean | 275 |
| time/ | |
| fps | 404 |
| iterations | 899 |
| time_elapsed | 4553 |
| total_timesteps | 1841152 |
| train/ | |
| approx_kl | 0.004820936 |
| clip_fraction | 0.0562 |
| clip_range | 0.2 |
| entropy_loss | -0.585 |
| explained_variance | 0.99786365 |
| learning_rate | 0.0003 |
| loss | 1.64 |
| n_updates | 3592 |
| policy_gradient_loss | -0.00275 |
| value_loss | 5.52 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 250 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 404 |
| iterations | 900 |
| time_elapsed | 4557 |
| total_timesteps | 1843200 |
| train/ | |
| approx_kl | 0.0035377743 |
| clip_fraction | 0.0271 |
| clip_range | 0.2 |
| entropy_loss | -0.559 |
| explained_variance | 0.90355617 |
| learning_rate | 0.0003 |
| loss | 410 |
| n_updates | 3596 |
| policy_gradient_loss | -0.00239 |
| value_loss | 377 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 246 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 404 |
| iterations | 901 |
| time_elapsed | 4561 |
| total_timesteps | 1845248 |
| train/ | |
| approx_kl | 0.0033690368 |
| clip_fraction | 0.0192 |
| clip_range | 0.2 |
| entropy_loss | -0.635 |
| explained_variance | 0.8861566 |
| learning_rate | 0.0003 |
| loss | 79.5 |
| n_updates | 3600 |
| policy_gradient_loss | -0.000501 |
| value_loss | 379 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 253 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 404 |
| iterations | 902 |
| time_elapsed | 4566 |
| total_timesteps | 1847296 |
| train/ | |
| approx_kl | 0.00090107834 |
| clip_fraction | 0.000977 |
| clip_range | 0.2 |
| entropy_loss | -0.585 |
| explained_variance | 0.86786056 |
| learning_rate | 0.0003 |
| loss | 271 |
| n_updates | 3604 |
| policy_gradient_loss | -0.000169 |
| value_loss | 460 |
-------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 254 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 404 |
| iterations | 903 |
| time_elapsed | 4570 |
| total_timesteps | 1849344 |
| train/ | |
| approx_kl | 0.004117831 |
| clip_fraction | 0.0446 |
| clip_range | 0.2 |
| entropy_loss | -0.589 |
| explained_variance | 0.9871657 |
| learning_rate | 0.0003 |
| loss | 2.69 |
| n_updates | 3608 |
| policy_gradient_loss | -0.00344 |
| value_loss | 15.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 252 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 404 |
| iterations | 904 |
| time_elapsed | 4574 |
| total_timesteps | 1851392 |
| train/ | |
| approx_kl | 0.0033420776 |
| clip_fraction | 0.0321 |
| clip_range | 0.2 |
| entropy_loss | -0.529 |
| explained_variance | 0.94523877 |
| learning_rate | 0.0003 |
| loss | 5.34 |
| n_updates | 3612 |
| policy_gradient_loss | -0.000942 |
| value_loss | 43.4 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 251 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 404 |
| iterations | 905 |
| time_elapsed | 4578 |
| total_timesteps | 1853440 |
| train/ | |
| approx_kl | 0.0017500257 |
| clip_fraction | 0.00745 |
| clip_range | 0.2 |
| entropy_loss | -0.617 |
| explained_variance | 0.80257916 |
| learning_rate | 0.0003 |
| loss | 400 |
| n_updates | 3616 |
| policy_gradient_loss | -0.00116 |
| value_loss | 767 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 251 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 404 |
| iterations | 906 |
| time_elapsed | 4582 |
| total_timesteps | 1855488 |
| train/ | |
| approx_kl | 0.0008659419 |
| clip_fraction | 0.00269 |
| clip_range | 0.2 |
| entropy_loss | -0.539 |
| explained_variance | 0.8032426 |
| learning_rate | 0.0003 |
| loss | 43.5 |
| n_updates | 3620 |
| policy_gradient_loss | -0.000648 |
| value_loss | 327 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 245 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 404 |
| iterations | 907 |
| time_elapsed | 4586 |
| total_timesteps | 1857536 |
| train/ | |
| approx_kl | 0.0030136236 |
| clip_fraction | 0.0234 |
| clip_range | 0.2 |
| entropy_loss | -0.621 |
| explained_variance | 0.8718165 |
| learning_rate | 0.0003 |
| loss | 436 |
| n_updates | 3624 |
| policy_gradient_loss | -0.00137 |
| value_loss | 379 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 263 |
| time/ | |
| fps | 405 |
| iterations | 908 |
| time_elapsed | 4591 |
| total_timesteps | 1859584 |
| train/ | |
| approx_kl | 0.0006219947 |
| clip_fraction | 0.00061 |
| clip_range | 0.2 |
| entropy_loss | -0.599 |
| explained_variance | 0.90436405 |
| learning_rate | 0.0003 |
| loss | 40.7 |
| n_updates | 3628 |
| policy_gradient_loss | -0.000233 |
| value_loss | 272 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1860000 to videos/step_1860000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 236 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 404 |
| iterations | 909 |
| time_elapsed | 4597 |
| total_timesteps | 1861632 |
| train/ | |
| approx_kl | 0.002393303 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -0.593 |
| explained_variance | 0.93224853 |
| learning_rate | 0.0003 |
| loss | 4.83 |
| n_updates | 3632 |
| policy_gradient_loss | -0.00139 |
| value_loss | 56.4 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 405 |
| iterations | 910 |
| time_elapsed | 4601 |
| total_timesteps | 1863680 |
| train/ | |
| approx_kl | 0.0014502569 |
| clip_fraction | 0.00647 |
| clip_range | 0.2 |
| entropy_loss | -0.575 |
| explained_variance | 0.87157905 |
| learning_rate | 0.0003 |
| loss | 118 |
| n_updates | 3636 |
| policy_gradient_loss | 0.000224 |
| value_loss | 351 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 405 |
| iterations | 911 |
| time_elapsed | 4605 |
| total_timesteps | 1865728 |
| train/ | |
| approx_kl | 0.0051596044 |
| clip_fraction | 0.0332 |
| clip_range | 0.2 |
| entropy_loss | -0.586 |
| explained_variance | 0.97924703 |
| learning_rate | 0.0003 |
| loss | 5.1 |
| n_updates | 3640 |
| policy_gradient_loss | -4.5e-05 |
| value_loss | 18.1 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 233 |
| ep_rew_mean | 261 |
| time/ | |
| fps | 405 |
| iterations | 912 |
| time_elapsed | 4609 |
| total_timesteps | 1867776 |
| train/ | |
| approx_kl | 0.0029927231 |
| clip_fraction | 0.0276 |
| clip_range | 0.2 |
| entropy_loss | -0.565 |
| explained_variance | 0.89742184 |
| learning_rate | 0.0003 |
| loss | 156 |
| n_updates | 3644 |
| policy_gradient_loss | -0.00115 |
| value_loss | 263 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 262 |
| time/ | |
| fps | 405 |
| iterations | 913 |
| time_elapsed | 4613 |
| total_timesteps | 1869824 |
| train/ | |
| approx_kl | 0.003498425 |
| clip_fraction | 0.0203 |
| clip_range | 0.2 |
| entropy_loss | -0.545 |
| explained_variance | 0.96629333 |
| learning_rate | 0.0003 |
| loss | 13.7 |
| n_updates | 3648 |
| policy_gradient_loss | -0.00319 |
| value_loss | 79 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 231 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 405 |
| iterations | 914 |
| time_elapsed | 4618 |
| total_timesteps | 1871872 |
| train/ | |
| approx_kl | 0.009469861 |
| clip_fraction | 0.0668 |
| clip_range | 0.2 |
| entropy_loss | -0.574 |
| explained_variance | 0.99053496 |
| learning_rate | 0.0003 |
| loss | 3.46 |
| n_updates | 3652 |
| policy_gradient_loss | 3.57e-05 |
| value_loss | 11.6 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 230 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 405 |
| iterations | 915 |
| time_elapsed | 4622 |
| total_timesteps | 1873920 |
| train/ | |
| approx_kl | 0.0015345097 |
| clip_fraction | 0.0219 |
| clip_range | 0.2 |
| entropy_loss | -0.65 |
| explained_variance | 0.8636657 |
| learning_rate | 0.0003 |
| loss | 182 |
| n_updates | 3656 |
| policy_gradient_loss | -0.000944 |
| value_loss | 417 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 232 |
| ep_rew_mean | 260 |
| time/ | |
| fps | 405 |
| iterations | 916 |
| time_elapsed | 4626 |
| total_timesteps | 1875968 |
| train/ | |
| approx_kl | 0.0005273888 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.594 |
| explained_variance | 0.8525693 |
| learning_rate | 0.0003 |
| loss | 124 |
| n_updates | 3660 |
| policy_gradient_loss | -0.000312 |
| value_loss | 469 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 231 |
| ep_rew_mean | 259 |
| time/ | |
| fps | 405 |
| iterations | 917 |
| time_elapsed | 4630 |
| total_timesteps | 1878016 |
| train/ | |
| approx_kl | 0.0014101019 |
| clip_fraction | 0.0189 |
| clip_range | 0.2 |
| entropy_loss | -0.547 |
| explained_variance | 0.96272933 |
| learning_rate | 0.0003 |
| loss | 4.02 |
| n_updates | 3664 |
| policy_gradient_loss | -0.000285 |
| value_loss | 39 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1880000 to videos/step_1880000.mp4
-----------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 404 |
| iterations | 918 |
| time_elapsed | 4642 |
| total_timesteps | 1880064 |
| train/ | |
| approx_kl | 0.008527126 |
| clip_fraction | 0.0668 |
| clip_range | 0.2 |
| entropy_loss | -0.581 |
| explained_variance | 0.99461424 |
| learning_rate | 0.0003 |
| loss | 3.04 |
| n_updates | 3668 |
| policy_gradient_loss | -0.00195 |
| value_loss | 7.39 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 405 |
| iterations | 919 |
| time_elapsed | 4646 |
| total_timesteps | 1882112 |
| train/ | |
| approx_kl | 0.0066773626 |
| clip_fraction | 0.0392 |
| clip_range | 0.2 |
| entropy_loss | -0.546 |
| explained_variance | 0.9856499 |
| learning_rate | 0.0003 |
| loss | 4 |
| n_updates | 3672 |
| policy_gradient_loss | 0.000683 |
| value_loss | 16.4 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 405 |
| iterations | 920 |
| time_elapsed | 4650 |
| total_timesteps | 1884160 |
| train/ | |
| approx_kl | 0.004666223 |
| clip_fraction | 0.0503 |
| clip_range | 0.2 |
| entropy_loss | -0.599 |
| explained_variance | 0.9978609 |
| learning_rate | 0.0003 |
| loss | 2 |
| n_updates | 3676 |
| policy_gradient_loss | 7.89e-05 |
| value_loss | 4.27 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 405 |
| iterations | 921 |
| time_elapsed | 4655 |
| total_timesteps | 1886208 |
| train/ | |
| approx_kl | 0.0046107145 |
| clip_fraction | 0.0493 |
| clip_range | 0.2 |
| entropy_loss | -0.603 |
| explained_variance | 0.99790525 |
| learning_rate | 0.0003 |
| loss | 2.15 |
| n_updates | 3680 |
| policy_gradient_loss | 0.000485 |
| value_loss | 4.74 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 405 |
| iterations | 922 |
| time_elapsed | 4659 |
| total_timesteps | 1888256 |
| train/ | |
| approx_kl | 0.004198748 |
| clip_fraction | 0.0447 |
| clip_range | 0.2 |
| entropy_loss | -0.613 |
| explained_variance | 0.9974743 |
| learning_rate | 0.0003 |
| loss | 1.43 |
| n_updates | 3684 |
| policy_gradient_loss | -0.00103 |
| value_loss | 3.51 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 269 |
| time/ | |
| fps | 405 |
| iterations | 923 |
| time_elapsed | 4663 |
| total_timesteps | 1890304 |
| train/ | |
| approx_kl | 0.0018898075 |
| clip_fraction | 0.0115 |
| clip_range | 0.2 |
| entropy_loss | -0.562 |
| explained_variance | 0.92799187 |
| learning_rate | 0.0003 |
| loss | 31.3 |
| n_updates | 3688 |
| policy_gradient_loss | -0.000147 |
| value_loss | 203 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 227 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 405 |
| iterations | 924 |
| time_elapsed | 4667 |
| total_timesteps | 1892352 |
| train/ | |
| approx_kl | 0.0038133203 |
| clip_fraction | 0.0209 |
| clip_range | 0.2 |
| entropy_loss | -0.579 |
| explained_variance | 0.982112 |
| learning_rate | 0.0003 |
| loss | 2.4 |
| n_updates | 3692 |
| policy_gradient_loss | -0.000273 |
| value_loss | 16.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 227 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 405 |
| iterations | 925 |
| time_elapsed | 4671 |
| total_timesteps | 1894400 |
| train/ | |
| approx_kl | 0.0033408464 |
| clip_fraction | 0.0266 |
| clip_range | 0.2 |
| entropy_loss | -0.591 |
| explained_variance | 0.9882088 |
| learning_rate | 0.0003 |
| loss | 2.39 |
| n_updates | 3696 |
| policy_gradient_loss | 0.000413 |
| value_loss | 9.22 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 228 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 405 |
| iterations | 926 |
| time_elapsed | 4675 |
| total_timesteps | 1896448 |
| train/ | |
| approx_kl | 0.0015369814 |
| clip_fraction | 0.0167 |
| clip_range | 0.2 |
| entropy_loss | -0.539 |
| explained_variance | 0.8757845 |
| learning_rate | 0.0003 |
| loss | 9.32 |
| n_updates | 3700 |
| policy_gradient_loss | 0.000242 |
| value_loss | 242 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 227 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 405 |
| iterations | 927 |
| time_elapsed | 4680 |
| total_timesteps | 1898496 |
| train/ | |
| approx_kl | 0.00137565 |
| clip_fraction | 0.00452 |
| clip_range | 0.2 |
| entropy_loss | -0.562 |
| explained_variance | 0.9445376 |
| learning_rate | 0.0003 |
| loss | 150 |
| n_updates | 3704 |
| policy_gradient_loss | -0.00153 |
| value_loss | 185 |
----------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1900000 to videos/step_1900000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 227 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 405 |
| iterations | 928 |
| time_elapsed | 4686 |
| total_timesteps | 1900544 |
| train/ | |
| approx_kl | 0.0033551925 |
| clip_fraction | 0.0403 |
| clip_range | 0.2 |
| entropy_loss | -0.552 |
| explained_variance | 0.9902085 |
| learning_rate | 0.0003 |
| loss | 4.83 |
| n_updates | 3708 |
| policy_gradient_loss | -0.00083 |
| value_loss | 11.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 228 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 405 |
| iterations | 929 |
| time_elapsed | 4690 |
| total_timesteps | 1902592 |
| train/ | |
| approx_kl | 0.0004718143 |
| clip_fraction | 0.0011 |
| clip_range | 0.2 |
| entropy_loss | -0.555 |
| explained_variance | 0.77297914 |
| learning_rate | 0.0003 |
| loss | 281 |
| n_updates | 3712 |
| policy_gradient_loss | -0.000104 |
| value_loss | 763 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 225 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 405 |
| iterations | 930 |
| time_elapsed | 4694 |
| total_timesteps | 1904640 |
| train/ | |
| approx_kl | 0.004694662 |
| clip_fraction | 0.0471 |
| clip_range | 0.2 |
| entropy_loss | -0.554 |
| explained_variance | 0.9952979 |
| learning_rate | 0.0003 |
| loss | 1.95 |
| n_updates | 3716 |
| policy_gradient_loss | 0.000839 |
| value_loss | 7.48 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 225 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 405 |
| iterations | 931 |
| time_elapsed | 4698 |
| total_timesteps | 1906688 |
| train/ | |
| approx_kl | 0.003371601 |
| clip_fraction | 0.0303 |
| clip_range | 0.2 |
| entropy_loss | -0.526 |
| explained_variance | 0.9974706 |
| learning_rate | 0.0003 |
| loss | 2.29 |
| n_updates | 3720 |
| policy_gradient_loss | -0.00115 |
| value_loss | 5.45 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 224 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 405 |
| iterations | 932 |
| time_elapsed | 4702 |
| total_timesteps | 1908736 |
| train/ | |
| approx_kl | 0.004781856 |
| clip_fraction | 0.0603 |
| clip_range | 0.2 |
| entropy_loss | -0.643 |
| explained_variance | 0.9987749 |
| learning_rate | 0.0003 |
| loss | 0.874 |
| n_updates | 3724 |
| policy_gradient_loss | -0.00168 |
| value_loss | 3.02 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 223 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 405 |
| iterations | 933 |
| time_elapsed | 4707 |
| total_timesteps | 1910784 |
| train/ | |
| approx_kl | 0.003527997 |
| clip_fraction | 0.0333 |
| clip_range | 0.2 |
| entropy_loss | -0.56 |
| explained_variance | 0.9983686 |
| learning_rate | 0.0003 |
| loss | 0.967 |
| n_updates | 3728 |
| policy_gradient_loss | -0.000795 |
| value_loss | 3.8 |
-----------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 223 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 406 |
| iterations | 934 |
| time_elapsed | 4711 |
| total_timesteps | 1912832 |
| train/ | |
| approx_kl | 0.00067814696 |
| clip_fraction | 0.0022 |
| clip_range | 0.2 |
| entropy_loss | -0.531 |
| explained_variance | 0.8708347 |
| learning_rate | 0.0003 |
| loss | 46.5 |
| n_updates | 3732 |
| policy_gradient_loss | 0.000118 |
| value_loss | 385 |
-------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 224 |
| ep_rew_mean | 273 |
| time/ | |
| fps | 406 |
| iterations | 935 |
| time_elapsed | 4715 |
| total_timesteps | 1914880 |
| train/ | |
| approx_kl | 0.00708913 |
| clip_fraction | 0.0455 |
| clip_range | 0.2 |
| entropy_loss | -0.577 |
| explained_variance | 0.9855586 |
| learning_rate | 0.0003 |
| loss | 2.89 |
| n_updates | 3736 |
| policy_gradient_loss | -0.00175 |
| value_loss | 14.9 |
----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 225 |
| ep_rew_mean | 276 |
| time/ | |
| fps | 406 |
| iterations | 936 |
| time_elapsed | 4719 |
| total_timesteps | 1916928 |
| train/ | |
| approx_kl | 0.004260849 |
| clip_fraction | 0.0387 |
| clip_range | 0.2 |
| entropy_loss | -0.579 |
| explained_variance | 0.99356955 |
| learning_rate | 0.0003 |
| loss | 1.9 |
| n_updates | 3740 |
| policy_gradient_loss | -0.000623 |
| value_loss | 7.52 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 225 |
| ep_rew_mean | 278 |
| time/ | |
| fps | 406 |
| iterations | 937 |
| time_elapsed | 4723 |
| total_timesteps | 1918976 |
| train/ | |
| approx_kl | 0.0033427905 |
| clip_fraction | 0.0375 |
| clip_range | 0.2 |
| entropy_loss | -0.537 |
| explained_variance | 0.9971184 |
| learning_rate | 0.0003 |
| loss | 2.51 |
| n_updates | 3744 |
| policy_gradient_loss | -0.00199 |
| value_loss | 6.04 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1920000 to videos/step_1920000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 224 |
| ep_rew_mean | 276 |
| time/ | |
| fps | 406 |
| iterations | 938 |
| time_elapsed | 4730 |
| total_timesteps | 1921024 |
| train/ | |
| approx_kl | 0.0036931352 |
| clip_fraction | 0.0236 |
| clip_range | 0.2 |
| entropy_loss | -0.572 |
| explained_variance | 0.99658376 |
| learning_rate | 0.0003 |
| loss | 2.16 |
| n_updates | 3748 |
| policy_gradient_loss | 0.000291 |
| value_loss | 7.55 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 232 |
| ep_rew_mean | 277 |
| time/ | |
| fps | 406 |
| iterations | 939 |
| time_elapsed | 4735 |
| total_timesteps | 1923072 |
| train/ | |
| approx_kl | 0.0006715879 |
| clip_fraction | 0.0259 |
| clip_range | 0.2 |
| entropy_loss | -0.595 |
| explained_variance | 0.8585626 |
| learning_rate | 0.0003 |
| loss | 323 |
| n_updates | 3752 |
| policy_gradient_loss | 0.00124 |
| value_loss | 488 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 230 |
| ep_rew_mean | 277 |
| time/ | |
| fps | 406 |
| iterations | 940 |
| time_elapsed | 4739 |
| total_timesteps | 1925120 |
| train/ | |
| approx_kl | 0.0046601435 |
| clip_fraction | 0.0439 |
| clip_range | 0.2 |
| entropy_loss | -0.731 |
| explained_variance | 0.9900083 |
| learning_rate | 0.0003 |
| loss | 2.58 |
| n_updates | 3756 |
| policy_gradient_loss | -0.00199 |
| value_loss | 15.5 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 231 |
| ep_rew_mean | 276 |
| time/ | |
| fps | 406 |
| iterations | 941 |
| time_elapsed | 4743 |
| total_timesteps | 1927168 |
| train/ | |
| approx_kl | 0.0031685128 |
| clip_fraction | 0.0248 |
| clip_range | 0.2 |
| entropy_loss | -0.552 |
| explained_variance | 0.9950303 |
| learning_rate | 0.0003 |
| loss | 3.08 |
| n_updates | 3760 |
| policy_gradient_loss | -0.000986 |
| value_loss | 8.74 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 232 |
| ep_rew_mean | 274 |
| time/ | |
| fps | 406 |
| iterations | 942 |
| time_elapsed | 4747 |
| total_timesteps | 1929216 |
| train/ | |
| approx_kl | 0.0046357634 |
| clip_fraction | 0.0558 |
| clip_range | 0.2 |
| entropy_loss | -0.507 |
| explained_variance | 0.9980434 |
| learning_rate | 0.0003 |
| loss | 1.4 |
| n_updates | 3764 |
| policy_gradient_loss | -0.00464 |
| value_loss | 4.07 |
------------------------------------------
----------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 406 |
| iterations | 943 |
| time_elapsed | 4751 |
| total_timesteps | 1931264 |
| train/ | |
| approx_kl | 0.00361666 |
| clip_fraction | 0.0416 |
| clip_range | 0.2 |
| entropy_loss | -0.535 |
| explained_variance | 0.8463491 |
| learning_rate | 0.0003 |
| loss | 399 |
| n_updates | 3768 |
| policy_gradient_loss | 0.00121 |
| value_loss | 501 |
----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 236 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 406 |
| iterations | 944 |
| time_elapsed | 4755 |
| total_timesteps | 1933312 |
| train/ | |
| approx_kl | 0.0023488286 |
| clip_fraction | 0.00708 |
| clip_range | 0.2 |
| entropy_loss | -0.505 |
| explained_variance | 0.8515509 |
| learning_rate | 0.0003 |
| loss | 55.1 |
| n_updates | 3772 |
| policy_gradient_loss | -0.000832 |
| value_loss | 514 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 406 |
| iterations | 945 |
| time_elapsed | 4759 |
| total_timesteps | 1935360 |
| train/ | |
| approx_kl | 0.0010557299 |
| clip_fraction | 0.00146 |
| clip_range | 0.2 |
| entropy_loss | -0.571 |
| explained_variance | 0.84740686 |
| learning_rate | 0.0003 |
| loss | 96.2 |
| n_updates | 3776 |
| policy_gradient_loss | -0.000134 |
| value_loss | 519 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 236 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 406 |
| iterations | 946 |
| time_elapsed | 4763 |
| total_timesteps | 1937408 |
| train/ | |
| approx_kl | 0.0015555376 |
| clip_fraction | 0.0198 |
| clip_range | 0.2 |
| entropy_loss | -0.594 |
| explained_variance | 0.8225767 |
| learning_rate | 0.0003 |
| loss | 54.4 |
| n_updates | 3780 |
| policy_gradient_loss | -0.000203 |
| value_loss | 521 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 406 |
| iterations | 947 |
| time_elapsed | 4767 |
| total_timesteps | 1939456 |
| train/ | |
| approx_kl | 0.0047367485 |
| clip_fraction | 0.0299 |
| clip_range | 0.2 |
| entropy_loss | -0.589 |
| explained_variance | 0.9923876 |
| learning_rate | 0.0003 |
| loss | 2.83 |
| n_updates | 3784 |
| policy_gradient_loss | -0.000365 |
| value_loss | 9.26 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1940000 to videos/step_1940000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 406 |
| iterations | 948 |
| time_elapsed | 4774 |
| total_timesteps | 1941504 |
| train/ | |
| approx_kl | 0.0033371304 |
| clip_fraction | 0.0201 |
| clip_range | 0.2 |
| entropy_loss | -0.546 |
| explained_variance | 0.98872054 |
| learning_rate | 0.0003 |
| loss | 2.21 |
| n_updates | 3788 |
| policy_gradient_loss | 0.000347 |
| value_loss | 9.68 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 406 |
| iterations | 949 |
| time_elapsed | 4778 |
| total_timesteps | 1943552 |
| train/ | |
| approx_kl | 0.006179631 |
| clip_fraction | 0.045 |
| clip_range | 0.2 |
| entropy_loss | -0.588 |
| explained_variance | 0.99618644 |
| learning_rate | 0.0003 |
| loss | 1.45 |
| n_updates | 3792 |
| policy_gradient_loss | -0.000807 |
| value_loss | 5.53 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 240 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 406 |
| iterations | 950 |
| time_elapsed | 4783 |
| total_timesteps | 1945600 |
| train/ | |
| approx_kl | 0.0018546742 |
| clip_fraction | 0.00793 |
| clip_range | 0.2 |
| entropy_loss | -0.529 |
| explained_variance | 0.9103193 |
| learning_rate | 0.0003 |
| loss | 6.15 |
| n_updates | 3796 |
| policy_gradient_loss | -0.000664 |
| value_loss | 75.9 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 406 |
| iterations | 951 |
| time_elapsed | 4787 |
| total_timesteps | 1947648 |
| train/ | |
| approx_kl | 0.0027506552 |
| clip_fraction | 0.0386 |
| clip_range | 0.2 |
| entropy_loss | -0.553 |
| explained_variance | 0.92629397 |
| learning_rate | 0.0003 |
| loss | 8.92 |
| n_updates | 3800 |
| policy_gradient_loss | -0.00158 |
| value_loss | 62.3 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 235 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 406 |
| iterations | 952 |
| time_elapsed | 4791 |
| total_timesteps | 1949696 |
| train/ | |
| approx_kl | 0.0035452878 |
| clip_fraction | 0.0364 |
| clip_range | 0.2 |
| entropy_loss | -0.533 |
| explained_variance | 0.99048495 |
| learning_rate | 0.0003 |
| loss | 2.33 |
| n_updates | 3804 |
| policy_gradient_loss | -0.00357 |
| value_loss | 8.91 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 406 |
| iterations | 953 |
| time_elapsed | 4796 |
| total_timesteps | 1951744 |
| train/ | |
| approx_kl | 0.005390859 |
| clip_fraction | 0.0492 |
| clip_range | 0.2 |
| entropy_loss | -0.559 |
| explained_variance | 0.9977774 |
| learning_rate | 0.0003 |
| loss | 1.25 |
| n_updates | 3808 |
| policy_gradient_loss | -0.00209 |
| value_loss | 3.77 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 407 |
| iterations | 954 |
| time_elapsed | 4800 |
| total_timesteps | 1953792 |
| train/ | |
| approx_kl | 0.0037935097 |
| clip_fraction | 0.0411 |
| clip_range | 0.2 |
| entropy_loss | -0.579 |
| explained_variance | 0.99792594 |
| learning_rate | 0.0003 |
| loss | 2.04 |
| n_updates | 3812 |
| policy_gradient_loss | -0.000181 |
| value_loss | 4.06 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 236 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 407 |
| iterations | 955 |
| time_elapsed | 4804 |
| total_timesteps | 1955840 |
| train/ | |
| approx_kl | 0.0067280633 |
| clip_fraction | 0.0605 |
| clip_range | 0.2 |
| entropy_loss | -0.526 |
| explained_variance | 0.9980719 |
| learning_rate | 0.0003 |
| loss | 2.29 |
| n_updates | 3816 |
| policy_gradient_loss | -0.000681 |
| value_loss | 3.36 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 407 |
| iterations | 956 |
| time_elapsed | 4808 |
| total_timesteps | 1957888 |
| train/ | |
| approx_kl | 0.0031404013 |
| clip_fraction | 0.016 |
| clip_range | 0.2 |
| entropy_loss | -0.545 |
| explained_variance | 0.84338784 |
| learning_rate | 0.0003 |
| loss | 31.6 |
| n_updates | 3820 |
| policy_gradient_loss | -0.00128 |
| value_loss | 459 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 407 |
| iterations | 957 |
| time_elapsed | 4812 |
| total_timesteps | 1959936 |
| train/ | |
| approx_kl | 0.0013297596 |
| clip_fraction | 0.0011 |
| clip_range | 0.2 |
| entropy_loss | -0.527 |
| explained_variance | 0.84983855 |
| learning_rate | 0.0003 |
| loss | 104 |
| n_updates | 3824 |
| policy_gradient_loss | -0.000514 |
| value_loss | 511 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1960000 to videos/step_1960000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 271 |
| time/ | |
| fps | 407 |
| iterations | 958 |
| time_elapsed | 4818 |
| total_timesteps | 1961984 |
| train/ | |
| approx_kl | 0.0004987758 |
| clip_fraction | 0.0022 |
| clip_range | 0.2 |
| entropy_loss | -0.527 |
| explained_variance | 0.8671855 |
| learning_rate | 0.0003 |
| loss | 84.5 |
| n_updates | 3828 |
| policy_gradient_loss | -0.000366 |
| value_loss | 476 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 239 |
| ep_rew_mean | 272 |
| time/ | |
| fps | 407 |
| iterations | 959 |
| time_elapsed | 4823 |
| total_timesteps | 1964032 |
| train/ | |
| approx_kl | 0.0061710747 |
| clip_fraction | 0.0453 |
| clip_range | 0.2 |
| entropy_loss | -0.531 |
| explained_variance | 0.9915919 |
| learning_rate | 0.0003 |
| loss | 4.62 |
| n_updates | 3832 |
| policy_gradient_loss | -0.00132 |
| value_loss | 11.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 407 |
| iterations | 960 |
| time_elapsed | 4827 |
| total_timesteps | 1966080 |
| train/ | |
| approx_kl | 0.0034670923 |
| clip_fraction | 0.0393 |
| clip_range | 0.2 |
| entropy_loss | -0.519 |
| explained_variance | 0.99446785 |
| learning_rate | 0.0003 |
| loss | 2.71 |
| n_updates | 3836 |
| policy_gradient_loss | -0.000929 |
| value_loss | 8.98 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 236 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 407 |
| iterations | 961 |
| time_elapsed | 4831 |
| total_timesteps | 1968128 |
| train/ | |
| approx_kl | 0.0025692694 |
| clip_fraction | 0.0444 |
| clip_range | 0.2 |
| entropy_loss | -0.53 |
| explained_variance | 0.86881316 |
| learning_rate | 0.0003 |
| loss | 93.3 |
| n_updates | 3840 |
| policy_gradient_loss | -0.00363 |
| value_loss | 385 |
------------------------------------------
-------------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 407 |
| iterations | 962 |
| time_elapsed | 4836 |
| total_timesteps | 1970176 |
| train/ | |
| approx_kl | 8.1713224e-05 |
| clip_fraction | 0 |
| clip_range | 0.2 |
| entropy_loss | -0.568 |
| explained_variance | 0.75372195 |
| learning_rate | 0.0003 |
| loss | 662 |
| n_updates | 3844 |
| policy_gradient_loss | -0.000175 |
| value_loss | 940 |
-------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 234 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 407 |
| iterations | 963 |
| time_elapsed | 4840 |
| total_timesteps | 1972224 |
| train/ | |
| approx_kl | 0.0010016304 |
| clip_fraction | 0.00195 |
| clip_range | 0.2 |
| entropy_loss | -0.57 |
| explained_variance | 0.83415884 |
| learning_rate | 0.0003 |
| loss | 211 |
| n_updates | 3848 |
| policy_gradient_loss | -0.000728 |
| value_loss | 422 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 243 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 407 |
| iterations | 964 |
| time_elapsed | 4844 |
| total_timesteps | 1974272 |
| train/ | |
| approx_kl | 0.0017162634 |
| clip_fraction | 0.0109 |
| clip_range | 0.2 |
| entropy_loss | -0.526 |
| explained_variance | 0.98234195 |
| learning_rate | 0.0003 |
| loss | 2.72 |
| n_updates | 3852 |
| policy_gradient_loss | -0.00055 |
| value_loss | 24.7 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 242 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 407 |
| iterations | 965 |
| time_elapsed | 4849 |
| total_timesteps | 1976320 |
| train/ | |
| approx_kl | 0.0064403675 |
| clip_fraction | 0.0225 |
| clip_range | 0.2 |
| entropy_loss | -0.655 |
| explained_variance | 0.98883444 |
| learning_rate | 0.0003 |
| loss | 1.01 |
| n_updates | 3856 |
| policy_gradient_loss | 0.000379 |
| value_loss | 9.37 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 241 |
| ep_rew_mean | 264 |
| time/ | |
| fps | 407 |
| iterations | 966 |
| time_elapsed | 4853 |
| total_timesteps | 1978368 |
| train/ | |
| approx_kl | 0.0063675153 |
| clip_fraction | 0.0403 |
| clip_range | 0.2 |
| entropy_loss | -0.551 |
| explained_variance | 0.9934892 |
| learning_rate | 0.0003 |
| loss | 2.67 |
| n_updates | 3860 |
| policy_gradient_loss | -0.00195 |
| value_loss | 10.9 |
------------------------------------------
WARNING:imageio_ffmpeg:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[Callback] Saved video frame at step 1980000 to videos/step_1980000.mp4
------------------------------------------
| rollout/ | |
| ep_len_mean | 241 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 407 |
| iterations | 967 |
| time_elapsed | 4859 |
| total_timesteps | 1980416 |
| train/ | |
| approx_kl | 0.0055375104 |
| clip_fraction | 0.0466 |
| clip_range | 0.2 |
| entropy_loss | -0.561 |
| explained_variance | 0.9974091 |
| learning_rate | 0.0003 |
| loss | 2.05 |
| n_updates | 3864 |
| policy_gradient_loss | -0.000545 |
| value_loss | 4.99 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 240 |
| ep_rew_mean | 267 |
| time/ | |
| fps | 407 |
| iterations | 968 |
| time_elapsed | 4863 |
| total_timesteps | 1982464 |
| train/ | |
| approx_kl | 0.0015281795 |
| clip_fraction | 0.0142 |
| clip_range | 0.2 |
| entropy_loss | -0.586 |
| explained_variance | 0.8587976 |
| learning_rate | 0.0003 |
| loss | 131 |
| n_updates | 3868 |
| policy_gradient_loss | 0.000122 |
| value_loss | 496 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 240 |
| ep_rew_mean | 269 |
| time/ | |
| fps | 407 |
| iterations | 969 |
| time_elapsed | 4867 |
| total_timesteps | 1984512 |
| train/ | |
| approx_kl | 0.0029085916 |
| clip_fraction | 0.0212 |
| clip_range | 0.2 |
| entropy_loss | -0.523 |
| explained_variance | 0.99119264 |
| learning_rate | 0.0003 |
| loss | 4.74 |
| n_updates | 3872 |
| policy_gradient_loss | -0.00118 |
| value_loss | 15 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 239 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 407 |
| iterations | 970 |
| time_elapsed | 4871 |
| total_timesteps | 1986560 |
| train/ | |
| approx_kl | 0.005256891 |
| clip_fraction | 0.0413 |
| clip_range | 0.2 |
| entropy_loss | -0.57 |
| explained_variance | 0.9962855 |
| learning_rate | 0.0003 |
| loss | 2.13 |
| n_updates | 3876 |
| policy_gradient_loss | 0.00175 |
| value_loss | 7.37 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 265 |
| time/ | |
| fps | 407 |
| iterations | 971 |
| time_elapsed | 4876 |
| total_timesteps | 1988608 |
| train/ | |
| approx_kl | 0.0029739114 |
| clip_fraction | 0.0145 |
| clip_range | 0.2 |
| entropy_loss | -0.572 |
| explained_variance | 0.91433555 |
| learning_rate | 0.0003 |
| loss | 52.4 |
| n_updates | 3880 |
| policy_gradient_loss | 1.71e-05 |
| value_loss | 102 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 270 |
| time/ | |
| fps | 407 |
| iterations | 972 |
| time_elapsed | 4880 |
| total_timesteps | 1990656 |
| train/ | |
| approx_kl | 0.003245582 |
| clip_fraction | 0.0411 |
| clip_range | 0.2 |
| entropy_loss | -0.548 |
| explained_variance | 0.99791026 |
| learning_rate | 0.0003 |
| loss | 2.61 |
| n_updates | 3884 |
| policy_gradient_loss | 0.000786 |
| value_loss | 5.57 |
-----------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 237 |
| ep_rew_mean | 266 |
| time/ | |
| fps | 407 |
| iterations | 973 |
| time_elapsed | 4884 |
| total_timesteps | 1992704 |
| train/ | |
| approx_kl | 0.003389536 |
| clip_fraction | 0.033 |
| clip_range | 0.2 |
| entropy_loss | -0.566 |
| explained_variance | 0.9866096 |
| learning_rate | 0.0003 |
| loss | 6.55 |
| n_updates | 3888 |
| policy_gradient_loss | -0.000513 |
| value_loss | 12.7 |
-----------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 238 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 408 |
| iterations | 974 |
| time_elapsed | 4889 |
| total_timesteps | 1994752 |
| train/ | |
| approx_kl | 0.0011488361 |
| clip_fraction | 0.0131 |
| clip_range | 0.2 |
| entropy_loss | -0.559 |
| explained_variance | 0.7534488 |
| learning_rate | 0.0003 |
| loss | 774 |
| n_updates | 3892 |
| policy_gradient_loss | -0.000511 |
| value_loss | 951 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 232 |
| ep_rew_mean | 269 |
| time/ | |
| fps | 408 |
| iterations | 975 |
| time_elapsed | 4893 |
| total_timesteps | 1996800 |
| train/ | |
| approx_kl | 0.0037559792 |
| clip_fraction | 0.0255 |
| clip_range | 0.2 |
| entropy_loss | -0.561 |
| explained_variance | 0.96748424 |
| learning_rate | 0.0003 |
| loss | 2.38 |
| n_updates | 3896 |
| policy_gradient_loss | -0.00213 |
| value_loss | 27.2 |
------------------------------------------
------------------------------------------
| rollout/ | |
| ep_len_mean | 232 |
| ep_rew_mean | 268 |
| time/ | |
| fps | 408 |
| iterations | 976 |
| time_elapsed | 4897 |
| total_timesteps | 1998848 |
| train/ | |
| approx_kl | 0.0074330773 |
| clip_fraction | 0.0573 |
| clip_range | 0.2 |
| entropy_loss | -0.567 |
| explained_variance | 0.99268043 |
| learning_rate | 0.0003 |
| loss | 3.13 |
| n_updates | 3900 |
| policy_gradient_loss | -0.00352 |
| value_loss | 8.25 |
------------------------------------------
-----------------------------------------
| rollout/ | |
| ep_len_mean | 232 |
| ep_rew_mean | 269 |
| time/ | |
| fps | 408 |
| iterations | 977 |
| time_elapsed | 4901 |
| total_timesteps | 2000896 |
| train/ | |
| approx_kl | 0.008311205 |
| clip_fraction | 0.0612 |
| clip_range | 0.2 |
| entropy_loss | -0.531 |
| explained_variance | 0.9964366 |
| learning_rate | 0.0003 |
| loss | 1.68 |
| n_updates | 3904 |
| policy_gradient_loss | -0.00202 |
| value_loss | 5.16 |
-----------------------------------------
[Callback] Total episodes completed: [6348]
<stable_baselines3.ppo.ppo.PPO at 0x7c052d2b4e90>
model.save("ppo-LunaLander-v2")
# Evaluate the agent
# First wrap the model in Monitor
eval_env = Monitor(env)
# Eval
mean_reward, std_reward = evaluate_policy(model, model.get_env(), n_eval_episodes=10)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")
mean_reward=296.07 +/- 16.74673333348448
200 reward means pass!
!cp "/content/ppo-LunaLander-v2.zip" "/content/drive/MyDrive/Colab Notebooks/ppo-LunaLander-v2.zip"
Now let’s push this model to hub with huggingface_sb3.
from huggingface_hub import notebook_login
notebook_login()
VBox(children=(HTML(value='<center> <img\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…
trained_model = PPO.load("ppo-LunaLander-v2")
!ls -lrt
total 156
drwxr-xr-x 1 root root 4096 Jul 1 21:04 sample_data
drwx------ 5 root root 4096 Jul 3 13:55 drive
drwxr-xr-x 2 root root 4096 Jul 3 15:22 videos
-rw-r--r-- 1 root root 147362 Jul 3 15:27 ppo-LunaLander-v2.zip
# Let's use package_to_hub to build a model_card, render env and push to hub
repo_id = "JpChi/ppo-Lunar-Lander-v2"
env_id = "LunarLander-v2"
model_name="ppo-LunarLander-v2"
eval_env = Monitor(env)
model_architecture="PPO"
commit_message = "deep-rl-unit1-hands-on"
package_to_hub(
model=trained_model,
model_name=model_name,
model_architecture=model_architecture,
env_id=env_id,
eval_env=eval_env,
repo_id=repo_id,
commit_message=commit_message,
)
[38;5;4mℹ This function will save, evaluate, generate a video of your agent,
create a model card and push everything to the hub. It might take up to 1min.
This is a work in progress: if you encounter a bug, please open an issue.[0m
Saving video to /tmp/tmpf_7c9ozf/-step-0-to-step-1000.mp4
Moviepy - Building video /tmp/tmpf_7c9ozf/-step-0-to-step-1000.mp4.
Moviepy - Writing video /tmp/tmpf_7c9ozf/-step-0-to-step-1000.mp4
Moviepy - Done !
Moviepy - video ready /tmp/tmpf_7c9ozf/-step-0-to-step-1000.mp4
[38;5;4mℹ Pushing repo JpChi/ppo-Lunar-Lander-v2 to the Hugging Face Hub[0m
Upload 2 LFS files: 0%| | 0/2 [00:00<?, ?it/s]
ppo-LunarLander-v2.zip: 0%| | 0.00/148k [00:00<?, ?B/s]
replay.mp4: 0%| | 0.00/149k [00:00<?, ?B/s]
[38;5;4mℹ Your model is pushed to the Hub. You can view your model here:
https://huggingface.co/JpChi/ppo-Lunar-Lander-v2/tree/main/[0m
CommitInfo(commit_url='https://huggingface.co/JpChi/ppo-Lunar-Lander-v2/commit/285a5ef57ad7900feebbbe1da39be2d5432a8f36', commit_message='deep-rl-unit1-hands-on', commit_description='', oid='285a5ef57ad7900feebbbe1da39be2d5432a8f36', pr_url=None, repo_url=RepoUrl('https://huggingface.co/JpChi/ppo-Lunar-Lander-v2', endpoint='https://huggingface.co', repo_type='model', repo_id='JpChi/ppo-Lunar-Lander-v2'), pr_revision=None, pr_num=None)
Next, let’s try out a different model, log metrics, save video per epoch(with callback) and visualize them. Try out a differnt model perhaps.
Huggy
In this we’re gonna use Huggy environment from Unity-Technolgies/Ml-Agents. This toolkit enables to use games and simulations as environments to train Agents.
Here we’re gonna train Huggy(dog) with motor joints to fetch stick.
Environment state:
- Target(stick) position.
- Relative position between Huggy and Target.
- Orientation of legs.
Action: Moves huggy can do.
In RL we’ve to maximize the expected return(cumulative reward). Reward is to fetch the stick with Huggy.
We want Huggy to get in least possible time. Below are the rewards and penalties(i.e reward function):
- Orientation bonus: Reward to getting close to target.
- Time Penalty: Fixed time penalty to push Huggy to get to the target asap.
- Penalty for spinning at a single place or spinning too much: Rotation penalty.
- Target reward: Getting the stick.
- RL Loop: State_0(Target, Huggy-distance-target, legs orientation) –> Action_0(motor joints movement) –> Next_state_1(Target, Huggy-distance-target, legs orientation) –> Reward_0(above 4 points)
- Environment: Stick is spawned randomnly, when Huggy reaches it.
Let’s dive into code. Python version should match with ml-agents python version. python_requires=”>=3.10.1,<=3.10.12” - This is in ml-agents/setup.py.
!python --version
# Install virtualenv and create a virtual environment
!pip install virtualenv
!virtualenv myenv
# Download and install Miniconda
!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
!chmod +x Miniconda3-latest-Linux-x86_64.sh
!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local
# Activate Miniconda and install Python ver 3.10.12
!source /usr/local/bin/activate
!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here
# Set environment variables for Python and conda paths
!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/
!export CONDA_PREFIX=/usr/local/envs/myenv
!python --version
Now python version matches ml-agents python version. Let’s install the packages.
!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents
# Go inside the repository and install the package (can take 3min)
%cd ml-agents
!pip3 install -e ./ml-agents-envs
!pip3 install -e ./ml-agents
Environment is available in a zip file. Let’s download the env and place it in a directory.
!mkdir ./trained-env-execs
!mkdir ./trained-env-execs/linux
# Download
!wget "https://github.com/huggingface/Huggy/raw/main/Huggy.zip"
!mv "Huggy.zip" "./trained-env-execs/linux/Huggy.zip"
!unzip -d ./trained-env-execs/linux ./trained-env-execs/linux/Huggy.zip
# Check executable is accesible
!chmod -R 755 ./trained-env-execs/linux/Huggy
To train Agent with ml-agents framework, we’ve to create a yaml file in the respective algorithm directory. It has for algorithms:
- imitation
- poca
- ppo
- sac
We’re gonna train with PPO, let’s create ml-agents/config/ppo/Huggy.yaml with below parameters.
behaviors:
Huggy:
trainer_type: ppo
hyperparameters:
batch_size: 2048
buffer_size: 20480
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: linear
network_settings:
normalize: true
hidden_units: 512
num_layers: 3
vis_encode_type: simple
reward_signals:
extrinsic:
gamma: 0.995
strength: 1.0
checkpoint_interval: 200000
keep_checkpoints: 15
max_steps: 2e6
time_horizon: 1000
summary_freq: 50000
Now let’s train this agent. We’ve env, agent, parameters to train.
!mlagents-learn ./ml-agents/config/ppo/Huggy.yaml --env=./trained-env-execs/linux/Huggy/Huggy --run-id="Huggy1" --no-graphics
from huggingface_hub import notebook_login
notebook_login()
# Let's push this model to hub
!mlagents-push-to-hf --run-id="HuggyTrain" --local-dir="./results/Huggy1" --repo-id="JpChi/Huggy" --commit-message="Huggy RL"