Kezhi's Blog

本文主要对基于策略优化定理的强化学习算法及相关变体进行一个梳理和总结。

This article mainly reviews and summarizes reinforcement learning algorithms based on policy optimization theorems and related variants.

深度学习的实验通常都需要跑很久，如果没有主动推送的手段，那么就会出现以下两种情况：

• 过一段时间检查一下实验结果，无法专心干别的事情。
• 撇下不管，实验跑g了也不知道，跑完了也不知道。

为解决以上问题，就需要给实验程序加上监控进程，在实验结束后进行通知并发送实验结果。本文的脚本主要包含三部分内容：进程监控，tensorboard 数据提取，企业微信机器人通知。

Deep learning experiments usually take a long time to run. Without proactive notification, two situations may arise:

• Check the experiment results after a period of time, unable to focus on other things.
• Abandon the experiment, not knowing whether it is running or completed.

To solve the above problems, it is necessary to add a monitoring process to the experimental program, notify and send the experimental results after the experiment ends. This article's script mainly includes three parts: process monitoring, tensorboard data extraction, and enterprise WeChat robot notification.

本文主要进行了一元高斯分布熵的推导：

This article mainly derives the entropy of a univariate Gaussian distribution:

我过去经常帮身边的朋友指导电脑硬件配置。然而，对于个人电脑来说，如果胡乱装软件，或者用了不合适的软件，会让电脑的使用效率事倍功半。同时，在我个人拿到一台新电脑时，也需要根据我的习惯初始化配置很多软件到新电脑上。由此，便有了写一份软件配置清单的想法，一来可以给大家做一些软件推荐，二来我自己也留一份作为参考。

I used to help my friends around me with computer hardware configurations. However, for personal computers, if you install software randomly or use inappropriate software, it will greatly reduce the efficiency of computer usage. At the same time, when I get a new computer, I also need to initialize and configure many software according to my habits. Therefore, I had the idea of writing a software configuration checklist, which can not only provide software recommendations for everyone, but also serve as a reference for myself.

在使用公共服务器时，通常用户只有普通权限账户，无法按照自己的想法任意对服务器进行配置。为满足这种需求，常使用docker来实现在容器内使用root权限来自由配置环境。VNC是一种常用的远程桌面，可以通过端口通信实现服务器渲染画面的回传。这在进行一些有界面的实验（如mujoco，habitat, igibson环境）时非常有用。本文将主要介绍如何在docker中配置vnc桌面，并给出了一个打包好的镜像：kezhiadore/ubuntu-vnc - Docker Image | Docker Hub

When using public servers, users typically have regular user accounts with limited permissions, unable to configure the server freely according to their preferences. To meet this demand, Docker is often used to allow root privileges within containers for environment configuration. VNC, a commonly used remote desktop tool, facilitates server-side rendering of graphical interfaces, which is particularly useful for experiments requiring interfaces (such as mujoco, habitat, igibson environments). This article mainly discusses configuring VNC desktop within Docker and provides a pre-packaged image: kezhiadore/ubuntu-vnc - Docker Image | Docker Hub