Kezhi's Blog

深度强化学习算法相比于深度学习，有更多的超参数需要调节，本文主要对DDPG算法中的actor网络和critic网络学习率的设置对学习效果的影响进行了探讨。

Deep reinforcement learning algorithms have more hyperparameters to tune compared to deep learning. This article mainly discusses the impact of the learning rate settings for the actor and critic networks in the DDPG algorithm on the learning performance.

前两天，由4位轮值主席组织的第一届游在长安活动正式落幕。在活动组织过程中，有一个小游戏的分组问题很有一次，在此记录。

A few days ago, the first "Game in Chang'an" event organized by four rotating chairpersons officially came to an end. There was a grouping problem in a small game during the event, and I will record it here.

年纪越大，感到时光流逝的就越快。转眼间，就已经到了2023年。比我大一届的师兄师姐们就要毕业了。这篇博客主要放照片用，没有什么太多的文字内容。

祝各位毕业的师兄师姐前程似锦，生活精彩。🎉🎉🎉

As we grow older, we feel that time passes by faster. In the blink of an eye, it's already 2023. The seniors who are one year older than me are about to graduate. This blog mainly consists of photos and doesn't have much text content.

I wish all the graduating seniors a bright future and a wonderful life. 🎉🎉🎉

最近，清华大学和商汤发表了一篇名为《Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory》的文章，简称GITM。很有意思，感兴趣的朋友可以读一下原文。

Recently, Tsinghua University and SenseTime published an article titled "Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory," abbreviated as GITM. It's quite interesting, and interested friends can read the original article.

深度强化学习的流程可以抽象为以下步骤的重复：

1. 智能体与环境交互产生并存储经验
2. 智能体从经验中进行学习

本文主要探讨在收集经验过程中，环境自然结束（Terminated，包括目标成功，失败等）和人为截断（Truncated，主要为达到一定步数结束）对经验收集和训练产生的影响，以及如何对其进行处理。并对其进行了部分实验来比较性能。

The process of deep reinforcement learning can be abstracted into the following steps:

1. The agent interacts with the environment and generates and stores experiences.
2. The agent learns from the experiences.

This article mainly discusses the impact of natural termination (Terminated), including successful or failed goals, and artificial truncation (Truncated), mainly ending after a certain number of steps, on experience collection and training. It also conducts some experiments to compare performance.