Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play
Por um escritor misterioso
Last updated 16 junho 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
Electronics, Free Full-Text
Win rate of QPlayer vs Random in Tic-Tac-Toe on different board
The future is here – AlphaZero learns chess
The future is here – AlphaZero learns chess
AlphaZero, a novel Reinforcement Learning Algorithm, in JavaScript
AlphaZero, a novel Reinforcement Learning Algorithm, in JavaScript
Train on Small, Play the Large: Scaling Up Board Games with
A general reinforcement learning algorithm that masters chess
Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect
Value targets in off-policy AlphaZero: a new greedy backup
Reinforcement learning is all you need, for next generation
AlphaGo Zero – How and Why it Works – Tim Wheeler
AlphaDDA: strategies for adjusting the playing strength of a fully
AlphaZero from scratch in PyTorch for the game of Chain Reaction
Acquisition of Chess Knowledge in AlphaZero – arXiv Vanity
Recomendado para você
-
AlphaZero Vs StockFish – A Literature Review.pptx16 junho 2024
-
Chess's New Best Player Is A Fearless, Swashbuckling Algorithm16 junho 2024
-
AlphaGo Zero Explained In One Diagram, by David Foster, Applied Data Science16 junho 2024
-
Has the Alpha Zero chess program been made to play the Evans Gambit against itself, in an attempt to discover whether that gambit, with best play, is theoretically sound or whether White16 junho 2024
-
DeepMind AlphaZero lernt übergreifend Spiele zu spielen16 junho 2024
-
Simple Alpha Zero16 junho 2024
-
GitHub - timvvvht/AlphaZero-Connect4: An asynchronous implementation of AlphaZero, a self-play reinforcement learning algorithm.16 junho 2024
-
PDF] Acquisition of chess knowledge in AlphaZero16 junho 2024
-
AI Summary: Finding Increasingly Large Extremal Graphs with AlphaZero and Tabu Search16 junho 2024
-
How AlphaZero Works – Augmented Lawyer16 junho 2024
você pode gostar
-
Umidade na sala de jogos: como resolver. - Thermomatic16 junho 2024
-
death note filme completo dublado em hd16 junho 2024
-
Anime Cyberpunk Edgerunners David Cosplay Necklace David Martinez Jewelry16 junho 2024
-
Seven Knights Idle Adventure adds new heroes and challenges in Halloween 2023 update16 junho 2024
-
Expansão Detective: Signature Series - Petty Officers16 junho 2024
-
The Raven Keeper, God of War Wiki16 junho 2024
-
Chess Courses Online - For All Levels - Chessable16 junho 2024
-
The Last of Us Part II Ellie Fake Tattoo - Kuru Store16 junho 2024
-
ice #hiehienomi #hiehie #onepiece #onepiecedevilfruit #onepiecedevil16 junho 2024
-
Animation - Grisaia no Kajitsu - Meikyu - Rakuen Blu-ray Box16 junho 2024