2023-04-11-chen23b.md

title	abstract	section	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Byzantine-Robust Online and Offline Distributed Reinforcement Learning	We consider a distributed reinforcement learning setting where multiple agents separately explore the environment and communicate their experiences through a central server. However, $\alpha$-fraction of agents are adversarial and can report arbitrary fake information. Critically, these adversarial agents can collude and their fake data can be of any sizes. We desire to robustly identify a near-optimal policy for the underlying Markov decision process in the presence of these adversarial agents. Our main technical contribution is COW, a novel algorithm for the robust mean estimation from batches problem, that can handle arbitrary batch sizes. Building upon this new estimator, in the offline setting, we design a Byzantine-robust distributed pessimistic value iteration algorithm; in the online setting, we design a Byzantine-robust distributed optimistic value iteration algorithm. Both algorithms obtain near-optimal sample complexities and achieve superior robustness guarantee than prior works.	Regular Papers	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	chen23b	0	Byzantine-Robust Online and Offline Distributed Reinforcement Learning	3230	3269	3230-3269	3230	false	Chen, Yiding and Zhang, Xuezhou and Zhang, Kaiqing and Wang, Mengdi and Zhu, Xiaojin	given family Yiding Chen given family Xuezhou Zhang given family Kaiqing Zhang given family Mengdi Wang given family Xiaojin Zhu	2023-04-11		Proceedings of The 26th International Conference on Artificial Intelligence and Statistics	206	inproceedings	date-parts 2023 4 11	https://proceedings.mlr.press/v206/chen23b/chen23b.pdf