2023-04-11-akhondzadeh23a.md

title	software	abstract	section	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Probing Graph Representations	https://github.com/msadegh97/probing-graph-representation	Today we have a good theoretical understanding of the representational power of Graph Neural Networks (GNNs). For example, their limitations have been characterized in relation to a hierarchy of Weisfeiler-Lehman (WL) isomorphism tests. However, we do not know what is encoded in the learned representations. This is our main question. We answer it using a probing framework to quantify the amount of meaningful information captured in graph representations. Our findings on molecular datasets show the potential of probing for understanding the inductive biases of graph-based models. We compare different families of models, and show that Graph Transformers capture more chemically relevant information compared to models based on message passing. We also study the effect of different design choices such as skip connections and virtual nodes. We advocate for probing as a useful diagnostic tool for evaluating and developing graph-based models.	Regular Papers	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	akhondzadeh23a	0	Probing Graph Representations	11630	11649	11630-11649	11630	false	Akhondzadeh, Mohammad Sadegh and Lingam, Vijay and Bojchevski, Aleksandar	given family Mohammad Sadegh Akhondzadeh given family Vijay Lingam given family Aleksandar Bojchevski	2023-04-11		Proceedings of The 26th International Conference on Artificial Intelligence and Statistics	206	inproceedings	date-parts 2023 4 11	https://proceedings.mlr.press/v206/akhondzadeh23a/akhondzadeh23a.pdf