Metrics gathering blocks if connection is timing out

[pryorda]

From @ipsecguy on September 5, 2017 11:31

Hi,

I have multiple ESXi Hosts im am scraping. Some of them are switched off most of the time.

It seems that if the vmware_exporter is in the process trying to connect to a currently switched off system it cannot process a second connection.

This can easily be reproduced using a web browser by connecting to a currenlty "off" target and opening a second tab trying to connect to an existent machine. Both fail.

Can the exporter process a single request only or is the problem mybe in the python vware library?

Andreas

Copied from original issue: rverchere/vmware_exporter#15

[pryorda]

From @rverchere on September 7, 2017 10:34

Hi,

Yes for now it handles only 1 request at a time.

I do not have availability yet to make it handle many connections, help is welcome ;)

[pryorda]

From @rverchere on September 10, 2017 12:20

Hi @ipsecguy,

coud you try the threads branch to check concurrent connections?

Thanks

[pryorda]

From @ipsecguy on September 10, 2017 13:4

Hi, just did. Seems to work in general (I am able to get metrics from the running ESXi machines while the exporter is trying to connect for one of the switched off machines).

What I have observed (Might or might not be related to the change) if I do a scrape on a running ESXI (in a browser) and at almost the same time to another (also running) I get most of the time a websever such as (not always the same object = '' part):

pyVmomi.VmomiSupport.vim.fault.NotAuthenticated: (vim.fault.NotAuthenticated) { dynamicType = , dynamicProperty = (vmodl.DynamicProperty) [], msg = '', faultCause = , faultMessage = (vmodl.LocalizableMessage) [], object = 'vim.VirtualMachine:16', privilegeId = 'System.View' }

Retrying the scrape then works. Might or might not be a race condition.

Cool stuff with the threading, I was looking a little but I soon realized my python/twisted skills are too poor.
Andreas

[pryorda]

From @rverchere on September 10, 2017 14:2

Hi,

thanks for your feedback. I have the same issue with the NotAuthenticated error, I guess it's a thread issue, but I'm not confortable enough - yet - with twisted and threads. I'll keep you in touch.

[pryorda]

From @britcey on February 16, 2018 20:54

I ran into the same issue with the lack of concurrent requests; are you committed to Twisted?

Moving it over to BaseHTTPServer was straight-forward and seems to work well:

https://github.com/britcey/vmware_exporter/blob/basic_webserver/vmware_exporter/vmware_exporter.py

I'll open a PR if you're willing to make this change.

[pryorda]

From @rverchere on February 19, 2018 20:3

@britcey great if BaseHTTPServer does the correct job.

I'll take a look when I'll have some free time!

[pryorda]

@ipsecguy is this an issue with the data not being split? Also, can you try with the new configuration set up?

[pryorda]

Closing because of inactivity.