Metrics gathering blocks if connection is timing out

[ipsecguy]

Hi,

I have multiple ESXi Hosts im am scraping. Some of them are switched off most of the time.

It seems that if the vmware_exporter is in the process trying to connect to a currently switched off system it cannot process a second connection.

This can easily be reproduced using a web browser by connecting to a currenlty "off" target and opening a second tab trying to connect to an existent machine. Both fail.

Can the exporter process a single request only or is the problem mybe in the python vware library?

Andreas

[rverchere]

Hi,

Yes for now it handles only 1 request at a time.

I do not have availability yet to make it handle many connections, help is welcome ;)

[rverchere]

Hi @ipsecguy,

coud you try the threads branch to check concurrent connections?

Thanks

[ipsecguy]

Hi, just did. Seems to work in general (I am able to get metrics from the running ESXi machines while the exporter is trying to connect for one of the switched off machines).

What I have observed (Might or might not be related to the change) if I do a scrape on a running ESXI (in a browser) and at almost the same time to another (also running) I get most of the time a websever such as (not always the same object = '' part):

pyVmomi.VmomiSupport.vim.fault.NotAuthenticated: (vim.fault.NotAuthenticated) { dynamicType = , dynamicProperty = (vmodl.DynamicProperty) [], msg = '', faultCause = , faultMessage = (vmodl.LocalizableMessage) [], object = 'vim.VirtualMachine:16', privilegeId = 'System.View' }

Retrying the scrape then works. Might or might not be a race condition.

Cool stuff with the threading, I was looking a little but I soon realized my python/twisted skills are too poor.
Andreas

[rverchere]

Hi,

thanks for your feedback. I have the same issue with the NotAuthenticated error, I guess it's a thread issue, but I'm not confortable enough - yet - with twisted and threads. I'll keep you in touch.

[britcey]

I ran into the same issue with the lack of concurrent requests; are you committed to Twisted?

Moving it over to BaseHTTPServer was straight-forward and seems to work well:

https://github.com/britcey/vmware_exporter/blob/basic_webserver/vmware_exporter/vmware_exporter.py

I'll open a PR if you're willing to make this change.

[rverchere]

@britcey great if BaseHTTPServer does the correct job.

I'll take a look when I'll have some free time!

[pryorda]

This issue was moved to pryorda/vmware_exporter#10