Log facility

[rizlas]

Is it possible to have separate log files for each source or at least a facility that identifies the source in each log line?

Thank you

[bb-Ricardo]

Ahh, you mean which source causes this issue?

Is it due to the "None" name issue?

[rizlas]

Is not about that issue but in general. We are currently running 3 different sources and more to come. Using DEBUG2 its very difficult to navigate through logs. For sure a good idea will be to lower the verbosity but I think it will be more useful to have a faster way to differentiate the different sources.

For example:

2022-03-29 03:10:57,815 - NAME_OF_THE_SOURCE - ERROR: Unfortunately updated item site for object

With something like this we could also use a grok pattern to filter out many log lines.

[bb-Ricardo]

This seems feasible. Would add it if $num_of_sources > 1.

It would be great if you could open another issue. Should be easily done.

[bb-Ricardo]

Ok, just figured out this was a lie. It is quite difficult do add this as they all use the same logger instance.

I'm able to give sources different log instances but then this ends up like this:

2022-03-29 12:24:32,211 - vcsim - INFO: Query data from vCenter: 'vcsim.local'
2022-03-29 12:24:32,219 - vcsim - DEBUG: vCenter returned '1' datacenter
2022-03-29 12:24:32,221 - vcsim - DEBUG: Parsing vCenter datacenter: NIC-London
2022-03-29 12:24:32,221 - DEBUG2: Parsing 'cluster group' data structure: NIC-London
2022-03-29 12:24:32,221 - INFO: Created new cluster group object: NIC-London
2022-03-29 12:24:32,225 - vcsim - DEBUG: vCenter returned '2' clusters
2022-03-29 12:24:32,229 - vcsim - DEBUG: Parsing vCenter cluster: Cluster_Intel-E5_Infra
2022-03-29 12:24:32,230 - vcsim - DEBUG2: Trying to find site name for cluster 'Cluster_Intel-E5_Infra'
2022-03-29 12:24:32,230 - vcsim - DEBUG: No site relation for 'Cluster_Intel-E5_Infra' found, using default site 'vCenter: vcsim'
2022-03-29 12:24:32,230 - DEBUG2: Parsing 'cluster' data structure: Cluster_Intel-E5_Infra
2022-03-29 12:24:32,230 - DEBUG2: Parsing 'cluster type' data structure: VMware ESXi
2022-03-29 12:24:32,230 - INFO: Created new cluster type object: VMware ESXi
2022-03-29 12:24:32,230 - DEBUG2: Parsing 'cluster group' data structure: NIC-Londen
2022-03-29 12:24:32,230 - DEBUG2: Parsing 'site' data structure: vCenter: vcsim
2022-03-29 12:24:32,230 - INFO: Created new site object: vCenter: vcsim
2022-03-29 12:24:32,230 - INFO: Created new cluster object: Cluster_Intel-E5_Infra

What do you think? It seems more confusing to me.

I mean the source are scraped in serial. Once the first source is done, then the next source is queried.

[rizlas]

Yeah it is a bit confusing. In particular because some lines doesn't report the source and also filtering will be difficult.
Otherwise is it possible to set the log file name for each source? Maybe the problem is the same even with this idea.

I'm asking this because right now I have narrowed a problem to a specific source and i want to see only the log for that.

[bb-Ricardo]

and running the script and disabling the other sources?

enabled = False

[rizlas]

I did this and of course is a good workaround. Still would be nice to have separate files, if possible of course.

[bb-Ricardo]

It is quite difficult to realise without rewriting a lot of code and passing around additional objects.

[bb-Ricardo]

I currently investigating an issue where the site is not properly assigned to a vm and the interface, may this is related.

[rizlas]

Ok, i know that is difficult but imho is worth a try. Anyway thank you a lot!