Query of riemann streams rate & instrumentation

[vipinvkmenon]

Can this metric be used to identify the stream rate for individual "events" for eg... I have an event of service ab.c.cpu and I would like to know what rate is this incoming into the system. Or for this, I would need to use something (where (service ab.c.cpu) rate). Also on a side note, is it possible to have multiple streams that can be tagged and monitored with different riemann streams rate.

if I were to have a riemann.config file as follows:

(instrumentation {:enabled? true})
(streams (where (service #"^a\.*") prn) ; stream a
(streams (where (service #"^b\.*") prn) ; stream b

How is this different from:

(streams 
 (where (where (service #"^a\.*") prn)
(where (where (service #"^b\.*") prn)
)

Are they the same since in both cases they are on the same Core (and same index?)? Could we have an example for running them in different cores? Also, what is the impact of reinject (I know this is not recommended)?. My understanding is that it would be sent back to the top-level streams. which means the event would be back in both scenarios.
Would this create 2 different top-level streams that can be individually instrumented? Is it possible to get to know the riemann streams rate individually? We have a lot of events coming in from multiple sources and occasionally we have spikes and we would like to know the source. I.E. which event is creating this?

Also, there is another metric that intrigued me: riemann index size. This I believe comes from the index.clj instrumentation.
But why it's interesting for me is the size. on our landscapes, it shows a size of 6-10 depending on load. And 0 in another riemann VM. What makes the size of the index change? index looks like its created from [:host :service]. (Meaning a concat of the host&service string?). then shouldn't the index size be large considering all the services?

Thanks for your help and support

[faxm0dem]

IIRC (streams (where...))(streams (where...)) is equivalent to (streams (where...)(where...)).

The riemann index size is simply the number of non-expired events in the index.
From the documentation:

The index is a table of the current state of all services tracked by Riemann. Each event is uniquely indexed by its host and service. The index just keeps track of the most recent event for a given (host, service) pair.

So you're correct in your assuption that the tuple [:host :service] has a big influence on the index size, as they are the primary keys: you can index many events with the same [:host :service] and the index size will increase at most by 1.