Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix service log calls #86

Merged
merged 6 commits into from
Dec 15, 2020
Merged

Fix service log calls #86

merged 6 commits into from
Dec 15, 2020

Conversation

rosswhitfield
Copy link
Member

@rosswhitfield rosswhitfield commented Dec 11, 2020

Need to test that this doesn't have negative performance implications before merging.

This was originally broken by 98711ca

Update with Testing performance:

If I have 10 component running simultaneously, all logging to the service every 0.01 seconds, there is no performance difference. This is the most extreme case that you might find in practice.

for i in range(10000):
            time.sleep(0.01)                                                                                                                                                                  
            self.services.info(f"sleep_log {i}")

Only in the most extreme cases do I see a difference, so 10 component running simultaneously, all logging to the service as fast as they can

for i in range(10000):
            self.services.info(f"sleep_log {i}")

then the total run time goes from ~5 seconds on the current master to ~15 seconds on this branch. But the output log file is now correct, master log file was 1642 lines long while this branch is 100044 lines. Given that this is unlikely how it will be used I think it is safe to go ahead with this fix.

Update with testing performance on Cori (working on $HOME):

Previous test was on my local computer

Do this same test

If I have 10 component running simultaneously, all logging to the service every 0.01 seconds, there is no performance difference. This is the most extreme case that you might find in practice.

I see no difference in total execution time (from IPS_START to IPS_END) on Cori, if was 103.6 seconds without this fix and 103.5 seconds with it and it now has the correct logs in the file . So this is good to go.

Also ipsframework.ipsLogging.IPSLogSocketHandler was able to be replaced with logging.handlers.SocketHandler (Ref #34)

"Changed in version 3.4: If port is specified as None, a Unix domain socket is created using the value in host" which was what IPSLogSocketHandler was implemented to do.
handle() is called in the SocketServer.StreamRequestHandler once for each connection. If you return from handle the connection is closed.

To handle more than one send/recv, you must loop until recv() returns 0, indicating the client closed the connection (or at least called shutdown() on sends).

This follows the guide from https://docs.python.org/3.8/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network
This should stop the dependence on the order that tests run
@codecov-io
Copy link

codecov-io commented Dec 11, 2020

Codecov Report

Merging #86 (625aa56) into master (4b5627f) will increase coverage by 0.41%.
The diff coverage is 57.14%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
+ Coverage   51.15%   51.56%   +0.41%     
==========================================
  Files          27       27              
  Lines        5806     5773      -33     
==========================================
+ Hits         2970     2977       +7     
+ Misses       2836     2796      -40     
Impacted Files Coverage Δ
ipsframework/configurationManager.py 80.29% <ø> (+0.69%) ⬆️
ipsframework/ipsLogging.py 44.33% <0.00%> (-5.25%) ⬇️
ipsframework/services.py 34.69% <75.00%> (+0.58%) ⬆️
ipsframework/ips.py 82.96% <100.00%> (+5.43%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b5627f...625aa56. Read the comment docs.

@rosswhitfield rosswhitfield marked this pull request as ready for review December 15, 2020 15:05
@rosswhitfield rosswhitfield merged commit 714bc25 into master Dec 15, 2020
@rosswhitfield rosswhitfield deleted the logging branch December 15, 2020 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants