future state is "running" after it has been resolved

[keithing]

On my machine (Ubuntu 14.04) it's possible for a future to be resolved yet have a state of "running". I'm not sure if this is intentional or not, since it seems possible to know whether a future is "finished" or "failed" before value is called, but maybe that's not the future design philosophy.

Here's a reproducible example:

library(future)                                                                 
                                                                                
# Run futures in a different process                                            
plan("multiprocess", workers=2)                                                 
                                                                                
# Create a future to sleep for 4 seconds                                        
fut <- future({                                                                 
  Sys.sleep(4)                                                                  
  2 + 2                                                                         
})                                                                              
                                                                                
# Test that I can do stuff while I'm waiting for future                         
print("Parallel!")                                                              
                                                                                
# Wait for future to be resolved                                                
while (!resolved(fut)) {                                                        
  Sys.sleep(1)                                                                  
}                                                                               
                                                                                
# This should be in a completed state like "finished" or "failed"               
print(fut$state)

[keithing]

FWIW, withplan("sequential"), behavior is different and the final line prints "finished" instead of "running".

[HenrikBengtsson]

Yes, the state only becomes "finished" when the value has been collected internally, e.g.

[...]
> print(fut$state)
[1] "running"

> value(fut)
[1] 4

> fut$state
[1] "finished"

Are you looking into this to understand what's happening under the hood or are you building something that relies on it? Please note that state and many other fields are "internal" fields not part of the official API, so be careful as they might change. Let me know if you needs something added to the official API that you can otherwise only get from the state field.

[keithing]

Awesome. I just came across as I was looking into handling errors that happen in a future. I didn't realize it was an internal field and it just surprised me. I'm not relying on it.

Let me know if you needs something added to the official API that you can otherwise only get from the state field.

Thanks for asking. Between resolved and wrapping value(fut) in a tryCatch I'm able to handle errors without needing any additions to the API.

[wlandau]

Naive question: is it safe to directly reference fut$state directly, or do you recommend a safer user-side API function similar to value()? I am planning a manual job scheduler for drake that will eventually replace the existing stage-based parallelism, and I am interested in monitoring jobs using future alone. It would be nice to not have to use drake's cache to monitor workers.

[wlandau]

Never mind: I should have requested state() in #172.