Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use Channel.interval() factory instead of watchPath #1009

Open
anoronh4 opened this issue Apr 30, 2024 · 1 comment
Open

use Channel.interval() factory instead of watchPath #1009

anoronh4 opened this issue Apr 30, 2024 · 1 comment
Labels
backburner probably won't address in a near future

Comments

@anoronh4
Copy link
Collaborator

anoronh4 commented Apr 30, 2024

See documentation here:
https://www.nextflow.io/docs/edge/channel.html#interval

Currently we're using a workaround to use the watchPath function in dsl2. this workaround still has limitations, as it requires us to touch -ca the file just to read it. this also has limitations, for example if we do not own or have permission to touch -ca.

This new factory should be able to replace the workaround. We should basically be able to set an interval and with each emission read an input file. This feature only seems to be available in the edge version right now.

for further context about why we needed a workaround in the first place, watchPath doesn't work by itself because it uses inotify, which requires the target of watchPath to be modified on the same machine as where nextflow is running. so if something is modified in one node of a cluster, and watchPath is running in another node, nothing will happen.

@anoronh4 anoronh4 added the backburner probably won't address in a near future label Apr 30, 2024
@anoronh4
Copy link
Collaborator Author

anoronh4 commented May 29, 2024

looks like watchPath doesn't work with touch -ca in the new version of nextflow anyways. the following doesn't trigger watchPath:

new Timer().schedule({
def cmd = "touch -ca hi" 
cmd.execute()
} as TimerTask, 1000, 5000 )
Channel.watchPath(file("hi"),'create,modify').view()

it works as late as 22.04.5 but stops working as early as 22.10.3. it also doesn't work with 24.03.0-edge, which is pretty recent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backburner probably won't address in a near future
Projects
None yet
Development

No branches or pull requests

1 participant