Watching more than expected.

[rquadling]

Hi.

I'm using grunt-contrib-watch which is using gaze v1.1.1.

My Gruntfile.js has a watch section in the config of:

        watch: {
            // build: {
            //     files: [
            //         'src/**/*.*',
            //         'third_party/**/*.*',
            //         'vendor/**/*.*'
            //     ],
            //     tasks: [
            //         'build',
            //         'notify:build_complete'
            //     ]
            // },
            configFiles: {
                cwd: '',
                files: [
                    'bower.json',
                    'package.json',
                    'Gruntfile.js'
                ],
                options: {
                    reload: true
                },
                tasks: [
                    'notify:watch_grunt'
                ]
            }
        }

If I add some simple debugging to grunt-contrib-watch/tasks/watch.js, just before the call to Gaze();

      grunt.log.writeflags(patterns);
      grunt.log.writeflags(target.options);

I get output of:

Flags: bower.json, package.json, Gruntfile.js
Flags: reload, target=null, cwd="C:\\wamp\\www\\epos", cliArgs=["-v"], interrupt=false, nospawn=false, spawn, atBegin=false, event=["all"]

which is exactly right for when I'm watching the config files.

The output I get next (provided by using -v option) is:

Watching bower.json for changes.
Watching .git for changes.
Watching .idea for changes.
Watching config for changes.
Watching logs for changes.
Watching node_modules for changes.
Watching public for changes.
Watching scripts for changes.
Watching src for changes.
Watching temp for changes.
Watching test for changes.
Watching third_party for changes.
Watching vendor for changes.
Watching package.json for changes.
Watching Gruntfile.js for changes.

But I can't get beyond this debugging as I don't know how to output content without grunt! If someone can point me to some easy debugging tools/info, I'll be happy to track this bug down.

If you were watching, you'll notice the commented out build section. If I use both sections, I still get the spurious entries. If I use ONLY the build section, all is well, leading me to think that there something wrong when only files are being watched.

I tried excluding directories, using relative paths, etc. all to no avail.

            configFiles: {
                cwd: '',
                files: [
                    '!*.*',
                    '!**/*.*',
                    '!**/',
                    './bower.json',
                    './package.json',
                    './Gruntfile.js'
                ],
                options: {
                    reload: true
                },
                tasks: [
                    'notify:watch_grunt'
                ]
            }
``

[FredLackeyOfficial]

Did this ever get solved?

[rquadling]

Sorry I haven't.

[ghost]

@rquadling @SkydiverFL This issue sticks very deep, but is located to this function here: https://github.com/shama/gaze/blob/master/lib/gaze.js#L272

And this line of code fucking the whole thing: https://github.com/shama/gaze/blob/master/lib/gaze.js#L286

If you do a console.log on 287

console.log(this._watched)

You will see the files mentioned in this issue, If you do it on the line above, everything is just fine.

So this line of code trigger this

helper.objectPush(this._watched, path.dirname(filepath) + path.sep, filepath);

// console log this too: path.dirname(filepath) + path.sep

Issue here is that the repo owner doesn't seem to fix this.

@shama Wont to fix this?

[shama]

This is not really a bug IMO. It watches certain folders that could potentially have matching files added to it. It just doesn't hide those files when reporting what files it is watching.

Also just because it says it's watching the node_modules folder doesn't mean it's watching every file within that folder... just that folder.

[ghost]

@shama In my case it was saying it was watching node_modules, and 23k files inside that folder. I know so because 23k was added when I started the watcher. And this is similar to what @rquadling was observing.

This is not happening when you add files the first time, only if you change the content of one file. Then I got 23k files extra to watch.

[rquadling]

What is odd is that I can watch specific files in a subdirectory and only those files are watched. Watch files in the local root and ALL files in the local root are watched.

[ghost]

@rquadling @shama I investigated a little more on this, and I found something interesting.

In my root folder. Say I just have a test.js file that I want to watch. I adding it like this:

gaze("test.js", function(err, watcher) {}

When I check what file(s) it's watching. In this case this files added to the watcher are:

• 3 different files
• 8 different folders

Note I only requested to watch test.js file.

Just to check if it's only watching one single file. I changed the content in a file inside on of the sub folders, and I'm now seeing this message in the console:

...was changed

Another strange things is there is no differene between star () and the globstar (*). A globstar adds sub dirs and files as well.

Example

/home/** would match /home/gus/data, but /home/* wouldn't.

In your module both * and * are equalent. See Glob wildcards regarding this and how it should work.

I also couldn't find any options for

• dot files
• hidden files (same as dot files on some file systems)
• file permissons
• hidden folders
• dot folders

so maybe it adds hidden and dot files as well? And this took me to another issue, out of topics of this issue ticket. E.g. in native fs.watch you have a persistent option. It's not included with gaze?

I didn't test this too much, but just scratched the surface of issues I found just now.

Update It does add dot files and dot directories by default.

 [ '.git\\',

This got auto-watched when gaze was detection changes when I was watching the test.js file mentioned above. Most of the time it only seem to list all folders and sub-folders and put their name in an array without watching the files inside it.

Making such huge array for nothing take a lot of resources, and why is it needed? What if I have 10k folders and watching one file in the root? The size of the array would be larger then what it can handle. I would either get a error telling me I'm running out of memory, or everything will slow down before it stops working.

[ghost]

@rquadling You are right about that. And it will not even trigger events. Do this to reproduce:

• create a folder in root named foo
• create a new folder inside foo and name it bar
• add a empty a.js file into foo folder.

Try to change content of the a.js file. No events are triggered!

You can also rename folder bar to bas, and add a b.js file inside it. No events triggered either!

@shama Is this going to be fixed?

update!
This seems to be related to GLOB pattern also. This module is not following the GLOB specs 100% it seems

// triggers nothing
gaze('foo', handler) 

// trigger only current folder and it sub files and folders
gaze('foo/*, handler)' 

// adding files from root, and the folder. No files and sub folder inside foo added
gaze('foo/, handler)' 

 // trigger current folder and it sub files and folders and you get a lot of files and folder from root too
gaze('foo/**, handler)'

[shama]

Feel free to dive into the issue history of this repo. A lot being investigated here has already been discussed in other issues. Sorry I don't have the energy to link it for you.

@shama Is this going to be fixed?

Feel free to send a PR and I'll take a look. Thanks!

[ghost]

@shama I have looked at most of the things already, and I just mention this to point out why things isn't worked as expected.

I also found that your rewrite from 0.6 where you rolled back to 0.5, you skipped an important thing - check for absolute paths.
27d3705#diff-5944dbe70e8c394d3975b1ad554261e2R20

I know this issue ticket regarding this matter: #203 But I think globule could handle it if you passed the option object.

No idea if this is related to this issue ticket, but worth mention. Chokidar e.g. checking for absolute paths.
But more I was looking into this, it also seems to be related to the globule module.

.dot files and this missing things could be passed as an option to the globule module itself, but doesn't seem to do so.

And just now I don't have time to anything. I just tried to figure out what all the issues was about. And there are too many issues with this module as I found, so I got confused.

Shame indeed. I really prefer this module over chokidar. I like your API and the way this is build.

@shama I could maybe have a done a PR for some of the issues, but when I noticed you have unsolved PR from 2 years back (soon 3 years), and also a pending one from august this year, I don't know.

[ghost]

@rquadling @shama I can confirm most of this issues is related to the globule module, and not gaze itself.
I wrote my own module just now - similiar to globule - and I don't have this issues anymore.
Also absolute paths that people reported before is solved. Plus other "issues" got solved.

But there are tons of other issues with this module, so better I develop my own and move on.

[shama]

@crazyhuggins Sounds good! Thanks and best of luck!