Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to exclude huge files #177

Open
2 tasks
pyhedgehog opened this issue Dec 16, 2024 · 1 comment
Open
2 tasks

Add option to exclude huge files #177

pyhedgehog opened this issue Dec 16, 2024 · 1 comment

Comments

@pyhedgehog
Copy link
Contributor

Background

Sometimes repositories contains source files too big to be parsed by pygount (I've stuck with 40Mb text file that seems to be source for ML-model). I propose simplest solution — option exclude huge files from processing.

Goals

  • When option --larger-to-skip is passed, every file that is larger than specified size is skipped from parsing.
  • Support parsing size specifiers like 10k, 20m and so on. Maybe borrow code from test.support._parse_memlimit()...
@roskakori
Copy link
Owner

Yes, this makes sense.

Not a high priority, but I will probably work on pygount during the coming holidays, so we will see.

Actually, in analysis.py you can already find this in SourceState:

    generated = 6
    # TODO: 'huge' = auto()  # source code exceeds size limit
    #: pygments does not offer any lexer to analyze the source
    unknown = 7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 Backlog
Development

No branches or pull requests

2 participants