-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tokenizer -x option is confusing #98
Comments
Hmmm true. But the point is to keep the interface pythonic, but I agree it's confusing. Let me think of a better wording for the feature =) |
What about something like
or just removing the shortened form I think it should at least have the "negation" on the help message because it is very confusing. |
Agreed, the option name and help text definitely do not make sense. But then, does the default behaviour need to be that special XML characters are escaped (legacy behaviour from SMT/Moses)? I totally understand if the argument is that sacremoses should behave exactly like the original Moses tokenizer. |
The
-x
option says on the usage:And it does the same as
-no-escape
option in Moses.The text was updated successfully, but these errors were encountered: