Provide a command line option to hc that allows the user to specify their own User-Agent string

[shawnmjones]

I've been reluctant to provide this functionality in case it might be misused, but I needed it today.

[ibnesayeed]

When people are running the application locally, they should be able to customize UA, which is actually preferred. It happened with us in the past when someone else's MemGator instance was causing extra load to an archive and we were contacted because they did not customize the UA. Now, MemGator logs a message when the server is run with default UA.

[shawnmjones]

I have not implemented this @ibnesayeed because I feel like Hypercane can be a potential resource hog on web archives themselves, especially for large numbers of mementos. I wrote a section about the importance of caching in the documentation (the docs may need an update). Even with caching and features like honoring the retry-after field, I fear that it is possible for Hypercane to abuse a web archive in the wrong hands, hence I have forced it to identify itself appropriately.

[ibnesayeed]

Thanks @shawnmjones for your description. I think it is not the responsibility of a tool to prevent people from intentionally abusing a system. Archives will have other ways to detect and prevent that, if they choose to do so. That said, even if we intend to enforce certain things in the UA string, it is still possible to allow some level of customization. In MemeGator, one can either overwrite the whole UA string or simply supply the extra information (e.g., contacts) to be included in the existing UA template. By enforcing a system-generated UA we limit the possibility of good players including information like the name or description of the project they are using the tool for and/or a contact information (e.g., email or Twitter handle) in case ops try to reach out to them.

[shawnmjones]

Thank you @ibnesayeed. That was the justification I needed. I will add this issue to the plan for the next release. 😀