doit.py, harness.py

[zachriggle]

The general model here is that doit.py is an exploit that gives a shell or reads a flag file, and harness.py verifies that.

First, the naming isn't obvious for anyone who's never used pwntools before and is casually browsing the repository. I'd suggest exploit.py instead of doit.py and test.py or test_harness.py instead of harness.py

Second, the harness.py using SILENT to doit.py. This makes it less useful for actual automated testing purposes, e.g. with travis-ci. Logging should be able to get cranked up all the way to DEBUG and still work. The issue is we need to see what's wrong, in the event that an exploit works locally but not on travis-ci.

One method of being able to verify that the flag was successfully retrieved, instead of scraping the exploit's output, may be to have the exploit check for a SAVEFLAG argument. The exploit would then write the flag to the specified file. For example, python exploit.py SAVEFLAG=foo. Then, the contents of foo and the real flag.txt can be verified for success (foo standing in for a temporary file path).

Instead of (or in addition to) echoing ok or not ok, what is currently called harness.py should use standard exit codes (0 for success, nonzero for failure).