Feature/milestone2 (#112)

* initial steps to supporting html stdout

* initial steps for removing the logging package - use Operations.imprimir everywhere

* improve some printing info

* step 1 of 2 - removing the logging package

* actually step 2 of 3 - next step to add verbose overrides back in

* actually, step 3 of 4 - rototill plus pylint cleanup

* trying to add more type annotation

* more printing and shellout cleanup and debugging

* yet more verbosity debugging

* chaser

* while here, might as well clean up the contest key names

* more Global and contest keyword cleanup

* first commit to remove the Contests iterator class - and the crowd goes wild :-); this alters the format of the ElectionData config.yaml file syntax

* more minor debugging repair

* casting a ballot is beginning to work (again)

* yet more generic debugging - submitting contestCVR's

* more cleanup post ballot/contest object rototill

* an all-at-once refactoring; 1) made verbosity consistent throughout; 2) fixed shell_out verbosity handling; removed the class Common (no longer used); 3) create a single verbosity default

* updating poetry

* limping around the inability to get singleton design pattern to work

* chaser - cleaning up more (mostly printing) leftover's from previous rototilling

* the default max for RCV contests should be 1 as well

* cleaning up some more CLI printing

* switching max to max_selections; cleaning up RCV bugs regarding max_selections

* more RCV tally debugging

* initial working pass of html coloring for tally

* initial tally and verify html support

* minor tweak to keep in sync with VTP-web-client

* looks like for the time being, no html support needed in backend (?)

_tools/build/setuptools_pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "votetrackerplus"
-version = "0.1.0"
+version = "0.2.0"
 description = "VoteTracker+ - a distributed, open-source, public ballot and Cast Vote Record integrity and tracking system"
 authors = [
   { name = "Sandy Currier", email = "[email protected]" }

docs/E2EV.md

@@ -178,7 +178,7 @@ Running "git rev-parse --show-toplevel"
 Running "git pull"
 Already up to date.
 Running "git log --topo-order --no-merges --pretty=format:%H%B"
-Scanned 348 contests for contest (US president) uid=0000, tally=rcv, max=1, win-by>0.5
+Scanned 348 contests for contest (US president) uid=0000, tally=rcv, max=1, win_by>0.5
 RCV: round 0
 Total vote count: 348
 [('Mitt Romney', 74), ('Kamala Harris', 62), ('Cory Booker', 61), ('Phil Scott', 60), ("Beta O'rourke", 46), ('Ron DeSantis', 45)]
@@ -212,7 +212,7 @@ Running "git rev-parse --show-toplevel"
 Running "git pull"
 Already up to date.
 Running "git log --topo-order --no-merges --pretty=format:%H%B"
-Scanned 348 contests for contest (US president) uid=0000, tally=rcv, max=1, win-by>0.5
+Scanned 348 contests for contest (US president) uid=0000, tally=rcv, max=1, win_by>0.5
 RCV: round 0
 Counted 793fc652bfea0cc8590e2c618bdcaf8605db26e7: choice=Beta O'rourke
 Total vote count: 348
@@ -236,7 +236,7 @@ Contest US president (uid=0000):
   ('Phil Scott', 0)
   ("Beta O'rourke", 0)
   ('Ron DeSantis', 0)
-Scanned 347 contests for contest (US senate) uid=0001, tally=rcv, max=1, win-by>0.5
+Scanned 347 contests for contest (US senate) uid=0001, tally=rcv, max=1, win_by>0.5
 RCV: round 0
 Counted c09d2105936fe408379fcd7332fb88020f107a53: choice=Alexandria Ocasio-Cortez
 Total vote count: 347
@@ -251,20 +251,20 @@ Contest US senate (uid=0001):
   ('Larry Hogan', 167)
   ('Pramila Jayapal', 0)
   ('Greg Abbott', 0)
-Scanned 347 contests for contest (governor) uid=0002, tally=plurality, max=1, win-by>0.5
+Scanned 347 contests for contest (governor) uid=0002, tally=plurality, max=1, win_by>0.5
 Plurality - one round
 Counted dfacf455e5081208b16a5f19bfa5b62465e8558d: choice=Bernie Sanders
 Contest governor (uid=0002):
   ('Brian Kemp', 186)
   ('Bernie Sanders', 161)
-Scanned 348 contests for contest (County Clerk) uid=0003, tally=plurality, max=1, win-by>0.5
+Scanned 348 contests for contest (County Clerk) uid=0003, tally=plurality, max=1, win_by>0.5
 Plurality - one round
 Counted c70ddbeaf2a035d6354288b54d8ac9a05f499a11: choice=Huckleberry Finn
 Contest County Clerk (uid=0003):
   ('Huckleberry Finn', 128)
   ('Jean-Luc Picard', 119)
   ('Peggy Carter', 101)
-Scanned 347 contests for contest (mayor) uid=0005, tally=rcv, max=1, win-by>0.5
+Scanned 347 contests for contest (mayor) uid=0005, tally=rcv, max=1, win_by>0.5
 RCV: round 0
 Counted 5f56fd3cd03ddc88e7934f2b58a267570d889212: choice=Twenty Eight
 Total vote count: 347
@@ -279,13 +279,13 @@ Contest mayor (uid=0005):
   ('Twenty Eight', 163)
   ('Jane Doe', 0)
   ('Twenty Seven', 0)
-Scanned 348 contests for contest (Question 1 - school budget override) uid=0006, tally=plurality, max=1, win-by>0.5
+Scanned 348 contests for contest (Question 1 - school budget override) uid=0006, tally=plurality, max=1, win_by>0.5
 Plurality - one round
 Counted 1d66b23c07c6368833527ecc5d39c158ef6f4430: choice=no
 Contest Question 1 - school budget override (uid=0006):
   ('no', 187)
   ('yes', 161)
-Scanned 347 contests for contest (Question 2 - new firehouse land purchase) uid=0007, tally=plurality, max=1, win-by>2/3
+Scanned 347 contests for contest (Question 2 - new firehouse land purchase) uid=0007, tally=plurality, max=1, win_by>2/3
 Plurality - one round
 Counted 46b0e916db2da2acfda582cbb539beecbc3bf23d: choice=no
 Contest Question 2 - new firehouse land purchase (uid=0007):

src/vtp/README.md

@@ -303,7 +303,7 @@ Running "git rev-parse --show-toplevel"
 Running "git pull"
 Already up to date.
 Running "git log --topo-order --no-merges --pretty=format:%H%B"
-Scanned 303 contests for contest (U.S. Senate) uid=0001, tally=rcv, max=1, win-by>0.5
+Scanned 303 contests for contest (U.S. Senate) uid=0001, tally=rcv, max=1, win_by>0.5
 RCV: round 0
 Total vote count: 303
 [('Gloria Gamma', 65), ('Anthony Alpha', 53), ('David Delta', 47), ('Emily Echo', 47), ('Francis Foxtrot', 47), ('Betty Beta', 44)]

src/vtp/cli/_arguments.py

@@ -17,6 +17,9 @@
 
 """Argument handling."""
 
+# local imports
+from vtp.core.common import Globals
+
 
 class Arguments:
 
@@ -116,7 +119,7 @@ def add_printonly(parser):
         )
 
     @staticmethod
-    def add_verbosity(parser, verbosity=3):
+    def add_verbosity(parser, verbosity=Globals.get("DEFAULT_VERBOSITY")):
         """Add verbosity option"""
         parser.add_argument(
             "-v",
@@ -125,3 +128,17 @@ def add_verbosity(parser, verbosity=3):
             default=verbosity,
             help=f"0 critical, 1 error, 2 warning, 3 info, 4 debug (def={verbosity})",
         )
+
+    @staticmethod
+    def add_output_style(parser):
+        """Set the STDOUT text style"""
+        parser.add_argument(
+            "-o",
+            "--output_style",
+            type=str,
+            default="text",
+            help=(
+                "'text' is normal text; 'html' will decorate output with html "
+                "markers (def='text')"
+            ),
+        )

src/vtp/cli/create_blank_ballot.py

@@ -70,6 +70,7 @@ def main():
         substreet=parsed_args.substreet,
         town=parsed_args.town,
         state=parsed_args.state,
+        csv=parsed_args.csv,
     )
 
     # do it

src/vtp/cli/run_mock_election.py

@@ -136,6 +136,7 @@ def main():
         substreet=parsed_args.substreet,
         town=parsed_args.town,
         state=parsed_args.state,
+        csv=parsed_args.csv,
     )
 
     # do it

src/vtp/cli/tally_contests.py

@@ -68,6 +68,7 @@ def parse_arguments():
         default="",
         help="a comma separated list of contests checks to track",
     )
+    Arguments.add_output_style(parser)
     Arguments.add_verbosity(parser)
     parsed_args = parser.parse_args()
 
@@ -94,6 +95,7 @@ def main():
     # do it
     tco = TallyContestsOperation(
         election_data_dir=parsed_args.election_data_dir,
+        output_style=parsed_args.output_style,
         verbosity=parsed_args.verbosity,
         printonly=False,
     )

src/vtp/cli/verify_ballot_receipt.py

@@ -72,6 +72,7 @@ def parse_arguments():
         action="store_true",
         help="display the contents of the CVRs when specifying a row",
     )
+    Arguments.add_output_style(parser)
     Arguments.add_verbosity(parser)
 
     parsed_args = parser.parse_args()
@@ -92,6 +93,7 @@ def main():
     # do it
     vbro = VerifyBallotReceiptOperation(
         election_data_dir=parsed_args.election_data_dir,
+        output_style=parsed_args.output_style,
         verbosity=parsed_args.verbosity,
         printonly=False,
     )

src/vtp/core/address.py

@@ -40,18 +40,20 @@ class supports return the Address as either a string or a
     _keys = ["number", "street", "substreet", "town", "state", "country", "zipcode"]
 
     @staticmethod
-    def convert_address_to_num_street(address):
+    def convert_address_to_num_street(address: str):
         """Convert a street address string to number and street"""
         return re.split(r"\s+", address, 1)
 
     @staticmethod
-    def create_generic_address(config, subdir, ggos):
+    def create_generic_address(config, subdir: str, ggos: list):
         """Will create/return a generic address nominally from the list
         of ggos
         """
-        # Walk the address in DAG order from root to the prescribed leafs
+        # ZZZ strange pylint error on config:dict above
+        # import pdb; pdb.set_trace()
         nodes = []
         count = 0
+        # Walk the address in DAG order from root to the prescribed leafs
         for _ in Globals.get("REQUIRED_GGO_ADDRESS_FIELDS"):
             # Get the basename of the node via its subdir
             name = subdir.split(os.path.sep)[((count + 1) * 3) - 1]
@@ -64,16 +66,16 @@ def create_generic_address(config, subdir, ggos):
     # pylint: disable=too-many-arguments
     def __init__(
         self,
-        address="",
-        number="",
-        street="",
-        substreet="",
-        town="",
-        state="",
-        country="",
-        zipcode="",
-        csv="",
-        generic_address=False,
+        address: str = "",
+        number: str = "",
+        street: str = "",
+        substreet: str = "",
+        town: str = "",
+        state: str = "",
+        country: str = "",
+        zipcode: str = "",
+        csv: str = "",
+        generic_address: bool = False,
     ):
         """At the moment only support a few ways of creating an
         Address: a csv string and reasonable set of specified fields.
@@ -98,6 +100,7 @@ def __init__(
         self.ballot_node = ""
         self.ballot_subdir = ""
 
+        #        import pdb; pdb.set_trace()
         if not csv:
             if not generic_address and address:
                 (
@@ -144,7 +147,7 @@ def __str__(self):
                 nice_string += " " + self.address[key]
         return nice_string.strip()
 
-    def get(self, name):
+    def get(self, name: str):
         """A generic getter - will raise a NameError if name is not defined"""
         if name in Address._keys:
             return self.address[name]
@@ -160,10 +163,9 @@ def get(self, name):
             return self.ballot_subdir
         raise NameError(f"Name {name} not accepted/defined for get()")
 
-    def set(self, name, value):
+    def set(self, name: str, value: str = ""):
         """A generic setter - will raise a NameError if name is not defined"""
         if name in Address._keys:
-            value = "" if value is None else value
             self.address[name] = value
         else:
             raise NameError(f"Name {name} not accepted/defined for Address.set()")
@@ -192,7 +194,7 @@ def match(self, regex):
             )
         )
 
-    def map_ggos(self, config, skip_ggos=False, ggos=None):
+    def map_ggos(self, config: dict, skip_ggos: bool = False, ggos: list = None):
         """Will map an address onto the ElectionConfig data.  If
         skip_ggos is True, will completely skip setting the
         active_ggos field.  If an explicit list of GGOs is provided,