Skip to content

Commit

Permalink
removing forgotten usage of isdigit
Browse files Browse the repository at this point in the history
appending values of passed digits for later use in code and removing
forgotten usage of isdigit
  • Loading branch information
elliewix committed Sep 7, 2016
1 parent 21ed38c commit 5a57ecd
Show file tree
Hide file tree
Showing 1,004 changed files with 64,079 additions and 64,076 deletions.
15 changes: 9 additions & 6 deletions data_profile.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,24 +66,27 @@ def review_csv(file, mode='rt', headers=True, index_row=True, missing=''):
info['missing'] = data[i].count(missing)
info['percent_missing'] = "{:.0%}".format(info['missing'] / len(data[i]))
# digits = len([d for d in data[i] if d.isdigit()])
dcount = 0
# dcount = 0
passed_digits = []
for d in data[i]:
try:
float(d)
dcount += 1
# dcount += 1
digits.append(d)
except:
pass # hahaha i'll pay for this
digits = dcount
digits = len(passed_digits)
totalvalues = len([d for d in data[i] if len(d) > 0])

if totalvalues == 0:
info['percent_digit'] = "no digits"
else:
info['percent_digit'] = "{:.0%}".format(digits / totalvalues)

if digits > 0:
digit_values = [float(d) for d in data[i] if d.isdigit()]
info['min_digit'] = min(digit_values)
info['max_digit'] = max(digit_values)
# digit_values = [float(d) for d in data[i] if d.isdigit()]
info['min_digit'] = min(passed_digits)
info['max_digit'] = max(passed_digits)
else:
info['min_digit'] = "no digits"
info['max_digit'] = "no digits"
Expand Down
62 changes: 31 additions & 31 deletions fakes/0_DataProfile.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Data Profile for fakedata/0.csv

Generated on: 2016-Aug-27 16:21:10
Generated on: 2016-Sep-07 14:05:48


Number of columns: 10
Expand All @@ -14,13 +14,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 1.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 103 (this includes missing values)
* max_digit: 996.0
* max_digit: no digits

**lhnpgel**
---------
Expand All @@ -29,13 +29,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 3.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 100 (this includes missing values)
* max_digit: 997.0
* max_digit: no digits

**dejgqsnl**
----------
Expand All @@ -44,13 +44,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 7.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 104 (this includes missing values)
* max_digit: 978.0
* max_digit: no digits

**ttzsbzzztt**
------------
Expand All @@ -59,13 +59,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 11.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 103 (this includes missing values)
* max_digit: 992.0
* max_digit: no digits

**fswewrmgbl**
------------
Expand All @@ -74,13 +74,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 0.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 103 (this includes missing values)
* max_digit: 994.0
* max_digit: no digits

**lqwtwpo**
---------
Expand All @@ -89,13 +89,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 8.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 107 (this includes missing values)
* max_digit: 991.0
* max_digit: no digits

**wnlat**
-------
Expand All @@ -104,13 +104,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 0.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 106 (this includes missing values)
* max_digit: 993.0
* max_digit: no digits

**jfgmzi**
--------
Expand All @@ -119,13 +119,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 3.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 98 (this includes missing values)
* max_digit: 981.0
* max_digit: no digits

**tqpvwxqsvk**
------------
Expand All @@ -134,13 +134,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 21.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 101 (this includes missing values)
* max_digit: 973.0
* max_digit: no digits

**kozisucyqc**
------------
Expand All @@ -149,11 +149,11 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 18.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 100 (this includes missing values)
* max_digit: 988.0
* max_digit: no digits

62 changes: 31 additions & 31 deletions fakes/100_DataProfile.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Data Profile for fakedata/100.csv

Generated on: 2016-Aug-27 16:21:10
Generated on: 2016-Sep-07 14:05:48


Number of columns: 10
Expand All @@ -14,13 +14,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 2.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 137 (this includes missing values)
* max_digit: 996.0
* max_digit: no digits

**lhnpgel**
---------
Expand All @@ -29,13 +29,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 1.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 141 (this includes missing values)
* max_digit: 989.0
* max_digit: no digits

**dejgqsnl**
----------
Expand All @@ -44,13 +44,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 10.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 141 (this includes missing values)
* max_digit: 988.0
* max_digit: no digits

**ttzsbzzztt**
------------
Expand All @@ -59,13 +59,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 28.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 140 (this includes missing values)
* max_digit: 995.0
* max_digit: no digits

**fswewrmgbl**
------------
Expand All @@ -74,13 +74,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 1.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 143 (this includes missing values)
* max_digit: 995.0
* max_digit: no digits

**lqwtwpo**
---------
Expand All @@ -89,13 +89,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 2.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 141 (this includes missing values)
* max_digit: 997.0
* max_digit: no digits

**wnlat**
-------
Expand All @@ -104,13 +104,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 1.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 143 (this includes missing values)
* max_digit: 980.0
* max_digit: no digits

**jfgmzi**
--------
Expand All @@ -119,13 +119,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 1.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 138 (this includes missing values)
* max_digit: 987.0
* max_digit: no digits

**tqpvwxqsvk**
------------
Expand All @@ -134,13 +134,13 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 3.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 142 (this includes missing values)
* max_digit: 995.0
* max_digit: no digits

**kozisucyqc**
------------
Expand All @@ -149,11 +149,11 @@ Using missing value of: (empty string)
* Description of data values and units:
* Reason for missing values:

* percent_digit: 100%
* percent_digit: 0%
* percent_missing: 0%
* min_digit: 22.0
* min_digit: no digits
* missing: 0
* unique_value_content: Not reported (More than 10 unique values)
* unique_values: 137 (this includes missing values)
* max_digit: 981.0
* max_digit: no digits

Loading

0 comments on commit 5a57ecd

Please sign in to comment.