unicode error

[eyaler]

this is on windows 10 64 + python 3.6.2 + tensorflow 1.3 + downloaded npy from dropbox

python pspnet.py -m pspnet50_ade20k -i example_images/ade20k.jpg -o example_results/ade20k.jpg
Using TensorFlow backend.
2017-10-14 16:39:03.003842: W C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-10-14 16:39:03.003972: W C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-14 16:39:03.347437: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.645
pciBusID 0000:01:00.0
Total memory: 8.00GiB
Free memory: 6.62GiB
2017-10-14 16:39:03.347582: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:976] DMA: 0
2017-10-14 16:39:03.348951: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:986] 0: Y
2017-10-14 16:39:03.349478: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0)
Namespace(flip=False, id='0', input_path='example_images/ade20k.jpg', model='pspnet50_ade20k', multi_scale=False, output_path='example_results/ade20k.jpg', sliding=False)
No Keras model & weights found, import from npy weights.
Building a PSPNet based on ResNet 50 expecting inputs of shape (473, 473) predicting 150 classes
PSP module will interpolate to a final feature map size of (60, 60)
Importing weights from weights\npy\pspnet50_ade20k.npy
Processing input_1
Processing conv1_1_3x3_s2
Processing conv1_1_3x3_s2_bn
Processing activation_1
Processing conv1_2_3x3
Processing conv1_2_3x3_bn
Processing activation_2
Processing conv1_3_3x3
Processing conv1_3_3x3_bn
Processing activation_3
Processing max_pooling2d_1
Processing activation_4
Processing conv2_1_1x1_reduce
Processing conv2_1_1x1_reduce_bn
Processing activation_5
Processing zero_padding2d_1
Processing conv2_1_3x3
Processing conv2_1_3x3_bn
Processing activation_6
Processing conv2_1_1x1_increase
Processing conv2_1_1x1_proj
Processing conv2_1_1x1_increase_bn
Processing conv2_1_1x1_proj_bn
Processing add_1
Processing activation_7
Processing conv2_2_1x1_reduce
Processing conv2_2_1x1_reduce_bn
Processing activation_8
Processing zero_padding2d_2
Processing conv2_2_3x3
Processing conv2_2_3x3_bn
Processing activation_9
Processing conv2_2_1x1_increase
Processing conv2_2_1x1_increase_bn
Processing add_2
Processing activation_10
Processing conv2_3_1x1_reduce
Processing conv2_3_1x1_reduce_bn
Processing activation_11
Processing zero_padding2d_3
Processing conv2_3_3x3
Processing conv2_3_3x3_bn
Processing activation_12
Processing conv2_3_1x1_increase
Processing conv2_3_1x1_increase_bn
Processing add_3
Processing activation_13
Processing conv3_1_1x1_reduce
Processing conv3_1_1x1_reduce_bn
Processing activation_14
Processing zero_padding2d_4
Processing conv3_1_3x3
Processing conv3_1_3x3_bn
Processing activation_15
Processing conv3_1_1x1_increase
Processing conv3_1_1x1_proj
Processing conv3_1_1x1_increase_bn
Processing conv3_1_1x1_proj_bn
Processing add_4
Processing activation_16
Processing conv3_2_1x1_reduce
Processing conv3_2_1x1_reduce_bn
Processing activation_17
Processing zero_padding2d_5
Processing conv3_2_3x3
Processing conv3_2_3x3_bn
Processing activation_18
Processing conv3_2_1x1_increase
Processing conv3_2_1x1_increase_bn
Processing add_5
Processing activation_19
Processing conv3_3_1x1_reduce
Processing conv3_3_1x1_reduce_bn
Processing activation_20
Processing zero_padding2d_6
Processing conv3_3_3x3
Processing conv3_3_3x3_bn
Processing activation_21
Processing conv3_3_1x1_increase
Processing conv3_3_1x1_increase_bn
Processing add_6
Processing activation_22
Processing conv3_4_1x1_reduce
Processing conv3_4_1x1_reduce_bn
Processing activation_23
Processing zero_padding2d_7
Processing conv3_4_3x3
Processing conv3_4_3x3_bn
Processing activation_24
Processing conv3_4_1x1_increase
Processing conv3_4_1x1_increase_bn
Processing add_7
Processing activation_25
Processing conv4_1_1x1_reduce
Processing conv4_1_1x1_reduce_bn
Processing activation_26
Processing zero_padding2d_8
Processing conv4_1_3x3
Processing conv4_1_3x3_bn
Processing activation_27
Processing conv4_1_1x1_increase
Processing conv4_1_1x1_proj
Processing conv4_1_1x1_increase_bn
Processing conv4_1_1x1_proj_bn
Processing add_8
Processing activation_28
Processing conv4_2_1x1_reduce
Processing conv4_2_1x1_reduce_bn
Processing activation_29
Processing zero_padding2d_9
Processing conv4_2_3x3
Processing conv4_2_3x3_bn
Processing activation_30
Processing conv4_2_1x1_increase
Processing conv4_2_1x1_increase_bn
Processing add_9
Processing activation_31
Processing conv4_3_1x1_reduce
Processing conv4_3_1x1_reduce_bn
Processing activation_32
Processing zero_padding2d_10
Processing conv4_3_3x3
Processing conv4_3_3x3_bn
Processing activation_33
Processing conv4_3_1x1_increase
Processing conv4_3_1x1_increase_bn
Processing add_10
Processing activation_34
Processing conv4_4_1x1_reduce
Processing conv4_4_1x1_reduce_bn
Processing activation_35
Processing zero_padding2d_11
Processing conv4_4_3x3
Processing conv4_4_3x3_bn
Processing activation_36
Processing conv4_4_1x1_increase
Processing conv4_4_1x1_increase_bn
Processing add_11
Processing activation_37
Processing conv4_5_1x1_reduce
Processing conv4_5_1x1_reduce_bn
Processing activation_38
Processing zero_padding2d_12
Processing conv4_5_3x3
Processing conv4_5_3x3_bn
Processing activation_39
Processing conv4_5_1x1_increase
Processing conv4_5_1x1_increase_bn
Processing add_12
Processing activation_40
Processing conv4_6_1x1_reduce
Processing conv4_6_1x1_reduce_bn
Processing activation_41
Processing zero_padding2d_13
Processing conv4_6_3x3
Processing conv4_6_3x3_bn
Processing activation_42
Processing conv4_6_1x1_increase
Processing conv4_6_1x1_increase_bn
Processing add_13
Processing activation_43
Processing conv5_1_1x1_reduce
Processing conv5_1_1x1_reduce_bn
Processing activation_44
Processing zero_padding2d_14
Processing conv5_1_3x3
Processing conv5_1_3x3_bn
Processing activation_45
Processing conv5_1_1x1_increase
Processing conv5_1_1x1_proj
Processing conv5_1_1x1_increase_bn
Processing conv5_1_1x1_proj_bn
Processing add_14
Processing activation_46
Processing conv5_2_1x1_reduce
Processing conv5_2_1x1_reduce_bn
Processing activation_47
Processing zero_padding2d_15
Processing conv5_2_3x3
Processing conv5_2_3x3_bn
Processing activation_48
Processing conv5_2_1x1_increase
Processing conv5_2_1x1_increase_bn
Processing add_15
Processing activation_49
Processing conv5_3_1x1_reduce
Processing conv5_3_1x1_reduce_bn
Processing activation_50
Processing zero_padding2d_16
Processing conv5_3_3x3
Processing conv5_3_3x3_bn
Processing activation_51
Processing conv5_3_1x1_increase
Processing conv5_3_1x1_increase_bn
Processing add_16
Processing activation_52
Processing average_pooling2d_4
Processing average_pooling2d_3
Processing average_pooling2d_2
Processing average_pooling2d_1
Processing conv5_3_pool6_conv
Processing conv5_3_pool3_conv
Processing conv5_3_pool2_conv
Processing conv5_3_pool1_conv
Processing conv5_3_pool6_conv_bn
Processing conv5_3_pool3_conv_bn
Processing conv5_3_pool2_conv_bn
Processing conv5_3_pool1_conv_bn
Processing activation_56
Processing activation_55
Processing activation_54
Processing activation_53
Processing lambda_4
Processing lambda_3
Processing lambda_2
Processing lambda_1
Processing concatenate_1
Processing conv5_4
Processing conv5_4_bn
Processing activation_57
Processing dropout_1
Processing conv6
Processing lambda_5
Processing activation_58
Set a total of 121 weights
Finished importing weights.
Writing keras model & weights
Traceback (most recent call last):
File "pspnet.py", line 265, in
weights=args.model)
File "pspnet.py", line 142, in init
input_shape=input_shape, weights=weights)
File "pspnet.py", line 48, in init
self.set_npy_weights(weights)
File "pspnet.py", line 129, in set_npy_weights
json_string = self.model.to_json()
File "C:\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2665, in to_json
model_config = self._updated_config()
File "C:\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2632, in _updated_config
config = self.get_config()
File "C:\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2326, in get_config
layer_config = layer.get_config()
File "C:\Anaconda3\lib\site-packages\keras\layers\core.py", line 659, in get_config
function = func_dump(self.function)
File "C:\Anaconda3\lib\site-packages\keras\utils\generic_utils.py", line 175, in func_dump
code = marshal.dumps(func.code).decode('raw_unicode_escape')
UnicodeDecodeError: 'rawunicodeescape' codec can't decode bytes in position 208-209: truncated \UXXXXXXXX escape

[eyaler]

lol. probably this: keras-team/keras#4135

[ljm355]

Have you got this fixed? I am having a similar issue:

File "C:\Miniconda3\lib\site-packages\keras\utils\generic_utils.py", line 140,
in deserialize_keras_object
list(custom_objects.items())))
File "C:\Miniconda3\lib\site-packages\keras\layers\core.py", line 699, in from
_config
function = func_load(config['function'], globs=globs)
File "C:\Miniconda3\lib\site-packages\keras\utils\generic_utils.py", line 224,
in func_load
raw_code = codecs.decode(code.encode('ascii'), 'base64')
UnicodeEncodeError: 'ascii' codec can't encode character '\xe3' in position 0: o
rdinal not in range(128)

Wonder if you have any suggestions?

[akshaychawla]

@ljm355 I solved this problem by using an older version of keras. My setup is as follows:
python 3.4.3
tensorflow-cpu (1.4.1)
keras - 2.0.6 (this is what made it work)
apparently there is a bug in keras 2.1.2 (the most recent one) where they were unable to deserialize old models. So i just used an older version of keras, it works well.

[ljm355]

Thanks!
I tried a number of different Keras and Python versions and finally worked it out with the following configuration：
Windows 7
python 3.5.0 (conda install python=3.5.0)
keras gpu 2.0.8 (conda install -c anaconda keras-gpu)
tensorflow-gpu 1.1.0 (conda install tensorflow-gpu)

One caveat about Python version:
Python 3.6 will cause 'SystemError: unknown opcode'

[tangsanli5201]

@akshaychawla I think there is an easier way to solve this problem ( and I found it's working):
In layers_builder.py, call the build_pspnet function as:
model = build_pspnet(nb_classes=19, resnet_layers=101, input_shape=(713, 713), activation='softmax')
print(model.to_json())
and replace the json string to the json file.
This will directly output a json string according to your keras version.

[akshaychawla]

@tangsanli5201 Cool! This is a much better solution as it works for all versions of Keras. Could you dump the json for the latest keras (via pip) and raise a PR? It will help others who are facing this problem.

[ktnguyen2]

I am having a similar issue on CentosOS:
$ python3.6 pspnet.py -m pspnet101_cityscapes -i example_images/cityscapes.png -o example_results/cityscapes.jpg
Using TensorFlow backend.
2018-11-11 19:09:55.178295: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Namespace(flip=True, glob_path=None, id='0', input_path='example_images/cityscapes.png', input_size=500, model='pspnet101_cityscapes', output_path='example_results/cityscapes.jpg', weights=None)
Keras model & weights found, loading...
XXX lineno: 18, opcode: 0
Traceback (most recent call last):
File "pspnet.py", line 188, in
weights=args.model)
File "pspnet.py", line 140, in init
input_shape=input_shape, weights=weights)
File "pspnet.py", line 33, in init
self.model = model_from_json(file_handle.read())
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/models.py", line 339, in model_from_json
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/layers/init.py", line 54, in deserialize
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/engine/topology.py", line 2450, in from_config
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/engine/topology.py", line 2445, in process_layer
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/engine/topology.py", line 596, in call
File "/home/knguyen2/.local/lib/python3.6/site-packages/Keras-2.0.6-py3.6.egg/keras/layers/core.py", line 647, in call
File "/media/tatsch/DataSSD1/workspace/PSPNet-Keras-tensorflow/pspnet/layers_builder.py", line 18, in Interp
return BatchNormalization(momentum=0.95, name=name, epsilon=1e-5)
SystemError: unknown opcode

I'm using tensorflow 1.12, keras 2.0.6, and python 3.6

[keepgoing365]

I'm using scipy==1.0.0，tensorflow-gpu==1.13.1，keras==2.0.6,opencv-python==4.1.0.25,python=3.5.2. it works!