test failures with tensorflow 1.9.0

[rekado]

Hi,

I just built the latest commit of AIscm with tensorflow 1.9.0 and I'm getting a total of 9 test failures:

test-name: derivative of variable
location: tests/test_tensorflow.scm:141
source:
+ (test-eqv
+   "derivative of variable"
+   -6.0
+   (let* ((s (make-session))
+          (w (tf-variable #:dtype <double> #:shape '()))
+          (c (tf-const #:dtype <double> #:value 0.0))
+          (x (tf-placeholder #:dtype <double>))
+          (g (tf-add-gradient (tf-square (tf-sub x w)) w)))
+     (run s '() (tf-assign w c))
+     (run s (list (cons x 3.0)) g)))
expected-value: -6.0
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: derivative for list of variables
location: tests/test_tensorflow.scm:150
source:
+ (test-equal
+   "derivative for list of variables"
+   (list 6.0)
+   (let* ((s (make-session))
+          (x (tf-placeholder #:dtype <double>))
+          (g (tf-add-gradient (tf-square x) (list x))))
+     (run s (list (cons x 3.0)) g)))
expected-value: (6.0)
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: convert array to tensor constant
location: tests/test_tensorflow.scm:162
source:
+ (test-equal
+   "convert array to tensor constant"
+   '((1 2) (3 4))
+   (let ((s (make-session))
+         (a (tf-const
+              #:dtype
+              <int>
+              #:value
+              (arr <int> 1 2 3 4))))
+     (to-list
+       (run s '() (tf-reshape a (arr <int> 2 2))))))
expected-value: ((1 2) (3 4))
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: convert scalar to tensor constant
location: tests/test_tensorflow.scm:167
source:
+ (test-eqv
+   "convert scalar to tensor constant"
+   5
+   (let ((s (make-session)))
+     (run s '() (tf-add 2 3))))
expected-value: 5
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: convert list of scalars to list of constants
location: tests/test_tensorflow.scm:169
source:
+ (test-eqv
+   "convert list of scalars to list of constants"
+   5
+   (let ((s (make-session)))
+     (run s '() (tf-add-n (list 2 3)))))
expected-value: 5
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: top-k returns two values
location: tests/test_tensorflow.scm:173
source:
+ (test-assert
+   "top-k returns two values"
+   (let ((s (make-session)))
+     (list? (run s '() (tf-top-kv2 (arr <int> 2 5 3) 2)))))
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: top-k returns two different values
location: tests/test_tensorflow.scm:176
source:
+ (test-equal
+   "top-k returns two different values"
+   '((5 3) (1 2))
+   (let ((s (make-session)))
+     (map to-list
+          (run s
+               '()
+               (tf-top-kv2 (arr 2 5 3) 2 #:sorted #t)))))
expected-value: ((5 3) (1 2))
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: get first output of operation by name
location: tests/test_tensorflow.scm:182
source:
+ (test-equal
+   "get first output of operation by name"
+   42
+   (let ((s (make-session))
+         (c (tf-const
+              #:dtype
+              <int>
+              #:value
+              42
+              #:name
+              "test-const")))
+     (run s
+          '()
+          (tf-graph-operation-by-name "test-const"))))
expected-value: 42
actual-value: #f
actual-error:
+ (misc-error
+   "tf-run"
+   "Node 'gradients/OnesLike' is not unique"
+   ()
+   #f)
result: FAIL

test-name: determine string length
location: tests/test_tensorflow.scm:231
source:
+ (test-eqv
+   "determine string length"
+   4
+   (let ((s (make-session)))
+     (run s '() (tf-string-length "test"))))
expected-value: 4
actual-value: #f
actual-error:
+ (unbound-variable
+   #f
+   "Unbound variable: ~S"
+   (tf-string-length)
+   #f)
result: FAIL

Most of these are the same error. I know that you're using a more recent version of Tensorflow than what I have here, but perhaps we can still make it work somehow. (We haven't been able to build anything more recent from source, because Bazel is very difficult to package.)

We also have Tensorflow Lite 2.5 in Guix, but the API differs enough that I can't use it as a drop-in replacement.

Do you happen to know what these test failures mean and how to work around them? And: would you be interested in supporting Tensorflow Lite going forward? That would be lovely, as it's quite a bit easier to build than the big Tensorflow package.

Thanks for your time!
Ricardo

[wedesoft]

Hi, sorry but I don't know how to fix the "'gradients/OnesLike' is not unique" issue. The last error with tf-string-length looks like that function is not defined, so one could just disable the test. Making the Tensorflow-C bindings was quite difficult and even then it is not complete (some functionality is only visible in Python). Tensorflow-Lite only supports inference which means that training neural networks would not be possible. I am rather inclined to drop the Tensorflow bindings if they turn out to be difficult to maintain.

[rekado]

Looks like there's a patch that was not included in the 1.9.0 release of Tensorflow, addressing this very problem. I'm rebuilding with that patch right now.

I now understand that Tensorflow Lite is not an alternative. Thanks for clarifying.

We will continue to try to build a later version of Tensorflow. I hope that the Tensorflow bindings can remain in AIscm, as that's the only way to use Tensorflow from Scheme.

[wedesoft]

Ok, thanks for your understanding and for your packaging effort.

[wedesoft]

Maybe somebody can look into Pytorch bindings and whether they are better.

[rekado]

The patch to Tensorflow helped; there's only one failing test due to the missing tf-string-length. I've disabled this test for now and pushed my changes to Guix. So now the latest version of AIscm can be installed with guix install guile-aiscm-next.

I'll try packaging a newer version of Tensorflow next and then re-enable the test.

[wedesoft]

Ok, good luck. Thanks for closing the issue.