Skip to content

NPT-14408: Unable to see data when importing tensorboard logs.  #70

@odinsbane

Description

@odinsbane

I am not able to view metrics from tensorboard generated logs.

1. To reproduce

Train a tensorflow model with keras using the tesnorboad callback.

import keras
import numpy

inp = keras.layers.Input( (16, ) )
op = keras.layers.Dense(4)(inp)

mdl = keras.models.Model( inputs=[inp], outputs = [op] )

x = numpy.random.random((128, 16))
y = (numpy.random.random((128, 4))>0.75) * 1.0

mdl.compile()

metrics = [ keras.metrics.BinaryAccuracy(name = "ba")]
mdl.compile( optimizer = keras.optimizers.Adam(0.00001), loss="mean_squared_error", metrics = metrics)

callbacks = [ keras.callbacks.TensorBoard(log_dir='./junk-logs'), keras.callbacks.EarlyStopping(patience=5, monitor="loss", mode='min')]

mdl.fit( x, y, callbacks=callbacks, epochs=100)

Then I try to synch the logs with the neptune.

export NEPTUNE_PROJECT="project/name"
export NEPTUNE_API_TOKEN="xxx"
./neptune-env/bin/neptune tensorboard junk-logs

The result is a new entry in my project, but I cannot find any of the values that were tracked.

Expected behavior

I can see graphs for the metrics in tensorboard, but I cannot see the graphs in neptune

3. Environment

Default environment that was installed when I used ./neptune-env/bin/pip install neptune-tensorboard

./neptune-env/bin/python --version

> Python 3.10.13

./neptune-env/bin/pip list

Package Version


absl-py 2.0.0
arrow 1.3.0
astunparse 1.6.3
attrs 23.2.0
boto3 1.34.14
botocore 1.34.14
bravado 11.0.3
bravado-core 6.1.1
cachetools 5.3.2
certifi 2023.11.17
charset-normalizer 3.3.2
click 8.1.7
contourpy 1.2.0
cycler 0.12.1
exceptiongroup 1.2.0
execnet 2.0.2
filelock 3.13.1
flatbuffers 23.5.26
fonttools 4.47.0
fqdn 1.5.1
fsspec 2023.12.2
future 0.18.3
gast 0.5.4
gitdb 4.0.11
GitPython 3.1.40
google-auth 2.26.1
google-auth-oauthlib 1.2.0
google-pasta 0.2.0
grpcio 1.60.0
h5py 3.10.0
idna 3.6
iniconfig 2.0.0
isoduration 20.11.0
Jinja2 3.1.2
jmespath 1.0.1
jsonpointer 2.4
jsonref 1.1.0
jsonschema 4.20.0
jsonschema-specifications 2023.12.1
keras 2.15.0
kiwisolver 1.4.5
libclang 16.0.6
Markdown 3.5.1
MarkupSafe 2.1.3
matplotlib 3.8.2
ml-dtypes 0.2.0
monotonic 1.6
mpmath 1.3.0
msgpack 1.0.7
neptune 1.8.6
neptune-tensorboard 1.0.2
networkx 3.2.1
numpy 1.26.3
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
oauthlib 3.2.2
opt-einsum 3.3.0
packaging 23.2
pandas 2.1.4
pillow 10.2.0
pip 23.0.1
pluggy 1.3.0
protobuf 4.23.4
psutil 5.9.7
pyasn1 0.5.1
pyasn1-modules 0.3.0
PyJWT 2.8.0
pyparsing 3.1.1
pytest 7.4.4
pytest-xdist 3.5.0
python-dateutil 2.8.2
pytz 2023.3.post1
PyYAML 6.0.1
referencing 0.32.1
requests 2.31.0
requests-oauthlib 1.3.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rpds-py 0.16.2
rsa 4.9
s3transfer 0.10.0
setuptools 65.5.0
simplejson 3.19.2
six 1.16.0
smmap 5.0.1
swagger-spec-validator 3.0.3
sympy 1.12
tbparse 0.0.8
tensorboard 2.15.1
tensorboard-data-server 0.7.2
tensorboardX 2.6.2.2
tensorflow 2.15.0.post1
tensorflow-estimator 2.15.0
tensorflow-io-gcs-filesystem 0.35.0
termcolor 2.4.0
tomli 2.0.1
torch 2.1.2
triton 2.1.0
types-python-dateutil 2.8.19.14
typing_extensions 4.9.0
tzdata 2023.4
uri-template 1.3.0
urllib3 2.0.7
webcolors 1.13
websocket-client 1.7.0
Werkzeug 3.0.1
wheel 0.41.2
wrapt 1.14.1

This is a 64bit linux, I think centos with 4.18 kernel.

Metadata

Metadata

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions