Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process for building Tensorflow that maximises use of system libraries? #42

Open
jdpipe opened this issue Oct 20, 2021 · 1 comment
Open

Comments

@jdpipe
Copy link

jdpipe commented Oct 20, 2021

Hi there, I raised a question about this on the Discord server, but was advised to come here and ask instead.

Are there any instructions/guidelines on how to build a smaller, lighter Tensorflow shared library (C bindings) and also Python module that maximises the use of standard system libraries, eg in Ubuntu 18.04, 20.04, CentOS or MSYS2?

With 29 dependencies listed in the THIRD_PARTY_TC_C_LICENSES, it seems to me that this 320 MB shared library could be potentially much smaller if it made use of system libraries instead of statically linked versions, and if there is a decent TensorFlow test suite then it should not be too hard to assure that the library is still functioning correctly.

I note https://github.com/tensorflow/build/tree/master/official_build_environments ('They should have as few external dependencies as possible'). With the dockers and statically linked shared libraries, it seems that TensorFlow community has a strong bias against this approach, preferring more self-contained builds with less risk of platform-specific divergence. I just wonder if, as the code's user base grows, whether there might be a case for more 'standard' OS packaging of this library (eg .deb, .rpm) and then also the associated Python bindings, which presumably in the 'standard install' also contains a separate copy of the shared library?

@bhack
Copy link
Contributor

bhack commented Nov 27, 2021

I suppose that is just a side effect of the monorepo approach:

https://jmmv.dev/2021/02/google-monorepos-and-caching.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants