docker

[MyraBaba]

Hi ,

Is there any docker container or any plan in near future ?

Best

[snexus]

Hi,

I might add support for docker in the future if there is enough demand

[MyraBaba]

There is a visible demand for enterprise search and llm mix. Also one of our firm is working on the cyber security and wondering that is llm-search could be useful for logs (firewall, servers etc) Best

…

On 1 Jan 2024, at 16:42, Denis Lapchev ***@***.***> wrote: Hi, I might add support for docker in the future if there is enough demand — Reply to this email directly, view it on GitHub <#75 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZH4JN5CJK7TVUR2KDHTYMK4LTAVCNFSM6AAAAABBHPZ4P2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZTGMZTGNBZGQ>. You are receiving this because you authored the thread.

[shuther]

in the docker (there is a DOCKERFILE), is it possible to install pandoc?

[snexus]

It is an outdated version and not in sync with the latest libraries. Will add pandoc once I update it.

[shuther]

thanks, indeed, I had an issue to load it.
Also, is it possible to remove the dependency on cuda? I expect the embeddings and llm to run on a different machine (openAI or through litellm)

[snexus]

CUDA is also required for sparse embeddings (SPLADE in this project), which are not optional at the moment. Theoretically, they can run on CPU but would have a large performance hit. Are there any sparse embeddings that can be served via lite llm?