Skip to content

@nndeploy AI推理部署加速

Change the repository type filter

All

    Repositories list

    6 repositories

    • nndeploy

      Public
      nndeploy是一款模型端到端部署框架。以多端推理以及基于有向无环图模型部署为基础，致力为用户提供跨平台、简单易用、高性能的模型部署体验。
      hpcparallelyoloeasy-to-usetensorrtmnnmodel-deploymentopenvinoascendrknn
      C++
      •
      Apache License 2.0
      •102•668•8•0•Updated Dec 25, 2024Dec 25, 2024

    • safetensors-cpp

      Public
      Header-only safetensors loader and saver in C++
      C++
      •
      MIT License
      •8•0•0•0•Updated Nov 19, 2024Nov 19, 2024

    • onnx-llm

      Public
      llm deploy project based onnx.
      C++
      •
      Apache License 2.0
      •4•0•0•0•Updated Oct 9, 2024Oct 9, 2024

    • tokenizers-cpp

      Public
      Universal cross-platform tokenizers binding to HF and sentencepiece
      C++
      •
      Apache License 2.0
      •66•1•0•0•Updated Jun 3, 2024Jun 3, 2024

    • Awesome-LLM-Inference

      Public
      💻A small Collection for Awesome LLM Inference [Papers|Blogs|Docs] with codes, contains TensorRT-LLM, streaming-llm, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
      GNU General Public License v3.0
      •206•2•0•0•Updated Dec 3, 2023Dec 3, 2023

    • onnx-simplifier

      Public
      Simplify your onnx model
      Python
      •
      Apache License 2.0
      •389•1•0•0•Updated Apr 27, 2022Apr 27, 2022