[ Task Excutor ] Add Task Executor for Load/Unload Tensor #3115
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Describe a commit content (Until 80 colums per line) in detail ASAP. **Changes proposed in this PR:** - Added TOC generator for README.md Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <[email protected]>
FSU For Window LLM Test **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <[email protected]>
This PR provide asynchronous Task Executor which works with tensor pool. The synchronize load and unlaod pair of Tasks are managed in Executor and Cache loader. **Changes proposed in this PR:** - Added TOC generator for README.md Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <[email protected]>
jijoongmoon
requested review from
myungjoo,
again4you,
jaeyun-jung,
leemgs,
wooksong,
gichan-jang,
anyj0527,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
baek2sm,
skykongkong8,
djeong20,
EunjuYang,
dkjung and
haehun
as code owners
EunjuYang
reviewed
Apr 14, 2025
| option('enable-biqgemm', type: 'boolean', value: false) | ||
| option('biqgemm-path', type: 'string', value: '../BiQGEMM') | ||
| option('enable-biqgemm', type: 'boolean', value: true) | ||
| option('biqgemm-path', type: 'string', value: '/home/jijoongmoon/WorkSpace1/nntrainer-i/BiQGEMM') |
There was a problem hiding this comment.
Suggested change
| option('biqgemm-path', type: 'string', value: '/home/jijoongmoon/WorkSpace1/nntrainer-i/BiQGEMM') | |
| option('biqgemm-path', type: 'string', value: '../BiQGEMM') |
| @@ -43,8 +43,8 @@ option('enable-fp16', type: 'boolean', value: false) | |||
| option('enable-cublas', type: 'boolean', value: false) | |||
| option('enable-openmp', type: 'boolean', value: true) | |||
| option('enable-opencl', type: 'boolean', value: false) | |||
| option('enable-biqgemm', type: 'boolean', value: false) | |||
| option('biqgemm-path', type: 'string', value: '../BiQGEMM') | |||
| option('enable-biqgemm', type: 'boolean', value: true) | |||
There was a problem hiding this comment.
Suggested change
| option('enable-biqgemm', type: 'boolean', value: true) | |
| option('enable-biqgemm', type: 'boolean', value: false) |
| @@ -4,7 +4,7 @@ option('install-app', type: 'boolean', value: true) | |||
| option('use_gym', type: 'boolean', value: false) | |||
| option('enable-capi', type: 'feature', value: 'auto') | |||
| option('enable-ccapi', type: 'boolean', value: true) | |||
| option('enable-test', type: 'boolean', value: true) | |||
| option('enable-test', type: 'boolean', value: false) | |||
Comment on lines
+403
to
+404
| std::cout << ">>>>>>>>>>>>>>>>>>> Forwarding Start " << node->getName()<<std::endl; | ||
| // std::cout << "Layer : " << node->getName() << std::endl; |
There was a problem hiding this comment.
Suggested change
| std::cout << ">>>>>>>>>>>>>>>>>>> Forwarding Start " << node->getName()<<std::endl; | |
| // std::cout << "Layer : " << node->getName() << std::endl; |
Comment on lines
+406
to
+412
| // print_rss(); | ||
| std::cout << ">>>>>>>>>>>>>>>>>>> Forwarding END " << node->getName()<<std::endl; | ||
| // model_graph.UnloadTensors(f); | ||
|
|
||
| // model_graph.LoadTensors(f); | ||
| // model_graph.checkLoadComplete(f); | ||
| // node->forwarding(training); |
There was a problem hiding this comment.
Suggested change
| // print_rss(); | |
| std::cout << ">>>>>>>>>>>>>>>>>>> Forwarding END " << node->getName()<<std::endl; | |
| // model_graph.UnloadTensors(f); | |
| // model_graph.LoadTensors(f); | |
| // model_graph.checkLoadComplete(f); | |
| // node->forwarding(training); |
I think this comes from include [FSU] FSU for window LLM TESET this two commit. i will check this change and merge into my version. FYI) [FSU] FSU for window LLM TESET this test commit closed and split to 3 PR for merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR provide asynchronous Task Executor which works with tensor
pool.
The synchronize load and unlaod pair of Tasks are managed in Executor
and Cache loader.
Changes proposed in this PR:
Resolves:
Self evaluation:
Signed-off-by: jijoong.moon [email protected]