[AOTI] Support InplaceBernoulliFallback in the ABI-compatible codegen #126183

desertfire · 2024-05-14T16:32:17Z

Stack from ghstack (oldest at bottom):

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. Fixes #121809

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @chauhang

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. [ghstack-poisoned]

pytorch-bot · 2024-05-14T16:32:21Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126183

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2ea22a7 with merge base ee8c155 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. ghstack-source-id: 244079be9160ce16845c7ce96d1f0fef0ed71793 Pull Request resolved: #126183

desertfire · 2024-05-14T16:42:59Z

torch/csrc/inductor/aoti_torch/generated/c_shim_cuda.h

@@ -112,8 +114,8 @@ AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_randperm(int64_t n, int32_t* dt
 AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_repeat_interleave_Tensor(AtenTensorHandle repeats, int64_t* output_size, AtenTensorHandle* ret0);
 AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_replication_pad1d_backward(AtenTensorHandle grad_output, AtenTensorHandle self, const int64_t* padding, int64_t padding_len_, AtenTensorHandle* ret0);
 AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_replication_pad2d_backward(AtenTensorHandle grad_output, AtenTensorHandle self, const int64_t* padding, int64_t padding_len_, AtenTensorHandle* ret0);
-AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_resize_(AtenTensorHandle self, const int64_t* size, int64_t size_len_, int32_t* memory_format, AtenTensorHandle* ret0);
-AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_resize_as_(AtenTensorHandle self, AtenTensorHandle the_template, int32_t* memory_format, AtenTensorHandle* ret0);
+AOTI_TORCH_EXPORT AOTITorchError aoti_torch_cuda_resize_(AtenTensorHandle self, const int64_t* size, int64_t size_len_, int32_t* memory_format);


It's ok to make this kind of one-time change, because aoti_torch_cuda_resize_ wasn't working properly previously.

…ble codegen" Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. Fixes #121809 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames chauhang [ghstack-poisoned]

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. ghstack-source-id: 81864809729cae55c4b103e1a352a4a81641f17b Pull Request resolved: #126183

…ble codegen" Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. Fixes #121809 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames chauhang [ghstack-poisoned]

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. ghstack-source-id: 655783073c5060f429b5faf6ab62aaa38a37f14a Pull Request resolved: #126183

desertfire · 2024-05-14T20:45:57Z

torchgen/gen_aoti_c_shim.py

 args, callsite_exprs = gen_arguments(
- [*schema.arguments.flat_non_out, *schema.arguments.out]
- if "at::native" in backend_call


This "at::native" logic is not needed anymore.

chenyang78 · 2024-05-15T01:51:31Z

The failure related to test_bernoulli1_cuda_cuda_wrapper seems to be real. Thanks.

…ble codegen" Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. Fixes #121809 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames chauhang [ghstack-poisoned]

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. ghstack-source-id: 373314464db98adf1af6c7150562b8026aa3449d Pull Request resolved: #126183

…ble codegen" Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. Fixes #121809 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames chauhang [ghstack-poisoned]

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. ghstack-source-id: b0c362906d55bb62fa01a56d05151aff8ff3b7d6 Pull Request resolved: #126183

desertfire · 2024-05-15T18:34:18Z

The failure related to test_bernoulli1_cuda_cuda_wrapper seems to be real. Thanks.

error fixed.

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. ghstack-source-id: b0c362906d55bb62fa01a56d05151aff8ff3b7d6 Pull Request resolved: pytorch#126183

angelayi · 2024-05-16T16:35:15Z

torch/_inductor/ir.py

+ wrapper.writeline(
+ f"{self.get_kernel_name()}({x}, {', '.join(map(repr, self.constant_args))}, NULL){wrapper.ending}"
+ )


Does this mean that if user passes in a generator function, we won't use it and will just pass null? Should we put a warning here so that they know?

Summary: The logic has been repeated several times in the code, so it's worth to write a common util function. Pull Request resolved: #126352 Approved by: https://github.com/chenyang78 ghstack dependencies: #126181, #126182, #126183

…pytorch#126183) Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. Fixes pytorch#121809 Pull Request resolved: pytorch#126183 Approved by: https://github.com/angelayi ghstack dependencies: pytorch#126181, pytorch#126182

Summary: The logic has been repeated several times in the code, so it's worth to write a common util function. Pull Request resolved: pytorch#126352 Approved by: https://github.com/chenyang78 ghstack dependencies: pytorch#126181, pytorch#126182, pytorch#126183

[AOTI] Support InplaceBernoulliFallback in the ABI-compatible codegen

ca3c971

Summary: Update the torchgen rule for inplace ops like bernoulli_, and update InplaceBernoulliFallback to codegen in the ABI-compatible mode. [ghstack-poisoned]

This was referenced May 14, 2024

[AOTI][torchgen] Update NativeFunctionsGroup mapping #125962

Closed

[AOTI][torchgen] Add a few more fallback ops #126013

Open

[AOTI][torchgen] Support at::Generator via C shim #126181

Closed

desertfire mentioned this pull request May 14, 2024

[AOTI] Refactor some fallback op util functions #126182

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels May 14, 2024

desertfire added the topic: not user facing topic category label May 14, 2024

desertfire commented May 14, 2024

View reviewed changes

desertfire requested review from chenyang78 and angelayi May 14, 2024 20:46

desertfire mentioned this pull request May 15, 2024

[AOTI] Support list of tensors as arg to fallback op #126282

Open

desertfire added the ciflow/trunk Trigger trunk jobs on your pull request label May 15, 2024

This was referenced May 15, 2024

[AOTI][refactor] Add aoti_torch_item as a util function #126352

Closed

[AOTI] Codegen None as empty tensor #126369

Closed

[AOTI] Disable stack allocation for OSS #125732

Closed

desertfire mentioned this pull request May 16, 2024

[AOTI][not for review] Test cpp_wrapper mode #125733

Open

angelayi reviewed May 16, 2024

View reviewed changes

angelayi approved these changes May 16, 2024

View reviewed changes

pytorchmergebot added the Merged label May 16, 2024

pytorchmergebot closed this in 0332b58 May 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AOTI] Support InplaceBernoulliFallback in the ABI-compatible codegen #126183

[AOTI] Support InplaceBernoulliFallback in the ABI-compatible codegen #126183

desertfire commented May 14, 2024 •

edited

pytorch-bot bot commented May 14, 2024 •

edited

desertfire May 14, 2024

desertfire May 14, 2024

chenyang78 commented May 15, 2024

desertfire commented May 15, 2024

angelayi May 16, 2024 •

edited

[AOTI] Support InplaceBernoulliFallback in the ABI-compatible codegen #126183

[AOTI] Support InplaceBernoulliFallback in the ABI-compatible codegen #126183

Conversation

desertfire commented May 14, 2024 • edited

pytorch-bot bot commented May 14, 2024 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126183

✅ No Failures

desertfire May 14, 2024

Choose a reason for hiding this comment

desertfire May 14, 2024

Choose a reason for hiding this comment

chenyang78 commented May 15, 2024

desertfire commented May 15, 2024

angelayi May 16, 2024 • edited

Choose a reason for hiding this comment

desertfire commented May 14, 2024 •

edited

pytorch-bot bot commented May 14, 2024 •

edited

angelayi May 16, 2024 •

edited