Dataflow compiler layer type support

[Parradox27]

Hi,
I am currently attempting to convert my own CNN models in ONNX to Hailo supported format (HEF). However, many operations that are used in vast majority of my networks are currently not supported, including transpose, reshape or squeeze. Is there any possibility for adding these operations for the dataflow parser? Or is there any plan on doing so in future releases of the software suite? The pretrained models in the zoo do contain these operations, so the AI accelerator must understand them. Thank you for your reply in advance.

[nadaved1]

Hi @Parradox27,
You're right not all ops are supported, mainly since it's not efficient for the Hailo HW. In most cases running your graph through a tool like onnx-simplifier can greatly help. If after this there are still unsupported ops, usually these are concentrated at the end of the net, you can just run this seperate from Hailo as these ops has low compute capacity.

[Parradox27]

Hi @Parradox27, You're right not all ops are supported, mainly since it's not efficient for the Hailo HW. In most cases running your graph through a tool like onnx-simplifier can greatly help. If after this there are still unsupported ops, usually these are concentrated at the end of the net, you can just run this seperate from Hailo as these ops has low compute capacity.

Hi @nadaved1,
thank you for your reply. I have tried using onnx simplifier, however none of the unsupported operations have been removed in the nets that I have tested. Some of them are not concentrated at the end nor the beginning, so removing them is not an option. You said that those operations are not efficient on Hailo HW, but not unsupported and since the hailo model zoo contains models with those operations, there must be a way to generate a HEF file from them. Without being able to compile our own nets, we will not be able to proceed with the AI accelerator. I would appreciate any assistance in this matter.

[nadaved1]

since this is something specific to your nets, I suggest that you'll open a support ticket in the hailo.ai developer zone. ‫בתאריך יום ב׳, 10 ביוני 2024 ב-16:01 מאת ‪Parradox27‬‏ <‪ ***@***.***‬‏>:‬

…

Hi @Parradox27 <https://github.com/Parradox27>, You're right not all ops are supported, mainly since it's not efficient for the Hailo HW. In most cases running your graph through a tool like onnx-simplifier can greatly help. If after this there are still unsupported ops, usually these are concentrated at the end of the net, you can just run this seperate from Hailo as these ops has low compute capacity. Hi @nadaved1 <https://github.com/nadaved1>, thank you for your reply. I have tried using onnx simplifier, however none of the unsupported operations have been removed in the nets that I have tested. Some of them are not concentrated at the end nor the beginning, so removing them is not an option. You said that those operations are not efficient on Hailo HW, but not unsupported and since the hailo model zoo contains models with those operations, there must be a way to generate a HEF file from them. Without being able to compile our own nets, we will not be able to proceed with the AI accelerator. I would appreciate any assistance in this matter. — Reply to this email directly, view it on GitHub <#98 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADBIQYD5EJRHTR7I3PTZ5PLZGWPRPAVCNFSM6AAAAABI2YQG36VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJYGI4DQMZZGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Regards, *Nadav Eden*

[havingfu]

Is there a list of all supported ops? so I'll know if my model is supported?
Also, I'm new to ML. Can you explain why it's not efficient for the Hailo HW?

Thanks,

[nadaved1]

Yes, we have that list in the data flow compiler user guide- Dataflow Compiler v3.28.0 (hailo.ai) <https://hailo.ai/developer-zone/documentation/v3-28-0/?sp_referrer=sdk/supported_layers.html> ‫בתאריך יום ב׳, 23 בספט׳ 2024 ב-18:46 מאת ‪Oak‬‏ ***@***.*** ‬‏>:‬

…

Is there a list of all supported ops? so I'll know if my model is supported? Also, I'm new to ML. Can you explain why it's not efficient for the Hailo HW? Thanks, — Reply to this email directly, view it on GitHub <#98 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADBIQYG6UDWVXWSCJZU2AFDZYAZUHAVCNFSM6AAAAABI2YQG36VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRYGY4DQMRTGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Regards, *Nadav Eden*