Engine Refactor part 1

[ashwinvaidya17]

Summary

Caution

Do not merge

• Converts Engine to NativeEngine.
• Introduce create_engine as the entrypoint.
• Based on the proposal in the design doc

How to test

import torch
from torchvision.transforms.v2 import Normalize, ToDtype

from otx.algo.classification.timm_model import TimmModelForMulticlassCls
from otx.core.config.data import SamplerConfig, SubsetConfig
from otx.core.data.module import OTXDataModule
from otx.core.data.transform_libs.torchvision import RandomResizedCrop, Resize
from otx.core.types.task import OTXTaskType
from otx.engine import create_engine


def run():
    datamodule = OTXDataModule(
        data_root="tests/assets/classification_dataset",
        data_format="imagenet_with_subset_dirs",  # Correct format for classification
        task=OTXTaskType.MULTI_CLASS_CLS,
        train_subset=SubsetConfig(
            subset_name="train",
            batch_size=64,
            num_workers=2,
            transform_lib_type="TORCHVISION",
            to_tv_image=False,  # Important for proper image conversion
            transforms=[
                RandomResizedCrop(scale=224, is_numpy_to_tvtensor=True),  # Convert numpy to tensor
                ToDtype(dtype=torch.float32, scale=False),
                Normalize(
                    mean=[123.675, 116.28, 103.53],
                    std=[58.395, 57.12, 57.375],
                ),
            ],
            sampler=SamplerConfig(),
        ),
        val_subset=SubsetConfig(
            subset_name="val",
            batch_size=64,
            num_workers=2,
            transform_lib_type="TORCHVISION",
            to_tv_image=False,
            transforms=[
                Resize(scale=224, is_numpy_to_tvtensor=True),  # Convert numpy to tensor
                ToDtype(dtype=torch.float32, scale=False),
                Normalize(
                    mean=[123.675, 116.28, 103.53],
                    std=[58.395, 57.12, 57.375],
                ),
            ],
            sampler=SamplerConfig(),
        ),
        test_subset=SubsetConfig(
            subset_name="test",
            batch_size=64,
            num_workers=2,
            transform_lib_type="TORCHVISION",
            to_tv_image=False,
            transforms=[
                Resize(scale=224, is_numpy_to_tvtensor=True),  # Convert numpy to tensor
                ToDtype(dtype=torch.float32, scale=False),
                Normalize(
                    mean=[123.675, 116.28, 103.53],
                    std=[58.395, 57.12, 57.375],
                ),
            ],
            sampler=SamplerConfig(),
        ),
    )
    model = TimmModelForMulticlassCls(label_info=datamodule.label_info, model_name="tf_efficientnetv2_s.in21k")
    engine = create_engine(model, datamodule)
    engine.train()



if __name__ == "__main__":
    run()

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have ran e2e tests and there is no issues.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[eugene123tw]

Thanks, @ashwinvaidya17! I think there's currently too much OV logic in NativeEngine. We'll be moving those to OVEngine, right?

[ashwinvaidya17]

Thanks, @ashwinvaidya17! I think there's currently too much OV logic in NativeEngine. We'll be moving those to OVEngine, right?

Yeah that's the plan