Skip to content

accel/amdxdna: aie4: restore power override across suspend/resume #1537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
NishadSaraf wants to merge 1 commit into
base: main
Choose a base branch
from
+62 −1
Open

accel/amdxdna: aie4: restore power override across suspend/resume #1537

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 62 additions & 1 deletion drivers/accel/amdxdna/aie4_pci.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,12 @@ static int aie4_query_aie(struct amdxdna_dev_hdl *ndev)
if (ret)
return ret;

ndev->pw_mode = POWER_MODE_DEFAULT;
/*
* Do not reset ndev->pw_mode here: this runs on every resume, and
* clobbering the cache would drop the user's power override. The
* probe-time default is set in aie4m_pcidev_init(), and the cached
* override is re-applied to firmware by aie4_restore_power_mode().
*/
ndev->total_col = min(AIE4_TOTAL_COLUMN, ndev->aie.metadata.cols);

ret = aie4_init_dpm_freq_table(ndev);
Expand All @@ -361,6 +366,36 @@ static int aie4_query_aie(struct amdxdna_dev_hdl *ndev)
return 0;
}

/*
* Firmware always boots in POWER_MODE_DEFAULT after a (re)load, so re-send the
* cached user override whenever the hardware starts. This keeps the driver
* cache (ndev->pw_mode) and the firmware power state consistent across
* suspend/resume and runtime PM cycles. On a fresh probe pw_mode is
* POWER_MODE_DEFAULT and this is a no-op. The override is a best-effort tuning
* knob, so a failure warns but does not fail hw start (mirrors ctx hysteresis).
*
* Power override is a per-VF property in firmware: each supervisor (VF) stores
* its own requested mode and the hypervisor arbitrates globally by taking the
* highest mode across all supervisors. A full firmware reload on suspend clears
* every supervisor override back to default, so each device type (PF, VF and
* classic) must re-send its own cached override on resume.
*/
static void aie4_restore_power_mode(struct amdxdna_dev_hdl *ndev)
{
int ret;

if (ndev->pw_mode == POWER_MODE_DEFAULT)
return;

ret = aie4_msg_set_power_mode(ndev, ndev->pw_mode);
if (ret)
XDNA_WARN(ndev->aie.xdna,
"Failed to restore power mode %d (%d), using fw default",
ndev->pw_mode, ret);
else
XDNA_DBG(ndev->aie.xdna, "Restored power mode %d", ndev->pw_mode);
}

static int aie4_pf_hw_start(struct amdxdna_dev_hdl *ndev)
{
int ret;
Expand Down Expand Up @@ -408,6 +443,8 @@ static int aie4_pf_hw_start(struct amdxdna_dev_hdl *ndev)
if (ret)
goto mbox_fini;

aie4_restore_power_mode(ndev);

return 0;

mbox_fini:
Expand Down Expand Up @@ -455,6 +492,8 @@ static int aie4_vf_hw_start(struct amdxdna_dev_hdl *ndev)
goto partition_fini;
}

aie4_restore_power_mode(ndev);

return 0;

partition_fini:
Expand Down Expand Up @@ -542,6 +581,8 @@ static int aie4_classic_hw_start(struct amdxdna_dev_hdl *ndev)
goto partition_fini;
}

aie4_restore_power_mode(ndev);

return 0;

partition_fini:
Expand Down Expand Up @@ -684,6 +725,7 @@ static int aie4m_pcidev_init(struct amdxdna_dev *xdna)
ndev->aie.xdna = xdna;
ndev->kernel_submit = true;
ndev->ctx_switch_hysteresis_us = AIE4_CTX_HYSTERESIS_US;
ndev->pw_mode = POWER_MODE_DEFAULT;
xdna->dev_handle = ndev;

xa_init_flags(&ndev->cert_comp_xa, XA_FLAGS_ALLOC);
Expand Down Expand Up @@ -1061,6 +1103,24 @@ static int aie4_get_array(struct amdxdna_client *client,
return ret;
}

/*
* Power override is a per-VF request but a single physical outcome. Firmware
* keeps a separate override per supervisor (one supervisor per VF mailbox under
* SR-IOV) and arbitrates them globally with a highest-wins policy: the applied
* DPM level is derived from max(override) across all supervisors, ordered
* DEFAULT < LOW < MEDIUM < HIGH < TURBO. Consequences when VFs disagree:
*
* - The device runs at the highest mode any VF has requested. Example: VF A
* requests TURBO and VF B requests LOW, the whole device runs at TURBO.
* - A VF cannot pull the device below a peer's request. Lowering (or
* resetting to DEFAULT) one VF only drops the physical DPM level once no
* other VF still holds a higher override.
* - Every VF's request is honored; there is no PF-only restriction on this
* opcode (unlike force preemption, which firmware rejects under SR-IOV).
*
* The driver therefore just records this VF's own request in ndev->pw_mode and
* forwards it; the cross-VF arbitration lives entirely in firmware.
*/
static int aie4_set_power_mode(struct amdxdna_client *client, struct amdxdna_drm_set_state *args)
{
struct amdxdna_drm_set_power_mode power_state;
Expand Down Expand Up @@ -1191,6 +1251,7 @@ static int aie4_hwctx_resume_all(struct amdxdna_dev_hdl *ndev)
* <- query_aie | - | - | Y | Y | Y
* <- partition_init | - | - | Y | Y | Y
* <- alloc_async_event | - | - | Y | Y | Y
* <- restore_power_mode | Y | Y | Y | Y | Y
* <- hwctx_resume_all | - | - | Y | Y | Y
* <- restore VFs | Y | Y | - | - | -
*
Expand Down