Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drivers/FPGA/DFL - Intel PAC N3000 missing modules select #22

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Drivers/FPGA/DFL - Intel PAC N3000 missing modules select #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ba1224e
Drivers/FPGA/DFL - Intel PAC N3000 missing modules select
hwspeedy Dec 13, 2023
f22d490
Update Kconfig
hwspeedy Dec 13, 2023
4463c1b
ADD: Missing DFL intel-m10-bmc-retimer driver from Xu Yilun, 2020
Dec 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions drivers/fpga/Kconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,24 @@ config FPGA_DFL_NIOS_INTEL_PAC_N3000
tristate "FPGA DFL NIOS Driver for Intel PAC N3000"
depends on FPGA_DFL
select REGMAP
select UIO_DFL
select MFD_INTEL_M10_BMC_CORE
select MFD_INTEL_M10_BMC_SPI
select SENSORS_INTEL_M10_BMC_HWMON
select SPI_ALTERA
select FPGA_DFL_FME
select FPGA_DFL_FME_MGR
select FPGA_DFL_FME_REGION
select FPGA_DFL_FME_BRIDGE
select FPGA_DFL_AFU
select FPGA_DFL_PCI
select ALTERA_FREEZE_BRIDGE
select FPGA_M10_BMC_SEC_UPDATE
select FPGA_M10_BMC_RETIMER
select ALTERA_PR_IP_CORE
select FPGA_MGR_ALTERA_CVP
select FPGA_MGR_ALTERA_PS_SPI

Comment on lines +206 to +223
Copy link
Contributor

@pcolberg pcolberg Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

select should not be used to express dependencies in general since

select should be used with care. select will force a symbol to a value without visiting the dependencies. By abusing select you are able to select a symbol FOO even if FOO depends on BAR that is not set. In general use select only for non-visible symbols (no prompts anywhere) and for symbols with no dependencies. That will limit the usefulness but on the other hand avoid the illegal configurations all over.

So then one might ask why not add these dependencies using depends on? If you take a look at dfl-n3000-nios.c, note how its minimal dependencies, FPGA_DFL and REGMAP, provide symbols used by the module, e.g., __dfl_driver_register() (through module_dfl_driver()) and regmap_read(), respectively. With only these dependencies selected, the dfl-n3000-nios module should load successfully since its symbols are satisfied. The dependencies you specified above may be useful to exploit the full functionality of the driver, but they are not needed to load the driver.

Copy link
Author

@hwspeedy hwspeedy Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your feedback.

Here are my 5 cents to DFL/OPAE and what I think need to happen for OFS success.

State of OFS - DFL/OPAE.
DFL contains a bunch of modules and almost no-one understands it.
Partial reconfiguration (PR) is a messy business – keeping part of the FPGA running while stalling another part and changing that. Isolation bridges for freezing the registers towards the running part, ensuring that the running part do not use the part that is being changed and the other way around, when loading the new part.
Why do you have to partition in special way – that is known only by the FPGA manufacture – but it has to do with chains used to configure the FPGA. For Arria 10 FPGA it is keep it slim and wide. The schemes are different for Xilinx chips – more like Stratix. The PR needs to be compiled for exactly what is running – loading something else is a crash?
So that is why the FPGA loading is validating - that could be Kernel or BMC. When a FPGA part is running it would be nice to could verify who made this part and if it is valid.
A simple way – upload known good image, another way is to check public/private keys. Security is important both for data centers, companies and consumers and the implementations need maturity - open source is quite good at that.
But all of this does not make entirely sense to most kernel maintainers/users – its ugly hardware solutions mostly used because the PCIe interface, embedded CPU or memory in FPGA needs to continue running while we update or replace a function.

So DFL and OPAE is trying to address this situation – but it is also seams messy.
Looking at the linux-fpga patchworks board - it seams Redhat as maintainer mostly gave up and I see why.

What is this? How does it benefit Linux? Where is going? Is it really open source?

My experience
I have a couple of Intel FPGA cards and installs them in a machine. Nothing works – no drivers.
Hacking the kernel and adding the Intel PAC n3000 card and finding the OPAE in the OFS gives a little life but no flashing LED yet.
Now you need to debug the entire DFL and Intel PAC n3000 private to discover lots of missing modules. Then add these – more life and mostly running as root – only a retimer module is missing.

Trying to get it to work as an user on the PC – Trouble again – PHY are accessed trough the kernel debug interface on the regmaps – Nice new feature by the way. But this leaves the driver incomplete and probably why the PAC n3000V is not working at 10G instead of the 25G, when the FPGA is made for 10G but the retimers are still 25G.

When we look at the FPGA and OFS to setup the retimers you end up needing a missing BMC source and FPGA source. But as OFS says on the webpage - you can get this package by contacting Intel.

FAE manager at Arrow was not able to get this package.
I need it to change the messy retimer thing with SPI bridge in both directions along with removing, what I believe is a CVE security problem – The fast Core Cache IP, but still have the BMC with Powerup seq, temp, overheating protection, FPGA loading, etc.

Can you get the package consisting of BMC controller RTL+FW+qsf and FPGA RTL bridge part for me?

Back to the open source idea, which has bought the world the Linux we know.

So Intel wants to be in the kernel that runs all data centers and seams to be contributing to open source – but where are the Xilinx PAC/NAC cards?

My 5 cents:

  1. DFL needs to be more user friendly and more complete, clear in the direction it is heading.
  2. DFL should embrace FW/FPGA developers of functions with a nice platform – the community effort made by AMD and Intel will pay back – There is a long way to reduce Cuda – OFS? Community?
  3. DFL needs to have lots of Intel and AMD FPGA cards – not just the PAC/NAC cards also a couple of other examples to enable the community.
  4. DFL should not be for data centers only – it should be for all developers of other FPGA products – like PicoEVB, Aller A7, etc.
  5. DFL should use standards already made in the Linux kernel – eg. use .dtb binary blobs in the FPGA instead of just a PR loading chain.
  6. DFL loading and security should be open sources so that all developers can reuse the same framework and add more solutions.
  7. DFL matters - if the same hardware can be different drivers - which is the case with FPGA solutions PAC/NAC or just a PCIe FPGA card - DFL could be the answer?

If the above succeeds so does Intel and AMD FPGA and we get more FPGA developers in the world.

Please share you comments and also let me know if I can get the above package, thanx.

help
This is the driver for the N3000 Nios private feature on Intel
PAC (Programmable Acceleration Card) N3000. It communicates
Expand Down Expand Up @@ -257,6 +275,18 @@ config FPGA_M10_BMC_SEC_UPDATE
(BMC) and provides support for secure updates for the BMC image,
the FPGA image, the Root Entry Hashes, etc.

config FPGA_M10_BMC_RETIMER
tristate "Intel(R) MAX 10 BMC ethernet retimer support"
depends on MFD_INTEL_M10_BMC_CORE
select PHYLIB
help
This driver supports the ethernet retimer (Parkvale) on
Intel(R) MAX 10 BMC, which is used by Intel PAC N3000 FPGA based
Smart NIC.

To compile this driver as a module, choose M here. The module
will be called intel-m10-bmc-retimer.

config FPGA_MGR_MICROCHIP_SPI
tristate "Microchip Polarfire SPI FPGA manager"
depends on SPI
Expand Down
2 changes: 2 additions & 0 deletions drivers/fpga/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ obj-$(CONFIG_ALTERA_PR_IP_CORE_PLAT) += altera-pr-ip-core-plat.o

# FPGA Secure Update Drivers
obj-$(CONFIG_FPGA_M10_BMC_SEC_UPDATE) += intel-m10-bmc-sec-update.o
# FPGA Retimer Drivers
obj-$(CONFIG_FPGA_M10_BMC_RETIMER) += intel-m10-bmc-retimer.o

# FPGA Bridge Drivers
obj-$(CONFIG_FPGA_BRIDGE) += fpga-bridge.o
Expand Down
11 changes: 9 additions & 2 deletions drivers/fpga/dfl-n3000-nios.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mfd/intel-m10-bmc.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
#include <linux/stddef.h>
Expand Down Expand Up @@ -159,6 +160,8 @@ struct n3000_nios {
struct regmap *regmap;
struct device *dev;
struct platform_device *altera_spi;
struct intel_m10bmc_platdata m10bmc_pdata;
struct intel_m10bmc_retimer_pdata m10bmc_retimer_pdata;
};

static ssize_t nios_fw_version_show(struct device *dev,
Expand Down Expand Up @@ -412,7 +415,8 @@ static struct spi_board_info m10_n3000_info = {
.chip_select = 0,
};

static int create_altera_spi_controller(struct n3000_nios *nn)
static int create_altera_spi_controller(struct n3000_nios *nn,
struct device *retimer_master)
{
struct altera_spi_platform_data pdata = { 0 };
struct platform_device_info pdevinfo = { 0 };
Expand All @@ -431,6 +435,9 @@ static int create_altera_spi_controller(struct n3000_nios *nn)
pdata.bits_per_word_mask =
SPI_BPW_RANGE_MASK(1, FIELD_GET(N3000_NS_PARAM_DATA_WIDTH, v));

nn->m10bmc_retimer_pdata.retimer_master = retimer_master;
nn->m10bmc_pdata.retimer = &nn->m10bmc_retimer_pdata;
m10_n3000_info.platform_data = &nn->m10bmc_pdata;
pdata.num_devices = 1;
pdata.devices = &m10_n3000_info;

Expand Down Expand Up @@ -549,7 +556,7 @@ static int n3000_nios_probe(struct dfl_device *ddev)
if (ret)
return ret;

ret = create_altera_spi_controller(nn);
ret = create_altera_spi_controller(nn, dfl_dev_get_base_dev(ddev));
if (ret)
dev_err(dev, "altera spi controller create failed: %d\n", ret);

Expand Down
9 changes: 9 additions & 0 deletions drivers/fpga/dfl.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,15 @@ void dfl_driver_unregister(struct dfl_driver *dfl_drv)
}
EXPORT_SYMBOL(dfl_driver_unregister);

struct device *dfl_dev_get_base_dev(struct dfl_device *dfl_dev)
{
if (!dfl_dev || !dfl_dev->cdev)
return NULL;

return dfl_dev->cdev->parent;
}
EXPORT_SYMBOL_GPL(dfl_dev_get_base_dev);

#define is_header_feature(feature) ((feature)->id == FEATURE_ID_FIU_HEADER)

/**
Expand Down
230 changes: 230 additions & 0 deletions drivers/fpga/intel-m10-bmc-retimer.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
// SPDX-License-Identifier: GPL-2.0
/* Intel Max10 BMC Retimer Interface Driver
*
* Copyright (C) 2018-2020 Intel Corporation. All rights reserved.
*
*/
#include <linux/device.h>
#include <linux/mfd/intel-m10-bmc.h>
#include <linux/module.h>
#include <linux/phy.h>
#include <linux/platform_device.h>

#define NUM_CHIP 2
#define MAX_LINK 4

#define BITS_MASK(nbits) ((1 << (nbits)) - 1)

#define N3000BMC_RETIMER_DEV_NAME "n3000bmc-retimer"
#define M10BMC_RETIMER_MII_NAME "m10bmc retimer mii"

struct m10bmc_retimer {
struct device *dev;
struct intel_m10bmc *m10bmc;
int num_devs;
struct device *base_dev;
struct mii_bus *retimer_mii_bus;
};

#define RETIMER_LINK_STAT_BIT(retimer_id, link_id) \
BIT(((retimer_id) << 2) + (link_id))

static u32 retimer_get_link(struct m10bmc_retimer *retimer, int index)
{
unsigned int val;

if (m10bmc_sys_read(retimer->m10bmc, PKVL_LINK_STATUS, &val)) {
dev_err(retimer->dev, "fail to read PKVL_LINK_STATUS\n");
return 0;
}

if (val & BIT(index))
return 1;

return 0;
}

static int m10bmc_retimer_phy_match(struct phy_device *phydev)
{
if (phydev->mdio.bus->name &&
!strcmp(phydev->mdio.bus->name, M10BMC_RETIMER_MII_NAME)) {
return 1;
}

return 0;
}

static int m10bmc_retimer_phy_probe(struct phy_device *phydev)
{
struct m10bmc_retimer *retimer = phydev->mdio.bus->priv;

phydev->priv = retimer;

return 0;
}

static void m10bmc_retimer_phy_remove(struct phy_device *phydev)
{
if (phydev->attached_dev)
phy_disconnect(phydev);
}

static int m10bmc_retimer_read_status(struct phy_device *phydev)
{
struct m10bmc_retimer *retimer = phydev->priv;

phydev->link = retimer_get_link(retimer, phydev->mdio.addr);

phydev->duplex = DUPLEX_FULL;

return 0;
}

static int m10bmc_retimer_get_features(struct phy_device *phydev)
{
linkmode_set_bit(ETHTOOL_LINK_MODE_10000baseT_Full_BIT,
phydev->supported);
linkmode_set_bit(ETHTOOL_LINK_MODE_10000baseSR_Full_BIT,
phydev->supported);
linkmode_set_bit(ETHTOOL_LINK_MODE_10000baseLR_Full_BIT,
phydev->supported);

linkmode_set_bit(ETHTOOL_LINK_MODE_25000baseCR_Full_BIT,
phydev->supported);
linkmode_set_bit(ETHTOOL_LINK_MODE_25000baseSR_Full_BIT,
phydev->supported);

linkmode_set_bit(ETHTOOL_LINK_MODE_40000baseCR4_Full_BIT,
phydev->supported);
linkmode_set_bit(ETHTOOL_LINK_MODE_40000baseSR4_Full_BIT,
phydev->supported);
linkmode_set_bit(ETHTOOL_LINK_MODE_40000baseLR4_Full_BIT,
phydev->supported);

linkmode_set_bit(ETHTOOL_LINK_MODE_Asym_Pause_BIT, phydev->supported);

return 0;
}

static struct phy_driver m10bmc_retimer_phy_driver = {
.phy_id = 0xffffffff,
.phy_id_mask = 0xffffffff,
.name = "m10bmc retimer PHY",
.match_phy_device = m10bmc_retimer_phy_match,
.probe = m10bmc_retimer_phy_probe,
.remove = m10bmc_retimer_phy_remove,
.read_status = m10bmc_retimer_read_status,
.get_features = m10bmc_retimer_get_features,
.read_mmd = genphy_read_mmd_unsupported,
.write_mmd = genphy_write_mmd_unsupported,
};

static int m10bmc_retimer_read(struct mii_bus *bus, int addr, int regnum)
{
struct m10bmc_retimer *retimer = bus->priv;

if (addr < retimer->num_devs &&
(regnum == MII_PHYSID1 || regnum == MII_PHYSID2))
return 0;

return 0xffff;
}

static int m10bmc_retimer_write(struct mii_bus *bus, int addr, int regnum, u16 val)
{
return 0;
}

static int m10bmc_retimer_mii_bus_init(struct m10bmc_retimer *retimer)
{
struct mii_bus *bus;
int ret;

bus = devm_mdiobus_alloc(retimer->dev);
if (!bus)
return -ENOMEM;

bus->priv = (void *)retimer;
bus->name = M10BMC_RETIMER_MII_NAME;
bus->read = m10bmc_retimer_read;
bus->write = m10bmc_retimer_write;
snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii",
dev_name(retimer->base_dev));
bus->parent = retimer->dev;
bus->phy_mask = ~(BITS_MASK(retimer->num_devs));

ret = mdiobus_register(bus);
if (ret)
return ret;

retimer->retimer_mii_bus = bus;

return 0;
}

static void m10bmc_retimer_mii_bus_uinit(struct m10bmc_retimer *retimer)
{
mdiobus_unregister(retimer->retimer_mii_bus);
}

static int intel_m10bmc_retimer_probe(struct platform_device *pdev)
{
struct intel_m10bmc_retimer_pdata *pdata = dev_get_platdata(&pdev->dev);
struct intel_m10bmc *m10bmc = dev_get_drvdata(pdev->dev.parent);
struct m10bmc_retimer *retimer;

retimer = devm_kzalloc(&pdev->dev, sizeof(*retimer), GFP_KERNEL);
if (!retimer)
return -ENOMEM;

dev_set_drvdata(&pdev->dev, retimer);

retimer->dev = &pdev->dev;
retimer->m10bmc = m10bmc;
retimer->base_dev = pdata->retimer_master;
retimer->num_devs = NUM_CHIP * MAX_LINK;

return m10bmc_retimer_mii_bus_init(retimer);
}

static int intel_m10bmc_retimer_remove(struct platform_device *pdev)
{
struct m10bmc_retimer *retimer = dev_get_drvdata(&pdev->dev);

m10bmc_retimer_mii_bus_uinit(retimer);

return 0;
}

static struct platform_driver intel_m10bmc_retimer_driver = {
.probe = intel_m10bmc_retimer_probe,
.remove = intel_m10bmc_retimer_remove,
.driver = {
.name = N3000BMC_RETIMER_DEV_NAME,
},
};

static int __init intel_m10bmc_retimer_init(void)
{
int ret;

ret = phy_driver_register(&m10bmc_retimer_phy_driver, THIS_MODULE);
if (ret)
return ret;

return platform_driver_register(&intel_m10bmc_retimer_driver);
}
module_init(intel_m10bmc_retimer_init);

static void __exit intel_m10bmc_retimer_exit(void)
{
platform_driver_unregister(&intel_m10bmc_retimer_driver);
phy_driver_unregister(&m10bmc_retimer_phy_driver);
}
module_exit(intel_m10bmc_retimer_exit);

MODULE_ALIAS("platform:" N3000BMC_RETIMER_DEV_NAME);
MODULE_AUTHOR("Intel Corporation");
MODULE_DESCRIPTION("Intel MAX 10 BMC retimer driver");
MODULE_LICENSE("GPL");
MODULE_IMPORT_NS(INTEL_M10_BMC_CORE);
19 changes: 18 additions & 1 deletion drivers/mfd/intel-m10-bmc-spi.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
*
* Copyright (C) 2018-2020 Intel Corporation. All rights reserved.
*/
#define DEBUG
#include <linux/bitfield.h>
#include <linux/dev_printk.h>
#include <linux/init.h>
Expand All @@ -24,6 +25,20 @@ static const struct regmap_access_table m10bmc_access_table = {
.n_yes_ranges = ARRAY_SIZE(m10bmc_regmap_range),
};

static void m10bmc_init_cells_platdata(struct intel_m10bmc_platdata *pdata,
struct mfd_cell *cells, int n_cell)
{
int i;

for (i = 0; i < n_cell; i++) {
if (!strcmp(cells[i].name, "n3000bmc-retimer")) {
cells[i].platform_data = pdata->retimer;
cells[i].pdata_size =
pdata->retimer ? sizeof(*pdata->retimer) : 0;
}
}
}

static struct regmap_config intel_m10bmc_regmap_config = {
.reg_bits = 32,
.val_bits = 32,
Expand Down Expand Up @@ -60,7 +75,8 @@ static int check_m10bmc_version(struct intel_m10bmc *ddata)

static int intel_m10_bmc_spi_probe(struct spi_device *spi)
{
const struct spi_device_id *id = spi_get_device_id(spi);
struct intel_m10bmc_platdata *pdata = dev_get_platdata(&spi->dev);
const struct spi_device_id *id = spi_get_device_id(spi);
const struct intel_m10bmc_platform_info *info;
struct device *dev = &spi->dev;
struct intel_m10bmc *ddata;
Expand All @@ -72,6 +88,7 @@ static int intel_m10_bmc_spi_probe(struct spi_device *spi)

info = (struct intel_m10bmc_platform_info *)id->driver_data;
ddata->dev = dev;
m10bmc_init_cells_platdata(pdata, info->cells, info->n_cells);

ddata->regmap = devm_regmap_init_spi_avmm(spi, &intel_m10bmc_regmap_config);
if (IS_ERR(ddata->regmap)) {
Expand Down
1 change: 1 addition & 0 deletions include/linux/dfl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,4 +92,5 @@ void dfl_driver_unregister(struct dfl_driver *dfl_drv);
dfl_driver_unregister)

void *dfh_find_param(struct dfl_device *dfl_dev, int param_id, size_t *pcount);
struct device *dfl_dev_get_base_dev(struct dfl_device *dfl_dev);
#endif /* __LINUX_DFL_H */
Loading