-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for Mellanox Innova-2 SmartNIC #1667
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great! I'm surprised it was so easy to port!
TARGET_PROJECT: firesim | ||
DESIGN: FireSim | ||
TARGET_CONFIG: FireSimRocketConfig | ||
PLATFORM_CONFIG: BaseXilinxAlveoConfig |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In #1669 this PLATFORM_CONFIG
is not shared between FPGA types. I think that is a right approach. Can you create a copy of it and rename it to something like BaseMellanoxInnova2Config
?
deploy_quintuplet: null | ||
platform_config_args: | ||
fpga_frequency: 60 | ||
build_strategy: TIMING |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is unsupported right? I would just put UNSUPPORTED
for now to make it clear this does nothing (other cfgs should do the same thing - i.e the U250 ones but we haven't done that).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a copy of the other AXI tieoff? Dedup?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't quite an exact copy, it's got a changed AXI bus frequency since my implementation of the memory controller has different memory timings.
|
||
file copy -force ${root_dir}/vivado_proj/firesim.runs/${impl_run}/design_1_wrapper.bit ${root_dir}/vivado_proj/firesim.bit | ||
|
||
write_cfgmem -force -format mcs -interface SPIx4 -size 1024 -loadbit "up 0x01002000 ${root_dir}/vivado_proj/firesim.bit" -verbose ${root_dir}/vivado_proj/firesim.mcs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like some of this doesn't match across implementation files. Can you just delete this file (there should only be the implementation_<version>.tcl
files anyways.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto on deleting this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you simlink all files that are shared with the U250 flow?
sim/make/driver.mk
Outdated
# Compile Driver | ||
$(mellanox_innova_2): export CXXFLAGS := $(CXXFLAGS) $(common_cxx_flags) $(DRIVER_CXXOPTS) \ | ||
-idirafter ${CONDA_PREFIX}/include -idirafter /usr/include | ||
$(mellanox_innova_2): export LDFLAGS := $(LDFLAGS) $(common_ld_flags) -Wl,-rpath='$$$$ORIGIN' \ | ||
-L${CONDA_PREFIX}/lib -Wl,-rpath-link=/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu | ||
|
||
$(mellanox_innova_2): $(header) $(DRIVER_CC) $(DRIVER_H) $(midas_cc) $(midas_h) | ||
mkdir -p $(OUTPUT_DIR)/build | ||
cp $(header) $(OUTPUT_DIR)/build/ | ||
$(MAKE) -C $(simif_dir) driver MAIN=$(PLATFORM) PLATFORM=$(PLATFORM) \ | ||
DRIVER_NAME=$(DESIGN) \ | ||
GEN_FILE_BASENAME=$(BASE_FILE_NAME) \ | ||
GEN_DIR=$(OUTPUT_DIR)/build \ | ||
OUT_DIR=$(OUTPUT_DIR) \ | ||
DRIVER="$(DRIVER_CC)" \ | ||
TOP_DIR=$(chipyard_dir) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tabbing is different from the rest of the file.
|
||
file copy -force ${root_dir}/vivado_proj/firesim.runs/${impl_run}/design_1_wrapper.bit ${root_dir}/vivado_proj/firesim.bit | ||
|
||
write_cfgmem -force -format mcs -interface SPIx8 -size 128 -loadbit "up 0x0 ${root_dir}/vivado_proj/firesim.bit" -verbose ${root_dir}/vivado_proj/firesim.bin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Call this MCS (just to make sure this should be format mcs
not bin
right)?
if self.BOARD_NAME == "mellanox_innova_2": | ||
local(f"cp {local_cl_dir}/vivado_proj/firesim*.mcs {tar_staging_path}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this needed? Shouldn't the MCS file be called firesim.mcs
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is needed because the mellanox uses two QSPI chips in parallel so you end up with a firesim_primary.mcs
and a firesim_secondary.mcs
. The VCU118 looks like it has a similar situation, but that code still expects that the first MCS file is firesim.mcs
, hence the bit of hack
Also if you could add a description to the PR that would be great (you can see examples in other PRs). |
Also if this can be rebased on the most recent main that would be great (there are a lot of flow improvements to the Alveo flow that apply here). |
…duplicated AXI Tieoff since mellanox is now using same DRAM Frequency as Alveo
Adds support for the Mellanox Innova-2 SmartNIC, an FPGA accelerated 25 Gb NIC which has become EOL and is now readily available on Ebay for ~140 USD. These cards contain an XCKU15P, a Xilinx UltraScale+ FPGA with 1143450 logic cells, around half of an Alveo U200 and 8 gigabytes of DDR4-2666.
Related PRs / Issues
None
UI / API Impact
Adds example mellanox config to example config_build_recipies.yaml
Verilog / AGFI Compatibility
None
Contributor Checklist
changelog:<topic>
label?ci:fpga-deploy
label?Please Backport
label?Reviewer Checklist (only modified by reviewer)
Note: to run CI on PRs from forks, comment
@Mergifyio copy main
and manage the change from the new PR.changelog:<topic>
label?