[HW/SW] Cheshire integration - Linux on FPGA

[mp-17]

Add FPGA and bare-metal/OS Cheshire flow to Ara

Main PR: pulp-platform/cheshire#160

With this PR, we add FPGA support to Ara. The SoC and scripts used are from Cheshire, which instantiates Ara as a Bender submodule.

The relative paths in ${ARA_ROOT}/cheshire/sw get a meaning only when Ara gets instantiated by Cheshire.

Changelog

Fixed

• Fix vector slicing in operand requesters.

Added

• Add bare-metal FPGA flow through Cheshire.
• Add cva6-sdk submodule.
• Add Linux FPGA flow through Cheshire.

Changed

• Bump the artifact-related GitHub actions for the CI.

Checklist

• Automated tests pass
• Changelog updated
• Code style guideline is observed

[mrbilandi]

Hi mp-17,
have you implemented the MMU for ARA?
What is the current status of the work? Is it now capable of running Linux apps on RVV (ARA)?
Thanks

[mp-17]

Hwy @mrbilandi; Ara will soon be able to run applications on Linux. We plan to release PRs and instructions on how to do it using the Cheshire SoC.

[mrbilandi]

Hi @mp-17,
I have a VCU118 and am interested in participating, at least in evaluating the design on the FPGA. If you think I can do something helpful, please let me know.

[fulcrum34]

Hi @mp-17
I hope this is the right place to ask. I'm happy to suggestions otherwise.

I'm experiencing issues with FPGA build flow.
I tried multiple times to build the bitstream for vcu128 using all default config and flow. However, I am facing Vivado bitgen failure.

VIVADO bitgen fails over a timing violation.
See log:
bitgen-failed-log.txt
When I open the failed design in GUI, I see
vivado-error-messages.txt

So far, I tried this with fresh clones on VIVADO 2019.1, 2020.2, 2023.2.
(2023.2 however had other issues before starting synth such as : https://adaptivesupport.amd.com/s/question/0D52E00006iHjbgSAC/vivado-20211-abnormal-program-termination-11?language=en_US)

Any clues what I might be missing here?

For context,

• I following this guide: https://github.com/pulp-platform/ara/tree/mp/pulp-v1-os-fpga-backup-moimfeld/cheshire
• Cheshire Commit and Branch: pulp-platform/cheshire@dc2cbd1

Modifications I did:

ara/cheshire/Makefile

Line 17 in e0a2de9

VIVADO ?= 'vitis-2020.2 vivado'

Changed to:
VIVADO = vivado

where vivado path is set in my env. variable.

I had to remove this snippet

||  (vlen_t'(ara_req_d.stride) >= csr_vl_q)

from

if (|ara_req_d.stride[$bits(ara_req_d.stride)-1:$bits(csr_vl_q)] ||
                      (vlen_t'(ara_req_d.stride) >= csr_vl_q)) null_vslideup = 1'b1;

as a temporary fix to Illegal Cast operation issue Mentioned in : pulp-platform/cheshire#112 (comment)

Other than that, its all default and clean.

Any clues, pointers to the fixes are extremely appreciated. 😊

[mp-17]

Hi @fulcrum34,

Just as an easy check, do you have VIVADO 2022.1, by any chances?

[fulcrum34]

Hi @fulcrum34,

Just as an easy check, do you have VIVADO 2022.1, by any chances?

Hi @mp-17
Thanks for checking out. 2022.1 is not available to me unfortunately. I tried with 2022.2 and it fails the same way 2023.2 does. Will try luck with 2021.2 as well.

INFO: [Synth 8-226] default block is never used [/home/asus/Documents/ara-linux/original_cheshire/cheshire/.bender/git/checkouts/ara-2c7b103275a16c87/hardware/src/lane/simd_div.sv:170]
	Parameter WIDTH bound to: 64 - type: integer 
	Parameter STABLE_HANDSHAKE bound to: 1 - type: integer 
INFO: [Synth 8-226] default block is never used [/home/asus/Documents/ara-linux/original_cheshire/cheshire/.bender/git/checkouts/ara-2c7b103275a16c87/hardware/src/lane/vmfpu.sv:1428]
	Parameter NrLanes bound to: 32'b00000000000000000000000000000010 
	Parameter VLEN bound to: 32'b00000000000000000000100000000000 
	Parameter INPUT_WIDTH bound to: 32'b00000000000000000000000000001011 
Abnormal program termination (11)
Please check '/home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/build/vcu128.cheshire/hs_err_pid10481.log' for details
segfault in /opt/Xilinx/Vivado/2022.2/bin/unwrapped/lnx64.o/vivado -exec vivado -mode batch -log ../cheshire.vcu128.log -jou ../cheshire.vcu128.jou -source /home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/scripts/impl_sys.tcl -tclargs vcu128 cheshire /home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/build/vcu128.clkwiz/out.xci /home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/build/vcu128.vio/out.xci /home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/build/vcu128.ddr4/out.xci, exiting...
make[1]: *** [/home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/xilinx.mk:62: /home/asus/Documents/ara-linux/original_cheshire/cheshire/target/xilinx/out/cheshire.vcu128.bit] Error 139
make[1]: Leaving directory '/home/asus/Documents/ara-linux/original_cheshire/cheshire'
make: *** [Makefile:36: ara-chs-xilinx] Error 2

here is the hs_err_pid10481.log
hs_err_pid10481.log

And this happens with Any project I have with ARA present. Even the ara_soc.sv from ara/main
To make sure the tool is all right, I generated bitstream from the default flow of OpenHW/CVA6 and it was successful.

However, from the timing failure issue in the above comment, I noticed it originates from AXI. How do you manage the dependency conflicts in bender? Is there a guide or instructions about it?

Thank you very much 😊

[mp-17]

Hey @fulcrum34,

I start with bender since it can be a problem that propagates to the implementation.

What flow are you using? By issuing bender checkout, bender should checkout all the IP dependencies using the commit hashes written in the Bender.lock file.

This cannot create any conflicts by definition, and it should guarantee reproducibility.

If you use bender update instead, you are asking bender to create a set of commit hashes that satisfy the constraints written in the Bender.yml file. In the case of conflicts with the constraints, bender asks you to solve them manually. This means you are likely going into unexplored territory by choosing which IP versions will work together from now on when you will bender checkout again. Indeed, bender update updates the Bender.lock used by bender checkout.

Are you using bender update to change AXI version? If you use this flow, you only bender checkout.

[fulcrum34]

Hi @mp-17
Thank you very much for pointing in the right direction. Actually what made me use bender update after bender checkout was this error. I was wrong.

.
.
.
Checkout unbent (https://github.com/pulp-platform/unbent.git)
     Cloning unbent (https://github.com/pulp-platform/unbent.git)
error: Git command (Command { std: cd "/home/asus/Documents/ara-linux/cheshire/.bender/git/db/ara-55c77ee6cf862ad0" && 
"git" "clone" "/home/asus/Documents/ara-linux/cheshire/.bender/git/db/ara-55c77ee6cf862ad0" "/home/asus/Documents/ara- 
  linux/cheshire/.bender/git/checkouts/ara-2c7b103275a16c87" "--recursive" "--branch" "bender-tmp- 
  19985464b2d8801f21900c1f9d6d7dd356f92b8b", kill_on_drop: false }) in directory "/home/asus/Documents/ara- 
  linux/cheshire/.bender/git/db/ara-55c77ee6cf862ad0" failed with exit code 1:   

Cloning into '/home/asus/Documents/ara-linux/cheshire/.bender/git/checkouts/ara-2c7b103275a16c87'...
done.
Note: switching to '19985464b2d8801f21900c1f9d6d7dd356f92b8b'.

.
.
.

I spun up a quick GitHub Actions build to reproduce it here Clone Cheshire Repository - Line 374

Anyways, I ignored it and moved forward, got the linux image and bitsream (with vivado 2020.2).
Sorry for making the noise 🙂 and Thanks again for this amazing project.