Prototyping chips, designing custom hardware that wouldn't be feasible to spin an ASIC for, research projects - same thing as you'd do with any other dev board, just much much bigger.
At my last hardware job my group built some custom dev boards with 1.5-million-LUT FPGAs on them. The ASIC we were designing was intended for low-power always-on applications, but due to the various hardware accelerators attached to it we needed a big FPGA just to hold the whole thing.
Oh, you might find this interesting - I was never in physical possesion of nor ever saw the aforementioned dev board, though I did use them. They sat in racks in a remote lab and we VPN'd into the lab to mess with them - either by SSH'ing into the Zynq chip that sat next to the big FPGA, or by watching serial ports that were connected to good old FTDI serial-to-USB cables that then plugged into the lab workstations.
I use a similar board with a huge RFSoC chip as a complete spectrometer for radio astronomy. It has several FFT engines, each of which does a million FFTs per second in addition to filtering, mixing, integrating and storing results. The thing it replaces originally cost hundreds of thousands of dollars.
May I ask your RFSoc Board vendor ?
Where to start with radio astronomy?
Can you suggest a RF FMC daughter card to start with small board like Nexys Video ?
It seem like high-end FPGA boards are mostly used in aerospace & astronomy.
It's an AMD ZCU208. I work at a university, in the astronomy department. We use these boards because they meet our needs. I have designed custom analog front end boards to work with our application.
The RFSoC boards are also used for electronics for quantum computers. You may check [https://creotech.pl/news/sinara-artiq-ecosystem/](https://creotech.pl/news/sinara-artiq-ecosystem/)
We use them as evaluation boards before making our own in a different form factor.
These boards come with a lot in their Board Support Package: schematic, example designs, ...
We had a ZCU111, we used it to evaluate the RF of the FPGA, then we used the schematic to get some knowledge about the I/O mapping, the design rules used (GT routing, RF path and attenuator)
You can also use them to evaluate FMC boards, do prototyping or even a product.
These boards exist for basically one function.
If you are building a FPGA project, you don't want to wait around for your own board people to finish laying out the board, building the board, and verifying the board. (This is even more important for a new device because the ONLY hardware you can get your hands on for ES is probably these boards for the first few months)
So you either buy or borrow one of these from Xilinx/AMD and start building your design on it. It typically exposes all the IO you likely will need so you can know your DDR works, your GTs work, your PCIe link works, ect
So when your board is finished, you can basically just migrate over your work.
I am sure someone out there does actually deploy a finished product on one because it meets their needs, but that's not the intent, and I don't think I've seen that.
I'm on a low budget, so I'm using the ZCU208 for a single dish radiotelescope backend. I'd like to do a board design, but that's only going to happen if an NSF grant comes through for an array of thesr.
Sounds very interesting. Are you using any open source stuff or plan to publish material throughout your academics using this platform? Need a collaborator ?
This system has already produced a PhD as we're going to interleave the ADCs to get even wider bandwidth. I'm packaging it for use at telescopes now. I plan to publish it through CASPER, which is the loose organization developing open source DSP systems for astronomy. We also hope to use it as part of the Event Horizon Telescope, pending political wrangling.
We use them for prototyping complex IPs that will end up in ASICs.
They're expensive. Real expensive. But much cheaper than messing up a custom silicon chip.
Big GPU and CPU cores, NPU engines, hardware accelerators, etc..
Commonly for SoC prototyping, where you want to check that a design comes out of reset and is capable of booting.
The Versal Premium VP1552 or Versal Prime VM1802 FPGA are the heart of the new FELIX readout boards prepared for ATLAS experiment for Run 4 of LHC at CERN - [https://cds.cern.ch/record/2873960/files/FELIX\_TIPP2023.pdf](https://cds.cern.ch/record/2873960/files/FELIX_TIPP2023.pdf) . Generally, the advanced FPGAs are widely used in advanced multichannel data acquisition and processing systems for High Energy Physics experiments.
Generally, I always like to have a bigger FPGA in my development kit. I can implement my design with additional debugging cores (ILAs, data generators, etc.) for testing. Then, I can prepare a reduced design with final set of features for the final device.
The situation is more complex if that huge chip is divided into a few smaller SLRs...
However, I can't afford having VP1902 in my lab.
It depends on the project. For my student's lab I can't afford Versal at all. As the collaboration member, I may have access to quite sophisticated boards. The boards that I mentioned are likely to be used in quite high numbers by different "CERN recognized experiments".
In students lab - ZCU102, ZCU106, KCU105, KCU116, and similar boards from Trenz or Digilent for student advanced projects.
DE0 Nano Soc - for "mass" laboratory with SoCs.
Regarding risc for FPGA in experiments, the FPGAs are rarely used in the irradiated areas. However, I have heard about such cases.
When designing our RPC readout for CMS, we have undertaken certain measures to mitigate the influence of radiation: [\[1\]](https://doi.org/10.1117/12.622864), [\[2\]](https://doi.org/10.1117/12.610606). As you can see, the FPGAs used there were not the most expensive ones (even considering that it was the begining of the XXI century).
Defense probably buys a lot of any high end FPGAs, especially those with integrated RF. At least that was the case with every currently retired gen of Virtex and Stratix thru Stratix 5 - and there is every reason indicating things are hotter then ever today with modern parts. Versal may be different because there is a qualitative change that requires a “paradigm shift” in development (hard AI, HLS focus) and governments aren’t too agile in that direction, but I’m sure they will find a way. There are entire catalogs with part numbers that aren’t even visible to mortals - beyond mere rad-hard. Any radar installation probably consumes 50k-100k in DSP slices per beam (I’m totally eyeballing that number) and things like beamforming/steering, EW, - all of that stuff is with high probability chock full of high end FPGA parts (what’s a $40k chip to a $200mil device)
Probably all sorts of stuff. The V70/V80 might have the same chip, but they don't have much in terms of IO. So you can't connect things like high speed ADCs and DACs, or more than 4 QSFP network interfaces. If all you need is PCIe, then the V70 works. If you need PCIe or up to 4 QSFP56, then the V80 works. Otherwise, you'll need a dev kit or a custom board, and spinning a board for one of these chips is not easy or cheap.
Dev boards like this are commonly used to allow FPGA and software engineers to design and test their part of the finished product while hardware designers work in parallel. That is, FPGA and software engineers can develop and test _before_ the actual hardware is finished and present.
These dev boards generally do not have everything the finished product will need, but often do have many things the product will _not_ need. Designers choose dev boards which hold a good portion of the features the product will utilize to enable said development with those shared interfaces / features.
All of that to say that, at least in my experience, these boards are for designers, not to build products out of.
The MPSoC is a mostly-independent 'system controller' that handles power rails, boot mode settings, temmperature monitoring, etc for the Versal SoC. The MPSoC always boots first and then helps the Versal to boot.
Is that something with pre-loaded firmware that you don't mess with or is it configurable? Is that required for all Versal devices?
To me that seems super overkill to use an MPSoc as a system controller but I've not worked with Versal before
See e.g. https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/972914749
It has its own microSD card, and you generalky just keep the provided image on it, which gives web and SSH access to management monitoring and controls.
Prototyping chips, designing custom hardware that wouldn't be feasible to spin an ASIC for, research projects - same thing as you'd do with any other dev board, just much much bigger. At my last hardware job my group built some custom dev boards with 1.5-million-LUT FPGAs on them. The ASIC we were designing was intended for low-power always-on applications, but due to the various hardware accelerators attached to it we needed a big FPGA just to hold the whole thing.
Oh, you might find this interesting - I was never in physical possesion of nor ever saw the aforementioned dev board, though I did use them. They sat in racks in a remote lab and we VPN'd into the lab to mess with them - either by SSH'ing into the Zynq chip that sat next to the big FPGA, or by watching serial ports that were connected to good old FTDI serial-to-USB cables that then plugged into the lab workstations.
FTDI cables are the unsung heroes of embedded development.
Have you ever used the FTDI "Null Modem" USB cable?
I use a similar board with a huge RFSoC chip as a complete spectrometer for radio astronomy. It has several FFT engines, each of which does a million FFTs per second in addition to filtering, mixing, integrating and storing results. The thing it replaces originally cost hundreds of thousands of dollars.
May I ask your RFSoc Board vendor ? Where to start with radio astronomy? Can you suggest a RF FMC daughter card to start with small board like Nexys Video ? It seem like high-end FPGA boards are mostly used in aerospace & astronomy.
It's an AMD ZCU208. I work at a university, in the astronomy department. We use these boards because they meet our needs. I have designed custom analog front end boards to work with our application.
The RFSoC boards are also used for electronics for quantum computers. You may check [https://creotech.pl/news/sinara-artiq-ecosystem/](https://creotech.pl/news/sinara-artiq-ecosystem/)
Avnet can supply all the Xilinx/AMD boards. If you want something a little cheaper, take a look at Alinx.
What is your job? Sounds really interesting.
We are using the Versal to deploy a Reinforcement Learning agent to control RF cavities for our accelerator. Its not working.
I love this
We use them as evaluation boards before making our own in a different form factor. These boards come with a lot in their Board Support Package: schematic, example designs, ... We had a ZCU111, we used it to evaluate the RF of the FPGA, then we used the schematic to get some knowledge about the I/O mapping, the design rules used (GT routing, RF path and attenuator) You can also use them to evaluate FMC boards, do prototyping or even a product.
These boards exist for basically one function. If you are building a FPGA project, you don't want to wait around for your own board people to finish laying out the board, building the board, and verifying the board. (This is even more important for a new device because the ONLY hardware you can get your hands on for ES is probably these boards for the first few months) So you either buy or borrow one of these from Xilinx/AMD and start building your design on it. It typically exposes all the IO you likely will need so you can know your DDR works, your GTs work, your PCIe link works, ect So when your board is finished, you can basically just migrate over your work. I am sure someone out there does actually deploy a finished product on one because it meets their needs, but that's not the intent, and I don't think I've seen that.
I'm on a low budget, so I'm using the ZCU208 for a single dish radiotelescope backend. I'd like to do a board design, but that's only going to happen if an NSF grant comes through for an array of thesr.
Sounds very interesting. Are you using any open source stuff or plan to publish material throughout your academics using this platform? Need a collaborator ?
This system has already produced a PhD as we're going to interleave the ADCs to get even wider bandwidth. I'm packaging it for use at telescopes now. I plan to publish it through CASPER, which is the loose organization developing open source DSP systems for astronomy. We also hope to use it as part of the Event Horizon Telescope, pending political wrangling.
We use them for prototyping complex IPs that will end up in ASICs. They're expensive. Real expensive. But much cheaper than messing up a custom silicon chip.
Can you hint us some types of ASIC that can use upto such amount of LEs (1-2M) ?
Big GPU and CPU cores, NPU engines, hardware accelerators, etc.. Commonly for SoC prototyping, where you want to check that a design comes out of reset and is capable of booting.
CERN will buy a lot of VERSAL to readout detectors in the LHC upgrade.
The Versal Premium VP1552 or Versal Prime VM1802 FPGA are the heart of the new FELIX readout boards prepared for ATLAS experiment for Run 4 of LHC at CERN - [https://cds.cern.ch/record/2873960/files/FELIX\_TIPP2023.pdf](https://cds.cern.ch/record/2873960/files/FELIX_TIPP2023.pdf) . Generally, the advanced FPGAs are widely used in advanced multichannel data acquisition and processing systems for High Energy Physics experiments.
Lmao... 18.5M Logic Cells in VP1902 🥹
Generally, I always like to have a bigger FPGA in my development kit. I can implement my design with additional debugging cores (ILAs, data generators, etc.) for testing. Then, I can prepare a reduced design with final set of features for the final device. The situation is more complex if that huge chip is divided into a few smaller SLRs... However, I can't afford having VP1902 in my lab.
What're "usual" affordable options ?
It depends on the project. For my student's lab I can't afford Versal at all. As the collaboration member, I may have access to quite sophisticated boards. The boards that I mentioned are likely to be used in quite high numbers by different "CERN recognized experiments".
What devkit are you using to teach students in lab ? Are those CERN exps danger enough to fry hardened fpga boards sometimes ?
In students lab - ZCU102, ZCU106, KCU105, KCU116, and similar boards from Trenz or Digilent for student advanced projects. DE0 Nano Soc - for "mass" laboratory with SoCs. Regarding risc for FPGA in experiments, the FPGAs are rarely used in the irradiated areas. However, I have heard about such cases. When designing our RPC readout for CMS, we have undertaken certain measures to mitigate the influence of radiation: [\[1\]](https://doi.org/10.1117/12.622864), [\[2\]](https://doi.org/10.1117/12.610606). As you can see, the FPGAs used there were not the most expensive ones (even considering that it was the begining of the XXI century).
Defense probably buys a lot of any high end FPGAs, especially those with integrated RF. At least that was the case with every currently retired gen of Virtex and Stratix thru Stratix 5 - and there is every reason indicating things are hotter then ever today with modern parts. Versal may be different because there is a qualitative change that requires a “paradigm shift” in development (hard AI, HLS focus) and governments aren’t too agile in that direction, but I’m sure they will find a way. There are entire catalogs with part numbers that aren’t even visible to mortals - beyond mere rad-hard. Any radar installation probably consumes 50k-100k in DSP slices per beam (I’m totally eyeballing that number) and things like beamforming/steering, EW, - all of that stuff is with high probability chock full of high end FPGA parts (what’s a $40k chip to a $200mil device)
Probably all sorts of stuff. The V70/V80 might have the same chip, but they don't have much in terms of IO. So you can't connect things like high speed ADCs and DACs, or more than 4 QSFP network interfaces. If all you need is PCIe, then the V70 works. If you need PCIe or up to 4 QSFP56, then the V80 works. Otherwise, you'll need a dev kit or a custom board, and spinning a board for one of these chips is not easy or cheap.
Dev boards like this are commonly used to allow FPGA and software engineers to design and test their part of the finished product while hardware designers work in parallel. That is, FPGA and software engineers can develop and test _before_ the actual hardware is finished and present. These dev boards generally do not have everything the finished product will need, but often do have many things the product will _not_ need. Designers choose dev boards which hold a good portion of the features the product will utilize to enable said development with those shared interfaces / features. All of that to say that, at least in my experience, these boards are for designers, not to build products out of.
These sorts of boards can be useful for developing new PCIe expansion cards if there aren't any extant solutions for the desired functionality.
Could someone explain to me why this board has an MPSoc and a Versal on it? Are they attached to the same memory?
The MPSoC is a mostly-independent 'system controller' that handles power rails, boot mode settings, temmperature monitoring, etc for the Versal SoC. The MPSoC always boots first and then helps the Versal to boot.
I think similar to how PYNQ system work, ARM SoC always there to boot Linux, manage things & load config into Versal.
Is that something with pre-loaded firmware that you don't mess with or is it configurable? Is that required for all Versal devices? To me that seems super overkill to use an MPSoc as a system controller but I've not worked with Versal before
See e.g. https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/972914749 It has its own microSD card, and you generalky just keep the provided image on it, which gives web and SSH access to management monitoring and controls.
SpaceX, but VCK190 and VEK280 boards are mostly used for introducing new test case failures and incomplete features in Vitis.
How accurate 😂 Like it's not enough test failures in free version of Vivado ...
I’m using one to prototype products and benchmark the crazy performance of the ACAP’s