Andy Reddig on I/O processing trends
Andy Reddig is the founder, CEO and CTO of TEK Microsystems Inc., a real-time I/O processing specialist serving signal processing applications in defense, intelligence and elsewhere. He recently discussed I/O processing trends with Open Architecture Review, along with interface evolution, the pluses and minuses of FPGAs, fabric protocols and other issues.
OAR: TEK Micro was an early supporter of the RACEway and RACE++ switched fabrics from Mercury Computer Systems within the VMEbus environment. Do you still support that legacy? How strong is that legacy among your customers?
Reddig: Surprisingly, we still get a fair amount of our business from RACEway. Some, as you might expect, is for things that were designed-in ten years ago that people are just making more of. But what’s even more surprising is that we are still getting new design activity with RACEway today. In some cases, it’s very risk averse customers who want to stick with what’s been working for them for years. In others, it’s customers who have a big investment in infrastructure and want to continue to reuse that. Maybe ten to twenty percent of our new designs have something to do with RACEway.
OAR: That’s pretty surprising.
Reddig: One design win we got recently was for a hybrid: a VXS card with a RACEway grafted onto it. The customer is hedging, working with RACEway to sort of look backwards and with VXS to look forwards. The FPGA on our board talks to the [VME] P0 connector for VXS and also to the RACEway connector [on part of VME P2]. Their migration plan is to use RACEway today in an architecture that lets them use the VXS interconnect if they need more bandwidth in the future. And they can make the change by just changing the firmware.
OAR: Can I assume that most previous VME/RACEway customers have now moved to VXS?
Reddig: We are seeing a pretty strong market in VXS from entirely new customers. We actually also deliver a lot of VXS boards to customers who are retrofitting an existing VME system and continue to use just VME, not VXS. They’re not driven by I/O off the card but rather by how much functionality they can put on the card. They’re just replacing an old VME card with a new VME/VXS card with much better performance. Frequently, though, they do want VXS as an option for the future.
OAR: It’s said that VXS is the evolutionary path forward from VME and VPX is the revolutionary path. Now that you’ve announced your intention to start offering VPX boards this year, do you expect VXS to just shrivel up and die as VPX becomes established?
Reddig. From the customers we talk to, it looks like there will be a long-term market for people continuing to upgrade VME in an evolutionary way, and we don’t plan to suspend activity in VXS for a long time. Nevertheless, VPX has some advantages–such as about twice the speed on the backplane–and as that ecosystem becomes more mature, for new designs that don’t have any legacy concerns, VPX is what people should use. But we’ll be offering both VXS and VPX for quite some time, I think.
OAR: Will your experience with VXS simplify the design and development of VPX boards?
Reddig: Certainly. If you look at one of our 6U VXS cards and imagine our 6U VPX card sitting next to it, there’s not going to be a lot of difference between them. It’s still FPGA based and still using the same modular I/O on the front, all the same software, all the same firmware, all the same system management. We are literally designing our 6U VPX products by taking the artwork for our VXS products and hacking off the [VMEbus] connectors and redoing that part. Our customers don’t want us to change anything on our cards. They pretty much want our 6U architecture except with VPX connectors.
OAR: Why is that?
Reddig: Our customers have the same issues that we do. Their biggest investment is in firmware and software, and if they’ve architected their design to base itself on our firmware and our “funnel” architecture [two FPGAs at the front end and one at the backend], their preference is not to have to change any of that. A large part of our value add is that if you migrate your application to a different card or form factor, everything that can possibly stay the same stays the same. Also, we spend a lot of time and energy on layering so that our firmware hides all the [hardware] details and protects user investment.
OAR: But what about the VMEbus on your VXS boards?
Reddig: Our current Virtex-2 and -5 [Xilinx FPGA] products don’t use VMEbus for anything, although they plug into a VMEbus chassis. We don’t even have a VME interface on those cards. We just use the VME [P1] connector for power and reset.
OAR: What about the P2 connector?
Reddig: We use P2 I/O for a bunch of random little things.
OAR: Such as?
Reddig: For A/D cards, for example, customers frequently need to have a PPS [pulse per second] input or a 10 MHz time input or something like that, so those things get done through a rear transition module on P2, and we hook up P2 to the FPGA.
OAR: So if you don’t have VME, what do you do for a control plane?
Reddig: We never really saw a lot of value to VME as the control plane, and the [interface] chip costs a lot of money and takes up a lot of space. But of course, we’re driven by customers, and if they say they want a VME control plane, we’ll give them a VME control plane.
What people tend to use to control our cards is Gigabit Ethernet. Our firmware and software make it really easy for an SBC to talk to the FPGAs on our cards over Gig E, which abstracts you completely from the bus.
Fairly early on in the old RACEway days, we had a PMC carrier with both VME and RACEway interfaces; actually, we still sell a fair number of them. What we found was that 95% of the customers just talked over RACEway, and VME was irrelevant. Ironically, though, some customers have now requested that we put a VME interface back in, so our next VXS card will have VME on it.
OAR: What do you think about that?
Reddig: There are customers migrating their applications who said they want to continue doing control operations through VME, so we have to accommodate them. Instead of putting a VME interface chip on, though, what we’re doing is putting a soft VME core in a small FPGA in the next card, so we can talk VME to people who want to talk VME, but we don’t make a big investment on the card in terms of power or space.
OAR: With the OpenVPX/VITA 65 activity, there are apparently now a select number of legal “profiles.” Do you have to decide which SBC companies’ profiles you want to interoperate with?
Reddig: That remains to be seen, but being FPGA based gives us an advantage in that we can be protocol agnostic and support multiple protocols just by changing firmware.
OAR: In terms of the high speed point-to-point (P2P) links on the VXS P0 connector, which protocol or protocols does Tek Micro support, and do you expect that to change?
Reddig: In our applications, it’s very common to use the P0 connector to do FPGA to FPGA links using Aurora [protocol from Xilinx], but customers have asked us about just about every protocol. We support Aurora, serial FPDP and Gigabit E.
OAR: Is there any appeal to PCI Express?
Reddig: We have done work with PCI Express, but there aren’t yet any PCI Express enabled VXS SBCs, so there isn’t anything to talk to over it yet. So far, that ecosystem hasn’t developed, though I expect it will. But if you’re talking from FPGA to FPGA, it’s much lower overhead to use Aurora as a high-speed spigot, and PCI Express doesn’t actually add a lot of value.
OAR: What about RapidIO?
Reddig: We’ve had customers ask us about Serial RapidIO, and we’ve confirmed that they can integrate a RapidIO core onto our card. But we’ve never had a customer ask us for it, and we don’t have any direct experience with it.
I think there’s a chance 10 Gbit Ethernet will supplant RapidIO, but I’m not sure how soon that will happen. It’ll happen a lot faster as soon as Xilinx puts a 10 Gbit E core on an FPGA, and I mean a hard core because a soft core uses up a lot of gates. I think RapidIO has a place these days, but it hasn’t been driven nearly as strongly as PCI Express, 10 Gig E, 40 Gig E and those sorts of things. So while it has a niche, it will be a declining niche over time.
OAR: What’s missing?
Reddig: The way people in defense electronics really get bang for the buck these days is by figuring out clever ways to leverage existing technologies whose performance and cost are being driven by other market forces. There is now a second-gen RapidIO spec, but how many people are making chips with it? How many processors? How many switches? It’s a huge investment to get a semiconductor manufacturer to make chips with the technology in them, and until you actually have processors and switches that make use of that [second-gen speedup], board vendors can’t take advantage it. How soon will they get there? Three years from now? By that time, 10 Gig E or 40 Gig E or whatever will be out there and faster.
OAR: Unlike VXS, VPX allows both 3U and 6U form factors. Do you have any plans for a 3U line of products?
Reddig: Yes, and 3U presents some interesting tradeoffs because it’s less space efficient than 6U in terms of the physical overhead of connectors, card guides, front panels and so forth. That makes it especially important to pick the right feature set, and there are some tradeoffs we’re working through. What’s the tradeoff between having one large vs two smaller FPGAs? How do you tradeoff FPGA density vs memory? Some applications need very deep memory buffers, some don’t need memory at all, and that makes it tough for board designers to come up with something that meets everybody’s needs.
OAR: How do you see mezzanine boards evolving over time? Will hybrid XMC boards, with both PMC and XMC connectors on them, be a short- or long-term phenomenon?
Reddig: In the early days of VITA 42, which defined the XMC mezzanine standard, I predicted that very few people would make a hybrid module, and now I find myself making one! The reason is that we had a set of functional requirements and found we could meet them by making one hybrid product rather than making two products, one with PMC and one with XMC, so we did it. But going forward, I think our products will be XMC only because SBCs will migrate away from hybrids, and PCI [for PMC] will become more and more problematic to implement for board vendors.
OAR: What’s the primary appeal of FPGAs in the markets you serve?
Reddig: If you’re size and power constrained, they give you bang for the buck that you couldn’t get with general-purpose processors. The flip side of that is that for a lot of application that are not so constrained, things move from things like VME, VXS and VPX to servers and blades, 1U things, because the cost effectiveness is ten times as good as anything you’re going to get in a traditional kind of rack-mounted configuration. But if you really need absolutely maximum performance within a given volume or power or both, FPGAs offer between a 5x and 10x performance capability than general-purpose process if you’re willing to take the time and effort to deal with the firmware and code up an FPGA.
OAR: Are there any tradeoffs to FPGAs?
Reddig: The axiomatic thing is that they’re much harder to program then general-purpose processors. There’s a dream of being able to take your C code and punch it through some kind of VHDL C compileresque type of thing, or being able to put your application into some graphical form and crunch it to come up with something about as efficient as a human being writing firmware. But we’re an order of magnitude away from that. You can take general-purpose C code, turn it into FPGA code and run it on an FPGA, but you wind up with an FPGA that’s about ten times faster running on code that’s about ten times slower, so the result is just about the same as having a general-purpose processor, only it’s a lot more expensive.
OAR: In what kinds of applications are FPGAs the most suitable approach?
Reddig: We see two classes of problems in embedded. One class is more or less fixed and, over time, what was a 20-slot chassis becomes a 10-slot chassis, which then becomes a five-slot chassis and so on, so the problem gets easier and easier to solve. The other class of problem is infinite, but there’s a certain size or power constraint, and you want to do more and more over time within the constraints—have more resolution or more signal bins, listen to more simultaneous channels or whatever. Here’s where FPGAs fit. If they can give you, say, 10x the performance in a particular size box, that’s worth almost any price to the customer. There’s a fair amount of pain and anguish involved in getting them programmed, but they solve problems that would not otherwise be solvable within the size or power constraints.
OAR: How do the FPGAs shape up powerwise?
Reddig: That’s a very interesting issue. Customers always ask how much power a board uses and with, for example, a PowerPC, that might be 62W if not much was going on and, say, 68W if the processor was very busy. But with FPGAs, it’s not so straightforward. If you fully load up an FPGA, it might max out at 40 to 50W, but for a different application, it might use only 20W.
OAR: What are the implications of that?
Reddig: It’s brought a very interesting change to systems engineering over the past three to five years. It’s no longer sufficient to just ask your vendors how much power a board uses and then figure out the chassis that’s going to cool it. You now have to think very hard about modeling how much power your application is going to use.
We’ve hit the point, particularly with the new Virtex-6, where we have the ability to put on the order of 300W of FPGA power burning capability on a 6U card, and if you fill up all the FPGA gates and work them very hard, there’s no way you’re going to cool that, at least not in an air-cooled environment. You have to spray cool it or liquid cool it. We have customers doing these kinds of things, but it’s a totally different world.
Five years ago, the challenge was fitting all the chips on the board, and the constraint was physical. Today, we can fit more chips on a board that you can easily cool, and power and thermal management is the constraint. Chassis design used to be a much simpler problem, but today, it’s become critical to work closely with customers to understand the implications of using FPGAs.