Engineering constraints of all kinds mean that chips with substantial compute capacity always have to be divided into blocks. Multicore designs appeared soon after deep learning processors themselves. How should multiple matrix cores be organized into a whole? Every approach I have seen follows straightforward engineering intuition: take some general-purpose NoC and network topology, fit it to the hardware, and leave the hard problems to software. There are only so many hardware structures to choose from; the basic idea is that getting everything connected counts as a victory.
I have always found this an inelegant part of chip design. My earlier work on fractal architectures was largely about wrapping up and hiding these problems that had been left to software. It was like stuffing a tangled mess into a cupboard and then claiming to have solved the problem.
As I continued developing the fractal theory and tried to prove some theorems, I came across Leslie Valiant’s Universal Circuit (UC). Its overhead at the gate level is staggering. To allow logic gates to be combined arbitrarily, the accompanying routing switches can take up tens of times as much space—a vivid illustration of the Turing tax. Because of this enormous overhead, the UC has remained a theoretical tool. Since it was proposed in 1976, it has been regarded as a circuit structure with no practical value, and consequently it is not widely known.
But a simple intuition told me that it really could be useful in deep learning. The defining idea of a deep learning processor is to use relatively large blocks of matrix computation as its atomic units. Alongside a collection of large systolic arrays, the overhead of a UC might be diluted to the point of being almost negligible. This would yield a new multicore interconnect with appealing properties: arbitrary, non-blocking connectivity for DAGs with bounded vertex counts and in-/out-degrees; static, deterministic behavior; no need for FIFOs or dynamic routing; and an asymptotically optimal routing structure.
This research began in 2022. Zhe Fan was the first person tasked with implementing it, but he was approaching the end of his PhD and under considerable pressure, and he was unable to finish the work. He is now an engineer at Cambricon. The project then lay dormant for quite some time, until Zhihao Yang completed it this year and it was published at MICRO 2026.
The basic assumption is this. Whatever set of computations we want to map simultaneously onto a large multicore architecture, each task’s computation flow graph within a round of collaboration should be a directed acyclic graph (DAG). Combining several independent DAGs still produces a DAG. So, if a graph structure can universally embed any DAG with bounded size and in-/out-degrees, it can provide the interconnect required by any such set of computations.
The paper implements an Edge-Universal Graph (EUG) with maximum in-degree and out-degree both equal to 2, using it in place of a NoC that requires dynamic routing. Any scattered set of idle cores on the chip can have computations assigned in topological order and be reorganized into a large virtual compute array. Compared with constrained interconnects such as Mesh and Torus, its non-blocking circuit switching avoids leaving unusable fragments because forwarding paths are missing. Compared with Crossbar and Butterfly, it needs only forward connections in topological order, eliminating reverse connections that serve no real purpose in the architecture and significantly reducing overhead. This is particularly effective for multi-tenancy, which was the scenario we targeted in writing the paper.
Here is an example with four vertices and maximum in-/out-degree 1: the EUG on the right requires only three 2:1 multiplexers. The actual implementation uses a sixteen-vertex EUG with maximum in-/out-degree 2.
The paper also applies the retiming lemma to analyze timing and determine where to insert registers. This breaks up long combinational paths while ensuring that, after different computation graphs are embedded, the additional interconnect latency does not disrupt their original data synchronization.
For the physical implementation, we place the compute cores along a Hilbert curve and rotate some of them to shorten wires within the same pipeline stage. The final design contains 16 cores, each comprising a 32×32 systolic array and its associated memory and control circuitry. After place-and-route with FreePDK45, the critical path is 2 ns in the inter-core network and 2.5 ns within a core, so the interconnect does not become the frequency bottleneck. Switching the network to a new configuration takes only 9 cycles. Running tasks do not have to pause as long as their communication requirements remain unchanged.
The entire inter-core network occupies 1.06 mm², equivalent to 6.44% of the combined area of the 16 compute cores. For comparison, SARA’s interconnect occupies 3.17 mm², while the evaluated Butterfly and Crossbar interconnects are both around 5.85 mm². These routing switches are essentially the same as those in the original UC structure. Simply replacing the individual logic gates they connect with entire matrix cores dilutes the overhead until it is no longer so daunting.
We perform cycle-accurate simulation using seven multitask workload sets from XRBench, testing the maximum frame-rate multiplier that can be sustained while at least 95% of tasks finish on time. FUSE’s geometric-mean throughput is about 11% higher than SARA’s. The utilization statistics are even more direct: aggregating across workload sets and QoS settings, all 16 cores are active in 93.6% of the cycles during which at least one core is working.
Following this line of thought, we could also consider further specialization for Mesh embeddings. Related research is already underway, and I hope we can publish it soon.
In the era of large models, little ideas like this are becoming less significant. Once models become large enough, how things are interconnected on-chip hardly seems to matter; everything runs up against the need for higher density, greater capacity, and wider memory access. We have surely entered the Dharma-ending age of computer architecture. Yet I still value this research a great deal. I simply hope more people will learn about this structure, because it is beautiful, and people deserve to know about it.
Published at MICRO 2026. [PDF]