In the 2017 'Attention Is All You Need' transformer architecture, each column from bottom to top is a pure feed-forward network with no search, iteration or conventional algorithm.

This is judged against the architecture described in Vaswani et al. 2017. True if, in a single forward pass for one position, the computation runs from the bottom layer to the top through a fixed stack of attention and feed-forward layers, with no recurrence, iterative loop, search or conventional algorithm inside the pass; 'feed-forward' is read in the network sense of an acyclic computation. Mostly true if this holds but the description misstates a minor detail, for example by ignoring that attention mixes information across positions, so the 'column' is not isolated. Mixed if the architecture involves real iteration that substantially contradicts the claim, such as autoregressive decoding loops being treated as part of the column. Mostly false or false if the paper's architecture includes recurrence, search or algorithmic iteration within a layer stack's forward pass.

Set before the research began.

1. E1 Attention is All you Need NeurIPS Proceedings (NIPS 2017) · 2017

   “Given z, the decoder then generates an output sequence (y1, ..., ym) of symbols one element at a time. At each step the model is auto-regressive [9], consuming the previously generated symbols as additional input when generating the next.”

2. E2 Attention is All you Need NeurIPS Proceedings (NIPS 2017) · 2017

   “Each position in the encoder can attend to all positions in the previous layer of the encoder.”

3. E3 Attention is All you Need NeurIPS Proceedings (NIPS 2017) · 2017

   “In addition to attention sub-layers, each of the layers in our encoder and decoder contains a fully connected feed-forward network, which is applied to each position separately and identically.”

4. E4 arxiv-1706.03762.md attention-wiki knowledge-base (Hugging Face datasets; publisher not otherwise identified on page)

   “An encoder maps an input sequence to continuous representations; a decoder generates the output autoregressively.”

E3 describes the encoder and decoder as stacks of layers, each with attention sub-layers and a position-wise feed-forward network. This matches a fixed bottom-to-top computation with no search or conventional algorithm inside a single pass. E2, however, shows that self-attention lets each position draw on every position in the previous layer, so a 'column' is not isolated. The criteria treat that as a minor misstatement. The autoregressive decoding in E1 and E4 is a loop across output steps, not iteration within one forward pass through the stack, so it does not contradict the claim about each column.