ShiftCNN shifts the channels based on the shift matrix(kernel)
I have a question here: the code seems to use a single fixed shift pattern, simply applying a fixed shift to each channel. No matter how many times this ShiftConv layer is used, the shifts are always the same (data in the same channel is always shifted to the same position)
Shift
This operation shifts different channels (the shift direction is fixed for data in the same channel)
Paper
To reduce the state space, we use a simple heuristic: divide the M channels evenly into DK2 groups, where each group of ⌊M/DK2 ⌋ channels adopts one shift
(Divide the M channels into groups of M/D^2; every M/D^2 channels share one shift direction)
I.e. and e.g. are both Latin abbreviations: i.e. stands for id est, meaning “that is”, and e.g. stands for exempli gratia, meaning “for example”
However, finding the optimal permutation, i.e., how to map each channel-m to a shift group, requires searching a combinatorially(in a combinatorial way) large search space.
we introduce a modification(restriction) to the shift operation that makes input and output invariant to channel order: We denote a shift operation with channel permutation π as Kπ(I don't fully understand this part; K denotes the channel permutation),
G ̃=Pπ2(Kπ(Pπ1(F)))
wherePπi are permutation operators and◦ denotes operator composition(I don't understand this: Pπ denotes a permutation operator, but Pπ(K)Pπ combined makes up the whole Shift operation)
Because shift is discrete, a point-wise convolution P1(F) is added to handle parameter updates, since weights are hard to update through a discrete operation; the shift matrix itself is not updated either
So long as the shift operation is sandwiched between two point-wise convolutions, different permutations of shifts are equivalent. Thus, we can choose an arbitrary permutation for the shift kernel, after fixing the number of channels for each shift direction.
(This says that once Shift is sandwiched between two PW convolutions, channel order no longer matters; in practice it certainly does have an effect)
This convolution comprises(contains) ..