Just got to know that. It made me thinking why is it enough to know only one response in the standard basis to derive response for any function if we should know responses for all basis vectors? Take for instance Fourier basis - we must know system responses to all sines and cosines to compute responses of functions, expanded in this basis. Why single delta function covers all sines?
Ok, delta function shows up as a constant in the Fourier space. It is a sum of all sines and cosines. But how knowledge of response to a sum of sines is equivalent to knowing response of every separate sine? Yet, it seems possble to recover individual complex exponentials from the entangled response.
I have found the reason. Time-Invariant (or translation-invariant, LTI) systems are represented by Toeplitz matrices, aka convolutions, which have complex exponentials as their eighenvectors. Diagonalized, nxn components of a matrix can be represented by only n eighenvalues when diagonalized.