Initial values

The recurrence $$f_n = a f_{n-1} + b f_{n-2}$$ has generating function $$f(x) = {f_0+x(f_1 - a f_0) \over 1-ax - bx^2} = \sum_0^\infty (C_1 \lambda_1 + C_2\lambda_2)x^n = \\ = {1\over \lambda_1 - \lambda_2} ({f_1-f_0\lambda_2 \over 1-\lambda_1 x} + {f_0\lambda_1 - f_1 \over 1-\lambda_2 x}) = {C_1+C_2+x(\lambda_1 C_2 + C_1 \lambda_2) \over 1-(\lambda_1+ \lambda_2)x - \lambda_1\lambda_2\ x^2}$$

which allows to relate initial values with eigendecomposition $$\begin{matrix}C_1 = {f_1-f_0\lambda_2\over\lambda_1-\lambda_2} & C_2 = {f_0\lambda_1 - f_1\over\lambda_1-\lambda_2} \\ f_0 = C_1 + C_2 & f_1 = C_1\lambda_1 + C_2\lambda_2\end{matrix}.$$ Eigencoordinaces $C_1$ and $C_2$ can be computed either this way or in the course of partial fraction decomposition.

Identifying initial values with inputs

We have just got that our linear system produces non-causal sequence (the output is 0 for n < 0). This sequence is also a solution of the recurrence. It is curious that initial values, $a_0$ and $a_1$ can be identified with inputs. Let's first define left and right shift: $$R(y) = R(y_0 + y_1 z + y_2 z^2 + \cdots) = 0 + y_0 z + y_1 z^2 + \cdots = a y,$$ $$L(y) = L(y_0 + y_1 z + y_2 z^2 + \cdots) = y_1 + y_2 z + y_3 z^2 + \cdots = (y-y_0)/a,$$ $$R(y)_n = y_{y-1}$$ $$L(y)_n = y_{y+1}$$ Therefore, $y_n = a y_{n-1} + x_n$ can be rewritten as z-transform $Y = y(z) = azy(z) + x(z) = X/(1-az)$ where x is input at time t and output, our sequence, is identically zero if no input is applied. Same result can be obtained from eq $y_{n+1} = a y_{n} + x_{n+1}$: $$(Y - y_0)/z = aY + (x(z)-x_0)/z \Rightarrow Y - y_0 = azY + (X - x_0) $$ $$\Rightarrow Y = {X + y_0-x_0 \over 1-az} = = {X + y_{-1} \over 1-az} = {X \over 1-az}.$$ The latter is because $y_{-1} = 0$ since we have causal sequences. For the second order system, $$y_n = a y_{n-1} + b y_{n-2}$$ we have $$y = a (zy + y_{-1}) + b (z^2y + zy_{-1} + y_{-2}) = {a y_{-1} + by_{-2} + b zy_{-1} \over 1 - az - bz^2} = {y_0 + bzy_{-1} \over 1 - az - bz^2} = 0$$ since $y_{-1} = y_{-1} = 0$ again and we again can add some input X to kick the sequence off zero. With $y_0 = x_0$ and $y_1 = a y_0 + x_1 \Rightarrow x_1 = y_1 - a y_0$ given, $$y = {x \over 1 - az - bz^2} = {x_0 + x_1z \over 1 - az - bz^2} = {y_0 + (y_1 - a y_0)z \over 1 - az - bz^2}.$$ $y_1 - a y_0 = b y_{-1},$ which we have before.

In general, equation $a_n y^n + a_{n-1} y^{n-1} + \cdots + a_0 y_0 = b_m x^m + b_{m-1} y^{m-1} + b_0 x_0$ translates into $Y = {\sum_0^m x_i z_i \over \sum_0^{n-1} a_i z_i} = X\cdot H$ where nominator X can be interpreted as either as input delta-pulses (in case of zero-state interpretation) or initial values (in case of zero-input interpretation).

See also What is the difference between zero-input and natural response.

Eigenvectors of recurrence equations

I have watched the Gilber Strang's great course. Lec 22 | MIT 18.06 Linear Algebra, explains everything but what the eigenvectors are. After some struggling I have figured out that companion matrix turns eighenvector [a b c d] into [λa λb λc λd]. Herein, λ is vector's the eigenvalue. Now, what are the [a, b, c, etc]? They are [λ³ λ² λ 1] because a element of the input vector turns into λa at the next iteration! Next time, λa turns into λ²a. This way, you must conclude that the train of values is the vector [λ³ λ² λ 1]. If we recall that it corresponds to [y₃  y₂  y₁  y₀], we'll understand that we start in y₀, next state variable y₁ = λy₀, y₂ = λ²y₀ and progress so on. But, we can factor the y₀ out of the eigenvector, [y₃  y₂  y₁  y₀] = y₀ [λ³ λ² λ 1] and, because all eigenvectors are identical up the scale factor, we can safely eliminate the y₀, yielding the eigenvector of  [λ³ λ² λ 1]. It is perfect that this vector is a geometric series (aka exponential function), which is an eigenfunction of any differential and difference equation.

Self-Reference

Edit: Begging a question says that "there is no fallacy if the missing premise is acknowledged, and if not, there is no circle". That is, A in A→A may be true or false, as I've said in what fallows.

I recently watched a lecture on Russel Paradox and got to know that self-referential definition is not "well founded". So that you define A based on B, A→B, then it is ok but A ↺A is not. I have also got to know that A ↺A is the simplest self-reference known as "Quine Atom". And, it seems to me that you cannot scratch your back with 0-length hand. This is a recurrence in space domain. But it is also mandatory in time -- you cannot read a value as soon as you produced it because infinite frequency implies uncertainty.

I was taught to implement the logic in digital circuits and optimize them. If you represent the Liar paradox by a circuit of logic gates, you'll get a basic oscillator. Start by optimizing

b = not a

a = b

is identical to

b = not b

What is b? If that would be a function b(x) and x was a known input value, i.e. ground voltage 0 or 1, then we could say what b(x) value is. If x would be a function itself, we could unroll it and reach the inputs. But, b refers to itself rather than staying on constants.

The quine atom, b = b

is the basic memory element, it feeds the input to its own output. This is known as positive feedback in saturation and just keeps its value rather than growing.

In the case b = not b, the inverse of output is submitted to the input. It means that output value  is inverted at every step.

If you do not lie, you lie and if you do lie, you do not. At which frequency? The feedback reaction speed is dependent of the physical length of the feedback. But, it also can be delayed and synchronized with something external. So what self-reference does is memory and evolution of the state in addition to simple combinatorial evolution of the function, given the arguments. It is a dynamical system. Self-reference also needs initial state specification and synchronization circuits. Otherwise it will have unpredictable result, oscillating at max speed. The shorter is the feedback the faster it will switch.

You cannot refer to yourself immediately. The paradox appears when you try. It is like asking what is position of pendulum oscillating at infinite frequency or much faster than you pick the answer. If you don't ask such stupid thing, you get (a slowly) evolving system. To heel yourself your hand must be should have quite powerful "service part" that can operate autonomously while your first part is in service. Self-repairing circuits are redundant. You cannot make the self-reference infinitely simple. You must have some matter for it. The snake bitying its tail can be shorter and shorter and shorter but you cannot make the loop of zero length infinitely short (to grasp yourself). Oh dear, what I'm talking here? Just wanted you to know about the logic and paradox modelling with logic gates.

Update I have got to know that Liar's paradox is also known as Liar's circle or just Circuit. So, my idea to draw circuits was right and the most insightful. See also Circular definition and infinite regress.

Moreover, It is cohesitivist approach to truth so state that a thing is true if it is not self-contradictory. Therefore, inverter-based generator is liar but trush-teller is true.

Somebody refers Norbert Weiner "A machine to answer this question would give the successive temporary answers: 'yes,' 'no,' 'yes,' 'no,' and so on, and would never come to equilibrium"

Green's function = Impulse Response

Just got to know that. It made me thinking why is it enough to know only one response in the standard basis to derive response for any function if we should know responses for all basis vectors? Take for instance Fourier basis - we must know system responses to all sines and cosines to compute responses of functions, expanded in this basis. Why single delta function covers all sines?

Ok, delta function shows up as a constant in the Fourier space. It is a sum of all sines and cosines. But how knowledge of response to a sum of sines is equivalent to knowing response of every separate sine? Yet, it seems possble to recover individual complex exponentials from the entangled response.

I have found the reason. Time-Invariant (or translation-invariant, LTI) systems are represented by Toeplitz matrices, aka convolutions, which have complex exponentials as their eighenvectors. Diagonalized, nxn components of a matrix can be represented by only n eighenvalues when diagonalized.

Why nth order differential eq can be represented by a sisytem of n equations of the first order?

For the same reason as an n bit vector can be represented as a sum of n 1-bit elements. Every order in diff eq is a memory element.

BTW, derivative operator maps  (k+1)-dimensional vector into k-dimensional space. That is why inverse requires one boundary condition to resolve uncertainty.

Why weighted mean works?

Just an illustration. Consider a population, a fraction of which favors somthing. We take two samples of size (aka weight) m1 and m2 with favor fractions of p₁ and p₂ correspondingly. These fractions are statistic probabilities p_n derived by dividing the number supporters f_n by total number m_n: p_n = f_n/m_n. With the higher sample size, the statistic must converge to the true fraction of the population. In the two polls, different number of people were sampled (m₁ != m₂) and weighting is used to combine them rather than simple averaging. In the combined poll, the level of support is p = f/m = (f₁+f₂) / (m₁+m₂) = (p₁m₁ + p₂m₂) / (m₁+m₂). The p_nm_n product is a number of supporters in each distinct poll.

As follows from the latter, the weighting naturally occures in combination of the results, whereby, the weighted sum is nothing more than the number of supporters in the aggregate sample size. In fact, any group result aggregates individual ones and, thus, involves binary weighting: a group consists of m sampled individuals, each with weight 1 and vote of either 0 (zero support) or 1 (full support). Summing the ones (supporters) we do weighted summation (vector dot product).

An agreement can also be seen when aggregating two series resistances: i=u1/r1=u2/r2=(u1+u2)/(r1+r2). The last equation u1/r1=(u1+u2)/(r1+r2) can be derived from the first u1/r1=u2/r2. Ok, the fraction (i) can be > 1 in this case :)

A Question on Switching Power Math

A switching converter/amplifier is a system with an inductor and capacitor and, thus, is described by a dynamic system with a harmonic oscillation solution. Designers, however, use a simplified, straight line approximations, like (for Boost):

ΔI = (V_in-Vout)/L × ΔT_onΔI = Vout/L × ΔToff

The linear approximation is used in calculus for infinitesimal increments. Yet, the system switching time is quite finite. The second thing that bothers is that Vout changes as much as the the current. Why only current delta is accounted?

Because the high-order non-linearities are not involved, ΔT is small enough. The deltas are used when they are small in comparison with the value itself. Therefore, ΔV does not affect the current delta. Actually, it is accounting the deltas that arises the non-linearities.

Why nobody uses ANNs for audio amplifiers?

The amplification is a simple control task. I think that it is not an easy task to find an optimal control function for a d-class switch. Especially, because the task is complicated by unknown output stage characteristics, which vary from circuit to circuit.

A 100 mhz ANN running on an FPGA could easily control a 1 MHz gate, IMO. It could also monitor and take the efficiency into account (because the best quality is at infinite frequency, where the efficiency is zero) by comparing the input and load currents. Is it an overkill for such a simple task? Do existing d-amplifiers provide much better price-quality relationship?

1/u = du/dt -- capacitor constant power discharge

Yesterday, I have discovered the magnitudes which change at their value speed: v=v′. Today, playing with capacitor discharging equations, a solution of 1/u=u′ has surprisingly came.

There are three types of discharge I have analyzed (plots to be added):

1. Via a constant resistor: U_r(t) = U₀*e^-t/RC or dU/dt = - kU (minus is because of discharge)


2. Constant current: U_i(t) = U₀ - I*t/C or dU/dt = - k


3. Constant power: ? or dU/dt = - k/U

The latest is not known to the general public (like me). It is the fastest discharge because the constant resistor in (1) drops both capacitor voltage and discharge current current (slope becomes less vertical), while the constant current discharge (2) reduces the voltage and thus the power, the energy extracted from the capacitor per time unit, P = U*I, drops too.

How do we compute the U_p(t)? The energy comes down at const power rate P:
E(t) = E₀ - Pt
The capacitor's Energy is defined by its U: E_cap = 1/2 CU². So,
1/2 CU² = 1/2 CU₀² - Pt
Moving 1/2 C to the left and taking the root we derive the U(t) sought:
U_p(t) = √‾(U₀² - 2Pt/C)‾

Initially, I have made a bad U(t) plot here drawing y=y₀ -√‾x rather than y=√‾(y₀ -x). This turned me to think about the current. The first nuance I have mentioned is that voltage must start discharging slowly and speed up to I= ∞ when U drops to zero. I knew that the current is the speed of voltage change I = -CU′. At the same time, the power formula is P = UI, where P is constant. Therefore, the differential equation that naturally comes:
I = P/U = -CU′ or U′ = -P/C * U

Suddenly, we have got u(t)=sqrt(U0² + 2kt) as a u′ = k/u diff equation solution. This extends my considerations of capacitor (dis-) charging at electronix.ru.

Philosophic conclusions: we have discovered how i connects sin/cos (mutual growth) with exp (self-growth) and sqrt (inverse growth).

exp and sin/cos similarity

It is known to exist deep. They can be expressed on through another. Harmonic oscillation degenerates into exp attenuation. It seems, I have finally comprehended it.

Any two magnitudes are related as sin and cos if the one is a growth of another. The exponent is a speed of change of itself.

In other words, I believe that a=sint and b=cost follows from the system
(1) a′(t) = b(t)
(2) b′(t) = k * a(t)

like f = e^x follows from the f(t)′ = f(t).


The angle Phi in exp(i*phi) signifies the balance between self-dependence and interdependence. Moving from zero (e^0) to in the imaginary cycle we shift to two-variables (e^i=sin) system and moving further we return to single-variable (e^(i*i)=e^-1)

Disclaimer: I'm a profane in the math. The philosophy must be taken as a prompt to study it.