MMSE Interference in Gaussian Channels 1

Transcription

1 MMSE Interference in Gaussian Channels Shlomo Shamai Department of Electrical Engineering Technion - Israel Institute of Technology 202 Information Theory and Applications Workshop 5-0 February, San Diego The talk is based on recent studies done jointly with Ronit Bustin Shlomo Shamai ITA, February 202 /26

2 The Framework The Scalar Additive Gaussian Channel Our framework is the scalar additive Gaussian channel: Y = snrx + N where N N (0, ). Through which we transmit length n codewords. Constraint We limit our investigation to power constrained codes: x C n n n x 2 i i= Shlomo Shamai ITA, February 202 2/26

3 The Framework The Scalar Additive Gaussian Channel Our framework is the scalar additive Gaussian channel: Y = snrx + N where N N (0, ). Through which we transmit length n codewords. Constraint We limit our investigation to power constrained codes: x C n n n x 2 i i= Shlomo Shamai ITA, February 202 2/26

4 Optimal Point-to-Point Codes Theorem [Peleg, Sanderovich and Shamai, ETT 2007] For every capacity achieving code-sequence, C n, over the Gaussian channel, the mutual information, when n, is as follows: and the MMSE is: I (X ; γx + N) = MMSE c (γ) = { 2 2 log( + γ), γ snr log( + snr), o/w { +γ, γ snr 0, o/w Shlomo Shamai ITA, February 202 3/26

5 Optimal Point-to-Point Codes - Cont. The mutual information (and MMSE) of optimal point-to-point codes follow the behavior of an i.i.d. Gaussian input up to snr. 0.9 I opt (γ) mutual information \ MMSE MMSE opt (γ) γ Shlomo Shamai ITA, February 202 4/26

6 What is the effect on an unintended receiver? X R Y? Z Assumption: the unintended receiver, Z, has smaller snr, that is, snr z < snr. How should we measure the effect (disturbance)? For optimal point-to-point codes both the mutual information and MMSE are completely known. But what about non-optimal code (that do not attain capacity)? Shlomo Shamai ITA, February 202 5/26

7 How to measure the disturbance? Bandemer and El Gamal, 20: measure the disturbance at the unintended receiver using the mutual information at Z. That is, assuming this mutual information is at most R d what is the maximum possible rate to the intended receiver, Y. In this work we measure the disturbance at the unintended receiver using the MMSE of the input, X, at Z. That is, β assuming the MMSE is constrained to be at most +βsnr z what is the maximum possible rate to the intended receiver, Y. Shlomo Shamai ITA, February 202 6/26

8 How to measure the disturbance? Bandemer and El Gamal, 20: measure the disturbance at the unintended receiver using the mutual information at Z. That is, assuming this mutual information is at most R d what is the maximum possible rate to the intended receiver, Y. In this work we measure the disturbance at the unintended receiver using the MMSE of the input, X, at Z. That is, β assuming the MMSE is constrained to be at most +βsnr z what is the maximum possible rate to the intended receiver, Y. Shlomo Shamai ITA, February 202 6/26

9 Definitions I n (γ) = I (X; Y(γ)) n I (γ) = lim n I (X; Y(γ)) n MMSE cn (γ) = n Tr(E X(γ)) MMSE c (γ) = lim n MMSEcn (γ) where E X (γ) is the MMSE matrix of estimating X from Y(γ) = γx + N. Shlomo Shamai ITA, February 202 7/26

10 The I-MMSE approach. The I-MMSE relationship [Guo, Shamai and Verdú, IT 2005] A fundamental relationship between the mutual information and the MMSE in the Gaussian channel: I n (γ) = 2 Taking the limit of n : I (γ) = 2 snr 0 snr 0 MMSE cn (γ)dγ MMSE c (γ)dγ Shlomo Shamai ITA, February 202 8/26

11 The I-MMSE approach 2. The single crossing point property The property was originally derived for the scalar case in [Guo, Wu, Shamai and Verdú, IT 20]. Several MIMO extensions are given in [Bustin, Payaró, Palomar and Shamai <arxiv > ]. We require the simplest extension. Define the following function for an arbitrary random vector X: q A (X, σ 2, γ) = σ2 + σ 2 γ Tr (A) Tr (AE X(γ)) where A is some n n general weighting matrix. Shlomo Shamai ITA, February 202 9/26

12 The I-MMSE approach 2. The single crossing point property The property was originally derived for the scalar case in [Guo, Wu, Shamai and Verdú, IT 20]. Several MIMO extensions are given in [Bustin, Payaró, Palomar and Shamai <arxiv > ]. We require the simplest extension. Define the following function for an arbitrary random vector X: q A (X, σ 2, γ) = σ2 + σ 2 γ Tr (A) Tr (AE X(γ)) where A is some n n general weighting matrix. Shlomo Shamai ITA, February 202 9/26

13 The I-MMSE approach - Cont. Theorem [Bustin, Payaró, Palomar and Shamai <arxiv > ] Let A S n + be a PSD matrix. Then, the function q A (X, σ 2, γ), has no nonnegative-to-negative zero crossings and, at most, a single negative-to-nonnegative zero crossing in the range γ [0, ). Moreover, let snr 0 [0, ) be that crossing point. Then, q A (X, σ 2, 0) 0. 2 q A (X, σ 2, γ) is a strictly increasing in γ [0, snr 0 ). 3 q A (X, σ 2, γ) 0 for all γ [snr 0, ). 4 lim γ q A (X, σ 2, γ) = 0. In this work we set A = I, the identity matrix: n q I(X, σ 2, γ) = σ2 + σ 2 γ n Tr (AE X(γ)) = mmse G (γ) MMSE cn (γ) where mmse G (γ) assumes an independent Gaussian input N (0, σ 2 ). Shlomo Shamai ITA, February 202 0/26

14 The I-MMSE approach - Cont. Theorem [Bustin, Payaró, Palomar and Shamai <arxiv > ] Let A S n + be a PSD matrix. Then, the function q A (X, σ 2, γ), has no nonnegative-to-negative zero crossings and, at most, a single negative-to-nonnegative zero crossing in the range γ [0, ). Moreover, let snr 0 [0, ) be that crossing point. Then, q A (X, σ 2, 0) 0. 2 q A (X, σ 2, γ) is a strictly increasing in γ [0, snr 0 ). 3 q A (X, σ 2, γ) 0 for all γ [snr 0, ). 4 lim γ q A (X, σ 2, γ) = 0. In this work we set A = I, the identity matrix: n q I(X, σ 2, γ) = σ2 + σ 2 γ n Tr (AE X(γ)) = mmse G (γ) MMSE cn (γ) where mmse G (γ) assumes an independent Gaussian input N (0, σ 2 ). Shlomo Shamai ITA, February 202 0/26

15 Superposition codes I (γ) and MMSE c (γ) - known exactly [Merhav, Guo and Shamai, IT 200]: A superposition codebook designed for (snr, snr 2 ) with the rate-splitting coefficient β <. I (γ) = 2 log ( + γ), if 0 γ < snr 2 log ( +snr +βsnr ) + 2 log ( + βγ), if snr γ snr 2 2 log ( +snr +βsnr ) + 2 log ( + βsnr 2), if snr 2 < γ MMSE c +γ, 0 γ < snr β (γ) = +βγ, snr γ snr 2 0, snr 2 < γ Shlomo Shamai ITA, February 202 /26

16 Superposition codes - Cont. Example: mutual information \ MMSE snr = 2 snr 2 = 2.5 β = 0.4 MMSE c I(γ) I opt (γ) MMSE opt γ Shlomo Shamai ITA, February 202 2/26 Student Version of MATLAB

17 Main Result Theorem Assuming snr < snr 2 the solution of the following optimization problem, max I (snr 2 ) s.t. MMSE c (snr ) β + βsnr for some β [0, ], is the following I (snr 2 ) = 2 log ( + βsnr 2) + ( ) + 2 log snr + βsnr and is attainable when using the optimal Gaussian superposition codebook designed for (snr, snr 2 ) with a rate-splitting coefficient β. Shlomo Shamai ITA, February 202 3/26

18 Proof Sketch Optimal Gaussian superposition codebook comply with the above MMSE constraint and attain the maximum rate. We need a tight upper bound on the rate. We prove an equivalent claim: assume a code of rate R c = 2 log ( + αsnr 2), designed for reliable transmission at snr 2, lower bound MMSE c (γ). Then specify for γ = snr. αsnr 2 γ The lower bound is trivially zero using the optimal Gaussian codebook designed for αsnr 2. This is equivalent to setting β = 0. γ αsnr 2 I (snr 2 ) I (γ) I (snr 2 ) 2 log ( + γ) = R c log ( + γ) 2 Shlomo Shamai ITA, February 202 4/26

19 Proof Sketch Optimal Gaussian superposition codebook comply with the above MMSE constraint and attain the maximum rate. We need a tight upper bound on the rate. We prove an equivalent claim: assume a code of rate R c = 2 log ( + αsnr 2), designed for reliable transmission at snr 2, lower bound MMSE c (γ). Then specify for γ = snr. αsnr 2 γ The lower bound is trivially zero using the optimal Gaussian codebook designed for αsnr 2. This is equivalent to setting β = 0. γ αsnr 2 I (snr 2 ) I (γ) I (snr 2 ) 2 log ( + γ) = R c log ( + γ) 2 Shlomo Shamai ITA, February 202 4/26

20 Proof Sketch Optimal Gaussian superposition codebook comply with the above MMSE constraint and attain the maximum rate. We need a tight upper bound on the rate. We prove an equivalent claim: assume a code of rate R c = 2 log ( + αsnr 2), designed for reliable transmission at snr 2, lower bound MMSE c (γ). Then specify for γ = snr. αsnr 2 γ The lower bound is trivially zero using the optimal Gaussian codebook designed for αsnr 2. This is equivalent to setting β = 0. γ αsnr 2 I (snr 2 ) I (γ) I (snr 2 ) 2 log ( + γ) = R c log ( + γ) 2 Shlomo Shamai ITA, February 202 4/26

21 Proof Sketch - Cont. Using the I-MMSE relationship: 2 snr2 γ MMSE c (τ) dτ 2 log ( + αsnr 2) log ( + γ) 2 Defining d through the following equality: 2 log ( + αsnr 2) 2 log ( + γ) = 2 log ( + dsnr 2) log ( + dγ) 2 we have: 2 snr2 γ MMSE c (τ) dτ 2 log ( + αsnr 2) log ( + γ) 2 = 2 log ( + dsnr 2) log ( + dγ) 2 = 2 snr2 γ mmse G (τ) dτ, X G N (0, d), i.i.d. Shlomo Shamai ITA, February 202 5/26

22 Proof Sketch - Cont. Using the I-MMSE relationship: 2 snr2 γ MMSE c (τ) dτ 2 log ( + αsnr 2) log ( + γ) 2 Defining d through the following equality: 2 log ( + αsnr 2) 2 log ( + γ) = 2 log ( + dsnr 2) log ( + dγ) 2 we have: 2 snr2 γ MMSE c (τ) dτ 2 log ( + αsnr 2) log ( + γ) 2 = 2 log ( + dsnr 2) log ( + dγ) 2 = 2 snr2 γ mmse G (τ) dτ, X G N (0, d), i.i.d. Shlomo Shamai ITA, February 202 5/26

23 Proof Sketch - Cont. Using the single crossing point property and the above inequality we can conclude: The single crossing point of mmse G (τ) and MMSE c (τ), if exists, will occur somewhere in the region (γ, ). Thus, we have the following lower bound: MMSE c (γ) d(γ) + d(γ)γ = αsnr 2 γ snr 2 γ + γ Specifically for γ = snr we obtain MMSE c (snr ) αsnr 2 snr snr 2 snr + snr. Deriving α as a function of the constraining β, and substituting it in R c = 2 log ( + αsnr 2) results with the superposition rate. Shlomo Shamai ITA, February 202 6/26

24 Proof Sketch - Cont. Using the single crossing point property and the above inequality we can conclude: The single crossing point of mmse G (τ) and MMSE c (τ), if exists, will occur somewhere in the region (γ, ). Thus, we have the following lower bound: MMSE c (γ) d(γ) + d(γ)γ = αsnr 2 γ snr 2 γ + γ Specifically for γ = snr we obtain MMSE c (snr ) αsnr 2 snr snr 2 snr + snr. Deriving α as a function of the constraining β, and substituting it in R c = 2 log ( + αsnr 2) results with the superposition rate. Shlomo Shamai ITA, February 202 6/26

25 The effect at other snrs Theorem 2 From the set of reliable codes of rate R c = 2 log ( + βsnr 2) + 2 log ( + snr + βsnr complying with the MMSE constraint at snr : ) MMSE c (snr ) β + βsnr the superposition codebook provides the minimum MMSE for all snrs. Shlomo Shamai ITA, February 202 7/26

26 Extension to two MMSE constraints Theorem 3 [Bustin and Shamai, IZS 202] Assuming snr 0 < snr < snr 2 the solution of, max I (snr 2 ) s.t. MMSE c (snr ) β + β snr, MMSE c (snr 0 ) β 0 + β 0 snr 0 for some positive β, β 0 such that β + β 0 and β < β 0, is I (snr 2 ) = ( 2 log ( + β snr 2 ) + β ) 0snr + snr 0 + β snr + β 0 snr 0 and is attainable when using the optimal three-layers Gaussian superposition codebook designed for (snr 0, snr, snr 2 ) with rate-splitting coefficients (β 0, β ). When β 0 < β the first constraint can be removed and we return to the case of a single constraint given in Theorem. Shlomo Shamai ITA, February 202 8/26

27 Proof Sketch Optimal Gaussian three-layers superposition codebook comply with the above MMSE constraints and attain the maximum rate. We need a tight upper bound on the rate. Using Theorem Considering only the constraint on MMSE c (snr 0 ) we obtain the following upper bound on the rate at snr : I (snr ) 2 log ( + β 0snr ) + ( ) + 2 log snr0 + β 0 snr 0 Shlomo Shamai ITA, February 202 9/26

28 Proof Sketch Optimal Gaussian three-layers superposition codebook comply with the above MMSE constraints and attain the maximum rate. We need a tight upper bound on the rate. Using Theorem Considering only the constraint on MMSE c (snr 0 ) we obtain the following upper bound on the rate at snr : I (snr ) 2 log ( + β 0snr ) + ( ) + 2 log snr0 + β 0 snr 0 Shlomo Shamai ITA, February 202 9/26

29 Proof Sketch - Cont. The I-MMSE approach I (snr 2 ) I (snr ) = 2 snr2 snr MMSE c (τ)dτ 2 snr2 snr mmse G (τ)dτ where X G N (0, β ) and i.i.d. This is valid since according to the constraint on MMSE c (snr ) we have MMSE c (snr ) β + β snr = mmse G (snr ) and according to the single crossing point property MMSE c (τ) mmse G (τ), τ snr Putting the two upper bounds together, we obtain the desired result. Shlomo Shamai ITA, February /26

30 Proof Sketch - Cont. The I-MMSE approach I (snr 2 ) I (snr ) = 2 snr2 snr MMSE c (τ)dτ 2 snr2 snr mmse G (τ)dτ where X G N (0, β ) and i.i.d. This is valid since according to the constraint on MMSE c (snr ) we have MMSE c (snr ) β + β snr = mmse G (snr ) and according to the single crossing point property MMSE c (τ) mmse G (τ), τ snr Putting the two upper bounds together, we obtain the desired result. Shlomo Shamai ITA, February /26

31 Mutual information disturbance: single constraint Theorem 4 [Bandemer and El Gamal, ISIT 20] Assuming snr < snr 2 the solution of the following optimization problem, max I n (snr 2 ) s.t. I n (snr ) 2 log ( + α snr ) for some α [0, ], is the following I n (snr 2 ) = 2 log ( + α snr 2 ). Equality is attained, for any n, by choosing X Gaussian with i.i.d. components of variance α. For n equality is also attained by a Gaussian codebook designed for snr 2 with limited power of α. Shlomo Shamai ITA, February 202 2/26

32 Mutual information disturbance: single constraint Alternative I-MMSE proof Since, 0 I n (snr ) 2 log ( + snr ) there exists an α [0, ] such that I n (snr ) = n log ( + α snr ). = MMSE cn (γ) and mmse G (γ) of X G N (0, α ) cross in [0, snr ]. Using the I-MMSE I n (snr 2 ) = 2 log ( + α snr ) + 2 log ( + α snr 2 ) snr2 snr MMSE cn (γ)dγ due to the single crossing point property which ensures MMSE cn (γ) mmse G (γ), γ [snr, ) Shlomo Shamai ITA, February /26

33 Mutual information disturbance: single constraint Alternative I-MMSE proof Since, 0 I n (snr ) 2 log ( + snr ) there exists an α [0, ] such that I n (snr ) = n log ( + α snr ). = MMSE cn (γ) and mmse G (γ) of X G N (0, α ) cross in [0, snr ]. Using the I-MMSE I n (snr 2 ) = 2 log ( + α snr ) + 2 log ( + α snr 2 ) snr2 snr MMSE cn (γ)dγ due to the single crossing point property which ensures MMSE cn (γ) mmse G (γ), γ [snr, ) Shlomo Shamai ITA, February /26

34 Mutual information disturbance: multiple constraints Theorem 5 Assuming snr < snr 2 < < snr K the solution of max I n (snr K ) s.t. i {,, K }, I n (snr i ) 2 log ( + α isnr i ) for some α i [0, ], is the following I n (snr K ) = 2 log ( + α lsnr K ) where α l, l {,, K }, is defined such that i {,, K } 2 log ( + α lsnr i ) 2 log ( + α isnr i ) The maximum rate is attained, for any n, by choosing X Gaussian with i.i.d. components of variance α l. For n equality is also attained by a Gaussian codebook designed for snr K with limited power of α l. Shlomo Shamai ITA, February /26

35 Summary and Outlook These results provide the engineering insight to the good performance of the Han and Kobayashi scheme on the interference channel. We show that the Han and Kobayashi scheme is optimal MMSE-wise. The results can be easily extended to K MMSE constraint [Bustin and Shamai, submitted ISIT 202]. The engineering advantage of the MMSE disturbance measure over the mutual information measure in the Gaussian channel are demonstrated. Single code I-MMSE tradeoff, bounded by the optimal superposition coding tradeoff [Bennatan, Shamai, Calderbank, <arxiv: v-200>]. Interesting challenges: optimization of I n (snr 2 ), under the MMSE cn (snr ) constraint when block-length n is finite. For n =, conjecture: the optimal X is discrete [Shamai, ISIT 20]. Shlomo Shamai ITA, February /26

36 Thank You! Shlomo Shamai ITA, February /26

37 MMSE interference in Gaussian Channels Ronit Bustin and Shlomo Shamai We consider the scalar Gaussian channel, and address the problem of maximizing the average mutual information of a power constraint n component (n ) input random vector at a given signal-to-noise ratio (snr), satisfying a minimum mean square error (MMSE) constraint at another lower snr value. We use the MMSE as an effective interference (disturbance) measure, motivated by interference networks, where codes are expected not only to optimize performance for the intended user but inflict minimum interference on other users. We show via the information-estimation relation, that superposition coding is optimal in this respect, providing further intuition to the effectiveness of the Han-Kobayashi coding strategy on the interference channel, and performance of bad codes. Moreover, the MMSE function of those codes, attaining the best rate at some snr, subjected to a prescribed MMSE demand at some other snr, is completely defined for all snr, and is the one obtained by the corresponding superposition codebooks. Extensions to two MMSE constraints, are discussed, and compared to the results for a mutual information disturbance measure. Some challenges for this class of interference problems will also be discussed. Shlomo Shamai ITA, February /26