Signal to Noise Ratio

The first parameter obtained from the input signals is the signal to noise ratio (SNR). It was computed using the stnr tool from the NIST Speech Quality Assurance Package (SPQA) (NIST Speech tools and APIs, 2006) which is also used in the acoustic beamforming system evaluated in the experiments section. This program estimates the SNR of a file, defined as

$$10 log \frac{{\rm peak\_speech\_power}}{{\rm mean\_noise\_power}}$$

where power refers to the Root Mean Square (RMS) of the signal over a sliding window of 20ms, with a scrolling size of 10ms. A histogram is created using the RMS values and then the noise and speech values are computed.

To determine the noise average power, a raised cosine function is fitted to the peak in the left hand side of the histogram (lowest values) using a search algorithm to minimize the Chi-Square distance between the histogram and the function. The midpoint of such function is considered the mean noise power. Then the obtained raised cosine is subtracted from the histogram in order to estimate the speech power distribution. The peak speech power is defined as the histogram bin midpoint where the 95% of the power falls below it. Given that the speech power contains additive noise, the computed noise power is subtracted from it to use it in the SNR formula.

As speech and noise do not exist independently in the recorded signal, this method is only an approximation of the SNR. The result from this tool might not be comparable to the result from other tools, but according to the authors it is consistent to results using the same tool and therefore adequate to compare the quality difference of several signals as it is intended in this section. It must be noted that for a few cases this algorithm is known to give erroneous results, therefore it should be taken as an information source and the average should be taken to avoid misreadings.

In order to compute the SNR values, both for the meetings and for the broadcast news recording, only the regions determined to be part of the evaluation were considered. As pointed out before, some of the recordings contain more acoustic data than the evaluated region, which sometimes is excluded due to problems with the microphones (in meetings) or because it contains commercials or very noisy acoustics (in broadcast news).

First of all, the SNR is computed for the files in the RT04f data set. As it can be seen in table 3.4 the speech peak power remains constant at a very high value, with an average of 65db, while the noise average power is very variable and ranges from around 15db to around 62db. Such averages are taken over the SNR values in log domain, and aim at indicating the overall quality of the dataset.

Some of the shows contain news material where reporters give their chronicles from the field, with a high level of background noise, while the conductor is in the studio, with a very good quality microphone in a controlled environment. The shows that contain less or none of the field recordings achieve a very good SNR (around 50db) while others perform very poorly (for example the CNN headline news, ABC and CNBC shows).

Table 3.4: Estimated SNR for RT04f Broadcast News shows

Show_ID	Speech Peak Power	Noise ave. Power	SNR
20031202_050216_CNN_ENG	66.5	14.75	51.75
20031202_203013_CNBC_ENG	60	57.75	2.25
20031203_183814_ABC_ENG	66.75	60.75	6
20031204_130035_CNN_ENG	67.25	53.75	13.5
20031206_163852_CSPAN_ENG	61	30.5	30.5
20031209_193152_ABC_ENG	65	61.25	3.75
20031209_193946_PBS_ENG	63.25	20.75	42.5
20031215_204057_CNNHL_ENG	68.25	61.75	6.5
20031215_231058_WBN_ENG	67	16.5	50.5
20031217_184122_ABC_ENG	66.75	22.75	44
20031218_004126_PBS_ENG	65	55.5	9.5
20031219_202502_CNBC_ENG	62	32	30
average	64.89	40.66	24.22

In the meetings domain the SNR is computed separately for the conference and lecture room sets. On the conference room set each of the rooms contains a variable number of microphones, mostly separated into 2 groups: the microphones situated in the middle of the table (labelled MDM) and the head-mounted microphones, worn by some of the participants (labelled IHM). Although the speaker diarization system presented in this thesis does not analyze the IHM case, the SNR for these microphones is also computed for comparison purposes.

Tables 3.5 and 3.6 show average SNR for the MDM and IHM channels in the RT06s meetings in the conference room. In both tables the number of microphones available is indicated in the second column. Then, the third though fifth columns indicate the average (in the linear domain) of the SNR values for all channels in each meeting. As the variety of microphones causes them to have very diverse quality levels, the last two columns indicate the maximum and minimum SNR values to give an idea of how disperse these are. Finally, the averages (in the log domain, as done in the broadcast news results) are computed for all meetings.

The speech quality for all cases is approximately the same (around 65db). The noise level for the MDM channels is much higher than the Broadcast news channels, which causes a decrease in SNR of almost 5db. The Average noise level is lower for the IHM channels than for the MDM or the broadcast news shows which leads to an overall better SNR. This is due to the proximity of these microphones to the speakers and that a meeting room contains less noise than some broadcast news shows. It is interesting to point out the outstanding quality of the IHM channels used in the Edimburgh recordings (within the AMI project), but at the same time these meetings have some of the worse quality MDM microphones.

In overall, the MDM channels in the conference room are of less quality than the average in the broadcast news, but they remain more constant in quality across meetings.

Table 3.5: Estimated SNR for RT06s Conference Meetings, MDM channels

Show_ID	# channels	Ave. Speech	Ave. Noise	Ave.	Min.	Max.
		Power	Power	SNR	SNR	SNR
CMU_20050912-0900	2	67.95	49.22	18.52	18	18.75
CMU_20050914-0900	2	61.72	47.56	14.02	13.5	14.25
EDI_20050216-1051	16	56.06	39.45	18.30	7.25	19.5
EDI_20050218-0900	16	60.56	41.88	18.43	16.5	19
NIST_20051024-0930	7	73.07	56.41	25.15	15.75	26
NIST_20051102-1323	7	72.02	55.48	19.47	15.5	20.25
VT_20050623-1400	4	59.72	36.51	26.96	21.25	27.5
VT_20051027-1400	4	63.64	37.89	25.34	22.25	25.75
average		64.34	45.55	20.77	-	-

Table 3.6: Estimated SNR for RT06s Conference Meetings, IHM channels

Show_ID	# channels	Ave. Speech	Ave. Noise	Ave.	Min.	Max.
		Power	Power	SNR	SNR	SNR
CMU_20050912-0900	4	70.14	42.39	40.39	16	41
CMU_20050914-0900	4	72.89	40.89	46.39	32	47
EDI_20050216-1051	4	57.39	13.89	63.14	43.5	63.75
EDI_20050218-0900	4	60.89	23.39	61.39	33.5	62
NIST_20051024-0930	9	70.59	41.36	38.79	18.5	39.76
NIST_20051102-1323	8	68.91	41.79	39.59	16	40.5
VT_20050623-1400	5	68.24	42.30	36.06	25.5	36.75
VT_20051027-1400	4	58.19	25.40	41.12	31	41.75
average		65.91	33.93	45.86	-	-

Finally, table 3.7 shows the computed averages for the RT06s meetings in the lecture room dataset. In the same way as in 3.5, for each recording several distant microphones are available. The Average between microphones is done in the linear domain while the average over all recordings is done in the log domain.

Table 3.7: Estimated SNR for RT06s Lecture Room Meetings, MDM channels

Show_ID	#	Ave. Speech	Ave. Noise	Ave.	Min.	Max.
	channels	Power	Power	SNR	SNR	SNR
AIT_20051010/Segment1	4	51.15	23.14	27.90	24.25	28.5
AIT_20051011_B/Segment1	4	51.65	22.30	28.91	25.5	29.5
AIT_20051011_C/Segment1	4	48.22	22.81	26.66	22.25	27.25
AIT_20051011_D/Segment1	4	49.90	22.64	27.16	24	27.75
IBM_20050819/Segment1	3	58.84	46.75	12.09	9.75	12.5
IBM_20050822/Segment1	3	61.77	47.57	13.77	10	14.25
IBM_20050823/Segment1	3	59.03	48.02	13.02	9	13.5
IBM_20050830/Segment1	3	60.59	47.41	13.30	11	13.75
ITC_20050503/Segment1	5	59.25	42.78	16.85	14.75	17.5
ITC_20050607/Segment1	5	60.54	45.28	36.30	15	37
UKA_20050420_A/Segment1	5	77.55	58.55	22.30	13.25	23
UKA_20050420_A/Segment2	5	77.55	58.55	20.70	14	21.25
UKA_20050427_B/Segment1	5	81.30	64.05	18.30	13	19
UKA_20050504_A/Segment1	5	82.80	67.30	17.25	14.5	17.75
UKA_20050504_B/Segment1	5	82.80	69.05	13.63	13	14
UKA_20050504_B/Segment2	5	81.55	67.30	14.83	12.25	15.5
UKA_20050525_A/Segment1	5	82.55	67.30	15.49	6.25	16
UKA_20050525_A/Segment2	5	82.80	67.80	14.84	8.75	15.25
UKA_20050525_B/Segment1	5	78.05	67.80	10.07	4.25	10.5
UKA_20050525_C/Segment1	5	82.55	68.80	14.24	-2.5	14.5
UKA_20050615_A/Segment1	5	79.30	66.30	13.60	12.25	14.25
UKA_20050622_B/Segment1	5	67.64	59.33	11.30	8.25	12.0
UKA_20050622_C/Segment1	5	74.07	60.35	16.55	13.0	17.25
UKA_20050622_C/Segment2	5	75.06	61.31	15.31	12.25	16
UPC_20050706/Segment1	4	79.66	51.65	31.89	27.5	32.5
UPC_20050720/Segment1	4	71.15	56.40	16.65	6.25	17.25
UPC_20050722/Segment1	4	71.15	53.90	17.07	16.75	17.25
UPC_20050727/Segment1	4	67.90	50.42	17.18	16.75	17.5
average		69.87	53.03	18.47	-	-

Although all meeting recordings were done within the CHIL project, the specifications on the room layout and on the acoustic environment change within each lecture room. The speech average peak power changes immensely among the rooms (from around 50db on AIT recordings to around 80db on UKA decordings) remaining stable within the same lecture room. The same thing happens with the noise average power, which is the lowest for the AIT recordings and the highest for the UKA. This indicates that the recording settings were not set equally for all settings, being such difference possibly due solely to the amplification applied to the signal by the recording equipment.

Regarding the SNR over all the channels, the AIT recordings are constantly achieving SNR values on the twenties, while the other shows are usually on the tens, with a global average of 18.47, which is slightly lower in average than the meetings in the conference room subdomain. The differences between minimum and maximum SNR values remain in the same line as in 3.5.