Asynchronous microphone arrays calibration and sound source tracking

Abstract

In this paper, we proposed an optimisation method to solve the problem of sound source localisation and calibration of an asynchronous microphone array. This method is based on the graph-based formulation of the simultaneous localisation and mapping problem. In this formulation, a moving sound source is considered to be observed from a static microphone array. Traditional approaches for sound source localisation rely on the well-known geometrical information of the array and synchronous readings of the audio signals. Recent work relaxed these two requirements by estimating the temporal offset between pair of microphones based on the assumption that the clock timing of each microphone is exactly the same. This assumption requires the sound cards to be identically manufactured, which in practice is not possible to achieve. Hereby an approach is proposed to jointly estimate the array geometrical information, time offset and clock difference/drift rate of each microphone together with the location of a moving sound source. In addition, an observability analysis of the system is performed to investigate the most suitable configuration for sound source localisation. Simulation and experimental results are presented, which prove the effectiveness of the proposed methodology.

Appendices

Jacobian of the observation model of a 2D microphone array

In this appendix, Jacobian matrix $J_{mic\_n}^{p-l}$, $J_{k\_x}^{p-l}$ and $J_{k\_y}^{p-l}$ of the observation model of a 2D microphone array are formulated.

Jacobian matrix $J_{mic\_n}^{p-l}$ in Eq. (19) is only nonzero at row n and this nonzero row is computed as

$$\begin{aligned} \begin{aligned}&J_{mic\_n}^{p-l}(n,:)\\&=\begin{bmatrix} \dfrac{x_{mic\_n}^x - x_{src\_k}^x}{c\sqrt{(x_{mic\_n}^x - x_{src\_k}^x)^2 + (x_{mic\_n}^y - x_{src\_k}^y)^2}}\\ \dfrac{x_{mic\_n}^y - x_{src\_k}^y}{c\sqrt{(x_{mic\_n}^x - x_{src\_k}^x)^2 + (x_{mic\_n}^y - x_{src\_k}^y)^2}}\\ 1\\ k{\varDelta }t \end{bmatrix}^{T}. \end{aligned} \end{aligned}$$

(32)

Jacobian matrix $J_{k\_x}^{p-l}$ and $J_{k\_y}^{p-l}$ are computed as

$$\begin{aligned} J_{k\_x}^{p-l}= & {} \begin{bmatrix} \dfrac{x_{src\_k}^x - x_{mic\_2}^x}{c\sqrt{(x_{mic\_2}^x - x_{src\_k}^x)^2 + (x_{mic\_2}^y - x_{src\_k}^y)^2}} \\ \vdots \\ \dfrac{x_{src\_k}^x - x_{mic\_N}^x}{c\sqrt{(x_{mic\_N}^x - x_{src\_k}^x)^2 + (x_{mic\_N}^y - x_{src\_k}^y)^2}} \end{bmatrix}\nonumber \\&-\begin{bmatrix} \dfrac{x_{src\_k}^x}{c\sqrt{{x_{src\_k}^x}^2 + {x_{src\_k}^y}^2}}\\ \vdots \\ \dfrac{x_{src\_k}^x}{c\sqrt{{x_{src\_k}^x}^2 + {x_{src\_k}^y}^2}}\\ \end{bmatrix}, \end{aligned}$$

(33)

$$\begin{aligned} J_{k\_y}^{p-l}= & {} \begin{bmatrix} \dfrac{x_{src\_k}^y - x_{mic\_2}^y}{c\sqrt{(x_{mic\_2}^x - x_{src\_k}^x)^2 + (x_{mic\_2}^y - x_{src\_k}^y)^2}} \\ \vdots \\ \dfrac{x_{src\_k}^y - x_{mic\_N}^y}{c\sqrt{(x_{mic\_N}^x - x_{src\_k}^x)^2 + (x_{mic\_N}^y - x_{src\_k}^y)^2}} \end{bmatrix}\nonumber \\&-\begin{bmatrix} \dfrac{x_{src\_k}^y}{c\sqrt{{x_{src\_k}^x}^2 + {x_{src\_k}^y}^2}}\\ \vdots \\ \dfrac{x_{src\_k}^y}{c\sqrt{{x_{src\_k}^x}^2 + {x_{src\_k}^y}^2}}\\ \end{bmatrix}. \end{aligned}$$

(34)

Jacobian of the observation model of a 3D microphone array

In this appendix, Jacobian matrix $J_{mic\_n}^{p-l}$, $J_{k\_x}^{p-l}$, $J_{k\_y}^{p-l}$ and $J_{k\_z}^{p-l}$ of the observation model of a 3D microphone array are formulated.

For a 3D microphone array, $J_{mic\_n}^{p-l}$ is Eq. (32) is rewritten as

$$\begin{aligned} \begin{aligned} J_{mic\_n}^{p-l}(n,:)= \begin{bmatrix} \dfrac{x_{mic\_2}^x - x_{src\_k}^x}{c{d}_{n,k}}\\ \dfrac{x_{mic\_2}^y - x_{src\_k}^y}{c{d}_{n,k}}\\ \dfrac{x_{mic\_2}^z - x_{src\_k}^z}{c{d}_{n,k}}\\ 1\\ k{\varDelta }t \end{bmatrix}^{T}. \end{aligned} \end{aligned}$$

(35)

$J_{k\_x}^{p-l}$, $J_{k\_y}^{p-l}$ and $J_{k\_z}^{p-l}$ in Eq. 20 can be formulated as follows,

$$\begin{aligned}&\begin{aligned} J_{k\_x}^{p-l}=\begin{bmatrix} \dfrac{x_{src\_k}^x - x_{mic\_2}^x}{c{d}_{n,k}} \\ \vdots \\ \dfrac{x_{src\_k}^x - x_{mic\_N}^x}{c{d}_{n,k}} \end{bmatrix} -\begin{bmatrix} \dfrac{x_{src\_k}^x}{c{d_k}}\\ \vdots \\ \dfrac{x_{src\_k}^x}{c{d_k}}\\ \end{bmatrix}, \end{aligned} \end{aligned}$$

(36)

$$\begin{aligned}&\begin{aligned} J_{k\_y}^{p-l}=\begin{bmatrix} \dfrac{x_{src\_k}^y - x_{mic\_2}^y}{c{d}_{n,k}} \\ \vdots \\ \dfrac{x_{src\_k}^y - x_{mic\_N}^y}{c{d}_{n,k}} \end{bmatrix} -\begin{bmatrix} \dfrac{x_{src\_k}^y}{c{d_k}}\\ \vdots \\ \dfrac{x_{src\_k}^y}{c{d_k}}\\ \end{bmatrix}, \end{aligned} \end{aligned}$$

(37)

$$\begin{aligned}&\begin{aligned} J_{k\_z}^{p-l}=\begin{bmatrix} \dfrac{x_{src\_k}^z - x_{mic\_2}^z}{c{d}_{n,k}} \\ \vdots \\ \dfrac{x_{src\_k}^z - x_{mic\_N}^z}{c{d}_{n,k}} \end{bmatrix} -\begin{bmatrix} \dfrac{x_{src\_k}^z}{c{d_k}}\\ \vdots \\ \dfrac{x_{src\_k}^z}{c{d_k}}\\ \end{bmatrix}, \end{aligned} \end{aligned}$$

(38)

where ${d_k}$ is the distance from the sound source position at the kth time instance to the origin of the global coordinate frame, which is formulated as follows,

$$\begin{aligned} d_k = \sqrt{{x_{src\_k}^x}^2 + {x_{src\_k}^y}^2 + {x_{src\_k}^z}^2}. \end{aligned}$$

(39)