What is the most rigorous definition of a matrix?

Let $I_n:=\{1,\cdots, n\}$.

A $n \times m$-matrix with coefficients in a ring $\mathcal{R}$ is a function $a: I_n \times I_m \to \mathcal{R}$. We define $a_{i,j}:=a(i,j)$.

In other words, a rectangular array.

Sidenote: In my point of view, I think it is particularly useful to see the above definition as a model for the concept of a matrix, and retain subconsciously the relevant idea of a rectangular array as the core concept. If you truly understand the core concept and has relative comfort with mathematics, coming up with the above definition is a simple matter.

Let $m$ and $n$ be positive integers and let $F$ be a field. An $m $ by $n $ matrix with entries in $F $ is a function from $S $ into $F $, where $S $ is the set of all ordered pairs of integers $(i,j) $ such that $1\leq i \leq m $ and $1\leq j \leq n $.

Let $R$ be a commutative ring with identity, and $n,m$ be positive integers. The set of $n\times m$ matrices $R^{n\times m}$ is defined as an $R$-module (vector space, if $R$ is a field) freely generated by elements $\{e_{i,j}\}$, where $1\leq i \leq n$ and $1 \leq j \leq m$. The element $e_{i,j}$ represents the matrix with all entries equal to zero, except the entry in $i$-th row and $j$-th column, which is equal to one.

For example, for $n=5, m=6$

$$e_{2,4}=\begin{pmatrix}0&0&0&0&0&0\\0&0&0&1&0&0\\0&0&0&0&0&0\\0&0&0&0&0&0\\0&0&0&0&0&0\end{pmatrix}$$

Any matrix is a linear combination of $e_{i,j}$, with the coefficients being just the matrix entries.

This is still just an $n \times m$-dimensional $R$-module, but there is also a natural multiplication map $\times : R^{n\times m} \times R^{m \times p} \rightarrow R^{n \times p}$, defined by $$e_{i,j} \times e_{k,l} = \begin{cases} e_{i,l} & j = k \\ 0 & j \neq k \end{cases}$$ and extended by linearity. In fact, it is the standard matrix product.

In case $n=m$, this multiplication makes $R^{n\times n}$ into an $R$-algebra, isomorphic to the algebra of all endomorphisms on $R^n$.

The standard way to define a table of values in some set $S$ as a mathematical object is to encode the positions as an index set $S$, and to represent the table as the map $I\to S$ that associates to each position it value in the table. Then matrices with $n$ rows and $m$ columns and values in $R$ could be represented (taking the usual convention of putting the row index before the column index) as maps $([n]\times[m])\to R$, where $[n]$ is an abbreviation for the $n$-element set $\{1,\ldots,n\}$, and "$\times$" denotes the Cartesian product.

However this definition has a drawback that the values $n,m$ cannot always be recovered from a matrix, even though the domain can be recovered from a map. The problem is that $[n]\times[m]=\emptyset$ whenever $n=0$ or $m=0$, so for a matrix that is the unique map $\emptyset\to R$, one cannot reconstruct the values of $n,m$. Bourbaki even explicitly states that there is a unique empty matrix, overlooking the fact that this causes problems with definitions that immediately follow in their own presentation. In everyday use of matrices, it is very common to assume that given an $n\times m$ matrix one knows what $n$ and $m$ are, but with the definition of a matrix as a map, this is simply not true in the case of an empty matrix. And empty matrices are necessary for encoding linear maps between finite dimensional vector spaces, in case a vector space of dimension$~0$ is the domain or codomain of the linear map.

One reason this is a real problem is that matrix multiplication is defined in such a way that any product of a $n\times k$ matrix with a $k\times m$ matrix should be an $n\times m$ matrix, each of whose entries is obtained as a sum of $k$ terms. If $k=0$ then both operands of the multiplication are empty matrices, so each entry of the product being a sum of $0$ terms must be $0$, giving the zero matrix of size $n\times m$. However this requires knowing the values of $n$ and $m$, and with the definition of matrix as map, this is not possible. One simply cannot define the product of the unique empty matrix with itself to be a different size zero matrix according to the circumstances.

So the conclusion is that for a rigorous definition of a matrix, one must record the dimensions as components of the matrix itself. So a proper and rigorous definition of a matrix is the following.

A matrix with entries in $R$ is a triple $(n,m,f)$ where $n,m\in\Bbb N$ and $f$ is a map $([n]\times[m])\to R$.

The other answers define a matrix in a way that is just a rectangular array of numbers. This not only ignores the beauty of the structures, but it makes proving many basic facts (eg. that the determinant is multiplicative, among innumerable others) very cumbersome.

Of course matrices can have very different mathematical structures depending on how you interpret them. (eg. linear transformations, bilinear forms, etc.) Since it's the most common, here's how you could define a matrix as a linear transformation:

Let $V$ and $W$ be finite dimensional vector spaces over a field $F$ with respective bases $B$ and $C$. Let $T:V\to W$ be a linear map. Then the "matrix" for $T$ can be defined as the map $B\times C\to F$, which maps $(\beta,\gamma)$ to the coefficient of $T(\beta)$ in the (unique) expansion of $\gamma$ in terms of image of $B$ under $T$, which is a basis for $W$.

Actually it's now obvious how you'd extend this definition to "infinite matrices."

Depending on how abstract/general/formal you want to get, you might want to check out (google) Chu spaces, e.g., https://en.wikipedia.org/wiki/Chu_space Your phrase "array of symbols and expressions" suggests you might want to discuss more abstract arrays than numbers/fields/etc. If the wikipedia page whets your appetite, I'd say the most comprehensive yet easy-to-follow introduction is http://boole.stanford.edu/pub/coimbra.pdf