Why are variables of train and test data defined using the capital letter (in Python)?

Question

I hope this question is the most suitable in this site...

In Python, usually the class name is defined using the capital letter as its first character, for example

class Vehicle:
    ...

However, in machine learning field, often times train and test data are defined as X and Y - not x and y. For example, I'm now reading this tutorial on Keras, but it uses the X and Y as its variables:

from sklearn import datasets

mnist = datasets.load_digits()
X = mnist.data
Y = mnist.target

Why are these defined as capital letters? Is there any convention (at least in Python) among machine learning field that it is better to use the capital letter to define these variables?

Or maybe do people distinguish the upper vs lower case variables in machine learning?

In fact the same tutorial later distinguish these variables like the following:

from sklearn.cross_validation import train_test_split

train_X, test_X, train_y, test_y = train_test_split(X, Y, train_size=0.7, random_state=0)

Answer 1

The X (and sometimes Y) variables are matrices.

In some math notation, it is common practice to write vector variable names as lower case and matrix variable names as upper case. Often these are in bold or have other annotation, but that does not translate well to code. Either way, I believe that the practice has transferred from this notation.

You may also notice in code, when the target variable is a single column of values, it is written y, so you have X, y

Of course, this has no special semantic meaning in Python and you are free to ignore the convention. However, because it has become a convention, it may be worth maintaining if you share your code.

Answer 2

I think this has nothing to do with Python but with mathematics. X is a matrix and y is a vector (most of the time). Usually upper case letters are used for matrices and lower case letters are used for vectors.

That's why you often see something like this (from sklearn examples):

digits = datasets.load_digits(n_class=10)
X = digits.data
y = digits.target

or that (from the same example):

x_min, x_max = np.min(X_red, axis=0), np.max(X_red, axis=0)
X_red = (X_red - x_min) / (x_max - x_min)

Here X_red is an m x n matrix (upper case) and x_min is a vector (lower case) of length n.

Answer 3

To understand when to use lowercase or uppercase, we need to know what is represented in X_train or X_test. It is a capital letter X to represent a 2-D matrix.

Mathematically, it is a common notation for Linear Algebra to use uppercase Latin letters for matrices (e.g. matrix X) and lowercase Latin letters for vectors (vector y).

In data science, the feature matrix X is a collection of many columns of feature values. For example a df with 1 target, 20 features and 1000 data records will have the shape of shape (1000, 21). So we will define the feature matrix X to have the shape (1000, 20). Whereas the target label y is a column of values having the shape (1000, 1).

After applying train_test_split() on X and y with test_size=0.25, I would expect:
X_train to be a 2-D matrix (750, 20)
y_train to be a 1-D vector (750, 1)