LT;DR: Executing SGD (Stochastic Gradient Descent) is almost like executing GD (Gradient Descent) but computationally cheaper, and probably the only way to execute GD on big datasets. By evaluating at each sample - each game - you are losing the advantages of SGD and one could say you're "rolling your own" version of a computationally cheap GD.
Gradient Descent Summary
After you evaluate the error/loss function of you model, with whatever criterion you may use, you will then attempt to take the gradient of your loss function at the point where your network is. To simply things let's assume that you are using cross-entropy as the criterion of the loss (which makes some sense if you are evaluating an entire matrix of probability results). Let's say that the loss function is, where $H$ is cross-entropy, $f$ the NN evaluation, $X$ input data and $Y$ the labels:
$$
E = H(f(X), y)
$$
This is what nn.CrossEntropyLoss does (or any other criterion).
We want to find the minima of $E$, for that we can use $\nabla E$ but we cannot get the general formula for $\nabla E$ because we do not have the actual formula for $f$. Yet, we can find the (vector) value of $\nabla E$ at the location we are in (where location is the value of every parameter/weight in our NN). We can find $\nabla E$ for one specific configuration of $W$ which will be the matrix of all weights in the network.
$$
\nabla E_{W} = \frac{2}{N} X^T(XW - y)
$$
This is oversimplified - to actually find the matrix $W$ we need to perform backpropagation. In pytorch we get that by doing a huge chain rule with E.backward(). i.e. this is what backward() does, it gets the matrix $W$.
Now we have a handful of very expensive matrix multiplications to get from $W$ to $\nabla E_W$. Possibly prohibitively expensive if the dataset is big ($X$ is the full dataset). Enters SGD.
Stochastic Gradient Descent
The only difference in SGD from GD is that SGD will not use the entire $X$ in the calculation above. Instead SGD will select just a handful of samples (rows) from $X$ and use that as an estimation of $\nabla E_W$. Often it is said that SGD takes only a single sample from $X$ but implementations vary (e.g. ASGD).
Finally step() will apply $\nabla E_W$ times the learning rate to all weights/parameters in the network.
Your Approach
Since you are going sample by sample and performing backward() and step() on each sample you are not really doing SGD (despite calling that function). In your case the SGD optimizer has only a single sample to select from every time, therefore you are uniformly trying all samples in your dataset (as opposite to Stochastically). (That uniformity will reduce the variance of your model, which may be dangerous in other ways, although not very relevant here)
Therefore yes: To use SGD you do need to feed batches of input, and it will "average" (sum) the gradients across the sampled rows of $X$. It will average/sum/accumulate (sum all components) the gradients thanks to the fact that some samples may result in opposite gradient values in some directions; and hopefully the taken samples will agree on the main direction of the gradient and increase the magnitude in that direction.
P.S. The fact that pytorch accumulates the gradients from each selected sample is also one of the reasons we need to call zero_grad() in the training loop. You want to accumulate gradients within an SGD step but not accumulate across steps (SO question on exactly that).
Code
To actually generate batches you can use torch.randperm() I personally like this answer on SO about randperm but I'll add a summary here:
Having the matrix $X$ above as X (in code), one can create a batch of 256 samples with:
batch_size = 128
n_samples = X.size()[0]
permutation = torch.randperm(n_samples)
for i in range(0, n_samples, batch_size):
indices = permutation[i:i+batch_size]
batch_x, batch_y = X[indices], Y[indices]
# nn(X), criterion(), backward(), step(), zero_grads(), ...
Which you need to perform per epoch (refer to linked answer for that and more - do not want to copy that code in full since I do not deserve the merit).
The tricky bit may be the fact that you need to generate the batch from both $X$ and $Y$, and keep the order between them. Using indices generated with randperm allows for that.
