Batch Size
Number of training examples processed together in one forward/backward pass. Affects training speed and memory usage.
Key Concepts
Batch
A subset of the training data.
Epoch
One complete pass through the entire training dataset.
Iteration
One forward and backward pass of a single batch.
Detailed Explanation
The batch size is a hyperparameter that defines the number of samples to work through before updating the internal model parameters.
The batch size can have a significant impact on the training process. A small batch size can lead to a more noisy training process, but it can also help the model to generalize better. A large batch size can lead to a more stable training process, but it can also lead to the model getting stuck in local minima.
The optimal batch size depends on the specific problem being solved. It is often a good idea to experiment with different batch sizes to find the one that works best for your problem.
Real-World Examples & Use Cases
Image Recognition
A batch size of 32 is often used for image recognition tasks.
Natural Language Processing
A batch size of 64 is often used for natural language processing tasks.