Keep the files the way they are for long term storage.
For experimenting, it might be beneficial to deal with just one format.
Depending on what resources you have available issues may be:
- supported formats
- disk space/speed
- decompression speed
- random access
Depending on the requirements of your analyses, you might want to reduce the quality of your tracks to gain speed advantages.
Supported Formats
If the software you use for analysis supports both MP3 and M4A, you're set. If not, you'll need to convert using tools like ffmpeg or sox.
Disk Space/Speed
If you were to convert your files to uncompressed WAV, this will eat up a lot of disk space. If you don't have this space, WAV is not a good format. Instead convert to another lossless format like FLAC or Apple Lossless. You can do so without loosing any quality.
Besides disk space, you need to pay attention to disk speed. For some deep learning tasks you will have to provide your network with lots and lots of different samples, which you need to read from disk. With WAV being roughly 10 times as large on disk, disk I/O may become a bottleneck, so minimizing file size on disk may become desirable.
Decompression Speed
Using a compressed format like FLAC or MP3 comes at a (small) price. You will have to decompress your audio files on-the-fly before analysis.
Random Access
If you need to quickly access audio starting at a certain timestamp, say at 10.455s, formats like WAV have a distinct advantage, as they allow random access of audio very easily. This is usually not possible (as quickly) with compressed formats.
So what format should you choose for your experiments?
It all depends on your requirements.
Do you need quick random access to the audio? Do you really need stereo quality at 44,100 Hz? You can reduce the problem size a lot, by converting to mono 22,050 Hz. If that's still good enough for your purposes, go for it and save yourself a lot of trouble (but keep your original files around!).
Also, to get more technical, if you are going to analyze your data with something like Python and you know you need spectrograms, there's nothing wrong with preprocessing your MP3/M4A files with a library like librosa and simply store numpy arrays, if that's what you ultimately need.