How to use time-sequence data with "meta data" of single value per sample?

Question

I'm, trying to estimate a fish weight by a time-sequence signal of the change in resistance when the fish goes through a gate with electrodes installed.

When the fish pass through the gate there's a spike in resistance, and the shape of it supposedly derives from the fish weight.

Here is an example of resistance readout of 340 gram fish experiment:

And here is an example of the extracted spikes from the same 340 gram fish experiment:

However, there are other parameters that governs the signal's shape such as:

• Water Temperature
• Water Salinity
• Gate formation (the proximity of the electrodes)

I wonder how to incorporate this type of data, which is single-valued per sample (for example, for all of fish #12 samples, temperature=28.2 Celsius, salinity=410, gate formation=80mm) with the time-series data of the sample itself (which is ~80 data points series).

So far I tried mostly neural networks (fully connected and 1-D convolutional architectures) and linear regression.

I searched the internet quite a while and couldn't find a suggested architecture for such data.

Would adding a channel for each parameter with the fixed value work?

Or alternatively, adding these parameters as part of the spike vector?

Any ideas / reference to some research?

Many thanks!

Answer 1

My impression is that this is as much an instrumentation problem as a DS/ML one. Without knowing the details of your setup (geometry, position of the electrodes, number of fishes in the tank at once and how you are planning to use it passed what looks like a calibration phase it's a bit hard to get a good intuition of what you are actually measuring and how a ML model could help you.

It's hard to believe there will be extensive literature on this, let alone any SOTA model. It is not a widely studied problem as far as I know.

You say yourself that the measurement obtained likely depends on "the size of the fish, its speed and proximity to the edges of the gate and probably other things like the water salinity and temperature.". Do you have access to these other variables? Also what does the actual setup look like?

You shared a table with resistance, salinity and temperature but neither the salinity and temperature vary in it. Do you have some calibration curves for them?

For a single fish dataset, I would imagine water salinity and temperature are likely nearly constant given the duration of your experiments. But for different experiments, depending how much time apart they were run, this could change. I would try to run controled experiment, maybe with a dead fish (I know it's somber but it should be informative) to try to understand how much variability these others variables introduce.

Given the amount of variability within a single fish experiment, it looks like your system isn't very constrained. You mentioned speed and proximity to electrode but, if you consider a measurement as the readings from these two electrode, I imagine the angle at which

Now, if you can ultimately actually measure the weight from a series of spikes intsead of a single spike, I think I would start by performing some EDA on these spikes. Construct features and insure there is some good correlation with fish weight. This is 100% Physics so the more work you do to understand/model your system which is equivalent to feature engineering, the less a complex NN will be necessary.

It's hard to say with the scale you used to present your data but it looks like there is a short delay between the spikes recorded by each electrode. Is that correct? If it is, it means that you can measure the fish speed $v = d / \tau$ using the distance between the electrode $d$ and the delay $\Delta t$ between the spike (using the time at which their value was maximum should work).

Now looking at individual spikes, the faster the fish swims, the shorter these spikes should be. Since you can get the fish swimming speed you can verify this. For each fish get a spike duration and plot them against the speed. To define the spike duration $\tau$, I would take the maximum value, and set a threshold for when the reading goes above and below x% of that value. You would have to do this for each electrode. Alternatively, you can look at the distribution of the resistance when there is no spike, and set a universal threshold.

Now the duration of the spike should be proportional to the fish length and inversely proportional to its speed. So a feature such as $\tau' = \tau / v$ should be useful.

I'm also guessing that the amplitude of the spike should be proportional to the fish thickness. So I would also use the maximum resistance for each spike and use that as a feature. If too noisy, the top 5 or 10th percentile of the spike values instead.

I would imagine that, for a given species of fish, if you have length and thickness, you can predict weight pretty well.

Finally, and this is out of the scope of your question but why not using camera? You could use the electrode to trigger the frame acquisition but you would probably be able to get a great weight estimate from the picture.