I am exploring an interdisciplinary research area involving multimodal data, focusing on agriculture. My study incorporates both visual and tabular data: crop and soil images from three distinct locations, alongside corresponding meteorological information (temperature, humidity, solar radiation, etc.), collected hourly from 9 am to 5 pm. The objective is to enhance soil moisture predictions by integrating these data modalities, aiming to estimate soil moisture accurately from crop images.

Could you please suggest methodologies for effectively combining these data types? I've experimented with or considered the following:

  1. Feature Concatenation, utilizing MLPs/Random Forest for meteorological data and CNNs for image analysis.
  2. Conv-LSTMs.
  3. Vision Transformers.

Additionally, I'm curious about strategies for incorporating sequential soil patch images into models like Conv-LSTMs to capture both temporal and visual features. Do you think this approach is viable?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.