Assume that there is a very large dataset of hundreds of sites which contains only the PM2.5 level, the site ID, and the Latitude and Longitude as features. The independent feature to be predicted is PM2.5. The dependent features are site ID and Latitude/Longitude.
Conversely, a second dataset pertains to only three of the above sites (A, B, C). It has the same features as the first. However, in addition to the location and PM2.5 features, there are also perhaps 50 environmental features collected daily for each site, such as daily high air temperature, daily max wind speed, and so on.
The goal is to determine which dataset can produce the better prediction for sites A, B, C on a given day in the future.
Question 1: Suppose that classifiers build using first dataset perform much better than classifiers built using the second dataset. Are classifiers built on the first dataset valid as they are basing some of their predictive value simply on the site ID? Is site ID (and to a similar extend Lat/Lon) a valid feature to include?
Question 2: Suppose that the task is now to predict PM2.5 at a fourth unseen location. There is no training data for the fourth location, only a test event. Both datasets do not include this location. The Lat/Lon of the fourth location is included in test. Site ID of this fourth location is N/A during test. The second dataset includes many environmental features for this fourth location during test. Suppose that based solely on location, the classifiers from the first dataset (many examples of Lat/Lon, site ID, previous day PM2.5 levels) perform much better at predicting PM2.5 at this new location than the models built using few locations. Is this a valid result?