Sampling is a totally good option, especially if the size of your data is bogging down the tool you're using to plot it.
If that's not the problem, a common issue is that plotting opaque markers will show you where data is located, but will disguise density information. For example, imagine a situation where every pixel of your plotting area is associated with at least ome observation (i.e. you have a uniformly colored plot) but one pixel is actually associated with 99% of your data. A good technique for situations like this is to try to visualize the density of the data. A simple approach is to add transparency to your markers (often by adjusting the "alpha" parameter), or you can model the density more directly with binning (e.g. a histogram or hexgrid) or with a kernel density estimate.
If you have discrete data, overplotting will likely be an issue but density might give you weird results. A good way to address this is to "jitter" your data by adding noise to one or more plotting dimensions to force your data to spread out more.
If you have time series data, you can resample to a coarser resolution: e.g., if you have a data point for every millisecond, your data will probably be easier to visualize if you aggregate by hour, day, or week.
Similarly, you can summarize the data by plotting a model. Plot $X$ vs $E[Y|X]$ instead of $X$ vs $Y$and throw in some error bands for good measure.
All that said: just try plotting it first and see what happens. Your visualization tool might do some stuff under the hood to render at least some of this manual effort unnecessary.