Although I could rely on DirectX to do MipMapping of textures in the distance, reducing their rendering time appropriately, the landscape geometry itself (the mesh) had no such support.
The problem is that the amount of landscape information you can see in reality is vast, but a 3d application is a simulation and must calculate everything you can see many times a second to maintain the illusion. There isn't time to calculate the relative positions of the millions of landscape heights visible over 5 miles. Even worse, the amount you can see in your field of view doesn't get linearly larger - it gets exponentially larger. I had to find a way of reducing the amount of detail in a mesh if it was further away.
At this point I realised my earlier victory with MipMapping was illusory - DirectX was indeed reducing the rasters so it rendered them fast when seen at a distance, but it still had to be loaded into RAM at its maximum resolution before DirectX had a chance to reduce it. I realised in both cases I was going to have to manage the distance and detail manually and load and unload data dynamically based on the viewers distance away. This wasn't difficult in theory because all I had to do was pre-calculate the various MipMaps, and various Mesh densities and save them in files, then load them as needed.
This approach was easy despite problems writing my first really serious multi-threaded disk access code, and the textures flew in and out of RAM with no performance drop. For mesh simplification I simply omitted every other point in the mesh for each scaled down version I stored.
The problem with this mesh reduction approach is that the landscape simply averages out, and this is not what we experience naturally - normally we see the features of the landscape that are most extreme at the longest distance. Peaks, ridges and undulations are what we see - we don't see and averaged landscape.
Further research was needed and I found CLOD - the Continuous Level Of Detail algorithm. Clearly this was a favourite of computer science colleges and there are lots of versions on the web. The problem for me was they attempted to provide 'continuous' level of detail and I wanted to highlight 'exceptional features'.
It took me about four months of experimenting before I found my answer. This was to carry out mesh reduction based on the amount of change between one triangle and another.
The imagery around mesh reduction is hard to imagine until you've seen it. I recommend a visit here to see examples of the kinds of problem I was hitting.
My solution was to implement a version of the edge-collapse algorithm which attempts to take adjacent triangles, measure their relative slopes and, where the slopes are nearly identical, to merge the triangles together merging four into one. Using this algorithm I preserved radical changes in ground slope and was able to progressively apply harsher and harsher simplification to generate my progressively less detailed landscape meshes for loading at runtime.
Above is 177k worth of mesh data, and it is progressively simplified through 56k, 18k, 4k and finally 3k. Notice how the 3d image degrades but preserves the most important features.
Although the number of mesh lines are reducing drastically when viewed in 3d the most important and radical features are preserved.
This approach worked beautifully but was extremely computationally intensive taking about 48 hours to process all the data in the UK. However this only needed doing each time I modified my algorithm but until I'd perfected it my electricity bill went up markedly !
The British Landscape Explorer Story
First Steps - The VB6 DirectX COM Wrapper
First Landscapes - VB6
The Managed DirectX 9
Mesh Madness
Towards the Product
Adding Features
The Finished Article