How can I optimize RAM usage when I need to show user a large amout of data?

I have an application in which I use DataGrid to show data to user. I use a DataTable object to store data in memory. DataTable have about 70 columns.

I have a very large data set. I want to display it in DataGrid and use filters and other DataGrid data management features (such as sorting). But I have a problem: when I load all my data in DataSource object, it consumes over 1GB of RAM (MS Excel consume only 60MB). This makes OS to intensively use swap, and of course, it slows down entire system.

My question is: is there any way to display data in DataGrid and use all advanced data management features without 开发者_高级运维complete loading all of my data in DataSource object?

Thank you in advance.

---

Since the user is never going to look at more than one screenfull at a time, there is no way you can explain him the need for gigabytes of ram to do that.

The database should be designed/indexed/optimized to allow partial retrieval of the data, for the 'most common' cases at least.

You can e.g. create one 'sorted' version of your data set for the 10 most commonly used sorting criteria. Feel free to store only the 1000 first and 1000 last rows. Use the knowledge of what exactly the user is going to do with those data to find more optimized data structures etc...

You can use techniques like in applied Ajax to provide a 'lively' impression.