tadoquery memory release

tianyifan

Table a has 500,000 data, all taken out at one time. Put into table b after processing

At the beginning of execution, the memory (512M) occupancy rate was 30%. As the amount of processed data increased, the memory occupancy rate continued to increase (almost all 512M was used up after processing about 80,000 data). I don't know what caused it. Is there any solution, please enlighten me. .

bbtcobra

It seems that you haven't done this before, if you update the statement will be better, otherwise it is best not to take out such a large

lolonois

Attendance Record?

xiaobai_11

1. There is a problem with the program itself, which has nothing to do with the amount of data;
2.If it is related to the amount of data, you can find an ID and process it in batches

njmumu

The quality of the program is high, and there are not many 500,000 records, but you can consider batch processing

stevevai

Follow the paging browse, you can divide the data into several batches, and choose the batching unit according to your memory size. Execute step by step, release the previous batch of data every time it is executed!

xclzc17

Set ADO with large amount of data to CursorType = ctOpenForwardOnly

May reduce the memory footprint, but the data set can only be moved in one direction

tianyifan

500,000 is already processed in batches. There are millions of common data.

There is a field in the a table to identify whether the record has been manipulated. The initial value is 0. I use select to fetch 500,000 data. Each time I process a record, I assign a value of 1 to this record. Then there is the problem of the title.

hawk_09

Learning~!

stevevai

It seems that this has something to do with your code. It is estimated that your processing is carried out one by one, so the records that have been processed in the north are not released, and will not be released until your batch of records is processed. Of course, the memory will continue to increase. And you also load a lot of fields, it is recommended that you do not load unnecessary fields. At the same time, further batching, according to your test, I think it should be better in 40,000 batches!