Using Top Values to export a range of records

[jscranton]

I have a very large table of 3 million records that I need to export out of ACCDB in chunks of 500k records at a time.

They are sorted in a particular order so their Primary IDs are not sequential.

I can easily export out the first 500k by setting the Top Values property to 500,000. Is there a way to then set the range to 500,001 to 1,000,000?

THanks,

Justin

 

[xenou]

Not tested (i rarely use top in Access) but should work like this

select top (500000) * from Table where SomeID < 500000 order by SomeID
select top (500000) * from Table where SomeID >= 500000 and SomeID < 1000000 order by SomeID
select top (500000) * from Table where SomeID >= 1000000 and SomeID < 15000000 order by SomeID

If the IDs are not really sequential then you can't be sure of getting exactly 500,000 per chunk.
If you need to get exactly 500,000 per chunk then you have to store the ID's of what is exported so you can start the next batch from the same place.

Probably not ideal. Best to test your algorithm on a small table with say 10 chunks of 10, with only 100 records (more or less).

Another possibility - export the relevant IDs into a temporary-ish table, partition them into the right number of groups, and use that table to control the exports. This would provide a better audit and probably a better way of ensuring reliability (which can also be helped by recording what is exporting and validating the counts).