October 22, 2024
Chicago 12, Melborne City, USA
SQL

Does using ORDER BY on an indexed column in a query cause issues if new rows are inserted during query execution


I have a table Users with id, this table has a million records or more. So when fetching data from the table I am using batching with offset & limit to reduce load on my server.

My question is if we are executing the below query of fetching the data from the table, simultaneously a query comes up inserting a record in the table. Will there be any consistency issues in data returned. The query will be executed until all the data rows in the table is finished.

Query:
SELECT * FROM Users ORDER BY id LIMIT 10000 OFFSET x;

The query will be run such that all the entries in table is returned.
Sample Code:

async function fetchDataInBatches(model, whereClause, batchSize = 1000) {
  let offset = 0;
  let moreDataAvailable = true;
  let allData = [];
  while (moreDataAvailable) {
    const results = await model.findAll({
      where: whereClause,
      limit: batchSize,
      offset: offset,
      order: [['id', 'ASC']], 
    });
    if (results.length === 0) {
      moreDataAvailable = false; 
      break;
    }
    allData = allData.concat(results);
    offset += batchSize;
  }
  return allData;
}



You need to sign in to view this answers

Leave feedback about this

  • Quality
  • Price
  • Service

PROS

+
Add Field

CONS

+
Add Field
Choose Image
Choose Video