"BadRequestError: cursor position out of range of original request" when receiving next batch of request

I have this handler to do some sort of process for all users of our application. Basically, it takes 1 packet, processes the records of that batch, and then enqueues a new task for the next batch.

QueueAllUsers(BaseHandler):
    FETCH_SIZE = 10
    FILTERS = [
        UserConfig.level != 0,
        UserConfig.is_configured == True
    ]
    ORDER = [UserConfig.level, UserConfig._key]

    def get(self):
        cursor_key = self.request.get('cursor')
        cursor = None
        if cursor_key:
            # if `cursor` param is provided, use it
            cursor = Cursor(urlsafe=str(cursor_key))
        q = UserConfig.query(*self.FILTERS).order(*self.ORDER)
        total = q.count()  # 31 total records
        logging.info(total)
        users, next_cursor, more = q.fetch_page(self.FETCH_SIZE,
                                                keys_only=True,
                                                start_cursor=cursor)
        self.process_users(users)
        if more:
            self.queue_next_batch(next_cursor)

    def queue_next_batch(self, next_cursor):
        # Call get() again but this time pass `cursor` param to process next batch
        logging.info(next_cursor.urlsafe())
        url = '/queue_all_users?cursor=%s' % (next_cursor.urlsafe())
        taskqueue.add(
            url=url,
            method='get',
            queue_name='cronjobs'
        )

    def process_users(self, users):
        logging.info(len(users))
        # trimmed

      

        
        
        
      

    

But when the task for the second batch is run, NDB throws an error BadRequest

saying that the cursor is out of range.

I don't understand why this is out of range? I fetched 10

records from the total 31

, so the cursor will still be valid.

Note that the bug is picked in the second batch (i.e. recording 11-20

).

So the stream looks like this:

• Call /queue_all_users
  
  for processing the first batch (without cursor). Everything works fine.
• Step 1 creates a task /queue_all_users?cursor=123456
  
  for the next batch.
• Call /queue_all_users?cursor=123456
  
  (cursor provided). fetch_page
  
  throws BadRequestErrror
  
  .

EDIT : I tried setting FETCH_SIZE

on 17

and the 2nd batch fetch worked! It seems that something below 17

is causing the error, and it works 17

. So ... what the hell?