Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-1171

Race conditions between WebServer and Scheduler while cancelling integration

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: P3: Somewhat important P3: Somewhat important
    • production
    • unversioned
    • Coin (obsolete)
    • None

      When cancelling an integration, the running integration task file is removed by the Scheduler. However, this creates a race condition between the Webserver and Scheduler, because the Webserver tries to read a symlink that might already been deleted by the Scheduler.

      To reproduce:
      1. In webui, add a new integration:
      2. In webui, cancel the integration.
      3. In tmux, compare scheduler and webserver pane outputs:

      2017-02-25 17:31:14,926 DEBUG:utils: Removed running symlink: /home/akeskimo/ci-working-dir/storage/running-integrations/run_builds_1488036666.thrift_bin

      webserver:2017/02/25 17:31:14.927219 websocket.go:183: Error reading symlink: readlink /home/akeskimo/ci-working-dir/storage/running-integrations/run_builds_1488036666.thrift_bin: no such file or directory /home/akeskimo/ci-working-dir/storage/running-integrations/run_builds_1488036666.thrift_bin

      As we compare the timestamps, we can see that the Webserver tries to read the symlink that has already been destroyed by the Scheduler.

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            aakeskimo Aapo Keskimölö
            aakeskimo Aapo Keskimölö
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved:

                There are no open Gerrit changes