Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-100244

Running a testcase on an unknown datatag should fail

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • P3: Somewhat important
    • None
    • 6.3.0 Alpha
    • Testing: qtestlib
    • None

    Description

      For example

      $ ./tst_qgraphicswidget focusWidget ; echo $?
      
      ********* Start testing of tst_QGraphicsWidget *********
      Config: Using QtTest library 6.3.0, Qt 6.3.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by GCC 11.2.1 20210816 [revision 056e324ce46a7924b5cf10f61010cf9dd2ca10e9]), opensuse-tumbleweed 20220122
      PASS   : tst_QGraphicsWidget::initTestCase()
      PASS   : tst_QGraphicsWidget::focusWidget(none)
      PASS   : tst_QGraphicsWidget::focusWidget(first)
      PASS   : tst_QGraphicsWidget::focusWidget(last)
      PASS   : tst_QGraphicsWidget::cleanupTestCase()
      Totals: 5 passed, 0 failed, 0 skipped, 0 blacklisted, 2ms
      ********* Finished testing of tst_QGraphicsWidget *********
      
      0    # SUCCESS exit code, as expected
      
      $ ./tst_qgraphicswidget focusWidget:blah ; echo $?
      
      ********* Start testing of tst_QGraphicsWidget *********
      Config: Using QtTest library 6.3.0, Qt 6.3.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by GCC 11.2.1 20210816 [revision 056e324ce46a7924b5cf10f61010cf9dd2ca10e9]), opensuse-tumbleweed 20220122
      PASS   : tst_QGraphicsWidget::initTestCase()
      Unknown testdata for function focusWidget: 'blah()'
      Available testdata:
      none
      first
      last
      PASS   : tst_QGraphicsWidget::cleanupTestCase()
      Totals: 2 passed, 0 failed, 0 skipped, 0 blacklisted, 1ms
      ********* Finished testing of tst_QGraphicsWidget *********
      
      0    # SUCCESS but should be ERROR instead
      

      Notice how the last line is 0 which means it succeeded, even though the datatag blah is unknown.

      It should return 1 which means error, just like when you ask to execute an unknown test function, for example tst_qgraphicswidget blah  returns error.

      Attachments

        Issue Links

          Activity

            People

              jimis Dimitrios Apostolou
              jimis Dimitrios Apostolou
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: