收集测试元数据

Circleci从XML文件中收集测试元数据,并使用它来提供对您的工作的见解。本文档介绍如何配置Circleci以将测试元数据输出为某些常用测试跑步者的XML和存储报告store_test_results.step.

要将测试结果视为工件,请使用上传它们store_artifactsstep.

The usage of thestore_test_results.key in your config looks like the following:

- store_test_results: path: test-results

Where thepath关键是您的绝对或相对的路径working_directorycontaining subdirectories of JUnit XML or Cucumber JSON test metadata files. Make sure that yourpath值不是隐藏文件夹(示例:。my_hidden_directorywould be an invalid format).

If you are using CircleCI Server配置Circleci以收集测试元数据后,最失败的测试通常会显示在详细信息页面上的列表中洞察in the CircleCI application to identify flaky tests and isolate recurring issues.

洞察for Failed Tests

The above screenshot applies to CircleCI Server only.

如果您使用的是Circleci云, see theAPI v2 Insights endpointsto find test failure information.

Enabling Formatters

Test metadata is not automatically collected in CircleCI 2.0 until you enable the JUnit formatters. For RSpec, Minitest, and Django, add the following configuration to enable the formatters:

  • rspec.requires the following be added to your gemfile:
GEM'rspec_junit_formatter'
  • 迷你requires the following be added to your gemfile:
gem 'minitest-ci'
  • Django should be configured using thedjango-nosetest runner.

Note:For detailed information on how to test your iOS applications, refer to the在MacOS上测试IOS应用程序page.

自定义测试步骤中的元数据集合

Write the XML files to a subdirectory if you have a custom test step that produces JUnit XML output as is supported by most test runners in some form, for example:

- store_test_results: path: /tmp/test-results

Custom Test Runner Examples

This section provides the following test runner examples:

黄瓜

对于自定义黄瓜步骤,您应该使用JUnit Formatter生成文件并将其写入黄瓜directory. Following is an example of the addition to your。Circleci./config.ymlfile:

steps:-run:姓名:保存测试结果command:|mkdir -p ~/cucumberBundle Exec Cucumber --format Junit --out〜/ cucumber / Junit.xml什么时候:always-store_test_results.:path:~/cucumber-store_artifacts:path:~/cucumber

The小路:相对于项目的根目录可怕吗ctory where the files are stored. CircleCI collects and uploads the artifacts to S3 and makes them available in the Artifacts tab of theJob page在申请中。

Alternatively, if you want to use Cucumber’s JSON formatter, be sure to name the output file that ends with。黄瓜并写到了/cucumberdirectory. For example:

steps:-run:姓名:保存测试结果command:|mkdir -p ~/cucumberbundle exec cucumber --format pretty --format json --out ~/cucumber/tests.cucumber什么时候:always-store_test_results.:path:~/cucumber-store_artifacts:path:~/cucumber

Maven Surefire Plugin for Java JUnit Results

如果你正在建造一个Maven基于项目,您可能更可能使用Maven Surefire plugin以XML格式生成测试报告。Circleci使得很容易收集这些报告。将以下内容添加到。Circleci./config.ymlfile in your project.

steps:-run:姓名:保存测试结果command:|mkdir -p〜/ /测试结果/ JUNIT /找 。-type f -regex“。* / target / surefire-reports /.* xml”-exec cp {}〜/ test-excuest / Junit / \;什么时候:always-store_test_results.:path:~/test-results-store_artifacts:path:~/test-results/junit

Gradle JUnit Test Results

如果你正在建造一个Java or Groovy based project withGradle, test reports are automatically generated in XML format. CircleCI makes it easy to collect these reports. Add the following to the。Circleci./config.ymlfile in your project.

steps:-run:姓名:保存测试结果command:|mkdir -p〜/ /测试结果/ JUNIT /找 。-type f -regex ".*/build/test-results/.*xml" -exec cp {} ~/test-results/junit/ \;什么时候:always-store_test_results.:path:~/test-results-store_artifacts:path:~/test-results/junit

node.js的mocha

使用您可以使用的Mocha Test Runner来输出JUnit测试mocha-junit-reporter

A working。Circleci./config.ymlsection for testing might look like this:

steps:-退房-run:npm install-run:mkdir ~/junit-run:command:mocha test --reporter mocha-junit-reporterenvironment:Mocha_file.:〜/ junit / test-execue.xml什么时候:always-store_test_results.:path:〜/ JUNIT.-store_artifacts:path:〜/ JUNIT.

Mocha with nyc

下面是公司mplete example for Mocha with nyc, contributed bymarcospgp

version: 2 jobs: build: environment: CC_TEST_REPORTER_ID: code_climate_id_here NODE_ENV: development docker: - image: circleci/node:8 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference environment: MONGODB_URI: mongodb://admin:password@localhost:27017/db?authSource=admin - image: mongo:4.0 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference environment: MONGO_INITDB_ROOT_USERNAME: admin MONGO_INITDB_ROOT_PASSWORD: password working_directory: ~/repo steps: - checkout # Update npm - run: name: update-npm command: 'sudo npm install -g npm@latest' # Download and cache dependencies - restore_cache: keys: - v1-dependencies-{{ checksum "package-lock.json" }} # fallback to using the latest cache if no exact match is found - v1-dependencies- - run: npm install - run: npm install mocha-junit-reporter # just for CircleCI - save_cache: paths: - node_modules key: v1-dependencies-{{ checksum "package-lock.json" }} - run: mkdir reports # Run mocha - run: name: npm test command: ./node_modules/.bin/nyc ./node_modules/.bin/mocha --recursive --timeout=10000 --exit --reporter mocha-junit-reporter --reporter-options mochaFile=reports/mocha/test-results.xml when: always # Run eslint - run: name: eslint command: | ./node_modules/.bin/eslint ./ --format junit --output-file ./reports/eslint/eslint.xml when: always # Run coverage report for Code Climate - run: name: Setup Code Climate test-reporter command: | # download test reporter as a static binary curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter chmod +x ./cc-test-reporter ./cc-test-reporter before-build when: always - run: name: code-coverage command: | mkdir coverage # nyc report requires that nyc has already been run, # which creates the .nyc_output folder containing necessary data ./node_modules/.bin/nyc report --reporter=text-lcov > coverage/lcov.info ./cc-test-reporter after-build -t lcov when: always # Upload results - store_test_results: path: reports - store_artifacts: path: ./reports/mocha/test-results.xml - store_artifacts: path: ./reports/eslint/eslint.xml - store_artifacts: # upload test coverage as artifact path: ./coverage/lcov.info prefix: tests

AVA.for Node.js

To output JUnit tests with theAVA.test runner you can use the TAP reporter withtap-xunit

A working。Circleci./config.ymlsection for testing might look like the following example:

步骤: - 运行:命令:|yarn add ava tap-xunit --dev # or you could use npm mkdir -p ~/reports ava --tap | tap-xunit > ~/reports/ava.xml when: always - store_test_results: path: ~/reports - store_artifacts: path: ~/reports

eslint.

To output JUnit results fromeslint.,你可以使用JUnit formatter

A working。Circleci./config.yml测试部分可能如下所示:

步骤: - 运行:命令:|mkdir -p ~/reports eslint ./src/ --format junit --output-file ~/reports/eslint.xml when: always - store_test_results: path: ~/reports - store_artifacts: path: ~/reports

PHPUnit

对于phpUnit测试,您应该使用该文件使用文件- junit.command line option and write it to the/phpunitdirectory. Your。Circleci./config.ymlmight be:

步骤: - 运行:命令:|mkdir -p〜/ phpunit phpunit --log-junit〜/ phpunit / junit.xml测试时间:始终 -  store_test_results:path:〜/ phpunit  -  store_artifacts:path:〜/ phpunit

Pytest.

To add test metadata to a project that usesPytest.you need to tell it to output JUnit XML, and then save the test metadata:

- run: name: run tests command: | . venv/bin/activate mkdir test-results pytest --junitxml=test-results/junit.xml - store_test_results: path: test-results - store_artifacts: path: test-results

rspec.

将测试元数据集添加到使用自定义的项目中rspec构建步骤,将以下GEM添加到您的Gemfile中:

GEM'rspec_junit_formatter'

并将您的测试命令修改为此:

步骤: - 结帐 - 运行:捆绑检查--Path =供应商/捆绑||捆绑安装--Path = vendor / bundle  -  jobs = 4 --retry = 3  - 运行:mkdir〜/ rspec  - 运行:命令:bundle exec rspec --format进度--format rspecjunitformatter -o〜/ rspec / rspec。XML何时:always  -  store_test_results:path:〜/ rspec

迷你

将测试元数据集添加到使用自定义的项目中minitest构建步骤,将以下GEM添加到您的Gemfile中:

gem 'minitest-ci'

并将您的测试命令修改为此:

步骤: - 结帐 - 运行:捆绑检查||捆绑安装 - 运行:命令:Bundle Exec Rake测试何时:始终 -  Store_Test_Results:Path:Test / Reports

看看minitest-ci READMEfor more info.

Test2JUNIT用于CLOJURE测试

Usetest2junitto convert Clojure test output to XML format. For more details, refer to thesample project

trx2junit.for Visual Studio / .NET Core Tests

Usetrx2junit.to convert Visual Studio / .NET Core trx output to XML format.

A working。Circleci./config.yml部分可能是这样的:

steps:-退房-run:dotnet build-run:dotnet test --no-build --logger "trx"-run:姓名:test results什么时候:alwayscommand:|dotnet tool install -g trx2junitexport PATH="$PATH:/root/.dotnet/tools"trx2junit测试/ ** / testresults / *。trx-store_test_results.:path:tests/TestResults-store_artifacts:path:tests/TestResults目的地:试验结果

Karma

使用您可以使用的Karma测试跑步者输出JUnit测试Karma-Junit-记者

A working。Circleci./config.yml部分可能是这样的:

steps:-退房-run:npm install-run:mkdir ~/junit-run:command:业力启动./karma.conf.js.environment:JUNIT_REPORT_PATH.:。/junit/JUNIT_REPORT_NAME:test-euctions.xml.什么时候:always-store_test_results.:path:。/junit-store_artifacts:path:。/junit
// karma.conf.js//其他配置...{reporters:['junit.'],JunitreePorter.:{outputDir:过程envJUNIT_REPORT_PATH.,outputFile:过程envJUNIT_REPORT_NAME,useBrowserName:false},}//其他配置...

Jest

To output JUnit compatible test data with Jest you can usejest-junit

A working。Circleci./config.yml部分可能是这样的:

steps:-run:姓名:安装JUnit报道记者command:yarn add --dev jest-junit-run:姓名:用JUnit作为记者运行测试command:jest --ci --runInBand --reporters=default --reporters=jest-junitenvironment:JEST_JUNIT_OUTPUT_DIR:./ reports/junit./-store_test_results.:path:./ reports/junit./-store_artifacts:path:./ reports/junit.

For a full walkthrough, refer to this article by Viget:在Circleci 2.0上使用JUnit和Eslint。请注意,jest cli参数的使用--testResultsProcessor在文章中已被取代--reporterssyntax, and JEST_JUNIT_OUTPUT has been replaced withJEST_JUNIT_OUTPUT_DIRandJEST_JUNIT_OUTPUT_NAME,如上所述。

Note:When running Jest tests, please use the- runinband.flag. Without this flag, Jest will try to allocate the CPU resources of the entire virtual machine in which your job is running. Using- runinband.将强制Jest仅在虚拟机中使用虚拟化构建环境。

For more details on- runinband., refer to thejest cli.文档。有关这些问题的更多信息,请参阅问题1524.andIssue 5239of the official Jest repository.

API

To access test metadata for a run from the API, refer to thetest-metadata API documentation

See Also

Using Insights

视频:测试跑步者故障排除