Pipeline Stage Library
This section discusses the available stages that can be called and added to the pipeline.
UpdateBOOTFiles
This stage downloads the needed files and then proceeds to update the files on the device’s SD card. After updating, the device reboots. For pluto and m2k, this stage downloads their firmware and then updates the device’s firmware.
![@startuml UpdateBOOTFiles Workflow
start
title UpdateBOOTFiles Stage Workflow
:Execute UpdateBOOTFiles stage;
:Print board name and branches;
partition "Try Block" {
if (Board is Pluto?) then (yes)
:Download firmware files via Nebula;
note right: dl.bootfiles ____board-name={board} ____branch={firmwareVersion} ____firmware
else (no)
:Download boot files via Nebula;
note right: dl.bootfiles ____board-name={board} ____source-root={source_root} ____source={source} ____branch={branches}
endif
:Get git SHA for board;
:Update boot files on device;
note right: manager.update-boot-files ____board-name={board} ____folder=outs
if (Board is Pluto?) then (yes)
:Set local NIC IP from USB device;
note right: uart.set-local-nic-ip-from-usbdev ____board-name={board}
endif
:Set elastic fields (success);
note right
set_elastic_field(board, 'uboot_reached', 'True')
set_elastic_field(board, 'kernel_started', 'True')
set_elastic_field(board, 'linux_prompt_reached', 'True')
set_elastic_field(board, 'post_boot_failure', 'False')
end note
}
partition "Exception Handling" {
if (Exception occurs?) then (yes)
:Log stack trace;
switch (Exception type?)
case (u-boot not reached)
:Set elastic fields;
note right
board, uboot_reached = False
board, kernel_started = False
board, linux_prompt_reached = False
end note
case (u-boot menu cannot boot kernel)
:Set elastic fields;
note right
board, uboot_reached = True
board, kernel_started = False
board, linux_prompt_reached = False
end note
case (Linux not fully booting)
:Set elastic fields;
note right
board, uboot_reached = True
board, kernel_started = True
board, linux_prompt_reached = False
end note
case (Ethernet/SSH issues)
:Set elastic fields;
note right
board, uboot_reached = True
board, kernel_started = True
board, linux_prompt_reached = True
board, post_boot_failure = True
end note
case (Other errors)
:Log unexpected failure;
endswitch
:Get git SHA for board;
if (send_results enabled?) then (yes)
:Set failing stage info;
note right
last_failing_stage = 'UpdateBOOTFiles'
last_failing_stage_failure = exception message
end note
:Call SendResults stage;
endif
:Throw UpdateBOOTFiles failed exception;
note right: Exception thrown but finally block still executes
endif
}
partition "Finally Block" {
:Archive UART logs;
note right
run_i("if [ -f ${board}.log ]; then mv ${board}.log uart_boot_" + board + ".log; fi")
archiveArtifacts artifacts: 'uart_boot_*.log', followSymlinks: false, allowEmptyArchive: true
end note
}
stop
@enduml
Success Case:
- uboot_reached: True
- kernel_started: True
- linux_prompt_reached: True
- post_boot_failure: False
end note
alt Exception occurs
UBF -> UBF: Analyze exception message
alt u-boot not reached
UBF -> ES: Set uboot_reached=False, kernel_started=False, linux_prompt_reached=False
else u-boot menu cannot boot kernel
UBF -> ES: Set uboot_reached=True, kernel_started=False, linux_prompt_reached=False
else Linux not fully booting
UBF -> ES: Set uboot_reached=True, kernel_started=True, linux_prompt_reached=False
else Ethernet/SSH issues
UBF -> ES: Set uboot_reached=True, kernel_started=True, linux_prompt_reached=True, post_boot_failure=True
else Other errors
UBF -> UBF: Log unexpected failure
end
UBF -> UBF: get_gitsha(board)
alt send_results enabled
UBF -> ES: Set last_failing_stage=UpdateBOOTFiles
UBF -> ES: Set last_failing_stage_failure message
UBF -> UBF: Call SendResults stage
end
UBF -> JP: Throw UpdateBOOTFiles failed exception
end
UBF -> UBF: Archive UART logs (rename {board}.log to uart_boot_{board}.log)
UBF -> JP: Stage completed
@enduml](_images/plantuml-0a2ade973ba447594bd017dcd10d8085ffd1f03d.png)
RecoverBoard
This stage enables users to recover boards when they can no longer be accessed. Reference files are first downloaded, then the board is recovered using the recovery device manager function of Nebula.
![@startuml RecoverBoard Workflow
start
title RecoverBoard Stage Workflow
:Execute RecoverBoard stage;
:Define reference branches;
note right: ref_branch = ['boot_partition', 'release']
if (Board is pluto?) then (yes)
:Log "Recover stage does not support pluto yet!";
stop
else (no)
:Enter recovery directory;
partition "Try Block" {
:Fetch reference boot files;
note right: 'dl.bootfiles - -board-name=' + board + ' - -source-root="' + gauntEnv.nebula_local_fs_source_root + '" - -source=' + gauntEnv.bootfile_source + ' - -branch="' + ref_branch.toString() + '"'
:Extract reference fsbl and u-boot;
note right
cd outs/
cp bootgen_sysfiles.tgz ..
tar -xzvf bootgen_sysfiles.tgz
cp u-boot-*.elf u-boot.elf
end note
:Execute board recovery;
note right: manager.recovery-device-manager ____board-name={board} ____folder=outs ____sdcard
}
partition "Exception Handling" {
if (Exception occurs?) then (yes)
:Log stack trace;
:Re-throw exception;
note right: throw ex
endif
}
partition "Finally Block" {
:Archive UART logs;
note right
run_i("if [ -f ${board}.log ]; then mv ${board}.log uart_recover_" + board + ".log; fi")
archiveArtifacts artifacts: 'uart_recover_*.log', followSymlinks: false, allowEmptyArchive: true
end note
}
endif
:Stage completed;
stop
@enduml](_images/plantuml-87f8bc0477ae687df67606dc1baf94346a248331.png)
SendResults
This stage sends the collected results from all stages to the elastic server which will then be processed for easy viewing.
Test Stages
There are also available test stages defined in the stage library.
LinuxTests
This stage checks for dmesg errors, checks iio devices, and runs diagnostics on boards.
![@startuml LinuxTests Workflow
start
title LinuxTests Stage Workflow
:Execute LinuxTests stage;
:Initialize variables;
note right
failed_test = ''
drivers_count = 0
missing_drivers = 0
end note
partition "Try Block" {
:Get board IP address;
note right: def ip = nebula('update-config network-config dutip --board-name='+board)
partition "DMESG Check" {
:Check DMESG for errors;
note right: nebula("net.check-dmesg - - ip='" + ip + "' - - board-name=" + board)
if (DMESG check fails?) then (yes)
:Add failure to failed_test;
note right: failed_test = failed_test + "[dmesg check failed: ${ex.getMessage()}]"
endif
}
partition "IIO Devices Check" {
:Check IIO devices;
note right: nebula('driver.check-iio-devices - - uri="ip:'+ip+'" - - board-name='+board, true, true, true)
if (IIO devices check fails?) then (yes)
:Add failure to failed_test;
:Parse missing devices;
:Write missing devices log;
:Set elastic field for missing drivers;
note right
failed_test += "[iio_devices check failed: ${ex.getMessage()}]"
missing_devs = Eval.me(ex.getMessage().split('\n').last().split('not found')[1].replaceAll("'\$",""))
missing_drivers = missing_devs.size()
writeFile(file: board+'_missing_devs.log', text: missing_devs.join(","))
set_elastic_field(board, 'drivers_missing', missing_drivers.toString())
end note
endif
}
:Get drivers enumerated;
note right: nebula('update-config driver-config iio_device_names -b '+board, false, true, false)
partition "Diagnostics Check" {
if (Board is not firmware board?) then (yes)
:Run network diagnostics;
note right: nebula("net.run-diagnostics - - ip='"+ip+"' - - board-name="+board, true, true, true)
:Archive diagnostic reports;
note right: archiveArtifacts artifacts: '*_diag_report.tar.bz2', followSymlinks: false, allowEmptyArchive: true
if (Diagnostics fails?) then (yes)
:Add failure to failed_test;
note right: failed_test = failed_test + " [diagnostics failed: ${ex.getMessage()}]"
endif
endif
}
if (Any tests failed?) then (yes)
:Throw Linux Tests Failed exception;
note right: if(failed_test && !failed_test.allWhitespace)
endif
}
partition "Exception Handling" {
if (Exception occurs?) then (yes)
:Throw NominalException;
note right: throw new NominalException(ex.getMessage())
endif
}
partition "Finally Block" {
:Count DMESG errors and warnings;
note right
set_elastic_field(board, 'dmesg_errs', sh(returnStdout: true, script: 'cat dmesg_err_filtered.log | wc -l').trim())
set_elastic_field(board, 'dmesg_warns', sh(returnStdout: true, script: 'cat dmesg_warn.log | wc -l').trim())
end note
:Rename and archive logs;
note right
run_i("if [ -f dmesg.log ]; then mv dmesg.log dmesg_" + board + ".log; fi")
run_i("if [ -f dmesg_err_filtered.log ]; then mv dmesg_err_filtered.log dmesg_" + board + "_err.log; fi")
run_i("if [ -f dmesg_warn.log ]; then mv dmesg_warn.log dmesg_" + board + "_warn.log; fi")
archiveArtifacts artifacts: '*.log', followSymlinks: false, allowEmptyArchive: true
end note
}
:Stage completed;
stop
@enduml](_images/plantuml-fc4fc6de42e5804dc01b0212b0653c4a6b599d4e.png)
PyADITests
This stage runs the pyadi-iio test on the target board.
![@startuml PyADITests Workflow
start
title Run Python Tests Stage Workflow
:Execute Run Python Tests stage;
partition "Try Block" {
:Get board configuration and declare variables;
note right
ip = nebula('update-config network-config dutip - - board-name=' + board)
serial = nebula('update-config uart-config address - - board-name=' + board)
def uri;
println('IP: ' + ip)
end note
:Clone pytest-libiio repository;
note right
run_i('git clone -b "' + gauntEnv.pytest_libiio_branch + '" ' + gauntEnv.pytest_libiio_repo, true)
end note
partition "pytest-libiio Setup" {
:Install pytest-libiio in its directory;
note right
dir('pytest-libiio') {
run_i('python3 setup.py install', true)
}
end note
}
:Clone pyadi-iio repository;
note right
run_i('git clone -b "' + gauntEnv.pyadi_iio_branch + '" ' + gauntEnv.pyadi_iio_repo, true)
end note
:Enter pyadi-iio directory;
note right: dir('pyadi-iio')
partition "pyadi-iio Setup (within dir)" {
:Install dependencies;
note right
run_i('pip3 install -r requirements.txt', true)
run_i('pip3 install -r requirements_dev.txt', true)
run_i('pip3 install pylibiio', true)
end note
:Create test directories;
note right
run_i('mkdir testxml')
run_i('mkdir testhtml')
end note
:Determine URI source;
if (iio_uri_source == "ip"?) then (yes)
:Set IP URI;
note right: uri = "ip:" + ip
else (no)
:Set serial URI;
note right: uri = "serial:" + serial + "," + gauntEnv.iio_uri_baudrate.toString()
endif
:Configure board markers;
note right
check = check_for_marker(board)
board = board.replaceAll('-', '_')
board_name = check.board_name.replaceAll('-', '_')
marker = check.marker
end note
:Build pytest command;
note right
cmd = "python3 -m pytest - - html=testhtml/report.html - - junitxml=testxml/" + board + "_reports.xml - - adi-hw-map -v -k 'not stress' -s - - uri='ip:"+ip+"' -m " + board_name + " - - capture=tee-sys" + marker
end note
:Execute pytest tests;
note right: statusCode = sh script:cmd, returnStatus:true
if (HTML report exists?) then (yes)
:Publish HTML report;
note right
publishHTML(target : [
escapeUnderscores: false,
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: true,
reportDir: 'testhtml',
reportFiles: 'report.html',
reportName: board,
reportTitles: board])
}
end note
endif
if (XML results exist?) then (yes)
partition "Parse pytest results (within try-catch)" {
:Define pytest metrics to parse;
note right
def pytest_logs = ['errors', 'failures', 'skipped', 'tests']
end note
:Parse each metric from XML;
note right
pytest_logs.each {
cmd = 'cat testxml/' + board + '_reports.xml | sed -rn \'s/.*'
cmd+= it + '="([0-9]+)".*/\\1/p\''
set_elastic_field(board.replaceAll('_', '-'), it, sh(returnStdout: true, script: cmd).trim())
}
end note
if (Parsing succeeds?) then (no)
:Log parsing failure;
note right
catch(Exception ex){
println('Parsing pytest results failed')
echo getStackTrace(ex)
}
end note
endif
}
endif
:Evaluate pytest status code;
if ((statusCode != 5) && (statusCode != 0)) then (yes)
:throw new NominalException('PyADITests Failed');
else (no)
endif
}
}
partition "Finally Block" {
:Archive test results;
note right: junit testResults: 'pyadi-iio/testxml/*.xml', allowEmptyResults: true
}
:Stage completed;
stop
@enduml](_images/plantuml-66fc22820fcbd8ca5a461f50ddead1feeabf351f.png)
LibAD9361Test
This stage runs the LibAD9361 tests available on the repository.
![@startuml LibAD9361Tests Workflow
start
title Test libad9361 Stage Workflow
:Execute LibAD9361Tests stage;
:Define supported boards list;
note right
def supported_boards = [
zynq-zed-adv7511-ad9361-fmcomms2-3',
zynq-zc706-adv7511-ad9361-fmcomms5',
zynq-adrv9361-z7035-fmc',
zynq-zed-adv7511-ad9364-fmcomms4',
pluto']
end note
if (Board supported and branch available?) then (yes)
note right: supported_boards.contains(board) && gauntEnv.libad9361_iio_branch != null
partition "Try Block" {
:Execute Test libad9361 stage;
:Get board IP configuration;
note right: def ip = nebula("update-config -s network-config -f dutip --board-name="+board)
:Clone libad9361-iio repository;
note right: run_i('git clone -b '+ gauntEnv.libad9361_iio_branch + ' ' + gauntEnv.libad9361_iio_repo, true)
:Enter libad9361-iio directory;
note right: dir('libad9361-iio')
partition "Build Setup (within dir)" {
:Create build directory;
note right: sh 'mkdir build'
:Enter build directory;
note right: dir('build')
partition "CMake Build Process" {
:Configure with CMake;
note right: sh 'cmake ..'
:Compile with Make;
note right: sh 'make'
:Run CTest with URI;
note right: sh 'URI_AD9361="ip:'+ip+'" ctest -T test --no-compress-output -V'
}
}
}
partition "Finally Block" {
:Process test results;
note right
dir('libad9361-iio/build') {
xunit([CTest(
deleteOutputFiles: true,
failIfNotNew: true,
pattern: 'Testing/**/*.xml',
skipNoTestFiles: false,
stopProcessingIfError: true
)])
}
end note
}
else (no)
:Skip board testing;
note right: println("LibAD9361Tests: Skipping board: "+board)
endif
:Stage completed;
stop
@enduml](_images/plantuml-2e99995e0aa308ee0ab6df0b822f244f066e89e3.png)
MATLABTests
This stage runs the MATLAB hardware test runner for the target boards.
