Difference between revisions of "Automated Testing - Suggestions"
From Rosalab Wiki
(Created a stub of the page to contain suggestions for our autom. testing. system) |
(Added description of the testing system and the initial set of suggestions) |
||
| Line 1: | Line 1: | ||
| − | + | == The System == | |
| − | + | The automated testing system for ROSA Linux distros operates as follows: | |
| − | + | # It looks for the new ISO images for the appropriate distros on ABF. If they are found, it downloads the images to a local machine. | |
| + | # It creates virtual machines (currently based on qemu-kvm) and checks that the OS boots in Live mode from the ISO images on these machines. | ||
| + | # It automatically installs the OS to these virtual machines from the ISO images and performs several basic adjustments and checks. For example, it checks whether SSH connection to the machine is working, tries to enable the default package repositories and perform a package update, etc. | ||
| + | # [http://autotest.github.com/ Autotest] is used to run functional tests on the live and installed systems. | ||
| + | # The results of the above steps are available [http://fba.rosalinux.ru/autotest/ at FBA] | ||
| + | |||
| + | All this is currently done for ROSA Desktop Fresh 2012, both 32- and 64-bit. | ||
| + | |||
| + | |||
| + | == Suggestions == | ||
| + | |||
| + | Listed here are the suggestions collected so far about the tests and the system features that could be useful, in no particular order. | ||
| + | |||
| + | ==== Profiles ==== | ||
| + | |||
| + | * Profile for ROSA Desktop Fresh GNOME Edition | ||
| + | * Profile for ROSA Desktop Fresh LXDE Edition | ||
| + | |||
| + | ==== Installation Options ==== | ||
| + | |||
| + | * /var on a separate partition | ||
| + | * Installation on a RAID | ||
| + | |||
| + | ==== Kernels ==== | ||
| + | |||
| + | * It could be useful to test the ROSA distros with different kernel flavours available for them. | ||
| + | |||
| + | ==== Tests ==== | ||
| + | |||
| + | * Xfstests - at least for ext4 FS, other file systems can also be tested this way in the future | ||
| + | * LSB test suites (see LSB Distribution Checker) | ||
| + | * Mozmill tests - at least for Firefox (similar to how it is done in OpenQA for OpenSUSE) | ||
| + | * Checking for errors in the kernel modules with KEDR tools | ||
| + | * Regression tests for some of the bugs fixed in the distros | ||
| + | * ? Fuzz tests like Trinity, etc. (may be destructive) | ||
| + | * ? (''What else?'') | ||
| + | |||
| + | ==== Benchmarks ==== | ||
| + | |||
| + | * ? (''Are they really needed here?'') | ||
| + | |||
| + | |||
| + | == Development == | ||
| + | |||
| + | The sources of the system are available at [https://abf.rosalinux.ru/spectre/rosa-autotest ABF]. | ||
Revision as of 15:49, 25 January 2013
Contents
The System
The automated testing system for ROSA Linux distros operates as follows:
- It looks for the new ISO images for the appropriate distros on ABF. If they are found, it downloads the images to a local machine.
- It creates virtual machines (currently based on qemu-kvm) and checks that the OS boots in Live mode from the ISO images on these machines.
- It automatically installs the OS to these virtual machines from the ISO images and performs several basic adjustments and checks. For example, it checks whether SSH connection to the machine is working, tries to enable the default package repositories and perform a package update, etc.
- Autotest is used to run functional tests on the live and installed systems.
- The results of the above steps are available at FBA
All this is currently done for ROSA Desktop Fresh 2012, both 32- and 64-bit.
Suggestions
Listed here are the suggestions collected so far about the tests and the system features that could be useful, in no particular order.
Profiles
- Profile for ROSA Desktop Fresh GNOME Edition
- Profile for ROSA Desktop Fresh LXDE Edition
Installation Options
- /var on a separate partition
- Installation on a RAID
Kernels
- It could be useful to test the ROSA distros with different kernel flavours available for them.
Tests
- Xfstests - at least for ext4 FS, other file systems can also be tested this way in the future
- LSB test suites (see LSB Distribution Checker)
- Mozmill tests - at least for Firefox (similar to how it is done in OpenQA for OpenSUSE)
- Checking for errors in the kernel modules with KEDR tools
- Regression tests for some of the bugs fixed in the distros
- ? Fuzz tests like Trinity, etc. (may be destructive)
- ? (What else?)
Benchmarks
- ? (Are they really needed here?)
Development
The sources of the system are available at ABF.
