FCREPO-1777 Provide Quality Assurance for fcrepo4-vagrant #29
Conversation
commit 7b69ee8 Author: jrgriffiniii <griffinj@lafayette.edu> Date: Fri Oct 30 19:13:45 2015 -0400 FCREPO-1777 #time 5m Modifying the README with instructions for executing the tests locally commit dce5b0a Author: jrgriffiniii <griffinj@lafayette.edu> Date: Fri Oct 30 19:01:18 2015 -0400 FCREPO-1777 #time 2h 46m Attempting to integrate Beaker for the automation of build environments
|
@jrgriffiniii for posterity, can you try and get TravisCI set-up with this on your account, so we know for sure whether or not it is possible? |
…tomation of acceptance tests
… Travis CI environment by installing Vagrant on the build slave
…ssues (these relate to kernel conflict issues within dpkg distributions offered through the default apt repositories)
…Box libsdl1.debian2 and libvpx1 are installed within the build slave environment
…ion issues (related to the integration of certain kernel headers) by using the "all distributions" installer
…rtualBox installer
… issues by restarting the vboxdrv service
…alling the kernel development headers
…efore the kernel headers are installed
…e VirtualBox is installed
…s executed with sudo privileges
…installation bin.
|
@ruebot after many attempts, it appears that I am encountering problems similar to those experienced by Omer Katz: Essentially, the kernel release within the Travis CI build environment (2.6.32, appearing to be based off of a release of Ubuntu 10.04 LTS [Lucid Lynx]) does not permit one to successfully install and run any releases of VirtualBox. I can continue to work to address this, but I shall first need to contact technical support at Travis CI in order to confirm that this is the case. |
|
Are we certain of the Ubuntu version? I know the PHP TravisCI box is Ubuntu 12.04. It might be worth investigating the box that is being built. I've going down this rabbit hole with some Islandora work 😄 ...but at the same time, it is a bit of a rabbit hole testing a virtualized box inside a virtualized box. Maybe worth a tech call discussion to see what we want to do as a group? |
|
@ruebot The problem lies not with the version of Ubuntu, but with the kernel which is integration with the Linux distribution which they are running within the Travis CI environment: I had assumed this to mean that I could assume a release of 2.6.32 (last supported by 10.04 LTS), but as indicated in the blog post by Omer Katz, this may not be the case. Further, as you've indicated, technical support at Travis CI has directed me away from attempting to undertake this approach due to concerns in relation to nested virtualization problems:
Hence, I am skeptical as to whether this can be undertaken in a stable and consistent manner between virtualized OpenVZ environments. |
.travis.yml
Outdated
| @@ -0,0 +1,23 @@ | |||
| language: ruby | |||
There was a problem hiding this comment.
Based on the discussion in this PR, we should probably remove this .travis.yml file, no?
…ting the Beaker acceptance test suite
… that the acceptance tests are *not* executed by Travis CI
README.md
Outdated
|
|
||
| One may validate the state of the environment provisioned by the BASH scripts using [beaker](https://github.com/puppetlabs/beaker). | ||
| The acceptance tests (implemented using [server-spec](http://serverspec.org/)) may be executed locally in the following manner: | ||
| * `rake beaker` |
There was a problem hiding this comment.
When you say "executed locally" in which directory do you mean? Ideally, this does not add any host-machine requirements... however, when I vagrant ssh into the box, rake is not found, and rake1.9.1 beaker does not appear to work regardless of which directory I am in.
Some more details here would be helpful.
There was a problem hiding this comment.
When you say "executed locally" in which directory do you mean? Ideally, this does not add any host-machine requirements
My apologies, this refers to the local environment into which the GitHub repository was cloned. It should be rephrased to explicitly state that the tests are executed outside of the VM instantiated.
There was a problem hiding this comment.
It is probably not realistic to expect host machines to have the rake application (and its dependencies) installed. Would it be possible to instead design your scripts to be run inside of the VM?
…ectory; Restructuring the acceptance test invocations into a shell script "acceptance.sh"; Adjusting the block of README text outlining the executing of the acceptance tests
…are relocated to the "spec" directory
|
@jrgriffiniii, as mentioned here ( #29 (comment) ), ideally this script would be completely self-contained and not require any additional requirements on the host system. Is it not possible to have the appropriate utilities installed within the Vagrant box itself? Here is the error I get when running Instead of debugging this, it would be great if the Vagrant box included everything that would be required to perform this verification from an ssh shell in the box. |
To the best of my understanding, the Vagrant Box serves as a system under test (SUT). Hence, following the approach outlined by Joe Ferris' article to which the previous link resolves, I'm fairly certain that the tests must always be run from the test harness (Beaker), lying external to the SUT. I shall contact members of the Beaker community in order to confirm that this is the case, and that my understanding is accurate. Once I have a response, I can then proceed with diagnosing and resolving the errors identified in your comment. |
I have attempted to implement an acceptance testing suite using the beaker framework, within which I am leveraging server-spec in order to validate the state of the virtualized environment provisioned using the BASH scripts.
Unfortunately, as this requires that a base box be downloaded for the provisioning process, it is highly unlikely that this could be integrated with Travis CI without incurring a timeout. Further, attempting to implement module testing for the BASH scripts themselves by using a testing framework such as the Bats would have required that I (significantly) restructure the scripts themselves.
It may be that the approach which I have undertaken is undesirable, that there are superior frameworks or design patterns, or that more modular testing can be implemented. Please freely override or correct my oversights if this is the case.