Right now, I'm feeling like a slave to a large bureaucracy. They'll run my test, but only if I write a bunch of wrapper code and fill out all of their forms in triplicate. They claim they are making my life easier by providing a "framework", but this framework feels more like an obstacle course.
Automation should:
- Be easy to create
- Be easy to run
- Give prompt feedback
- Make your life easier!
PowerShell is an excellent tool for this:
- Scripts can be written in any text editor, and a decent one is included with recent versions of Windows (PowerShell ISE)
- Scripts can be triggered locally or remotely from the command line, or scheduled to run at a later time, on any recent version of Windows. No compilation is required, so no special tools (like Visual Studio) are needed.
- PowerShell can easily parse XML (useful for loading configurations or saving results), interact with COM (which lets you control IE at the DOM level, among other things), and leverage .NET (which lets you do pretty much anything)
The system I'm learning at the moment seems to violate all of these rules:
- Entries for new tests must be created in a Web UI. These entries must reference separate metadata, which is created in with an internal tool. This metadata in turn references a dll containing the new test function(s). These new test functions must be added to a Visual Studio project of related tests, which will build the dll. (In my current case, these test functions are just wrappers that reference *another* dll...) Untangling this web is daunting to a new-comer, and likely still frustrating to those accustomed to it.
- A functioning local repository (whose setup is non-trivial) is required to interact with the service that runs automated tests. All test code editing must be done in this repository.
- The service that runs automated tests performs extensive setup (possibly including re-imaging the OS) on arbitrarily selected target machine(s) from a pool (after waiting for machine(s) to become available). Waiting 3 hours for official results is par for the course. Tests can be run against local machines, but tests that work here will not necessarily work during official runs.
- Adding a new test to the automation system is a non-trivial and frustrating process, which discourages use of said system.
Testers need to keep abreast of changes in the product, and so must their test scripts. Stifling layers of complexity make it hard to keep these scripts up to date, thus negating their usefulness. An automation system should be simple and flexible, and the path from idea to script should be as quick and painless as possible. Some basic common infrastructure may be necessary, but it should strive to stay out of the way as much as possible.
